Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,7 @@ the candidate mechanisms.
| [Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198), 2026-10-21; eight Git source inventories in the [owner metadata](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json) | Task/quality/workload separation across text, agentic coding, work agents, TTS, full-duplex interaction and image/video; selected pinned AIPerf/realtime/FDB/video sources rechecked and the current InferRSI metric contract inspected | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [task selection](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md) and [command templates](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md). Candidate guidance; source review and bounded smoke do not qualify all tasks or establish a performance gain. Replay scores, reference parity and task quality retain separate meanings. |
| [NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Speech.cpp/tree/20999cc921d2b14aa58708f61756a8b2e787ba20), inspected 2026-11-01 | Selected cache-aware encoder/config, runner, SDK bounds, endpoint policy, geometry test/CMake and ASR benchmark; complete 14-file readset and Apache-2.0 / parakeet MIT attribution retained in the reference | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [cache-aware ASR stream boundaries](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md): first/steady mel geometry and decoded clock, stream-slot reset/reuse, EOU/EOF/reset and model-test SKIP boundaries. Source-derived candidate / empirical not_run; benchmark figures remain author-reported with precision, FE and trial limits. No model/device/quality/long-audio/load/service qualification. |
| [yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizhang/faster-cosyvoice/tree/f1049c59196c2c1a890079abc56056b43f27cd1d), inspected 2026-11-03; overall source rights UNKNOWN, selected file headers retain Apache-2.0 and unresolved inherited attribution | Bounded PCM client/server-credit, chunk planner, session/worker, Token2Wav and timing/quality-report inspection; [exact readset and per-file rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json) | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md): route-origin credit versus client zero-buffer playback, same-session Mel dependencies, cumulative vocoding and partial-cost/token-derived denominators. Original analysis with immutable links only; candidate / empirical not_run / efficacy not_evaluated. Author reports remain separate; no copied implementation/media, new runtime or hardware/quality/service qualification. |
| NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives, hash-pinned SMI document; Triton v3.2.0 @ `9641643da6c52000c807b5eeed05edaec4402a67`; PyTorch 2.7; [robust-kbench paper v1](https://arxiv.org/html/2509.14279v1); [typed source inventory](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json) | Selected clock/cache/replay, timing/graph, metric-name, flush, extension-identity and benchmark-exploit contracts, inspected 2026-11-15 | Extend `inference-benchmark-design` with [GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md). Candidate / empirical not_run / efficacy not_evaluated; normalization/adoption is an unverified author proposal, four task-supplied cases lack raw evidence, and source review does not establish a gain or current-device metric availability. |

## SGLang versioned entry selection

Expand Down
8 changes: 8 additions & 0 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,14 @@ separate from the public release's inspected code paths.
| Accelerator / Host / service boundary | Observe / Verify: select task quality and reference parity; freeze workload, timing, SLOs, units and failure accounting; distinguish agent replay from live task success | [inference-benchmark-design task reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md) and [commands](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md) | Cross-task measurement guidance, not new measured hardware coverage. A text HTTP smoke cannot validate TTS, duplex, image/video quality, repeated A/B or production capacity. No runtime adapter or automatic evaluator is added. |
| Accelerator / Host / decoded-audio boundary | Observe / Verify: derive cache-aware ASR startup/steady mel geometry and decoded frontier; distinguish slot reset/reuse, EOU, EOF and full stream reset; retain optional-model test coverage | [inference-benchmark-design ASR reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md) | NeMo-Speech.cpp `20999cc921d2b14aa58708f61756a8b2e787ba20`; selected source and test definitions, author ASR benchmark only. Candidate / empirical not_run; no executed model, device, long-stream/load, ASR quality or service gain and no new runtime adapter. |

[GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md)
extends `inference-benchmark-design` with clock/cache controls, conditional
within-run attribution, work-count signatures and timing-trap checks. This is
evaluation guidance, not a hardware-resource optimization mechanism or added
numbered-layer coverage; the owner's N/A declaration remains applicable.
Candidate / empirical not_run / efficacy not_evaluated; task-supplied numerical
observations are unverified and establish no device or service gain.

## SGLang scope selection

Use the [versioned entry comparison](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md#choose-among-source-pins)
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:fbc4d29e40a03beefb6c204f5defd7f817f5d5f68fb102926b3e746bc8801eee`
- Input digest: `sha256:d076a0096bce36fa313827cd89396972001950a685f39a087ba5e4b5d0161664`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -132,7 +132,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
| [dense-performance-evidence](../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
| [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval |
| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>7 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)<br>Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 13 readset(s) | — |
| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>8 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>Task-specific benchmark selection (issue #60) (NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)<br>Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 14 readset(s) | — |
| [inference-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval |
| [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval |
| [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point<br>5 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"repository": "https://github.com/InferMatrixLab/InferRSI",
"source_revision": "1e9c140f489f83ca3ecfba0e6e8b416ddb5091cc",
"source_revision": "3671a98ad340fb19e6595d1c8d818c7e9ca2577a",
"registry": {
"path": "src/inferrsi/knowledge/data/skill_library/skill-index.json",
"sha256": "7904275c3383380db179287171e1430617aaa9ac4f439291eff8a71426987fbb"
Expand Down
Loading
Loading