perf(sglang): use DSpark6 for B300 DSV4 AgentX / B300 DSV4 AgentX 使用 DSpark6 - #2918
perf(sglang): use DSpark6 for B300 DSV4 AgentX / B300 DSV4 AgentX 使用 DSpark6#2918nvpohanh wants to merge 4 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked whether the new hardcoded MODEL_BASENAME == "DeepSeek-V4-Pro-0813" check in runners/launch_b300-dsxe.sh conflicts with the pre-existing STAGED_MODELS entry for that same basename — it doesn't: the new check runs first and simply reroutes this one model to the /data/models shared mount instead of /scratch/models, which is an intentional override, not a race or dead branch.
Extended reasoning...
The two inline findings already flagged are substantive: the new test in runners/test_slurm_utils.py freezes literal source strings (SLURM_ACCOUNT value, HF cache export lines, mount-dir strings) from launch_b300-dsxe.sh, which is brittle and against the repo's stated test-quality guidance of not pinning incidental implementation details; and switching spec-decoding from "mtp" to "draft_model" in configs/nvidia-master.yaml silently drops the CI scheduling priority boost since utils/ci_priority.py only recognizes mtp/eagle/eagle3 in configs/ci-priority.yaml, with no "draft_model" key added anywhere in this diff. Both are confirmed by reading the referenced files/diff.
I independently verified the one ruled-out candidate: STAGED_MODELS in runners/launch_b300-dsxe.sh does already contain "DeepSeek-V4-Pro-0813" (line 35), and the new basename check is placed as an if before the elif that checks STAGED_MODELS, so for this specific model the new branch simply takes precedence and reroutes to /data/models — it is an intentional, non-conflicting override rather than a bug, consistent with the PR's stated goal of using shared model weights.
Given the two confirmed findings are already surfaced as inline comments, I am not restating them here. This defer note only adds the one additional check I performed myself, which rules out a plausible ordering concern in the same file without duplicating information already presented.
| def test_b300_dsxe_draft_model_uses_public_paths_and_writable_hf_cache() -> None: | ||
| launcher = (REPO_ROOT / "runners/launch_b300-dsxe.sh").read_text() | ||
|
|
||
| assert 'SLURM_ACCOUNT="benchmark"' in launcher | ||
| assert 'SQUASH_DIR="/data/home/sa-gha-runner/squash"' in launcher | ||
| assert 'SHARED_MODEL_ROOT="/data/models"' in launcher | ||
| assert 'WRITABLE_MODELS_DIR="/data/home/sa-gha-runner/models"' in launcher | ||
| assert '[[ "$MODEL_BASENAME" == "DeepSeek-V4-Pro-0813" ]]' in launcher | ||
| assert 'MODEL_MOUNT_DIR="$SHARED_MODEL_ROOT"' in launcher | ||
| assert "nv-gha-runner" not in launcher | ||
| assert ( | ||
| '[[ "$SPEC_DECODING" == "mtp" || "$SPEC_DECODING" == "draft_model" ]]' | ||
| in launcher | ||
| ) | ||
| assert 'export HF_HOME="$HF_CACHE_CONTAINER_DIR"' in launcher | ||
| assert 'export HF_HUB_CACHE="$HF_CACHE_CONTAINER_DIR/hub"' in launcher | ||
| assert 'export HF_XET_CACHE="$HF_CACHE_CONTAINER_DIR/xet"' in launcher | ||
| assert '"$HF_CACHE_HOST_DIR:$HF_CACHE_CONTAINER_DIR"' in launcher |
There was a problem hiding this comment.
🟡 (optional) New test freezes literal source-code strings from launch_b300-dsxe.sh (e.g. SLURM_ACCOUNT="benchmark", SQUASH_DIR path, exact HF_HOME/HF_HUB_CACHE export lines) as raw substring assertions, directly violating AGENTS.md Test quality: "Do not freeze current recipe counts, hardware/framework inventories, image tags, pins, enum values, or source-code strings in assertions." Any harmless refactor of the launcher (renaming a var, reformatting a line, changing quoting) breaks this test even though behavior is unchanged. Fix: replace with a behavioral test that sources/executes the relevant launcher logic (as the file's other tests do via run_bash) and asserts on the resulting env vars/mount args, not on literal script text.
Extended reasoning...
The test does launcher = Path(...).read_text() then assert 'SLURM_ACCOUNT="benchmark"' in launcher and similar literal checks for MODEL_MOUNT_DIR, HF_HOME/HF_HUB_CACHE/HF_XET_CACHE export statements, and the container-mounts string. Every other test in this file (e.g. test_copy_fixed_sequence_results_, test_patch_srt_eval_dispatch_) instead runs the shell function/patch and asserts on observable output, per the file's own established pattern and AGENTS.md's explicit prohibition (line 23) on freezing source-code strings. A future contributor who reformats or renames these variables for clarity gets a spurious CI failure unrelated to any behavior change.
Verification: nit. The new test_b300_dsxe_draft_model_uses_public_paths_and_writable_hf_cache (runners/test_slurm_utils.py:151-168) does launcher = (REPO_ROOT / "runners/launch_b300-dsxe.sh").read_text() and then a series of raw substring assertions against the launcher's source text: 'SLURM_ACCOUNT="benchmark"', 'SQUASH_DIR="/data/home/sa-gha-runner/squash"',… | nit (test-quality convention…
| - { tp: 8, kv-offloading: none, spec-decoding: draft_model, conc-list: [1, 4, 8, 16, 32] } | ||
| - { tp: 8, ep: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, spec-decoding: draft_model, conc-list: [32, 64, 128, 256, 384, 512, 576], router: { name: sglang-router, version: "0.3.2" } } |
There was a problem hiding this comment.
🟡 (optional) Changing spec-decoding from mtp to draft_model for this recipe silently drops the CI scheduling priority boost: utils/ci_priority.py's calculate_priority looks up adjustments.spec-decoding[entry['spec-decoding']] against configs/ci-priority.yaml, which only defines mtp/eagle/eagle3 (each +0.75), not draft_model, so these two search-space rows now score +0 instead of +0.75 and queue behind other jobs with no functional reason. Pre-existing gap (3 other draft_model entries already hit it before this PR) but this diff doubles the affected population by converting 2 more rows from mtp to draft_model. Fix: add a draft_model key (matching mtp's weight, or the intended weight for non-MTP speculative decoding) to adjustments.spec-decoding in configs/ci-priority.yaml.
Extended reasoning...
utils/ci_priority.py:161-164 computes score += adjustments.get('spec-decoding', {}).get(str(entry.get('spec-decoding', '')), 0). configs/ci-priority.yaml:19-22 only has mtp/eagle/eagle3 mapped to 0.75; there is no 'draft_model' key anywhere in that file (grep confirms 0 hits, even after this PR). Before this diff, dsv4-fp4-b300-sglang-agentic-hicache-mtp's two search-space entries had spec-decoding: mtp and got +0.75 priority in run-sweep.yml's CI queue ordering; after this diff they read spec-decoding: draft_model and get +0, so the resulting benchmark jobs sort lower in the priority queue purely from this label swap, unrelated to the DSpark change's actual urgency. Verified: base commit already had 3 draft_model entries in nvidia-master.yaml (lines showing they predate this PR) that hit the same gap, so the mechanism is pre-existing, but this PR converts 2 more rows onto the ungapped path, widening exposure.
Verification: nit. The mechanism is real and reachable. configs/ci-priority.yaml only maps spec-decoding mtp/eagle/eagle3 to 0.75 (no draft_model key anywhere). utils/ci_priority.py:161-163 uses an exact-match lookup with default 0: score += _decimal(adjustments.get("spec-decoding", {}).get(str(entry.get("spec-decoding", "")), 0)) — and unlike framework-prefix there is no prefix fallback for… | nit.…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34351291078 |
…agentic-hicache-dspark6-0813-refresh # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34448330157 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34448330157 |
…agentic-hicache-dspark6-0813-refresh # Conflicts: # perf-changelog.yaml
1 participant
Summary
Note
Medium Risk
Changes benchmark serving flags, memory limits, and cluster launcher/model paths for a production AgentX config; mis-tuning could cause OOM or wrong throughput numbers, but scope is isolated to B300 DSXE AgentX.
Overview
B300 AgentX moves from DeepSeek-V4-Pro + native EAGLE/MTP to DeepSeek-V4-Pro-0813 with DSpark block size 6 (1 speculative step, 7 draft tokens). Throughput runs use golden synthetic acceptance length 3.77; eval still uses real verification.
The agentic SGLang recipe turns on
--enable-dp-lm-headand--enable-w4a4-mxfp4-megamoefor DP-attention MegaMoE, drops the old DeepGEMM FP4/MXF4 env toggles, and retunesmem-fraction-static(including 0.90 from concurrency 32 on DEP8) for MegaMoE workspace headroom.nvidia-masterpointsdsv4-fp4-b300-sglang-agentic-hicache-mtpat a newer nightly image,cluster:b300-dsxe,spec-decoding: draft_model, and expanded concurrency sweeps (e.g. 64/128 on the HiCache DP path).launch_b300-dsxe.shloads Pro-0813 from/data/models, treatsdraft_modellikemtpfor script selection, and mounts a writable Hugging Face/Xet cache into the container.perf-changelog.yamland a launcher regression test document the new paths.Reviewed by Cursor Bugbot for commit 3f52b08. Bugbot is set up for automated code reviews on this repo. Configure here.