Skip to content

[Klaud Cold] Update qwen3.5-fp8-b300-sglang-agentic-mtp SGLang image to nightly-dev-cu13-20260907-30705c00 and move to cluster:b300-dsxe #2881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
functionstackx wants to merge 1 commit into
base: main
Choose a base branch
from
+11 −2
Open

[Klaud Cold] Update qwen3.5-fp8-b300-sglang-agentic-mtp SGLang image to nightly-dev-cu13-20260907-30705c00 and move to cluster:b300-dsxe #2881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7201,10 +7201,10 @@ qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg:

# ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ----------
qwen3.5-fp8-b300-sglang-agentic-mtp:
image: lmsysorg/sglang:v0.5.16-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00
model: Qwen/Qwen3.5-397B-A17B-FP8
model-prefix: qwen3.5
runner: cluster:b300-nv
runner: cluster:b300-dsxe
precision: fp8
framework: sglang
multinode: false
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6921,3 +6921,12 @@
- "Pick up the latest automatic ROCm DeepSeek-V4 optimizations, including fused mHC post/pre plus RMSNorm, gfx950 C4A top-k dispatch, fused C4 compressor GEMMs, fused SWA q/kv RMSNorm plus q FP8 quantization, and medium-batch cooperative top-k tuning."
- "Keep the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings, and explicitly add VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 plus VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to both STP and MTP paths. The current checkpoint's shared-expert path does not satisfy the latest vLLM fusion conditions, so that fusion flag self-disables while preserving recipe parity."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2792

- config-keys:
- qwen3.5-fp8-b300-sglang-agentic-mtp
scenario-type:
- agentic-coding
description:
- "Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 (v0.5.16 release) to lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 (2026-09-07 cu13 dev nightly, digest sha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007, tag commit sgl-project/sglang@30705c00; Docker Hub last pushed 2026-09-07T01:43:42Z), the same tag the Qwen3.5 SGLang AgentX recipes on B200/H200/H100 moved to in #2861/#2862/#2868/#2869. benchmarks/single_node/agentic/qwen3.5_fp8_b300_sglang_mtp.sh is unchanged: fp8 quantization, fp8_e4m3 KV, trtllm_mha attention, flashinfer_trtllm MoE runner, native NEXTN MTP with golden AL 3.39; the TP4 and TP2/EP2 grids are unchanged. The qwen3.5-fp8-b300-sglang-agentic-power-ab key is deliberately not bumped: it is a controlled FP8/FP4 power matrix that requires an identical SGLang build across both precisions."
- "Move the recipe from the retired cluster:b300-nv fleet (launcher and runner labels removed in #2826) to cluster:b300-dsxe so the sweep has a runner to schedule on. The runner change means this is not an append-only bump: the whole curve reruns on DSXE."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2881
Loading