From a588190b26c839f5719a25215e9d65e004bbb469 Mon Sep 17 00:00:00 2001 From: functionstackx <47992694+functionstackx@users.noreply.github.com> Date: Mon, 7 Sep 2026 21:41:06 -0400 Subject: [PATCH 1/2] [Klaud Cold] Update glm5.2-fp4-b300-sglang-agentic-mtp SGLang image to nightly-dev-cu13-20260907-30705c00 and move to cluster:b300-dsxe Co-Authored-By: Claude Fable 5.1 --- configs/nvidia-master.yaml | 4 ++-- perf-changelog.yaml | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/configs/nvidia-master.yaml b/configs/nvidia-master.yaml index 0a6c86df6d..cd9fa442aa 100644 --- a/configs/nvidia-master.yaml +++ b/configs/nvidia-master.yaml @@ -8988,10 +8988,10 @@ glm5.2-fp8-h200-dynamo-sglang-agentic-mtp-2p2d: # HiCache the working set outgrows HBM past conc # 8 and the radix hit rate collapses to <0.1. glm5.2-fp4-b300-sglang-agentic-mtp: - image: lmsysorg/sglang:v0.5.16-cu130 + image: lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 model: nvidia/GLM-5.2-NVFP4 model-prefix: glm5.2 - runner: cluster:b300-nv + runner: cluster:b300-dsxe precision: fp4 framework: sglang multinode: false diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 6b249ea15e..8df180a3ae 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6921,3 +6921,12 @@ - "Pick up the latest automatic ROCm DeepSeek-V4 optimizations, including fused mHC post/pre plus RMSNorm, gfx950 C4A top-k dispatch, fused C4 compressor GEMMs, fused SWA q/kv RMSNorm plus q FP8 quantization, and medium-batch cooperative top-k tuning." - "Keep the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings, and explicitly add VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 plus VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to both STP and MTP paths. The current checkpoint's shared-expert path does not satisfy the latest vLLM fusion conditions, so that fusion flag self-disables while preserving recipe parity." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2792 + +- config-keys: + - glm5.2-fp4-b300-sglang-agentic-mtp + scenario-type: + - agentic-coding + description: + - "Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 (v0.5.16 release) to lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 (2026-09-07 cu13 dev nightly, digest sha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007, tag commit sgl-project/sglang@30705c00; Docker Hub last pushed 2026-09-07T01:43:42Z), the same tag the B200 GLM-5.2 SGLang AgentX recipes moved to in #2863/#2864. benchmarks/single_node/agentic/glm5.2_fp4_b300_sglang_mtp.sh is unchanged: TP8 low-latency arm, fp8 KV, cutedsl bf16 GEMM, EAGLE MTP at three speculative tokens with golden AL 2.99, 270 GB/rank HiCache target pool; the conc [1, 4, 8, 12, 16] grid is unchanged. Overlaps with the open #2829, which repoints the same key to DSXE and extends the grid to 48; whichever lands second rebases." + - "Move the recipe from the retired cluster:b300-nv fleet (launcher and runner labels removed in #2826) to cluster:b300-dsxe so the sweep has a runner to schedule on. The runner change means this is not an append-only bump: the whole curve reruns on DSXE." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2880 From f7ab3517d4511aea193230018161057d27d91fa9 Mon Sep 17 00:00:00 2001 From: functionstackx <47992694+functionstackx@users.noreply.github.com> Date: Mon, 7 Sep 2026 22:08:17 -0400 Subject: [PATCH 2/2] Re-pin to the 2026-09-08 cu13 nightly, which carries sglang#38318 (EAGLE DSA fp8 read-door None guard) Co-Authored-By: Claude Fable 5.1 --- configs/nvidia-master.yaml | 2 +- perf-changelog.yaml | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/configs/nvidia-master.yaml b/configs/nvidia-master.yaml index cd9fa442aa..94b5b50887 100644 --- a/configs/nvidia-master.yaml +++ b/configs/nvidia-master.yaml @@ -8988,7 +8988,7 @@ glm5.2-fp8-h200-dynamo-sglang-agentic-mtp-2p2d: # HiCache the working set outgrows HBM past conc # 8 and the radix hit rate collapses to <0.1. glm5.2-fp4-b300-sglang-agentic-mtp: - image: lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 + image: lmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b model: nvidia/GLM-5.2-NVFP4 model-prefix: glm5.2 runner: cluster:b300-dsxe diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 8df180a3ae..78d52b7a3b 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6929,4 +6929,5 @@ description: - "Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 (v0.5.16 release) to lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 (2026-09-07 cu13 dev nightly, digest sha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007, tag commit sgl-project/sglang@30705c00; Docker Hub last pushed 2026-09-07T01:43:42Z), the same tag the B200 GLM-5.2 SGLang AgentX recipes moved to in #2863/#2864. benchmarks/single_node/agentic/glm5.2_fp4_b300_sglang_mtp.sh is unchanged: TP8 low-latency arm, fp8 KV, cutedsl bf16 GEMM, EAGLE MTP at three speculative tokens with golden AL 2.99, 270 GB/rank HiCache target pool; the conc [1, 4, 8, 12, 16] grid is unchanged. Overlaps with the open #2829, which repoints the same key to DSXE and extends the grid to 48; whichever lands second rebases." - "Move the recipe from the retired cluster:b300-nv fleet (launcher and runner labels removed in #2826) to cluster:b300-dsxe so the sweep has a runner to schedule on. The runner change means this is not an append-only bump: the whole curve reruns on DSXE." + - "Re-pin from lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 to lmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b (2026-09-08 cu13 dev nightly, digest sha256:9a352a35c973a2357372e85f3bcb5388b6b3c46c1329165987260f3b089647dc; Docker Hub last pushed 2026-09-08T01:40:59Z, tag commit sgl-project/sglang@20ca564b). The 2026-09-07 build carries an unguarded kv_index_translator.translate_dcp_read_ids call on the DSA fp8 KV read path that the EAGLE draft backend never binds, so GLM-5.2 MTP runs crash intermittently with AttributeError (observed on the MI355X FP8 sibling in run 34173459478 after 74 minutes of serving). sgl-project/sglang#38318 (merged 2026-09-07T20:03Z) adds the None guard and is six commits behind 20ca564b." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2880