diff --git a/benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh b/benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh index 11e40a70eb..ee35417d9e 100644 --- a/benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh +++ b/benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh @@ -62,10 +62,8 @@ export AIPERF_HTTP_TCP_USER_TIMEOUT=900000 # Outlast the client pool so the race cannot occur. export SGLANG_TIMEOUT_KEEP_ALIVE=900 # The DSA indexer's top-k v2 kernel (default since v0.5.14) is JIT-compiled -# from CUDA-only source (cooperative_groups.h) and cannot build for gfx950; -# v1 dispatches to the precompiled HIP op in sgl-kernel (upstream MI355X CI -# runs DSA models the same way). -export SGLANG_OPT_USE_TOPK_V2=false +# PR #36684 & PR #36851 turned the v2 fused top-k on for GLM-5.x on ROCm +export SGLANG_OPT_USE_TOPK_V2=true # HiCache L2 (host DRAM), optionally extended with Mooncake L3. # KV_OFFLOADING=dram requires KV_OFFLOAD_BACKEND=hicache or mooncake. diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 390afc2ecf..7fb1599840 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6969,3 +6969,12 @@ - "Use SGLang's current W4A4 MegaMoE and DP LM-head flags for DP-attention instead of the deprecated MegaMoE environment variables." - "Resolve draft-model jobs through the existing B200 SGLang speculative recipe and the staged DeepSeek-V4-Pro-0813 checkpoint." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2821 + +- config-keys: + - glm5.2-fp4-mi355x-sglang-agentic-mtp + scenario-type: + - agentic-coding + description: + - "Set SGLANG_OPT_USE_TOPK_V2=true in benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh. The recipe forced the v1 top-k because the DSA indexer's v2 fused kernel would not build for gfx950; sgl-project/sglang PR #36684 and PR #36851 add the ROCm path, so v2 now dispatches natively." + - "No other recipe change: image, TP4/EP4 HiCache and TP8/EP1 resident arms, MTP, and concurrency grids are unchanged." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2915