From 4ef1fc15d3ae9fe4346e5c6a83f8825b732d1352 Mon Sep 17 00:00:00 2001 From: functionstackx <47992694+functionstackx@users.noreply.github.com> Date: Mon, 7 Sep 2026 20:40:57 -0400 Subject: [PATCH 1/2] [Klaud Cold] Update minimaxm3-fp8-mi300x-vllm-agentic-mtp vLLM ROCm image to nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 Co-Authored-By: Claude Fable 5.1 --- configs/amd-master.yaml | 2 +- perf-changelog.yaml | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/configs/amd-master.yaml b/configs/amd-master.yaml index cd10677cd7..2510dc3b8d 100644 --- a/configs/amd-master.yaml +++ b/configs/amd-master.yaml @@ -1533,7 +1533,7 @@ dsv4-fp4-mi355x-atom-disagg: - "DECODE_NODES=1" # 1P1D TP8 minimaxm3-fp8-mi300x-vllm-agentic-mtp: - image: vllm/vllm-openai-rocm:v0.27.1 + image: vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 model: MiniMaxAI/MiniMax-M3-MXFP8 model-prefix: minimaxm3 runner: cluster:mi300x-amd diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 6b249ea15e..075e858763 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6921,3 +6921,11 @@ - "Pick up the latest automatic ROCm DeepSeek-V4 optimizations, including fused mHC post/pre plus RMSNorm, gfx950 C4A top-k dispatch, fused C4 compressor GEMMs, fused SWA q/kv RMSNorm plus q FP8 quantization, and medium-batch cooperative top-k tuning." - "Keep the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings, and explicitly add VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 plus VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to both STP and MTP paths. The current checkpoint's shared-expert path does not satisfy the latest vLLM fusion conditions, so that fusion flag self-disables while preserving recipe parity." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2792 + +- config-keys: + - minimaxm3-fp8-mi300x-vllm-agentic-mtp + scenario-type: + - agentic-coding + description: + - "Update vLLM ROCm image from vllm/vllm-openai-rocm:v0.27.1 (v0.27.1 release) to vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 (2026-09-07 upstream ROCm nightly, digest sha256:74d4a95f3ae672ecddf9acb7296917def82d9eca51687fa2862ae72b03ff1907, tag commit vllm-project/vllm@d9105ea8; Docker Hub last pushed 2026-09-07T05:26:48Z). benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh is unchanged: TRITON_ATTN attention, fp8 KV, block-size 128, EAGLE3 speculative decoding with the Inferact MiniMax-M3 EAGLE3-GQA draft and the committed golden synthetic acceptance length, minimax_m3 tool-call and reasoning parsers; the search space is unchanged. The upstream commit-pinned ROCm nightly is the same vllm commit the B200 MiniMax-M3 AgentX recipe moved to in #2860. Note that vllm-openai-rocm commit-nightly tags have expired from Docker Hub within days in the past; node squash caches keep merged configs running, but a re-pin to a durable tag may be needed later." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2873 From 2e3b73a22f68dc2013d3fbabacc1fc31c0feb880 Mon Sep 17 00:00:00 2001 From: functionstackx <47992694+functionstackx@users.noreply.github.com> Date: Mon, 7 Sep 2026 20:57:49 -0400 Subject: [PATCH 2/2] Capture FULL_DECODE_ONLY CUDA graphs so the nightly boots MiniMax-M3 with breakable graphs off Co-Authored-By: Claude Fable 5.1 --- benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh | 6 ++++++ perf-changelog.yaml | 1 + 2 files changed, 7 insertions(+) diff --git a/benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh b/benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh index 4234a66490..32a77f2ee8 100755 --- a/benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh +++ b/benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh @@ -162,6 +162,12 @@ VLLM_CMD=( --trust-remote-code --block-size 128 --gpu-memory-utilization 0.90 + # The upstream nightly does not torch-compile MiniMaxM3SparseForConditionalGeneration, + # and with VLLM_USE_BREAKABLE_CUDAGRAPH=0 the default FULL_AND_PIECEWISE graph + # mode aborts at init ("piecewise CUDA graphs unavailable ... Set + # VLLM_USE_BREAKABLE_CUDAGRAPH=1 or cudagraph_mode=NONE/FULL"; run 34174124043). + # Capture full decode-only graphs, as the MI355X sibling does on its nightly (#2825). + --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' --enable-chunked-prefill --max-num-batched-tokens 16384 --language-model-only diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 075e858763..74791df51b 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6928,4 +6928,5 @@ - agentic-coding description: - "Update vLLM ROCm image from vllm/vllm-openai-rocm:v0.27.1 (v0.27.1 release) to vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 (2026-09-07 upstream ROCm nightly, digest sha256:74d4a95f3ae672ecddf9acb7296917def82d9eca51687fa2862ae72b03ff1907, tag commit vllm-project/vllm@d9105ea8; Docker Hub last pushed 2026-09-07T05:26:48Z). benchmarks/single_node/agentic/minimaxm3_fp8_mi300x_mtp.sh is unchanged: TRITON_ATTN attention, fp8 KV, block-size 128, EAGLE3 speculative decoding with the Inferact MiniMax-M3 EAGLE3-GQA draft and the committed golden synthetic acceptance length, minimax_m3 tool-call and reasoning parsers; the search space is unchanged. The upstream commit-pinned ROCm nightly is the same vllm commit the B200 MiniMax-M3 AgentX recipe moved to in #2860. Note that vllm-openai-rocm commit-nightly tags have expired from Docker Hub within days in the past; node squash caches keep merged configs running, but a re-pin to a durable tag may be needed later." + - "Add --compilation-config cudagraph_mode=FULL_DECODE_ONLY to the serve command. The upstream nightly does not torch-compile MiniMaxM3SparseForConditionalGeneration, so with VLLM_USE_BREAKABLE_CUDAGRAPH=0 the default FULL_AND_PIECEWISE mode aborts at engine init with piecewise CUDA graphs unavailable (first sweep, run 34174124043, eval cell); full decode-only graphs are what the MI355X MiniMax-M3 sibling runs on its nightly (#2825)." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2873