diff --git a/benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh b/benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh index 28d901417b..d0d732176a 100755 --- a/benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh +++ b/benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh @@ -72,6 +72,12 @@ VLLM_CMD=( --port "$PORT" --tensor-parallel-size "$TP" --gpu-memory-utilization 0.90 + # The upstream nightly does not torch-compile MiniMaxM3SparseForConditionalGeneration, + # and with VLLM_USE_BREAKABLE_CUDAGRAPH=0 the default FULL_AND_PIECEWISE graph + # mode aborts at init ("piecewise CUDA graphs unavailable ... Set + # VLLM_USE_BREAKABLE_CUDAGRAPH=1 or cudagraph_mode=NONE/FULL"; run 34174124043). + # Capture full decode-only graphs, as the MI355X sibling does on its nightly (#2825). + --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' --kv-cache-dtype fp8 --block-size 128 --language-model-only diff --git a/configs/amd-master.yaml b/configs/amd-master.yaml index cd10677cd7..2c54c8268c 100644 --- a/configs/amd-master.yaml +++ b/configs/amd-master.yaml @@ -1565,7 +1565,7 @@ glm5.2-fp8-mi325x-sglang-agentic-mtp: - { tp: 8, ep: 1, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 3, 4, 5, 6, 8] } minimaxm3-fp8-mi325x-vllm-agentic-mtp: - image: vllm/vllm-openai-rocm:v0.27.1 + image: vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 model: MiniMaxAI/MiniMax-M3-MXFP8 model-prefix: minimaxm3 runner: cluster:mi325x-amds diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 6b249ea15e..66c950bb81 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6921,3 +6921,12 @@ - "Pick up the latest automatic ROCm DeepSeek-V4 optimizations, including fused mHC post/pre plus RMSNorm, gfx950 C4A top-k dispatch, fused C4 compressor GEMMs, fused SWA q/kv RMSNorm plus q FP8 quantization, and medium-batch cooperative top-k tuning." - "Keep the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings, and explicitly add VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 plus VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to both STP and MTP paths. The current checkpoint's shared-expert path does not satisfy the latest vLLM fusion conditions, so that fusion flag self-disables while preserving recipe parity." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2792 + +- config-keys: + - minimaxm3-fp8-mi325x-vllm-agentic-mtp + scenario-type: + - agentic-coding + description: + - "Update vLLM ROCm image from vllm/vllm-openai-rocm:v0.27.1 (v0.27.1 release) to vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 (2026-09-07 upstream ROCm nightly, digest sha256:74d4a95f3ae672ecddf9acb7296917def82d9eca51687fa2862ae72b03ff1907, tag commit vllm-project/vllm@d9105ea8; Docker Hub last pushed 2026-09-07T05:26:48Z). benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh is unchanged: TRITON_ATTN attention, fp8 KV, block-size 128, EAGLE3 speculative decoding with the Inferact MiniMax-M3 EAGLE3-GQA draft and the committed golden synthetic acceptance length, minimax_m3 tool-call and reasoning parsers; the search space is unchanged. The upstream commit-pinned ROCm nightly is the same vllm commit the B200 MiniMax-M3 AgentX recipe moved to in #2860. Note that vllm-openai-rocm commit-nightly tags have expired from Docker Hub within days in the past; node squash caches keep merged configs running, but a re-pin to a durable tag may be needed later." + - "Add --compilation-config cudagraph_mode=FULL_DECODE_ONLY to the serve command. The upstream nightly does not torch-compile MiniMaxM3SparseForConditionalGeneration, so with VLLM_USE_BREAKABLE_CUDAGRAPH=0 the default FULL_AND_PIECEWISE mode aborts at engine init with piecewise CUDA graphs unavailable (first sweep, run 34174124043, eval cell); full decode-only graphs are what the MI355X MiniMax-M3 sibling runs on its nightly (#2825)." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2872