Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,12 @@ VLLM_CMD=(
--port "$PORT"
--tensor-parallel-size "$TP"
--gpu-memory-utilization 0.90
# The upstream nightly does not torch-compile MiniMaxM3SparseForConditionalGeneration,
# and with VLLM_USE_BREAKABLE_CUDAGRAPH=0 the default FULL_AND_PIECEWISE graph
# mode aborts at init ("piecewise CUDA graphs unavailable ... Set
# VLLM_USE_BREAKABLE_CUDAGRAPH=1 or cudagraph_mode=NONE/FULL"; run 34174124043).
# Capture full decode-only graphs, as the MI355X sibling does on its nightly (#2825).
--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
--kv-cache-dtype fp8
--block-size 128
--language-model-only
Expand Down
2 changes: 1 addition & 1 deletion configs/amd-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1565,7 +1565,7 @@ glm5.2-fp8-mi325x-sglang-agentic-mtp:
- { tp: 8, ep: 1, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 3, 4, 5, 6, 8] }

minimaxm3-fp8-mi325x-vllm-agentic-mtp:
image: vllm/vllm-openai-rocm:v0.27.1
image: vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36
model: MiniMaxAI/MiniMax-M3-MXFP8
model-prefix: minimaxm3
runner: cluster:mi325x-amds
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6921,3 +6921,12 @@
- "Pick up the latest automatic ROCm DeepSeek-V4 optimizations, including fused mHC post/pre plus RMSNorm, gfx950 C4A top-k dispatch, fused C4 compressor GEMMs, fused SWA q/kv RMSNorm plus q FP8 quantization, and medium-batch cooperative top-k tuning."
- "Keep the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings, and explicitly add VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 plus VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to both STP and MTP paths. The current checkpoint's shared-expert path does not satisfy the latest vLLM fusion conditions, so that fusion flag self-disables while preserving recipe parity."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2792

- config-keys:
- minimaxm3-fp8-mi325x-vllm-agentic-mtp
scenario-type:
- agentic-coding
description:
- "Update vLLM ROCm image from vllm/vllm-openai-rocm:v0.27.1 (v0.27.1 release) to vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 (2026-09-07 upstream ROCm nightly, digest sha256:74d4a95f3ae672ecddf9acb7296917def82d9eca51687fa2862ae72b03ff1907, tag commit vllm-project/vllm@d9105ea8; Docker Hub last pushed 2026-09-07T05:26:48Z). benchmarks/single_node/agentic/minimaxm3_fp8_mi325x_mtp.sh is unchanged: TRITON_ATTN attention, fp8 KV, block-size 128, EAGLE3 speculative decoding with the Inferact MiniMax-M3 EAGLE3-GQA draft and the committed golden synthetic acceptance length, minimax_m3 tool-call and reasoning parsers; the search space is unchanged. The upstream commit-pinned ROCm nightly is the same vllm commit the B200 MiniMax-M3 AgentX recipe moved to in #2860. Note that vllm-openai-rocm commit-nightly tags have expired from Docker Hub within days in the past; node squash caches keep merged configs running, but a re-pin to a durable tag may be needed later."
- "Add --compilation-config cudagraph_mode=FULL_DECODE_ONLY to the serve command. The upstream nightly does not torch-compile MiniMaxM3SparseForConditionalGeneration, so with VLLM_USE_BREAKABLE_CUDAGRAPH=0 the default FULL_AND_PIECEWISE mode aborts at engine init with piecewise CUDA graphs unavailable (first sweep, run 34174124043, eval cell); full decode-only graphs are what the MI355X MiniMax-M3 sibling runs on its nightly (#2825)."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2872
Loading