feat: add GB300 MiniMax-M3 AgentX recipes / 新增 GB300 MiniMax-M3 AgentX 配置 - #3232
RohitNagraj wants to merge 3 commits into
Conversation
新增 GB300 MiniMax-M3 Dynamo-vLLM AgentX 聚合与分离式部署配置，固定运行依赖，并更新 srt-slurm 以支持各工作进程独立配置 Mooncake 存储。
|
Thanks for the contribution!
中文感谢你的贡献！
|
将 GB300 MiniMax-M3 配置的变更日志关联到 PR #3232。
| #!/usr/bin/env bash | ||
| set -eo pipefail | ||
| # Download, verify, and install the published wheel inside each backend container. | ||
| uv pip install --system --no-deps --reinstall --require-hashes \ |
There was a problem hiding this comment.
thanks for the contribution @RohitNagraj but any specific reason that mooncake nightly need to installed here. When will vllm docker be updated instead with this new version?
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| set -eo pipefail | ||
| # Download, verify, and install the published wheel inside each backend container. | ||
| uv pip install --system --no-deps --reinstall --require-hashes \ | ||
| 'https://test-files.pythonhosted.org/packages/f3/00/fb2847f5564864be132d0f70384dde5688689d9f91b2c81d2609632f36e4/mooncake_transfer_engine_cuda13-0.3.14.dev20260910-cp312-cp312-manylinux_2_28_aarch64.whl#sha256=c55fcc42cf189fcdecdcdf9d776ef84b348a9970a9bd3aad2fad64cb2d9e6f82' |
There was a problem hiding this comment.
🟡 (optional) All four disaggregated GB300 MiniMax-M3 recipes fail their setup step once this wheel is pruned, since it is fetched from test-files.pythonhosted.org, TestPyPI's sandbox file host, not the production files.pythonhosted.org used elsewhere in this repo (e.g. benchmarks/benchmark_lib.sh:1457 for bfcl_eval). TestPyPI explicitly does not guarantee permanence of uploaded files, unlike PyPI proper. Fix: host the Mooncake wheel on a durable, project-controlled or production PyPI-backed location (or vendor it) so disagg jobs do not depend on a testing sandbox that can delete the file at any time.
Extended reasoning...
setup-disagg.sh runs uv pip install --require-hashes 'https://test-files.pythonhosted.org/packages/.../mooncake_transfer_engine_cuda13-0.3.14.dev20260910-...whl#sha256=...' inside every prefill/decode container for the four disagg recipes (dep4-tep4-c64, dep4-tp2-c90/c100/c110). test-files.pythonhosted.org backs test.pypi.org, which is documented as a sandbox with no durability guarantee for uploaded files. If the upload is removed or expires, the download 404s, set -eo pipefail aborts the script, and every disagg job using this setup script fails at startup with no fallback. Compare to benchmarks/benchmark_lib.sh:1457 which pins bfcl_eval from the production files.pythonhosted.org host for the same durability reason.
Verification: nit. Verified: benchmarks/multi_node/srt-slurm-recipes/configs/minimaxm3-mooncake/setup-disagg.sh:5 fetches the Mooncake wheel from https://test-files.pythonhosted.org/packages/.../mooncake_transfer_engine_cuda13-0.3.14.dev20260910-...whl#sha256=.... test-files.pythonhosted.org is TestPyPI's sandbox file host, not the production files.pythonhosted.org used everywhere else in the repo (e.g.…
仅对固定校验和的 Mooncake wheel 安装清除无关的软件包版本覆盖项，并保留哈希校验。同步主分支并保留变更日志历史内容。
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35371177969 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35371177969 |
2 participants
Add seven GB300 MiniMax-M3 Dynamo-vLLM AgentX configurations: aggregate TP8 at C1 and TP2 at C20/C24; disaggregated DEP4 prefill with four TEP4 decode workers at C64 or five TP2 decode workers at C90/C100/C110.
Pin the ARM64 vLLM nightly image and Dynamo package, use EAGLE3-GQA with three draft tokens, and install the checksum-verified Mooncake wheel for disaggregated serving. Throughput uses the repository's automatic golden acceptance selection; evals retain real speculative verification.
Update the shared srt-slurm dependency from v2.2.1 to v2.11.0 for process-local Mooncake store configuration, and synchronize the recipe guide's version references. This dependency update applies to all srt-slurm workloads.
Validation: exact configuration and eval matrix generation, focused repository and srt-slurm tests, changelog validation, and shell syntax checks pass. The updated-head hardware sweep and eval validation are pending.
AI model disclosure
AI assisted the original preparation and this recovery. The exact model identifiers could not be verified. The recovery included implementation, failure diagnosis, and independent AI review.
中文
新增七个 GB300 MiniMax-M3 Dynamo-vLLM AgentX 配置：聚合部署采用 TP8、并发数 1，以及 TP2、并发数 20/24；分离式部署采用 DEP4 预填充，搭配四个 TEP4 解码工作进程、并发数 64，或五个 TP2 解码工作进程、并发数 90/100/110。
固定 ARM64 vLLM nightly 镜像和 Dynamo 包版本，采用 EAGLE3-GQA 与三个草稿 token，并为分离式部署安装经过校验和验证的 Mooncake wheel。吞吐测试由仓库自动选择 golden acceptance，eval 保留真实的推测解码验证。
将共享的 srt-slurm 依赖从 v2.2.1 更新至 v2.11.0，以支持各工作进程独立配置 Mooncake 存储，并同步配置指南中的版本引用。此依赖更新适用于所有 srt-slurm 工作负载。
验证：精确配置与 eval 矩阵生成、仓库及 srt-slurm 针对性测试、变更日志校验和 Shell 语法检查均通过。更新后提交的硬件 sweep 和 eval 验证尚待完成。
AI 模型披露
原始准备和本次修复均使用了 AI 辅助，但无法核实确切的模型标识。本次修复中的 AI 工作包括实现、故障诊断和独立审查。