[samsja]

Builds DeepGEMM from vLLM's fork at e1f418c, the commit vLLM 0.31.0 vendors (2.8.0). Before this it built 2.5.0 (891d57b).

Why

• DeepSeek-V4.1 on H200. vLLM imports an installed deep_gemm ahead of its vendored copy, so prime-rl's 2.5.0 wheel replaces vLLM's 2.8.0. 2.5.0's SM90 paged MQA logits path supports only 64-token KV blocks, and V4.1's indexer uses 32, so serving fails (block_kv == 64 or (arch_major == 10 and block_kv == 32)).
• Wrong MoE weight gradients on H200. On SM90, 2.5.0's k_grouped_fp8_gemm_nt_contiguous returns wrong weight gradients for some experts, nondeterministically.
  • A 10-expert test went wrong on 6 of 6 runs with 2.5.0: 15–32% relative error, against about 3.6% FP8 error on the other experts.
  • It went wrong on 0 of 6 runs with 2.8.0.

Changes

• install_deep_gemm.sh: a new --repo option; the default repo and ref are now vllm-project/DeepGEMM at e1f418c.
• CI: installs elfutils-devel, because DeepJIT includes elfutils/libdwfl.h. It's needed at compile time only; the library is dlopened at runtime.

Checked on H200 with a local x86_64 build (deep_gemm-2.8.0+e1f418c, CUDA 13.0, torch 2.13.0):

• The paged MQA logits kernel accepts both 32- and 64-token KV blocks.
• prime-rl's FP8 blockwise linear and grouped GEMM, forward and backward, give the same bits as vLLM's vendored 2.8.0.
• With this wheel in place, DeepSeek-V4.1-Flash serves on 8×H200 in vLLM 0.31.0.

The aarch64 wheel and the smoke test are only checked by this PR's CI.

🤖 Generated with Claude Code