[ronenkat]

Purpose

Make DeepEPM2NAFDConnector's default deepgemm expert backend use the
DeepGEMM bundled in the vLLM wheel (vllm.third_party.deep_gemm) instead of a
separately built upstream deep_gemm package. With a stock vLLM v0.32.0
CUDA install, including the vllm/vllm-openai image, the connector then
needs only the afdep extra. The DeepGEMM source build (submodules, elfutils
headers, a --no-deps wheel install) is no longer part of the installation.

The connector always uses vLLM's copy, also when an upstream deep_gemm is
installed, so its experts run the DeepGEMM version the pinned vLLM ships.

Issue

• Related issue(s): [Feature]: Add GPU M2NConnector E2E coverage and integrate it into weekly CI #512
• Closing keyword, only if fully resolved: Closes #

Scope

• In scope:
  • afd_plugin/connectors/gpu/deepep_m2n/deepgemm.py: where DeepGEMM is
    imported from.
  • A CPU unit test for the import.
  • Installation text in the connector guide, docs/installation.md,
    tests/gpu/README.md, tests/e2e/README.md, the DeepEP M2N recipe README
    and afd_plugin/connectors/README.md.
• Out of scope:
  • CI image and pipeline changes (follow-up).
  • The fused_experts backend, connector config and APIs, other connectors,
    NPU.

Implementation Notes

• load_deepgemm() imports vllm.third_party.deep_gemm._C. The kernel
  wrappers in deepgemm.py are unchanged; the import stays
  functools.cached and deferred to first use, so fused_experts
  deployments never import DeepGEMM.
• A vLLM build without DeepGEMM (no vllm.third_party package) gets an error
  that points to expert_backend: fused_experts. Any other import failure
  inside the bundled copy is raised unchanged.
• An installed upstream deep_gemm is ignored by the connector. vLLM's own
  kernels still prefer it (vllm/utils/deep_gemm.py); both can be loaded in
  one process, since upstream registers its functions through pybind and the
  vLLM fork through TORCH_LIBRARY(deep_gemm).
• No vLLM private API, no monkey patch. The plugin imports the package path
  vLLM installs.

Test Plan

Environment: one node with 4x H200, stock vllm/vllm-openai:v0.32.0, the
plugin with the dev group and the locked afdep extra (afdEP-comm
v1.1.0). No upstream deep_gemm and no elfutils headers installed;
EP_DISABLE_GIN=1.

• pre-commit run --files <changed files>
• pytest -v tests/unit/connectors/test_deepep_m2n_connector.py -k deepgemm
• pytest tests/unit and the CI gate
  pytest -m "not gpu and not vllm_runtime" tests/unit
• load_deepgemm() resolution check in the pod
• pytest -sv -m gpu tests/gpu/test_deepep_m2n_gpu.py, cold JIT caches, then
  warm

Test Result

• pre-commit on the changed files: all hooks passed.
• DeepGEMM unit tests: 4 passed.
• tests/unit: 1568 passed, 52 skipped, 2 failed. CI gate: 1555 passed,
  51 skipped, 2 failed. The two failures are
  tests/unit/compat/npu/test_feature_validation.py::test_npu_feature_validation[gate-on-attention]
  and [async-graph], which fail on main as well and are unrelated.
• load_deepgemm() returns vllm.third_party.deep_gemm._C.
• tests/gpu/test_deepep_m2n_gpu.py: 33 passed, 2 skipped, cold in 32 min
  and warm in 7.9 min. The skipped cases are
  test_attention_tp_peers_dispatch_disjoint_slices[2F-2DPx2TP-{bf16,fp8}],
  which need 6 GPUs. The BF16 and FP8 DeepGEMM expert cases, CUDA graph and
  DBO cases all pass with the bundled DeepGEMM, and its JIT compiles without
  the elfutils headers.

Docs Impact

• Files updated:
  • docs/gpu/DEEPEP_M2N_CONNECTOR_USER_GUIDE.md: DeepGEMM comes with vLLM;
    the source build and elfutils steps are removed.
  • docs/installation.md
  • tests/gpu/README.md, tests/e2e/README.md
  • recipe/gpu/DeepEPM2NAFDConnector/README.md
  • afd_plugin/connectors/README.md

---

Essential PR Checklist

• Purpose is clear and linked to public context when possible.
• Scope is bounded.
• Compatibility with vLLM v0.32.0 and, for NPU changes, the pinned vLLM-Ascend source is considered, as applicable.
• No changes are made to the vLLM source checkout.
• Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
• Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.
• Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
• Validation evidence is included, including skipped GPU tests when applicable.
• Documentation impact is stated.

🤖 Generated with Claude Code

[ronenkat]

FYI. @Etelis