[Pingzii]

What changed

BF16 (quant_mode=0, default)
  Attention: attention → activation (BF16) ─────────────────▶ FFN: experts quantize locally

MXFP8 (quant_mode=4, opt-in)
  Attention: attention → quantize → E4M3 activation + E8M0 scale ─▶ FFN: experts use both

quant_mode=4 sends a pre-quantized activation over the synchronous A2E transport. Attention
quantizes to E4M3 with torch_npu.npu_dynamic_mx_quant inside CAMP2P's opaque send op, before that
op calls the native a2e, and the activation plus its E8M0 MX scale travel in their own regions of
the A2E packet; the FFN side keeps the two together through padding, gathering and sharding, then
hands both to the experts through ALLGATHER or MC2 without dequantizing.

quant_mode=0 keeps the original twelve-argument call, so an operator build that predates the MXFP8
arguments still runs. There is no silent fallback: a deployment that asks for mode 4 and cannot be
carried is refused at startup.

Where the change lands at this head (18 files, +2,177 / -72):

Kind	Files	Lines	What it is
Core	7	+791 / -42	The transport itself: camp2p/connector.py (+473) holds the construction gate, the two startup collectives and the fourteen-argument send with the six-value receive; deepseek_v4_external_routing_moe.py (+151) holds the pair contract, the per-path scale layout and the MC2 dispatcher subclass; camp2p/external_routing.py (+74) carries the scale through pad/gather/split; compat/npu/ops.py (+67) is the operator schema probe; the remaining three files (+26) are the field and the two-hop forwarding
Tests	8	+1,330 / -26	test_camp2p_token_ids.py (+669) covers mode 4 send/receive, construction and the startup handshake with per-collective stop points; test_deepseek_v4_external_routing_moe_cpu.py (+436) drives the real component, prepare transforms and runner forwarding; the probe, payload-transform and codec files account for the rest (+209)
Docs	3	+56 / -4	The connector guide gains the MXFP8 section and its requirements (+53); the payload contract and the support matrix follow (+3)

Deployment and dependency

• The operator build must come from [WIP] feat(npu): support MXFP8 datatype for A2E operator #444 and take x_scale plus quant_mode. The connector
  probes the operator schema at construction, but that probe only covers the input interface -- a
  build that registers the arguments and still cannot run them is a separate question, tracked by the
  capability marker requested in [WIP] feat(npu): support MXFP8 datatype for A2E operator #444. Until it lands, mode 4 needs an A5 device and an operator
  built with the E4M3 / E5M2 / E8M0 dtypes, as stated in the connector guide.

• Upgrade the Attention and FFN roles together. The startup handshake runs for quant_mode=0
  too, so it can catch a peer configured the other way; a role still on a version without the
  handshake never joins it and the newer side fails at startup rather than running a mixed pair.

• The handshake also exchanges afd.compute_gate_on_attention. The A2E ids slot carries routing ids with the gate on and token ids with it off, and both configurations pass construction, so a mixed pair used to exchange payloads and then read one as the other. The disagreement fails on both roles before the first A2E call.

• The selector pre-check runs whenever routing is carried: always for mode 4, and for mode 0 when the gate is on. A BF16 pair such as 8A8F with a bound beyond the MC2 capacity is therefore refused at startup instead of failing at prefill when the selector chooses ALLTOALL. A gate-off BF16 pair still exchanges one collective.

• The handshake exchanges every rank's mode and the batch bound the Attention role can send:

  • a mixed set fails on both roles;
  • the routing-id budget is judged from the bound Attention publishes, so the two roles may use
    different --max-num-batched-tokens;
  • the FFN role asks the pinned select_moe_comm_method about that same bound and refuses when it
    resolves to anything but ALLGATHER or MC2, which is what the external-routing consumer supports.

  All of it happens before afd_moe and p2p are created, and a selector that raises still
  publishes its verdict, so neither side waits for an answer that will not come.

• On the FFN side the routing slot is required in this mode: the router and shared experts still need
  the model dtype, so recv_routing is the only supported receive.

Verification

CPU only -- there is no device run behind this PR.

Local Windows run (CPU, no vLLM), full tests/unit: 953 passed, 89 skipped, 0 failed after rebasing onto the current main.

ruff check/format, mypy on the changed files and typos are clean.

The transport itself is pinned by: the payload byte-level tests (padding row = byte 127, per-rank
distinct bytes through the gather, scale following every row transform); the startup handshake, with
the stop point asserted per collective (mode mismatch stops after the first, an unsupported path or a
raising selector after the second, and a mode-0 pair never reaches the second); the fourteen-argument
send and the six-value receive, including the five-value build that must name the shortfall instead
of raising IndexError; the MC2 dispatch kwargs; and the runner forwarding, mutation-checked by
deleting the forwarding line and watching both comm paths fail.

Device acceptance, once a compatible A2E build is installed:

1. the real HCCL startup handshake, including a mode-0 pair and a peer on a version without it;
2. the FFN role calling with x_scale=None, with the activation, the MX scale, the expert ids and the
   routing weights compared row for row;
3. one real generation on ALLGATHER and one on MC2. For MC2: with A2E quant_mode=4 and MC2
   selected, verify that pre-quantized dispatch (quant_mode=0) returns E8M0 dynamicScalesOut
   aligned with the dispatched E4M3 activations on every valid expert row, and that GMM consumes the
   aligned pair.

Performance and accuracy

Historical measurements from the development stack: single node A5 4A4F, Graph, gate on Attention,
same #444 operator build as this branch expects. They are not results of this PR's CI.

Comparison	Basis	Result
MXFP8 vs BF16 transport, median throughput	same commit f683eb1, two runs per arm	328,835 -> 341,168 token/s, +3.75%
MXFP8 vs BF16 transport	cross-commit comparison	+6.07%
whole stack vs clean baseline		+4.32%

Accuracy, GSM8K with 1,319 problems: quant_mode=4 89.76% against quant_mode=0 90.90%, a
difference of -1.14 pp. It was measured on the development stack with the #444 operator build,
over the same problem set; the totals are all that was recorded.

[jiangkuaixue123]

[P2] Reject unsupported MXFP8 rank counts at connector construction (afd_plugin/connectors/npu/camp2p.py:554). With quant_mode=4, a 4A2F deployment can pass the AIV and schema checks and proceed to the first A2E call. The operator supplied by #444 explicitly requires equal positive Attention and expert rank counts, at most 32 each. The asymmetric padding path at line 1405 cannot satisfy that contract. Please enforce the rank constraint in the mode-4 validator, document it beside the 4A2F example, and cover asymmetric rejection and symmetric acceptance.

The GSM8K scores are from the same 1,319 questions. The two aggregate percentages alone cannot establish accuracy neutrality. Please report the paired counts (BF16-only correct and MXFP8-only correct) and a paired interval for the difference, or describe the observed 1.14-point drop without claiming equivalence. Static review of the incremental change and #444 operator contract. I did not reproduce device measurements.

[Pingzii]

Ready for re-review at 5041bd5. All three points from the last review are addressed; dev re-ran the probe, CAMP2P transport, payload and FFN wiring cases (84 passing).

1. Runner-level wiring coverage. tests/unit/model_executor/models/test_deepseek_v4_external_routing_moe_cpu.py now starts from the real _ffn_forward in v1/worker/npu/ffn_model_runner.py — the method body runs from the source file against stand-ins for the helpers it calls — with a payload handed over by a connector stand-in. From there the chain is real: compute_ffn_output → AFDExternalRoutingMoE → the prepare row transforms → the expert entry point, asserted as (4, 1, 2) on ALLGATHER and (4, 2) on MC2 with distinct per-row E8M0 bytes. Mutation check: deleting ffn_kwargs["activation_scale"] in the runner fails both cases on the pair contract.

2. Mode 4 send, receive and construction. tests/unit/connectors/test_camp2p_token_ids.py gains 11 cases: the fourteen-argument send carries the quantized activation together with its scale, the receive reads the sixth value, a build that returns five values now raises RuntimeError: ... returned 5 instead of IndexError (_A2E_MX_RETURN_VALUES guards the read of outputs[5], while outputs[3]/outputs[4] stay readable on a pre-MXFP8 build), the token-ids slot is refused, and construction accepts 4A4F while rejecting 4A2F, an FFN-side gate, more than 32 ranks per side, and a probe that answers False or cannot answer.

3. Accuracy wording. The body now reports GSM8K 89.76% for quant_mode=4 against 90.90% for quant_mode=0, −1.14 pp, and states that equivalence is not claimed: it would need per-question paired evidence, which has not been run. The device section also separates the schema gate (startup) from the substantive check (one real generation proving the sixth value reaches FFN), so a successful construction is not presented as mode-4 validation.

CPU verification on this head: full tests/unit 863 passed / 82 skipped / 8 failed, with the same eight environmental failures on the base commit; mypy-3.11, ruff check, ruff format and typos clean. There is still no device run: the operator that accepts x_scale and quant_mode is #444.

[Pingzii]

Two follow-ups are in at ee59d9d; all three checks are green on that head.

The handshake has a test that fails if it is deleted. Rather than adapting the torch_npu-gated init test -- which is skipped in exactly the environments where the check would otherwise run -- I put a locally runnable one next to the other CAMP2P tests: it drives the real init_afd_connector against recorded stub groups and mocked torch/all_gather_into_tensor, and asserts that the collective runs once, on the AFD main group, before afd_moe and p2p are created. Removing the handshake call no longer publishes the handshake event, so the test fails on the order contract rather than on an incidental attribute; the mixed set then fails with the mismatch error on the same path. A follow-up will run the two mixed cases through the real init once per role, so Attention and FFN each prove they refuse before afd_moe and p2p exist.

The guide states the startup behavior. The two bullets that had been joined are split, the quant_mode bullet describes the AFD-group collection, and the section says the handshake runs for quant_mode=0 too: a role running a version without the handshake never joins the collective, so the newer side fails at startup with a communication timeout even when both are configured for BF16. Upgrading the Attention and FFN roles together is what avoids that.

Two carry-overs from the earlier, now superseded check. The per-payload mode comparison is still in the code and will go away in the follow-up that makes the startup handshake the single protocol check. What stays regardless: construction requires an Ascend A5 device (_require_a5_device, because the operator registers the same fourteen-argument schema on every build and rejects mode 4 only at runtime), and the guide states the A5 plus E4M3/E5M2/E8M0 build requirements. The dtype half cannot be decided from Python without a marker from the operator; that request is #444 (comment), and until it exists this stays a documented requirement rather than a check.

Testing is down to the seams that own each behaviour. Five cases are gone at ee59d9d: the payload padding case (the ALLGATHER component case walks the same padded() and checks the original rows and the appended 127 bytes), the zero-block scale case (the connector already validates the column count before a routing payload exists), the MoE row-count row (the payload-level check rejects it first), and the MC2 scale-dtype/activation-dtype rows (the normal A2E to MoE path validates both before a dispatcher is reached). The gather case with distinct per-rank bytes stays, since that is what proves the order, and so does no-scale, which catches a runner that stops forwarding.

The convergence follow-up will also keep the JSON round-trip test (dropping only its quant_mode assertion), delete the field plus its standalone missing-field case, and remove the per-packet comparison sentence from the guide.

CPU verification at this head: connectors and model_executor 323 passed / 28 skipped; ruff check/format, typos and the changed-file mypy clean. Still unrun: the real HCCL startup handshake, mode 4 end-to-end on ALLGATHER and MC2, and the operator capability marker.

[Pingzii]

Both fixed at ec6409f (amended into the same commit, so the id changed); checks green.

The bound read is now inside the guard. int(rows[ffn_size:, 1].max()) sits in the same try as the import and the selector call, so a reduction or scalar-read failure on the NPU aggregate also produces an UNSUPPORTED verdict and still reaches the second collective. The device aggregate is the only source the FFN trusts, which is why it stays the read; nothing on the host has the Attention bound.

The fake no longer lets the preset answer for this rank. After copying the preset, it overwrites this rank's own slot with what it actually published (target[0].copy_(local) for FFN, slot 1 for Attention), so the peers are simulated and the local verdict is real. That makes your check reachable: an FFN that publishes PASS in the ffn-alltoall case now turns the aggregate all-OK and the case fails on the missing raise. The mixed-mode rows got stronger for the same reason -- a rank publishing the wrong mode no longer hides behind the preset.

Guide updated with the two missing statements: the FFN must resolve to ALLGATHER or MC2 for the largest batch Attention can send, rejected at the startup handshake on both roles before afd_moe and p2p; and the two roles may run different --max-num-batched-tokens, because the check uses the bound Attention publishes.

Local: connectors 45 cases in that file, connectors + model_executor 333 passed / 28 skipped; ruff check/format, typos and the changed-file mypy clean.

[Pingzii]

Presets reordered at 9e8b284 (same commit, new id). ffn-alltoall and the selector-exception case now publish [0, 1]: FFN slot 0 carries whatever the FFN actually sends, Attention slot 1 publishes PASS.

Mutation-checked that the case now bites: forcing the FFN to publish a passing verdict in the ALLTOALL branch makes test_startup_handshake_stops_where_the_pair_is_refused[ffn-alltoall] fail (nine other startup cases still pass), and reverting the mutation restores green. So the old [1, 0] blind spot is closed.

Local: connectors 45 cases in that file, connectors + model_executor 333 passed / 28 skipped; ruff/format and pre-commit clean. PR is still 10 commits.