[Pingzii]

Purpose

The synchronous CAMP2pAFDConnector transports the FFN activation as BF16 and
offers no quantization; the asynchronous connector has a dynamicQuant switch,
this one does not. AFD's payoff comes from aggregating many attention instances
onto fewer FFN instances, so the payload shipped on this path grows with the A
count — and gets considerably more expensive once A and F are placed on different
nodes (see Potential for larger gains at scale below).

This PR adds the missing mode. On the attention side the activation is quantized
to FP8 (E4M3) plus an E8M0 block scale and transported in that form; the
block-scale transport comes from the operator support in #444. The FFN side
restores the model dtype at the module boundary, because two consumers there need
it: the router reads the activation itself (MoeGatingTopKHash,
x=router_logits) and the shared experts quantize their own input
unconditionally. Keeping that restore keeps the change confined to the transport.

Result: +34.5% total token throughput at 128 concurrent requests on one A5
host. Details, and the reason the gain is conditional, are in
Test Result.

Issue

• Related issue(s): [WIP] feat(npu): support MXFP8 datatype for A2E operator #444 (operator support this PR depends on)
• Closing keyword: none — this is the service side of a two-part change.

Scope

• In scope: the CAMP2P connector's MXFP8 send/receive path, the quant_mode
  option it already exposes through connector_extra_config, the A5
  DeepSeek-V4-Flash recipe, and unit coverage.
• Out of scope: the a2e operator itself — that is [WIP] feat(npu): support MXFP8 datatype for A2E operator #444. The operator files are
  deliberately not duplicated here; this branch contains only the service
  side.

Implementation Notes

• connector_extra_config.quant_mode gains a second value: 0 keeps the BF16
  transport (the default), 4 moves a pre-quantized E4M3 activation plus its
  E8M0 block scale. This is the connector's existing field, so there is no
  additional switch and the two roles cannot drift apart through a second
  configuration surface.

• Mode 4 is rejected up front unless the A2E operator accepts x_scale and
  quant_mode and at least 3 AIV cores are configured, instead of failing later
  inside the kernel.

• The FFN restores the model dtype after the receive. This preserves the
  existing router and shared-expert input path and keeps quantization confined to
  the transport.

  Direct FP8 consumption was evaluated and is excluded from this PR: it measured
  6.8% lower throughput, added a per-layer collective for the transported scale,
  kept both FP8 and BF16 activations resident, had incomplete scale propagation on
  the DP=1 and SP/EP paths, and came in 1.59 GSM8K points below the baseline.
  Details are under Why the FFN restores BF16; the
  measurements are kept there to document the trade-off and guide future work.

Test Environment

item	value
host	one A5 node, 8× Ascend950DT_9574
topology	AFD 4A4F — attention DP4/TP1 on devices 0-3, FFN DP4/TP1 on devices 4-7
connector	CAMP2pAFDConnector, process-group rendezvous 127.0.0.1:6239
software	vLLM 0.30.0 (ced6857afa), vLLM-Ascend 8d4409d62, plugin built from this branch plus the #444 operator
serve flags	--max-num-batched-tokens 3072, --max-num-seqs 128, --max-model-len 9216, --kv-cache-dtype fp8, attention-config {"indexer_kv_dtype":"fp8"}, cudagraph_capture_sizes [8,16,32,64,128] with FULL_DECODE_ONLY, HCCL_BUFFSIZE 2048, PYTORCH_NPU_ALLOC_CONF expandable_segments:False
workload	aisbench vllm_api_general_chat + synthetic_gen: 128 requests, 8192 tokens in / 32 out, batch_size=128, RequestCount=128, temperature=0, ignore_eos=True

The gain depends on the load: the same configuration measured −0.3% at 8
concurrent requests, because each layer's round-trip wait is then dominated by its
fixed latency rather than by the activation payload. 128 concurrent is the load
this was validated at, not a threshold derived from the configuration —
--max-num-seqs is a capacity limit (128 per rank, so up to 512 in flight across
DP4) and says nothing about when the box saturates.

On --max-num-batched-tokens 3072. The default is 2048 and it is equally
correct; mode 4 works at any chunk. 3072 was chosen for measurement because it is
the largest value at which all four DP ranks fill to the cap — at 4096 only the
first step reaches it and later steps fall to ~2500, which lowers throughput for
both configurations (measured below).

Test Plan

• CPU-only unit coverage: the A5 rank shape the operator requires, the
  quant_mode values connector_extra_config accepts and rejects, the E8M0
  exponent reconstruction (including a hidden size that is not a multiple of the
  32-element group), the send path padding the scale with the activation, the
  receive path requesting both payloads, and deferred imports naming symbols that
  exist.
• Device validation on one A5 host: quant_mode=0 versus 4 at 128 concurrent
  requests, two runs each, plus GSM8K and an A-side torch profiler trace.

Test Result

Throughput — chunk 3072, 128 concurrent, two runs each

group	run 1	run 2	mean	TTFT median	duration
quant_mode=0 (BF16 transport)	233,977	238,778	236,378	3748 ms	4499 ms
quant_mode=4 (MXFP8)	326,684	310,354	318,519	2905 ms	3340 ms

Failed Requests = 0 and Max Concurrency = 128 in all four runs. Gain over
quant_mode=0: +34.7%.

Re-validated after rebasing onto main (2026-10-20, two runs each)

group	run 1	run 2	mean	TTFT median	duration
quant_mode=0	246,721	214,318	230,520	3581 ms	4636 ms
quant_mode=4	308,783	311,478	310,131	2972 ms	3429 ms

Gain: +34.5%. Both sides drifted by about 2.5% relative to the first table and
in the same direction. With two runs per group that is not enough to call the
result unchanged; what it does show is that the mean throughput gain remained
similar after rebasing.

chunk 4096 (one run each)

group	throughput	TTFT
quant_mode=0	218,646	3581 ms
quant_mode=4	297,245	3138 ms

+35.9%. The ratio holds, but both absolute numbers are below the chunk-3072 runs,
which is why 3072 is the configuration in the recipe.

Where the time goes (A-side torch profiler, chunk 3072)

E2a — the receive that blocks the attention side for the whole FFN round trip —
accounts for 49.5% of attention-side device time with the BF16 transport. That
is where the saving has to appear, not in A2e: the A-side send is
fire-and-forget (it writes its own window and an 8-byte flag), so it never waits
on the payload.

The two traces were captured over different step counts, so their per-call totals
are not comparable and this PR does not claim a per-operator delta from them. The
share above is the one number that is stable across both.

Accuracy (GSM8K, max_out_len=2048, ignore_eos=False, batch_size=8)

configuration	accuracy
quant_mode=0	93.86%
quant_mode=4	93.71%
quant_mode=4, earlier configuration	92.87%

About −0.15 points against a ±0.5 point spread on this benchmark; the same path
measured 92.87% once, so do not call it lossless. A 0.15-point move is inside
the spread at this sample size, and the third row shows the spread is real.

Unit tests

The added coverage is targeted rather than bulk: the operator capability probe
(supported, unsupported, not loaded), the send and receive call shapes — including
that quant_mode=0 still passes the original twelve arguments — the E8M0
reconstruction with a hidden size that is not a multiple of the 32-element group,
and the deferred-import surface. tests/unit no longer carries a line budget;
#474 removed that gate upstream, so this PR neither moves nor relies on it.

Scope of the run. The CPU-only run was on a host without torch_npu or a
usable vLLM, so the tests that need either were skipped, not passed. They were
later executed on an A5 host: tests/unit/connectors/test_camp2p_connector.py
reported 20 passed, which covers the operator probe, both send call shapes and
the receive dequantization. The four failures that remain (three in
test_ascend_config_lifecycle.py, plus
test_package.py::test_entry_point_is_registered) also fail on main in the same
environment, so they are pre-existing rather than introduced here.

Device validation, against a build whose a2e operator predates the MXFP8
arguments (len(schema.arguments) == 12, so a2e_accepts_mxfp8_arguments() is
False):

run	result
quant_mode=0, both roles	serves requests. Started F then A and completed a /v1/completions call, so the whole send/receive path works on the twelve-argument call — the backward-compatibility claim this PR makes for the default mode
quant_mode=4, F role	rejected at construction with A2E MXFP8 needs an a2e operator that accepts x_scale and quant_mode, before any device work — the capability probe reads the real extension, not a mock

Still open: the positive case. #444 has to land so a build accepts the pair; then
construction should succeed and mode 4 should run. The only CI check on this PR is
DCO.

Why the FFN restores BF16

The FFN restores the transported MXFP8 activation to BF16 before running MoE.
This preserves the existing router and shared-expert input path while keeping
quantization confined to A2E transport.

Direct FP8 consumption was evaluated but excluded from this PR. This variant
let routed experts reuse the transported FP8 activation and scale, skipping
internal re-quantization. It achieved 296,934 token/s versus 318,519 token/s for
the BF16-restoring path — 6.8% lower throughput.

Avoiding re-quantization introduced other costs. On the DP all-gather path, the
transported scale required its own pad-and-gather operation to stay aligned with
the gathered activation, adding one collective per layer. The variant also
retained both FP8 and BF16 activations: routed experts consumed FP8, while the
router still required BF16. These costs can offset the computation saved, although
their individual contributions to the slowdown were not isolated.

The implementation also had incomplete scale propagation for DP=1 and SP/EP paths.
GSM8K accuracy was 92.27%, compared with 93.86% for the BF16 baseline, a
1.59-percentage-point drop requiring further investigation; the result alone does
not establish the cause.

The PR therefore retains the BF16-restoring path. The direct-path measurements are
recorded here to document the trade-off and guide future work.

Potential for larger gains at scale with suitable A/F ratios and interconnects

All results above were measured on a single eight-device host, with Attention and
FFN communicating over the intra-node HCCL fabric. These measurements establish
the benefit in that configuration; larger deployments introduce two additional
factors that may increase the value of A2E quantization.

Higher A:F ratios can put greater pressure on the A2E path. One motivation
for AFD is to let multiple Attention instances feed fewer FFN instances,
combining distributed KV-cache capacity with larger batches for expert
computation. As more Attention instances contribute tokens, both the incoming
activation volume and the FFN workload increase. If FFN computation remains
efficient while activation delivery becomes a bottleneck, reducing A2E traffic
can help sustain the benefit of aggregation.

Bandwidth-constrained links can make payload reduction more valuable. When
A/F placement requires traffic to cross a lower-bandwidth interconnect,
transferring the same activations takes longer and consumes a larger share of the
available link capacity. MXFP8 roughly halves the activation-data portion of A2E
traffic relative to BF16, although scale metadata and other communication costs
remain. The resulting benefit may therefore be greater when A2E bandwidth is a
limiting resource.

These are expectations, not measured results. In the single-host experiment, E2A
accounted for 49.5% of Attention-side device time, and the measured throughput
improvement was +34.5%. That device-time share is not, by itself, an end-to-end
bottleneck measurement. Nevertheless, larger gains are plausible if higher A:F
ratios and more bandwidth-constrained links make A2E transfer a greater part of
the critical path, provided quantization overhead and other bottlenecks do not
offset the savings. Validating this expectation requires measurements across
deployment scales, A/F ratios, and interconnect configurations.

What this PR does not cover

• Decode. Every number above is prefill-dominated (8192 in / 32 out), and the
  recipe README says the same. A decode data point needs its own run.
• Topologies other than one host in 4A4F. Both roles use the intra-node
  fabric here.

Docs Impact

• docs/npu/CAM_P2P_CONNECTOR_USER_GUIDE.md: quant_mode documented as the
  single transport switch, with what the mode does and does not change. The
  transport section builds on text already in [WIP] feat(npu): support MXFP8 datatype for A2E operator #444, so it should be rebased after
  that PR lands.
• docs/design/module/connector_contracts.md and execution_platforms.md: the
  accepted quant_mode values, their prerequisites, and the load shape the
  measurement covers.
• recipe/npu/CAMP2pAFDConnector/deepseek_v4_flash/: the measured configuration, the A/B numbers, and what the measurement depends on. No launch script is carried in the tree.

[ksiyuan]

Reviewed at 7befb006 against 7ace946.

The gain itself is credible and I would keep it: halving the A2E payload is a real
effect, the +34.5% survives a rebase with both sides drifting by the same ~2.5%,
the variance warning is unusually honest, and E2a at 49.5% of Attention-side
device time is consistent with the throughput delta. What follows is about scope,
patch contract, and one accuracy number that does not fit its own explanation.

All of it is from code reading plus the numbers in this PR body — there is no
torch/torch_npu on my host, so I ran no device validation.

---

[P1] Drop the pass-through path (AFD_NPU_CAMP2P_MXFP8_DIRECT) from this PR

afd_plugin/envs.py:26, afd_plugin/connectors/npu/camp2p.py:442-448,871-872,890,
afd_plugin/model_executor/models/npu/deepseek_v4.py:483-516,
afd_plugin/compat/patches/npu/moe_mxfp8_scale.py:128-372

Pass-through is 6.8% slower and GSM8K is 92.27% vs 93.86% for mode 0 — 1.59
points, about 3x the ±0.5 spread this PR states for the benchmark. The cost is
explained; the accuracy is not, and mechanically it should be near-neutral:
pass-through hands the routed experts the same E4M3 data plus the scale it was
quantized with, and only skips one fused re-quantization. A 3-sigma drop points at
scale semantics, not at quantization cost. Two candidates:

1. The GMM's scale algorithm or grouping may differ from
   get_dynamic_mx_quant_scale_alg(), which makes feeding the transport's scale as
   pertoken_scale a systematic bias.
2. A stale scale is reused. moe_mxfp8_scale.py:191 early-returns without clearing
   AFD_MXFP8_SCALE_ATTR; :231-232 stages it on prepare_finalize and on
   moe_config, both cached by get_moe_comm_method per (comm type, config) for
   the process lifetime; :291-293 reads it back through getattr fallbacks.
   Reachable: run an all-gather batch (_moe_requires_model_dtype false, x_scale
   pinned at camp2p.py:890), then a dispatch batch (camp2p.py:871 returns true
   → direct=False → x_scale never set → the model pins None → forward
   early-returns), and :318 installs the previous batch's gathered scale as
   pertoken_scale. By this file's own docstring that makes the routing op take
   quant_mode=-1 permute-only and skip npu_dynamic_mx_quant, so a BF16
   activation is consumed as if it were already quantized.

Keeping the path "so the trade-off remains reproducible" does not need production
code — the numbers are already in this PR body. Removing it also removes all three
patched upstream functions, the runner.routed_experts /
runner.ascend_shared_experts hard dependency that deepseek_v4.py:483-516 adds
to the default path (5 setattr per layer for values that are always None when
DIRECT=0 plus 2 attribute reads), and the aivec error 263 operator limit the
recipe README documents at ~14,288 gathered rows.

If you keep it: clear the pinned attributes in the not pinned branch of
forward, reject DIRECT=1 in CAMP2PExtraInfo.validate_supported so it cannot
be switched on by accident, and fix or explain the accuracy.

[P1] The vLLM-Ascend patch does not meet the AGENTS.md patch contract

afd_plugin/compat/patches/npu/moe_mxfp8_scale.py:5,154,270,360,208-232,330-349

• No pinned upstream ref. The docstring at :5 names two modules only, while
  ascend_platform.py:5 pins a commit (8d4409d...). Use the
  vLLM-Ascend revision this PR's Test Environment table already records.
• All three patched functions are _original_* delegation instead of an upstream
  copy with # ### PATCH START/END markers: forward:154, _prepare_with_dp_group:270,
  _prepare_gate_up_input:360. Only forward calls out the exception; the other
  two do not, and _prepare_gate_up_input is small enough to expand locally. A
  reviewer cannot check the return-tuple contract or the quant_type /
  replace_allreduce ordering against the tag.
• The compatibility probes AGENTS.md tells us to avoid are pervasive: getattr
  at :208,216,224,228,230,291,293,330,334,362, except TypeError around
  get_moe_comm_method at :220, and dir() plus print of candidate names at
  :341-346.
• These probes demonstrably did not buy stability — this area needed 6b62e38,
  2198f91, 324c819, 3e48e9b, 6d996ce, 9b76102 on the source branch to
  chase exactly the signature and attribute churn they hide. And they do not even
  protect: deepseek_v4.py:484,496 reads the same attributes directly, so an
  upstream rename fails there first.

Fix: copy the three upstream bodies from the pinned revision, keep only the marked
AFD deltas, pin the commit in the docstring, drop the probes and the print.

[P2] The shipped configuration still hard-depends on select_moe_comm_method

afd_plugin/connectors/npu/camp2p.py:870-873

not self._moe_requires_model_dtype(batch_size) and camp2p_mxfp8_direct_enabled()
evaluates the left operand first, so with DIRECT=0 — the default, and the recipe
default — the receive path still imports MoECommType and
select_moe_comm_method from vllm_ascend.ascend_forward_context and calls the
selector on the first receive per batch size, then discards the result. A private
vLLM-Ascend symbol becomes a hard dependency of the default path for a decision
that is never used. Swapping the two operands fixes it.

[P2] The profiler table does not support the causal claim it is used for

The 48.6% is computed from max (11,366 → 5,842 µs) while mean moves the other
way by 6x (421 → 2,594 µs). The PR notes that the two traces cover different step
counts (45,279 vs 4,601 calls), which makes max non-comparable for the same
reason it makes the totals non-comparable.

The PR's own fit also disagrees with it: 3.63 ms + 0.72 µs × rows at 3072 rows
is 5.84 ms/call, and mode 0 at 2.52 µs × rows is 7.74 ms/call — that is
−24.5%, not −48.6%. And 43 layers × 7.74 ms is ~251 ms per step against a measured
~53 ms step (230,520 token/s ÷ 4 ranks ÷ 3072), which cannot be. The fit and the
trace window appear to describe different regimes.

Please either provide a matched A/B trace (same step count, same row distribution)
or drop the table and keep only E2a at 49.5% of Attention-side device time, which
is the number that actually supports the throughput delta.

[P2] Evidence provenance and gate state

• The A/B table was produced with "plugin built from this branch plus the #444
  operator". #444 is [WIP], open, mergeable: UNKNOWN, DCO ACTION_REQUIRED.
  This PR cannot exercise quant_mode=4 on its own, and the user-guide section
  explicitly needs a rebase after #444 lands, so the two halves are individually
  unverifiable. Either state #444 as a hard prerequisite and land it first, or
  review the two together.
• gh pr checks reports only DCO, and --required returns no required checks reported, so the Buildkite test-ready gate has not validated the 25 tests the
  PR body says pass locally.
• No decode data point. The mechanism predicts the gain disappears at small row
  counts — the PR's own fixed-round-trip argument, and the −0.3% at 8 concurrent —
  and recipe/README.md lists this recipe under Prefill while the other CAMP2P
  recipe is Decode. Please add a decode measurement or state the scope as
  prefill-only.

[P2] Design pages and the patch inventory were not updated

• docs/design/module/connector_contracts.md:95 still says the runtime "rejects
  gate-on-Attention and any nonzero quantization mode".
• docs/design/module/execution_platforms.md still pins
  connector_extra_config.quant_mode=0 for the Ascend V1 CAMP2P row, with no new
  row or evidence entry for mode 4.
• docs/design/module/compatibility_and_patches.md:129-133 lists every NPU patch
  module with its targets, validation paths and a removal condition;
  npu/moe_mxfp8_scale.py is absent, so the new patch has no documented removal
  plan, which the AGENTS.md patch requirement asks for.

[P2] Duplicated constants, and .gitattributes guards a path that does not exist

• 4 and 32 each live in more than one place: afd_plugin/envs.py:16-17,
  afd_plugin/connectors/npu/camp2p.py:88-96,
  afd_plugin/compat/patches/npu/moe_mxfp8_scale.py:43. And
  afd_plugin/compat/npu/feature_validation.py:71-72 imports the private
  _A2E_QUANT_MODE_MXFP8 from the connector while the public pair sits in envs.py.
• camp2p.py:185: raw.get("quant_mode", camp2p_quant_mode()) evaluates the
  default eagerly, so a malformed AFD_NPU_CAMP2P_QUANT_MODE raises even when
  connector_extra_config supplies quant_mode explicitly.
• .gitattributes:5-9 adds patches/**/*.patch -text and patches/**/*.md text eol=lf, but this tree has no patches/ directory, zero *.patch files,
  and nothing references such a path. The CRLF-versus-git apply hazard the
  commit describes is not addressed by it.
• tests/unit/compat/npu/test_a2e_mxfp8_contract.py:23 redeclares
  _A2E_MX_MAX_TOTAL_RANKS = 64 instead of importing it, so the boundary
  assertion tracks the test rather than the product; :100-110 cannot fail on a
  CPU host. mx_dequantize_a2e_activation (the E8M0 exponent reconstruction) is
  pure torch math with no unit test, where a wrong exponent bias would only ever
  show up as accuracy drift.

---

Happy to help with any of these. The smallest version of this PR that keeps the
measured win is: connector send/receive quantization, the two env switches, the
docs, the recipe, and the non-AFD baseline — with the patch file, the model-side
pinning, and the pass-through path removed.

[Pingzii]

Thanks — reworked against this review.

The branch was rebuilt from main rather than patched on top of f941030, so
please read the diff as new: 5 commits, 14 files, +1176/−37.

• The pass-through path is gone, and with it npu/moe_mxfp8_scale.py, the
  model-side pinning and the extra payload field.
• Your stale-scale analysis was right: the early return did not clear
  AFD_MXFP8_SCALE_ATTR, so a batch that produced no scale could inherit the
  previous one. Combined with the accuracy gap, that is why the path was dropped
  rather than repaired — both are recorded in the PR body so the reasoning is not
  lost.
• The per-operator profiler table is removed; that section now states only the
  49.5% device-time share. Your arithmetic on the fit — −24.5%, and 43 × 7.74 ms
  against a ~53 ms step — is what settled it.
• The .gitattributes block did guard nothing (there is no patches/ tree), so
  the rules are removed rather than fixed. Constants now have a single source, the
  three design pages are updated, and tests/unit/.loc-baseline moves 27468 →
  27920 with the reason stated in the PR body.

Still open: quant_mode=0 on a clean build without #444, and there is no decode
data point — the performance section now says so instead of extrapolating.

I have not added a matched A/B trace for the profile question. Say the word if you
think the operator table is worth capturing properly rather than dropped.

[ksiyuan]

Re-reviewed at e88eaae. All seven findings from the previous review are resolved —
and resolved by removal rather than by patching, which is what I would have asked
for:

• Pass-through is gone, and with it npu/moe_mxfp8_scale.py, the model-side
  pinning, the hidden_states_fp32 payload field and CAMP2PTransferState.x_scale.
  envs.py now carries only the two mode values, so
  connector_extra_config.quant_mode really is the single switch. I grepped for
  dangling references to the deleted module, the removed env switches, the deleted
  recipe scripts and inspect_afd_trace.py — all clean, and
  recipe/npu/CAMP2pAFDConnector/deepseek_v4_flash/ correctly holds only the
  README with no .sh reference left.
• _moe_requires_model_dtype is gone, so the default path no longer reaches
  select_moe_comm_method.
• The per-operator table is dropped and the 49.5% share is stated with the same
  caveat I raised. Showing the 92.87% run in the accuracy table is the right move —
  it makes the spread concrete instead of asserted.
• connector_contracts.md and execution_platforms.md are updated and accurate,
  including "measured for prefill-dominated load only".
• .gitattributes removed, single constant source in envs.py, and the eager
  raw.get(...) default fixed. test_rank_limit_matches_the_connector plus
  TestMxfp8Dequantization (127/128/126 → 1.0/2.0/0.5, and the short last group)
  close exactly the two gaps I named — the E8M0 exponent bias is now pinned against
  a value known by construction, which is the only way to catch it without a
  device run.

Three items left, all documentation.

[P2] The A5 sample command in the user guide cannot do what it claims on this tree

docs/npu/CAM_P2P_CONNECTOR_USER_GUIDE.md:228-233

The guide documents
AFD_TEST_DEVICES=2,3 AFD_TEST_QUANT_MODE=4 python tests/unit/compat/npu/a2e_e2a_a5.py
and states that it "checks both activation and scale bytes for both FP8 formats and
compute_gate=0 and 1", and that "default AFD_TEST_QUANT_MODE=0 retains the BF16
roundtrip". At e88eaae, tests/unit/compat/npu/a2e_e2a_a5.py reads only
AFD_TEST_DEVICES, AFD_TEST_E, AFD_TEST_A, AFD_TEST_WORLD and AFD_TEST_PORT;
it has no AFD_TEST_QUANT_MODE, no MXFP8 path, and the scales it passes are the
routing weights, not MX block scales. So the documented command silently runs the
BF16 roundtrip and reports success.

This paragraph is #444's text, and #444 is where the sample is upgraded. Suggest
moving that whole block (the AFD_TEST_* command and the three sentences after it)
out of this PR and leaving it to #444 — otherwise the two PRs also collide in this
file.

[P2] TTFT contradicts itself across two files this PR edits

recipe/README.md:40 says "−19% TTFT"; the model README says
TTFT median 3748 ms → 2905 ms (−22.5%). The previous version's 2995 ms would be
−20.1%, so −19% matches neither the current nor the previous median. If −19% is the
mean rather than the median, please name the metric — right now the same result is
two different numbers in two files this PR touches.

Nit: the quant_mode table row in the same guide says mode 4 transports
"E4M3/E5M2" activations, but the connector hardcodes E4M3
(_A2E_MX_ACTIVATION_DTYPE); E5M2 is operator-level only. One clause saying the
connector emits E4M3 would settle it.

---

On your question: I would not go back and capture a matched A/B trace for the
operator table. The table is gone, the 49.5% share is the number that carries the
argument, and a per-operator delta does not change any review conclusion here — it
would cost two more runs and buy a number that the throughput ratio already
constrains. If the A2E transport profile becomes its own investigation later, that
is the right moment for it.

Nothing else from me blocks this on the code. #444 sequencing, the guide conflict,
the missing test-ready run and the absent decode data point are all stated in the
PR body, which is the right place for them.

[jiangkuaixue123]

Static review at 2b34225. One additional E8M0 decoding issue below; no tests were executed.

[jiangkuaixue123]

[P2] Handle the two E8M0 endpoint encodings before shifting

E8M0 byte 0x00 represents 2^-127, not zero, and 0xff represents NaN, not infinity (see PyTorch's conversion). This shift therefore silently zeros a valid minimum-scale block: an E4M3 activation of 16 with scale byte 0 should restore to 2^-123, which is representable in BF16, but currently becomes 0. It also changes NaN-scale propagation. Please special-case these encodings in the reconstructed FP32 bits and cover bytes 0 and 255 in the dequantization tests.