GDN prefill runs the sequential recurrence on CPU/ROCm/Vulkan where vLLM runs the chunked algorithm on every device — 2.3e-4 out, 2.2e-3 state, and it is 100% algorithmic

[localai-org-maint-bot]

Row: -

No row owns this yet and no spec lists it under ## Owed. That is part of
the finding. Candidate owners are KERNEL-GDN-AOT-BF16 and KERNEL-SSM-MAMBA;
whoever claims it should add the Row: line here and the ## Owed entry in the
same change.

The one-line result

vt::GdnPrefill's CPU arm runs the sequential gated-delta recurrence. vLLM's
prefill runs the chunked WY decomposition, on every device, with no sequential
option. Measured against the committed chunked-oracle golden, our CPU
algorithm is 2.29e-04 (out) and 2.25e-03 (state) away from what vLLM
actually emits, and 100% of that distance is the algorithm choice, not our
precision: recomputing the same recurrence in float64 moves the distance by
2.7e-09.

The divergence is recorded — in .agents/specs/gdn-semantics.md §7,
.agents/porting-inventory.md:124, include/vt/ops.h:3644-3646 and
include/vllm/v1/attention/backends/gdn_attn.h:37-40 — but it was recorded in
July 2026 as a milestone-staged decision ("correctness-grade sequential; chunked
perf kernel M2.3"). M2.3 landed the chunked kernel on CUDA only. The CPU,
ROCm and Vulkan arms never moved, no record says so, and
.agents/porting-inventory.md §9 "Deviations from upstream (forced, recorded)"
carries no entry for it.

What vLLM does, at the pin 5559679229bc961848b121ccdeaa8fa5d79bec98

QwenGatedDeltaNetAttention._forward_core takes the chunked kernel on every
prefill and has no sequential branch:

• vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py:1424-1450 —
  if attn_metadata.num_prefills > 0: ... self.chunk_gated_delta_rule(...).
• :1460-1479 — elif attn_metadata.num_decodes > 0: is the only other arm,
  and it calls fused_sigmoid_gating_delta_rule_update.
• ChunkGatedDeltaRule (:212-336) resolves to forward_cuda (FlashInfer),
  forward_cutedsl, or forward_native — and forward_native (:266-294)
  calls fla_chunk_gated_delta_rule, the Triton chunk kernel. There is no
  torch-native and no CPU fallback. The chunked algorithm is the behaviour on
  every path.
• fused_recurrent_gated_delta_rule, the sequential kernel our CPU arm ports,
  is not referenced by the Qwen GDN layer at all at this pin. Its only callers
  are vllm/model_executor/layers/mamba/gdn/olmo_gdn_linear_attn.py:430 and
  :473, both decode.

These are at-pin citations, not forward references.
.agents/oracles/vllm.md and .agents/upstream-sync.md both pin
5559679229. The model qwen4_exp is not registered at the pin, but the GDN
layer it reuses through RunGdnBlockPaged is, and
qwen_gdn_linear_attn.py is its mirror source.

What we do

• CPU: src/vt/cpu/cpu_ops.cpp:1815 GdnPrefillKernel →
  GdnHeadTokenStep (:1765-1794), a strictly sequential per-token
  recurrence with the [Dv,Dk] state in f32 throughout. No dtype branch, no
  chunking.
• CUDA: src/vt/cuda/cuda_gdn.cu:6195 GdnPrefillKernelCuda takes
  GdnPrefillChunkedCuda (:6117) by default and falls back to the sequential
  GdnScanCuda only for VT_GDN_CHUNKED=0, non-WMMA dims, or < sm_80.
  This arm mirrors upstream.
• ROCm: src/vt/rocm/rocm_gdn_scan.hip:2-13 — sequential only, with the
  chunked prefill named as a deliberate non-port (docs/ROCM.md §6).
• Vulkan: src/vt/vulkan/vulkan_ops.cpp:1648, the recurrence.
• Tenstorrent: src/vt/tenstorrent/tenstorrent_ops.cpp:5442 — chunked, via
  ttnn::transformer::chunk_gated_delta_rule.

So three of five backends diverge from vLLM's prefill algorithm, and the
one users are told to prefer for exact ids is one of them.

The measurement

Committed goldens, identical inputs, no build needed. The f32 goldens were
dumped from upstream's sequential kernel; the bf16 golden from upstream's
chunked prefill kernel (.agents/specs/gdn-semantics.md §9). The same
sequential algorithm, run in numpy f32, against both:

golden	oracle it was dumped from	our sequential vs it (out)	(state)	committed tol
gdn_prefill_f32_realdims	fused_recurrent_gated_delta_rule	1.12e-08	5.96e-08	1e-05
gdn_prefill_f32_small	fused_recurrent_gated_delta_rule	2.98e-08	1.19e-07	1e-05
gdn_prefill_f32_small_noinit	fused_recurrent_gated_delta_rule	2.98e-08	8.94e-08	1e-05
gdn_prefill_bf16_realdims	chunk_gated_delta_rule	2.29e-04	2.25e-03	5e-03

Two things follow.

Our CPU arm is a faithful port — of the wrong kernel. It reproduces
upstream's sequential kernel to ~1e-08, i.e. to round-off. It is not buggy.
It implements a kernel vLLM does not use for prefill.

The golden tolerance is the recorded footprint of the divergence. The
sequential-oracle goldens gate at 1e-05. The chunked-oracle golden had to be
opened to 5e-03 — 500x looser — and its own manifest note says it started at
1.5e-02. Nothing else in that family needs that.

"More accurate" and "mirrors upstream" point in opposite directions here,
by four orders of magnitude. Taking the f64 sequential recurrence as ground
truth (the recurrence is the definition; the chunked form is an algebraically
equivalent reassociation):

	distance from the exact recurrence	distance from vLLM's actual output
our CPU algorithm (f32 sequential)	1.15e-08 out / 5.08e-08 state	2.29e-04 out / 2.25e-03 state
vLLM's chunked prefill output	2.29e-04 out / 2.25e-03 state	0

Our CPU arm is ~20,000x more accurate than the reference and ~20,000x further
from it. The f32-vs-f64 row is what settles attribution: 2.286453e-04
versus 2.286426e-04. Our rounding contributes 2.7e-09 of a 2.29e-04 gap.

Reproduce with tests/parity/goldens/gdn_prefill_* and any numpy; the numbers
also agree with what the gdn_prefill_bf16_realdims manifest already records
from the C++ CPU pass (2.29e-04 out, 2.2486746e-03 state), and
GdnHeadTokenStep's arithmetic has not changed since that dump — the one
commit touching it, 9ef45b34e, only made heads a parallel axis over disjoint
state blocks.

Why this matters, beyond tidiness

docs/USAGE.md:670 tells users "--device cpu is the arm to use when the
exact ids matter." On the mirror criterion that is backwards: the CPU arm is
the one whose prefill algorithm vLLM does not run. #2547 /
docs/bench-evidence/qwen4exp-cuda-prefill-divergence-20260902.md traced the
first CPU-vs-CUDA divergence to exactly this and concluded the CUDA arm mirrors
upstream and the CPU arm is the outlier. That conclusion is correct. What it did
not say is that the outlier is the one that should move.

What fixing it would cost, and what it would NOT buy

It would not be a small change and it is not obviously worth doing first.

• A CPU chunked prefill means porting FLA's five-stage pipeline
  (chunk_local_cumsum → chunk_scaled_dot_kkt → solve_tril →
  recompute_w_u → chunk_delta_h → chunk_fwd_o, FLA_CHUNK_SIZE=64) with
  upstream's dtypes: bf16 h/u/w/v_new, f32 A and final_state. The
  CUDA arm's LaunchChunkedPrefill is a local reference for the structure.
• It needs the CUDA arm's dtype-conditioned routing, because upstream's chunk
  wrapper rejects f32 (chunk.py:213) and three committed goldens are
  therefore sequential-oracle goldens. tests/parity/test_op_parity.cpp:594
  already forces VT_GDN_CHUNKED=0 for the f32 cases on CUDA for this reason.
• Every CPU-arm token output moves. Eleven tracked files carry the CPU control
  sequence 11751 13 15767 411 2029 11 1092 369, including docs/USAGE.md,
  five specs, four docs/bench-evidence/ files and a comment in
  src/vllm/model_executor/models/qwen4_exp_registry.cpp:212.
• The CPU arm is the oracle for the ROCm, Vulkan and Tenstorrent
  cross-device gates (tests/vt/test_backend_cross_device.cpp, ROCm's
  NMSE <= 5e-4 bar). Moving CPU to chunked while ROCm and Vulkan stay
  sequential inverts which side of those gates is the reference, and 2.25e-03
  on state is not obviously inside a 5e-4 NMSE bar.
• It is NOT proven to buy token agreement, and there is direct evidence
  against assuming it. docs/bench-evidence/qwen4exp-cuda-prefill-divergence-20260902.md
  §2 measured the opposite experiment: VT_GDN_CHUNKED=0 cut the layer-0
  divergence 332x and agreed on fewer token ids (3 of 8, down from 5).
  Agreement is an argmax over near-ties and is not monotone in residual. A CPU
  chunked kernel would also not be bit-identical to the WMMA CUDA one. And the
  same evidence file names a second, unremoved source: a 7.27e-05 per-layer
  MoE residue that survives with GDN removed.

So the case for the change is mirror compliance, which AGENTS.md makes a
first-order obligation, and not a predicted token win. Anyone claiming this row
should say which of the two they are buying.

What was not established

The CUDA arm was not re-measured here. No GPU lease was taken; the CUDA
figures above are quoted from #2547's evidence file, not reproduced. The
claim that a chunked CPU kernel would land within round-off of the chunked
oracle is untested — no chunked reference was implemented on CPU in this
investigation.

Investigated 2 September 2026 against origin/main 6faa9e482.

[localai-org-maint-bot]

Confirmed with the committed CPU kernel, not only in numpy

Built test_op_parity CPU-only from origin/main 6faa9e482
(cmake rc=0, ninja -j 4 rc=0) and ran the real vt::GdnPrefill CPU arm
against the committed goldens. Baseline: rc=0, 113 assertions, "46 op
cases ran" — a non-zero assertion count, so this is a pass and not a skip.

The gdn_prefill_bf16_realdims case was proven to run by mutation rather
than assumed from silence. The baseline log never names it, so its tolerance
was tightened in place and the tree restored byte-for-byte afterwards
(git status --porcelain on the golden directory: 0 files).

gdn_prefill_bf16_realdims tol	rc	first mismatch reported
atol=1e-05, rtol=1e-05	1	out[0]: got -0.0105125 want -0.0105591 (|d| = 4.66e-05)
atol=2.5e-04, rtol=0	1	out clean; state_out[128]: got -0.137035 want -0.137364 (|d| = 3.29e-04)
atol=2.5e-03, rtol=0	0	—

So the shipped CPU kernel's distance from vLLM's chunked prefill output is
bracketed in (4.66e-05, 2.5e-04] on out and (2.5e-04, 2.5e-03] on
state — consistent with the 2.29e-04 / 2.25e-03 computed independently,
and with what the manifest already recorded from the July 2026 CPU pass.

The 1e-05 row is the load-bearing one: it is the tolerance the three
sequential-oracle goldens pass at. The same kernel clears 1e-05 against
upstream's sequential kernel and misses it by 4.7x on the very first output
element against upstream's actual prefill kernel.

The CUDA arm was not re-measured. No GPU lease was taken.