Qwen3.8-27B Q4_K_M: p4/14 is the CPU tier diverging across ARCHITECTURES, not ROCm; #2590 narrows to two steps

[localai-org-maint-bot]

Row: QUANT-QWEN38-27B-GGUF-ARM

#2590 was opened owning three steps at which our ROCm and CPU tiers emit a
different token from an identical prefix. Run on ONE host, in one lease, from one
libvllm.so, only two of them reproduce.

step	recorded in #2590	measured on one host (strix:gpu0, rc job 5af7552a-5fba-4e51-92d8-f4cdfd3b21ca)
p1/45	ROCm 303, CPU 1521	ROCm 303, CPU 1521 — reproduces
p3/45	ROCm 25, CPU 393	ROCm 25, CPU 393 — reproduces
p4/14	ROCm 22486, CPU 4593	both tiers 22486; all 48 ids EQUAL

#2590's A vs D column compared a gfx1151 ROCm run on x86-64 against a
CPU-tier run on thor, aarch64, so it crossed an architecture as well as a
tier. With both arms on x86-64 our CPU tier emits 22486 at p4/14, agreeing with
the ROCm tier, with the HIP oracle and with the aarch64 oracle.

So p4/14 is the CPU tier's own architecture dependence, not a ROCm-local
term. The token the aarch64 CPU tier emitted there (4593) is one no other
side emits. This issue owns that, and #2590 is narrowed to two steps.

What is already known

• The step is a near-tie: the HIP oracle's top1 - top2 gap at p4/14 is
  0.092752, inside the 19-step population whose gap is below 0.20.
• The x86-64 CPU tier is bit-reproducible per layer against itself, and so is the
  ROCm tier: the run-to-run floor measured here is exactly 0 over 9360 compared
  per-layer rows.
• The cross-tier hidden-state divergence at layer 63 is 2.6-2.8% relative, which
  is 1.03 to 1.11 times what two independent bf16 residual streams must produce.
  A near-tie under roughly half a nat is decided by that, so a divergence at a
  0.0928 gap needs no further cause than a different reduction order.
• The one candidate that is NOT explained by that is why aarch64 and x86-64
  differ from each other, which is what this issue is for. QuantRepackForDevice
  is dev == kCPU only and the aarch64 lane carries the block_q8_0x4 interleave
  (quant_repack is set from a host-ISA probe with no device term: aarch64 i8mm + --device cuda stages ARM-repacked weights the CUDA kernel cannot read #2406); x86-64 does not repack Q4_K either, so the repack is a candidate and
  not a conclusion.

The next traceable step

Run the same per-layer hidden-state comparison this measurement built
(scripts/tier-hidden-delta.py, scripts/tier-delta-attribution.py,
VT_DUMP_ACT) with the two arms being aarch64 CPU and x86-64 CPU on the same
artifact and the same prompt, and read the per-layer profile the same way. The
instrument is committed and needs no GPU on either side.

Not admissible from this

No speed, latency or memory number. The arm's declared token gate is FAIL and
AGENTS.md §Gates admits none from it.

Evidence:
docs/bench-evidence/qwen38-27b-q4km-tier-divergence-bisect-20260902.md.
Spec: .agents/specs/rocm-tier-hidden-state-bisect.md.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]