feat(KV-FP8): W6 — the ROCm fp8-e4m3 KV cache store and read

[ghazni101]

What changed

The ROCm backend gains the fp8-e4m3 KV cache store kernel
(ReshapeAndCacheFp8KernelRocm in src/vt/rocm/rocm_dense_basic.hip) and
the fp8 dequant on the paged-attention read (LoadKv in
src/vt/rocm/rocm_paged_attn.hip). This closes the last non-Metal gap in
the fp8 KV cache surface.

OpId::kReshapeAndCacheFp8 is registered for DeviceType::kROCM in
src/vt/rocm/rocm_ops.hip. The fp8 read refusal in src/vt/ops.cpp
widens from kCPU || kCUDA to kCPU || kCUDA || kROCM. Metal remains
refused by name.

ROCm HIP has no __nv_cvt_float_to_fp8 intrinsic, so the store uses a
software codec (F32ToF8E4M3Dev) with frexpf/nearbyintf/ldexpf
arithmetic that mirrors the CPU vt::F32ToF8E4M3. The codec is
bit-identical and portable across all ROCm targets, unlike hardware fp8
conversion intrinsics that exist only on CDNA2+ (gfx940/941/942).

Why

The fp8 KV cache halves the KV footprint. The CPU arm (W1) and CUDA arm
(W2) already landed; the ROCm arm was the last refused backend. Issue
#2065 tracks this work.

How to verify

Build with HIP and run the test:

cmake -B /tmp/build -DVLLM_CPP_HIP=ON
make -j4 vllm test_rocm_fp8_kv_cache
ctest -R test_rocm_fp8_kv_cache

Measured on gfx1100 (RX 7900 XTX), hipcc 7.2.4:

• test_rocm_fp8_kv_cache: 7/7 cases, 28/28 assertions. G3 store
  byte-identical to the W1 CPU oracle; G4 f32 read NMSE < 1e-6; G4b bf16
  read NMSE < 1e-4; G5 e5m2 refused with the named message.
• test_ops_fp8_kv_cache: 8/8, 511 assertions (CPU regression).
• test_rocm_backend: 9/9, 1065 assertions (no regression).
• test_ops_paged_attn: 14/14, 1646 assertions (no regression).

RED-first proven: dropping *scale in the read dequant fails G4/G4b at
NMSE ~34000x; dropping /scale in the store fails G3 at the byte level.

Out of scope

fp8_e5m2 compute, per-attention-head scales, the Metal arm, fast-path
(tensor-core/rocWMMA) fp8 attention kernels, and the memory-halving e2e
measurement on a ROCm gate model.

Relationship with #1936 and landing order

#1936 (the GFX1100-TG200 campaign) cherry-picked this PR's W6 store
and read work (commit 87c6f2518516) so its fp8 decode attention arm
can be tested end-to-end. The decode read side in #1936 (commit
e43a39d45803) depends on this store side -- both are needed for fp8
KV cache to work.

Landing order:

1. This PR lands first -- the standalone W6 store kernel, closes
   KV-FP8 W6: the ROCm fp8-e4m3 KV cache store and read #2065.
2. perf(GFX1100-TG200): the 200 tok/s gfx1100 decode campaign and fp8 KV cache decode attention #1936 rebases onto the new main and drops the cherry-pick commit
   87c6f2518516 since the store work is then in main.

All 13 files in this PR are currently also in #1936. After this PR
lands and #1936 rebases, the overlap is zero.

Closes #2065

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:glm-5-2 [OMP]

[localai-org-maint-bot]

@mudler This ROCm fp8-KV arm is ready for CI review, but the current reconciled head has no checks because its workflows still need approval. The branch includes focused byte-identity, dequantization, refusal, and backend regression tests with gfx1100 evidence. Please approve the current-head workflows before a merge decision.