ABQuantGemmPipelineAgBgCrEightWaves produces silently wrong results — no assert, no diagnostic — for any B-quant granularity finer than one scale per warp tile.
Measured (gfx950, MI350X, example 38, fp8)
| config |
result |
time |
8-wave, group_size=1x128x128 (kN=128, as upstream exercises it) |
correct |
0.1516 ms |
8-wave, group_size=1x1x128 (kN=1, per-channel) |
fail, 80.5% wrong |
0.1442 ms |
Cause
Two pieces assume one scale per warp tile:
- the non-
BPreshuffleQuant consume branch (block_universal_gemm_as_aquant_bs_bquant_cr_eight_waves.hpp:355-375) applies a single scale across the whole 16-wide warp tile, with no cross-lane broadcast;
- the BQ load distribution (
R = sequence<MWarps, warp_size>) replicates the same value to all 64 lanes.
Both are correct exactly when kN >= WarpGemm::kN, which matches the measurements: fine granularity silently multiplies most columns by a neighbour's scale.
The ds_bpermute branch in that same file is not an optimisation of a working path — it is the only construction under which fine-grained 8-wave could be correct, and its load side was never implemented (see #3767). So today there is no correct 8-wave configuration at kN < WarpGemm::kN, by either route.
Ask
A static_assert in the policy (or a documented restriction) so this configuration cannot be selected accidentally. A wrong answer at full speed is the worst failure mode available: nothing in the current code tells the caller that the granularity they asked for is not supported.
Related: #3767 (the missing BPreshuffleQuant load path, and a validated fix for the fine-grained consume side), #3766, #3765.
ABQuantGemmPipelineAgBgCrEightWavesproduces silently wrong results — no assert, no diagnostic — for any B-quant granularity finer than one scale per warp tile.Measured (gfx950, MI350X, example 38, fp8)
group_size=1x128x128(kN=128, as upstream exercises it)group_size=1x1x128(kN=1, per-channel)Cause
Two pieces assume one scale per warp tile:
BPreshuffleQuantconsume branch (block_universal_gemm_as_aquant_bs_bquant_cr_eight_waves.hpp:355-375) applies a single scale across the whole 16-wide warp tile, with no cross-lane broadcast;R = sequence<MWarps, warp_size>) replicates the same value to all 64 lanes.Both are correct exactly when
kN >= WarpGemm::kN, which matches the measurements: fine granularity silently multiplies most columns by a neighbour's scale.The
ds_bpermutebranch in that same file is not an optimisation of a working path — it is the only construction under which fine-grained 8-wave could be correct, and its load side was never implemented (see #3767). So today there is no correct 8-wave configuration atkN < WarpGemm::kN, by either route.Ask
A
static_assertin the policy (or a documented restriction) so this configuration cannot be selected accidentally. A wrong answer at full speed is the worst failure mode available: nothing in the current code tells the caller that the granularity they asked for is not supported.Related: #3767 (the missing BPreshuffleQuant load path, and a validated fix for the fine-grained consume side), #3766, #3765.