Row: MODEL-MM-QWEN4-EXP
The gap
tests/vt/test_moe_router_tie_stability.cpp (landed by #2595, wave TIEBREAK of
#2586) guards each of its three device cases with the tree's permissive idiom:
if (!HasCuda()) {
MESSAGE("no CUDA backend registered; skipping");
return;
}A CUDA build on a host with no device therefore reports 4 cases | 488
assertions | 0 failed | rc 0 and reads as a pass while measuring nothing on
device. That is not hypothetical: the wave's own orin:gpu0 lease did exactly
this, and #2595's evidence file §8 records that the binary read green there with
a broken tie-break compiled in.
The tree already has the idiom that refuses instead.
tests/parity/test_qwen27n_fp8_tower_paged_engine.cpp:149 reads
VT_REQUIRE_27N_FP8_GATE and turns absence into a hard FAILURE, so a harness
that intends the run as evidence can never record the absence as a pass.
This is a tree-wide gap, not #2595's defect. grep -rl "no CUDA backend registered; skipping" tests/ finds 14 files, of which 13 are not this one, and
the broader permissive-CUDA-skip shape appears in 34 test files. #2595 is named
here only because its whole subject is the difference between a real result and
a vacuous one, so it is the change that made the gap legible.
The discriminator the records were missing, and now carry
Assertion count alone does NOT separate the three states, because two of them
read 488. Case count AND assertion count together do:
| state |
cases |
assertions |
rc |
CPU-only build (device cases #ifdef'd out) |
1 |
488 |
0 |
| CUDA build, no device (cases run, skip, return) |
4 |
488 |
0 |
| CUDA build with a device |
4 |
4652 |
0 |
The CPU-only row is measured on the dev box at #2595's head
(380ce21d4); the 4652 row is the thor:gpu0 result #2595 records. #2595's
Gates section previously taught only the assertion count and has been corrected
to teach both.
What closing this means
Either an env gate on the model of VT_REQUIRE_27N_FP8_GATE — one name, read by
every CUDA-device test, absence becomes FAILURE when it is set — or a harness
that asserts the expected case/assertion pair for the build it configured. It is
a tree-wide change across ~34 files and needs its own row, spec and gate; it is
NOT an in-flow fix to a test-only pull request.
Row:
MODEL-MM-QWEN4-EXPThe gap
tests/vt/test_moe_router_tie_stability.cpp(landed by #2595, wave TIEBREAK of#2586) guards each of its three device cases with the tree's permissive idiom:
A CUDA build on a host with no device therefore reports 4 cases | 488
assertions | 0 failed | rc 0 and reads as a pass while measuring nothing on
device. That is not hypothetical: the wave's own
orin:gpu0lease did exactlythis, and #2595's evidence file §8 records that the binary read green there with
a broken tie-break compiled in.
The tree already has the idiom that refuses instead.
tests/parity/test_qwen27n_fp8_tower_paged_engine.cpp:149readsVT_REQUIRE_27N_FP8_GATEand turns absence into a hard FAILURE, so a harnessthat intends the run as evidence can never record the absence as a pass.
This is a tree-wide gap, not #2595's defect.
grep -rl "no CUDA backend registered; skipping" tests/finds 14 files, of which 13 are not this one, andthe broader permissive-CUDA-skip shape appears in 34 test files. #2595 is named
here only because its whole subject is the difference between a real result and
a vacuous one, so it is the change that made the gap legible.
The discriminator the records were missing, and now carry
Assertion count alone does NOT separate the three states, because two of them
read 488. Case count AND assertion count together do:
#ifdef'd out)The CPU-only row is measured on the dev box at #2595's head
(
380ce21d4); the 4652 row is thethor:gpu0result #2595 records. #2595'sGates section previously taught only the assertion count and has been corrected
to teach both.
What closing this means
Either an env gate on the model of
VT_REQUIRE_27N_FP8_GATE— one name, read byevery CUDA-device test, absence becomes FAILURE when it is set — or a harness
that asserts the expected case/assertion pair for the build it configured. It is
a tree-wide change across ~34 files and needs its own row, spec and gate; it is
NOT an in-flow fix to a test-only pull request.