feat(MODEL-MM-GLM53-FLASH-KPOOL-CUDA): give GLM-5.3-Flash's k-pool indexer a device it can run on - #2432
feat(MODEL-MM-GLM53-FLASH-KPOOL-CUDA): give GLM-5.3-Flash's k-pool indexer a device it can run on#2432localai-org-maint-bot wants to merge 7 commits into
Conversation
…e the kernel down before writing it #2415 asks whether GLM-5.3-Flash's k-pool DSA indexer gets a CUDA kernel or stays on the host at eleven device-to-host round trips per step. This block takes the first answer and records the arithmetic behind it: the packed key history the pool grid is re-formed over is 257 floats per token per layer, so the host arm moves 371 MiB per step across the eleven DSA layers before the selection comes back. A device arm whose indexer is on the host is the CPU path with a copy tax. W9c-0 scopes two ops and nothing else. `vt::Glm5NextKpoolCompress` is upstream's `get_pooled_states` (`modular_glm5_next.py:897-970` @ transformers v5.16.1) with the `keep` compaction done on the device and `P` published as a device scalar, so neither op synchronises. `vt::Glm5NextKpoolSelect` is the selection half of `forward` (`:821-875`) with `get_visible_tokens` (`:877-895`) folded in as a predicate and `append_visible_tail` (`:972-1022`) folded in as the tail write. The section says why `PackIndexerStates` and `GetVisibleTokens` deliberately do NOT become ops, why the kernels are f32 where the host reference is double, and why there is no CPU provider. O36 names what the wave leaves unreached, the wave that owns the wiring and the issue that tracks it, because the ops land before anything calls them. FOLLOWING_AGENTS_PROTOCOL Refs: #2415, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…ce it can run on GLM-5.3-Flash's DSA indexer is a k-pool indexer, and until this change nothing in this tree computed one on a device. `include/vt/ops.h` defined exactly two indexer ops, both of them the DSA Lightning Indexer's, which score RAW TOKENS; `git grep -l 'kpool\|index_kpool\|compress_ape\|compress_gate' src/ include/` returned eleven files and every one was a `glm5_next_*` host file. Composition was not available either: this OpId inventory has no general softmax, no general reduction and no pooled gather, and its only top-k entries select experts and sampler tokens. `vt::Glm5NextKpoolCompress` is the learned pooling — `get_pooled_states`, `modular_glm5_next.py:897-970` at transformers v5.16.1, the lane pin this row already carries. `index_head_dim` independent softmaxes over a pool's members, one per channel, with the learned intra-pool position embedding added to each member's gate score, the pool grid starting at the first VALID token rather than at slot 0. `vt::Glm5NextKpoolSelect` is the selection half of `forward` (`:821-875`) with `get_visible_tokens` (`:877-895`) folded in as a predicate and `append_visible_tail` (`:972-1022`) as the tail write. THE COMPACTION IS ON THE DEVICE AND `P` NEVER REACHES THE HOST. `keep = pool_valid.any(0)` (`:968`) is not cosmetic: `select_k` reads the compacted width, so a `P` one too large moves the ragged tail's write offset by `index_kpool` columns and changes what the final truncation keeps. The compress op therefore does a per-pool validity pass, a single-block exclusive scan and a compacted write, and publishes `P` as a `[1]` i32 DEVICE scalar the select kernel reads on the device. Every buffer is sized at the static bound `ceil(kv_len / index_kpool)`. That is the whole point: eleven DSA layers stop paying a device-to-host round trip on every step, which is what #2415 asked to decide. Both ops register on kCUDA only. There is deliberately no CPU provider — the CPU answer is `glm5_next_dsa.cpp` and it is this family's ORACLE, so registering it again under these ids would make the seam its own golden. A CPU queue is refused by name through `vt::GetOp`, and `vllm::glm5_next::KpoolDeviceOpsAvailable()` mirrors `deepseek_v4::V4DeviceKernelsAvailable` so a forward can ask before it builds operands. RED FIRST, twice, on `bd2c14ce5` with the test in place and no implementation. The first build stopped at `fatal error: vllm/model_executor/models/ glm5_next_device.h: No such file or directory`. Adding only that seam moved the failure onto the ops themselves: `'kGlm5NextKpoolCompress' is not a member of 'vt::OpId'` and `'Glm5NextKpoolSelect' is not a member of 'vt'`, twelve diagnostics naming the two ids and the two free functions. Both builds exited 1. f32 throughout, because `:823` and `:960-964` are fp32 upstream. The host reference widens to `double` and the device deliberately does not follow it: a fp64 pool softmax would put the model path on the 1/64-rate pipe to be more precise than the thing it mirrors, and no token gate can see a dtype that is too wide. Every accumulation spells its rounding with `__fadd_rn` / `__fmul_rn` / `__fdiv_rn` in the host reference's own order, so nvcc's default contraction cannot decide the answer, and `head_scale` is narrowed once on the host in double rather than taken from `rsqrtf`. NOTHING CALLS EITHER OP YET. `ModelRegistry::Forward` still reaches the host indexer, `glm5_next_forward.cpp:231-238` still refuses a non-CPU queue by name, and the only callers are the new device gate and the availability probe. W9c-3 owns the wiring, on row `MODEL-MM-glm5-next-glm5-next-for-conditional-generation`, tracked by #2410, and the spec's O36 names all three. This is a staged slice under AGENTS.md "Nothing lands dead", not a delivered capability. FOLLOWING_AGENTS_PROTOCOL Refs: #2415, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…the margin the top-k decides on, not by a number The device gate compared `index_scores` against a flat 2e-5. That tolerance was chosen rather than derived and it was wrong in both directions: the golden score magnitudes on this fixture run to 45.17, so one f32 ULP there is already 3.8e-6, and a bound below that fails on arithmetic while one far above it bounds nothing. Top-k error is bimodal, so the only question a selection gate can actually answer about a score is whether the numerical difference is small against the gap the selection turns on. So the gate now asserts `score_delta * 4 < margin` and prints both, where the margin is the gap between the `select_k`-th and `select_k + 1`-th masked score over the rows that actually prune. A future fixture that narrows the margin fails here instead of quietly becoming a coin flip. Two records move with it. W9c-0 gains the pre-lease numpy transcription's numbers and, beside them, the bound that transcription does not cross: it fixes the algorithm, and it says nothing about the build, nvcc, the launch geometry, the shared-memory reductions or float rounding. And §W9c's device-gate block said `dgx:gpu0` was queued and the GB10 leg PENDING; the fleet has since answered it (`test_cuda_quant_dot` 17 cases / 177,284 assertions and the GLM-5.3-Flash MoE geometry case's 25, on `sm_121a` with FA2 at `[121a]`), so the sentence is corrected here rather than left asserting a PENDING that is no longer true. FOLLOWING_AGENTS_PROTOCOL Refs: #2415, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…ts/CMakeLists.txt additions `origin/main` moved 27 commits while this branch was written, and the only file both sides touched is `tests/CMakeLists.txt`, where each added its own `vllm_cpp_add_test` block at a different place. Git resolves that cleanly; the resolution is checked by asserting BOTH blocks are present rather than by trusting the clean exit, because a keyed record that automerges cleanly and drops a tail row is a failure this repository has already paid for. Nothing on `main` touches `include/vt/ops.h`'s enum tail, `src/vt/ops.cpp`, `src/vt/op_provider.cpp`'s name switch, `src/vt/cuda/` or `.agents/specs/glm5-next-flash.md`, so no claim in W9c-0 or O36 is falsified by this merge. The device gate queued on `dgx:gpu0` measures `83fa7c8fb`, the SHA recorded in the archive's `BASE_SHA`; the merge changes no file under test and the job's own source sha256 says which bytes it built. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…two kCUDA lookups, with the reason beside them `check-device-leakage.py` reds this branch at `kcuda: 2 > baseline 0`: `glm5_next_device.cpp` names `vt::DeviceType::kCUDA` twice, and it lives under `src/vllm`, which the checker scans as the device-agnostic shared layer. The two sites are the checker's own named exception rather than a hole in it. They are `vt::OpRegistered` lookups -- the "ask the op/provider table the question instead" the error message points at -- in a resolver TU that holds no CUDA code and is always compiled. This is the shape `laguna_device.cpp` already carries at the same count for the same reason, and `deepseek_v4_device.cpp` at four times it. So the entry goes in ALLOWLIST, per-file and per-bucket with an exact expected count, and the baseline is NOT raised. A later edit that changes the count reds as ALLOWLIST STALE, which is what keeps this from becoming a mute switch. The reason records what a reader of the count cannot infer: why there is no CPU provider to fall back to (the CPU answer is `glm5_next_dsa.cpp` and it is the ORACLE these kernels are gated against), what a CPU build therefore does (the probe returns false and `vt::Glm5NextKpoolCompress` refuses by name through `GetOp`), and which test asserts both. It also inherits the deferred follow-up the other two entries carry -- thread a runner `DeviceType` through the resolver instead of hardcoding kCUDA -- and says why it cannot be taken here: W9c-3 (#2410) owns the forward that would supply a runner, and until it lands there is no device to read from. FOLLOWING_AGENTS_PROTOCOL Refs: #2415, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
Status: the GB10 gate is queued, and one r1 attempt has already been thrown awayThe device gate is still PENDING and is not reported as a pass. Recording what has moved so the next reader does not re-derive it. r1 reached r3 also makes the source hash FATAL rather than printed. r1 and r2 printed The archive was restaged onto the branch head. The first archive was CPU evidence on the merged head
The one failure is Two failures in the first preflight run were mine and are fixed in What the queued job will produceLegs 1 and 2 (the device gate and the host reference suite), then a nine-mutation matrix on the CUDA TU — the Leg 3 is the |
Do not merge: the GB10 gate is queued behind a 5-hour job and has not returnedRecording the exact state so whoever reads this next does not re-derive it or, worse, read a queued gate as a taken one. Job Where its result will be: Read the assertion count, not the exit code. The three device cases in What is already asserted before any result is believed, because each of these has produced a false green here before: the CUDA arch is read off the device and refused if unreadable; The archive is Three things this PR does not claim, stated plainly rather than left to inference: no device gate has been taken; no throughput number was measured and there is no denominator; and no generation from this artifact was observed by this branch. The ops land unreached — |
…ken on GB10, and the two mutations that are not assertion kills say so The gate W9c-0 left PENDING has returned from `dgx:gpu0`. Recording it with the box named beside every number, because `sm_121a` and `thor`'s `sm_110` are different targets and neither transfers to the other. `test_glm5_next_kpool_device` runs 4 cases / 918 assertions / 0 failed with `KPOOL_SKIPPED_CASES=0`. The skip count is the load-bearing line and not the exit code: the same binary on a CPU-only build passes 4 cases with FOUR assertions and three cases skipped, so 918-against-4 is what says the device arm ran at all. `test_glm5_next_dsa` is unmoved at 10 / 1934. The device answer agrees with the transformers v5.16.1 RUN goldens to 3.58e-07 on the pooled keys and 6.68e-06 on the scores, and its selection is SET-equal and positionwise equal to both the oracle and the host reference at 0 mismatches of 462, against a smallest decision margin of 2.58e-03 over 17 pruning rows. That ratio is the point: the fp32 device result differs from the fp64 host reference by about a hundredth of the gap the top-k decides on, which is a statement a bimodal gate can make. `P == 5` from `np == 6` says the `keep` compaction ran. Nine mutations, each rebuilt with the BINARY's sha256 asserted to have moved before its result counted, each restored byte-for-byte with the source hash checked, and the pristine tree rebuilt afterwards to reproduce 918. Seven were killed by assertions. The other two are recorded rather than counted. M8 deleted the compress op's registration and SURVIVED AS A SKIP: the probe goes false, `HasCuda()` goes false, and the suite exits 0 with 4 assertions and 3 skipped cases. That is the reachability answer and not a kill, and it is exactly why the job reads the skip count. M5 was killed by the COMPILER, which is weaker. Blanking the tail write orphaned `valid` and `vis` and `-Werror=all-warnings` refused it with `error #177-D`. A compiler kill proves the code changed and nothing about whether the gate sees the defect, so it is rewritten to keep every operand live and re-submitted as job `3db96e7a`. Until that returns the ragged tail is the one guarantee here with no assertion-grade mutation behind it, and this entry says so rather than letting seven-of-nine read as nine. FOLLOWING_AGENTS_PROTOCOL Refs: #2415, #2410 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
The GB10 device gate is taken, and it passesEvery number below came from
The CUDA TU compiled clean on the first attempt under Leg 1 — the gate4 cases, 918 assertions, 0 failed,
Selection agreement is SET equality and positionwise equality, against the oracle and against the host reference, at 0 mismatches of 462. So the fp32 device answer differs from the fp64 host reference by about a hundredth of the gap the top-k turns on — the statement a bimodal gate can actually make. Leg 2 — the host reference, unmoved
Nine mutations, and the two that are not assertion killsEach applied to product code, rebuilt with the binary's sha256 asserted to have moved before its result counted, restored byte-for-byte with the source hash checked (9 restores, all verified), and the pristine tree rebuilt afterwards, reproducing 918.
M8 is the reachability result, and it is not a kill. With no provider the probe goes false, M5 was killed by the compiler, which is weaker, and it is owed. Blanking the tail write orphaned Still not claimedNo throughput number and no denominator. No generation from this artifact was observed. The ops remain unreached — Evidence recorded in |
…ot the refusal this row keeps citing W9c-0's lease drove `--device cuda` on the real `GLM-5.3-Flash-UD-Q2_K_XL` artifact as a by-product, and it falsifies a sentence this row has repeated for several waves. The model loads, auto-fits the KV cache to 256 blocks and `max_model_len` 8192, enters the engine, and dies in the KV binding: `'model.layers.3.self_attn. indexer.k_cache' resolved to attn_kv index 45 but only 22 cache(s) arrived` (`glm5_next_kv.cpp:127`). The guard at `glm5_next_forward.cpp:231-238` is still in the tree and NEVER FIRES, because `ResolveAttnCache` throws before the forward is entered. So "`--device cuda` still refuses by name at `glm5_next_forward.cpp:231-238`" describes code that exists but no behaviour a user can reach, and W9c-3 -- scoped as the compose that deletes that refusal -- will meet the KV binding first and earlier. The 22 is W5b-2c's own expected count (11 latents plus 11 indexer caches, O29), so the name index is producing indices from a larger space than the vector it indexes into. This is the FIRST run that reached the engine. The prior probe passed the checkpoint DIRECTORY rather than a shard, died in `hf_config` in five seconds over an artifact that carries no `config.json`, and captured nothing; it is not a baseline and is not cited as one. What is deliberately NOT concluded: whether `--device cpu` reproduces it. This lease drove the CUDA arm only, the auto-fit above is memory-dependent and may differ per device, and O30's ` Paris.` predates several waves and a different resolved config. A general KV-binding defect and a device-only one have different owners, so O37 leaves the polarity open and names the one `--device cpu` run that settles it rather than guessing. Filed as #2445 and listed under `## Owed` as O37. Not repaired in flow: the fix lives in the multi-KV index mapping W5b-2c owns, and the measurement above has to come first. It is also not a claim about the k-pool ops this branch lands -- nothing on that path reaches them, and O36 already says they are unreached. FOLLOWING_AGENTS_PROTOCOL Refs: #2445, #2415, #2410, #2348 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
Leg 3 (the by-product) found something, and it falsifies a sentence this row repeats
Driven correctly — first shard, not the directory — The refusal at This is the first run that reached the engine. The prior probe passed the checkpoint directory, died in Deliberately not concluded: whether Remaining before this is mergeableM5 re-run, job |
2 participants
feat(MODEL-MM-GLM53-FLASH-KPOOL-CUDA): give GLM-5.3-Flash's k-pool indexer a device it can run on
#2415 asked one question: does the k-pool DSA indexer get a CUDA kernel, or does
the indexer stay on the host and every DSA layer pay a device-to-host round trip
on every step? This branch answers it with the kernel and closes #2415 with the
argument.
Why the kernel, and not the round trip
The host arm is not a cheap fallback on this model. The pool grid is re-formed
over the WHOLE packed key history on every call, and that row is
2 * index_head_dim + 1 = 257floats per token per layer — 33.7 MiB per layerper step at 32k context, 371 MiB per step across the eleven DSA layers, plus a
[B, S, 2051]selection coming back. That is a synchronous stall eleven times onthe critical path of a 101.24 GiB model whose entire reason for having a device
arm is that its weights are already resident. A device arm whose indexer is on
the host is the CPU path with a copy tax. The alternative is not staged, not
flagged and not kept as a fallback.
What the gap was
Measured, not inherited.
include/vt/ops.hdefined exactly two indexer ops,kDsaIndexerLogitsandkDsaTopkSelect, and its own comment calls them the DSA"Lightning Indexer" selection pair; they score RAW TOKENS.
git grep -l 'kpool\|index_kpool\|compress_ape\|compress_gate' src/ include/returned eleven files and every one was a
glm5_next_*host file — nothing inthe seam, nothing under
src/vt/, nothing in any other model. Composition wasnot a way out either: this
OpIdinventory has no general softmax, no generalreduction and no pooled gather, and its only top-k entries select experts and
sampler tokens.
What this adds
Two ops, and a probe.
vt::Glm5NextKpoolCompressis upstream'sget_pooled_states(
modular_glm5_next.py:897-970@ transformers v5.16.1, the lane pin this rowalready carries):
index_head_dimindependent softmaxes over a pool's members,one per channel, with the learned intra-pool absolute-position embedding added to
each member's gate score. The pool grid starts at the first VALID token rather
than at slot 0, so a left-padded row groups differently.
vt::Glm5NextKpoolSelectis the selection half offorward(:821-875), withget_visible_tokens(:877-895) folded in as a predicate rather thanmaterialised — a
[B, S, kv_len]visibility tensor is 2.7 GiB at 32k context forsomething a thread evaluates in two instructions, and upstream materialises it
only because torch has no other way to gather under it — and
append_visible_tail(:972-1022) folded in as the tail write, because thetail's write offset is
select_k * index_kpooland only this op knowsP.vllm::glm5_next::KpoolDeviceOpsAvailable()mirrorsdeepseek_v4::V4DeviceKernelsAvailableso a forward can ask before it buildsoperands rather than after it throws.
vLLM implements
glm5_nextat no revision — re-verified rather than inherited:git grep -n 'glm5_next\|Glm5Next' -- vllm/exits 1 with no output at the paritypin
5559679229, and vllm#53906 is still open and therefore inadmissible.The compaction is on the device and
Pnever reaches the hostkeep = pool_valid.any(0)(:968) is not cosmetic.select_kreads theCOMPACTED width, so a
Pone too large moves the ragged tail's write offset byindex_kpoolcolumns and changes what the final[..., :output_width]truncation keeps. The compress op therefore does a per-pool validity pass, a
single-block exclusive scan over the
keeppredicate and a compacted write, andpublishes
Pas a[1]i32 DEVICE scalar the select kernel reads on the device.Neither op synchronises; every buffer is sized at the static bound
ceil(kv_len / index_kpool), andtopk_indicesis a fixed 2051 wide on thischeckpoint — not 2048, and sizing it 2048 truncates the tail the model always
keeps.
No CPU provider, on purpose
The CPU answer already exists as
glm5_next_dsa.cppand it is this family'sORACLE. Registering it a second time under these ids would make the seam its own
golden. A CPU queue is refused by name through
vt::GetOp, and the gate assertsthat refusal.
f32, because that is upstream's own arithmetic
:823scores in fp32 and:960-964takes the pool softmax in fp32. The hostreference widens to
double, and the device deliberately does not follow it: afp64 pool softmax would put the model path on the 1/64-rate pipe to be MORE
precise than the thing it mirrors, and no token gate can see a dtype that is too
wide. Every accumulation spells its rounding with
__fadd_rn/__fmul_rn/__fdiv_rnin the host reference's own order, so nvcc's default contractioncannot decide the answer, and
head_scaleis narrowed once on the host in doublerather than taken from
rsqrtf.Red first, twice, and captured
On
bd2c14ce5, with the test in place and no implementation. The first buildstopped at
fatal error: vllm/model_executor/models/glm5_next_device.h: No such file or directory. Adding only that seam moved the failure onto the opsthemselves —
'kGlm5NextKpoolCompress' is not a member of 'vt::OpId','Glm5NextKpoolSelect' is not a member of 'vt', twelve diagnostics naming thetwo ids and the two free functions. Both builds exited 1.
The gate, and the bound it is written against
tests/vllm/models/test_glm5_next_kpool_device.cppruns the SAME fixturegeometry
test_glm5_next_dsagates on the host —seq_len21 againstindex_topk8, one row left-padded by three — and reusesglm5_next_dsa_goldens.incverbatim, which is the RUN output of the unmodifiedGlm5NextTextIndexerat v5.16.1 and carries the intermediates as well as theresult. So the device arm answers to the oracle directly and to the host arm as
well.
np = ceil(21/4) = 6againstkNumPools = 5makes thekeepcompactionobservable; four short cases exercise
P == 0.Selection error is BIMODAL, so the score bound is TIED TO THE MARGIN rather than
chosen: golden score magnitudes here run to 45.17, where one f32 ULP is already
3.8e-6, so an absolute tolerance below that fails on arithmetic and one far above
it bounds nothing. The gate asserts
score_delta * 4 < marginand prints both,alongside SET equality of the selected token indices and the positionwise
comparison. Every float comparison is
isfinite-guarded on both operands first,because an all-NaN forward on this row once read as a perfect match and the model
then emitted token id 0 eight times.
What is measured, and what is still PENDING
CPU, x86_64, on the merged head:
test_glm5_next_kpool_device4 cases / 4assertions with the three device cases explicitly SKIPPED and saying so;
test_glm5_next_dsa10 / 1934;test_ops_dsa_indexer10 / 176;test_op_provider14 / 494;test_glm5_next_moe13 / 9619;test_glm5_next_bridge20 / 32562;test_glm5_next_layer10 / 1656;test_glm5_next_forward23 / 167;test_glm5_next_attn14 / 160. Siblinginertness, since
include/vt/ops.h,src/vt/ops.cppandsrc/vt/op_provider.cppare shared:
test_deepseek_v4_dsa13 / 38,test_qwen4_exp_qsa_device12 / 4697,test_dots3_note_attn51 / 6888,test_kimi_kda14 / 36, andtest_glm_moe_dsa_forward7 / 5258 under its CTestVT_MOE_EXPERT_STREAM=1environment. One mutation was killed on the host: adding a CPU provider reds both
CHECK_FALSE(OpRegistered(..., kCPU))and theCHECK_THROWS, restoredbyte-for-byte by sha256 and rebuilt green.
THE DEVICE GATE IS PENDING AND IS NOT REPORTED AS A PASS. It is queued on
dgx:gpu0(sm_121a) as job6954476b-1d86-42c1-b4b0-12afc911efbc, buildingwith
-DVLLM_CPP_FLASH_ATTN=ONplus CUTLASS asserted FATAL at configure time,and carrying a nine-mutation matrix that rebuilds and asserts the binary's sha256
moved before believing any post-mutation result.
sm_110andsm_121aaredifferent targets and a number from one does not transfer to the other. An
untaken device gate is PENDING, never a pass, and this PR does not merge before
it returns.
No speed claim, and no token claim. Nothing here was measured for
throughput, there is no denominator, and no generation from this artifact was
observed by this branch.
Nothing lands dead: this lands UNREACHED, and here is what owns the wiring
ModelRegistry::Forwardstill reaches the HOST indexer.glm5_next_forward.cpp:231-238still refuses a non-CPU queue by name. The onlycallers of either op are the device gate and the availability probe. This is the
shape
.agents/reachability.mdcalls "the test-only driver", and it is stageddeliberately rather than disclosed after the fact.
DeviceType::kCUDAregistration ofkGlm5NextKpoolCompressandkGlm5NextKpoolSelectas reached FROM THIS MODEL,and
KpoolDeviceOpsAvailable(), which no production predicate consults.MODEL-MM-glm5-next-glm5-next-for-conditional-generation, wave W9c-3 — thecompose that constructs a CUDA queue for this forward, produces the packed
indexer row on a device, and stops de-paging
k_passinto host memory atglm5_next_kv.cpp:450-456.The row's spec lists it under
## Owedas O36. There is no production callsite to delete, so the reachability question is already answered; the mutation
run in its place deletes the
RegisterOpline and is EXPECTED to survive as aSKIP, which is the finding and not a kill — and is exactly why the device job
reads the skipped-case count instead of the exit code.
Out of scope and untouched: W9c-1 (the MLA route), W9c-2 (the two CPU-only
refusals at
glm5_next_kda.cpp:322-325andglm5_next_moe.cpp:222-225), W9b(residency).
Closes #2415
FOLLOWING_AGENTS_PROTOCOL
Refs: #2410
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]