feat(vulkan): moe_indexed matmul for IQ4_NL and Q5_0 routed expert banks (#407)

[jamesburton]

Closes #407.

Sub-issue of #344; hard blocker for #402.

The gap

moe_indexed_matmul_* existed for f32, q4_k, q5_1, q5_k, q6_k, q8_0 — not for IQ4_NL or Q5_0. Both types already had complete-or-partial dense Vulkan paths, but a routed expert bank of either type hit MoeRoutedRawDeviceQuantType's F32 fallback and was host-dequantised on upload: 7.1x for IQ4_NL, 5.8x for Q5_0.

Nemotron-3.5-Lightning's published GGUFs are ~91% IQ4_NL / ~76% Q5_0 by tensor count, with ~93% of parameters in routed experts — so essentially every expert bank expanded, turning an 18.9 GB model into ~100 GB of host allocation. That is a load failure, not a slow run.

What landed

Shaders (4-binding q8_0/q6_k family — no per-expert scale binding; the 5-binding q5_1 variant is Gemma-4-specific):

• moe_indexed_matmul_q5_0_f32.comp — symmetric d*(q-16), 22-byte block, 5th bit of each weight from the separate 32-bit qh field.
• moe_indexed_matmul_iq4_nl_f32.comp — 18-byte block, nibbles index ggml's non-linear kvalues_iq4nl codebook.

Unpack logic is lifted from the in-tree, already-verified dense paths rather than re-derived. Both block strides are odd multiples of 2, so every read goes through the straddle-safe readByte/readHalf/readWord/read4Bytes helpers. .spv blobs committed.

Wiring — without it the kernels are unreachable, which #344 calls out explicitly:

• MoeOverlayKeepsQ5_0 / MoeOverlayKeepsIq4Nl (K % 32 == 0), consumed by MoeRoutedRawDeviceQuantType.
  Deliberately not added to MoeOverlayKeepsQuantized — that predicate governs the router-gate / shared-expert overlay path, which has no matmul kernel for either type.
• Routed per-expert byte sizing split out into RoutedBankUploadBytes, driven by the already-resolved type. The shared MoeOverlayUploadBytes is left alone: adding the two types there alone would have sized a buffer for packed bytes that the gate path then fills with F32. This was a real hazard caught mid-implementation, not a hypothetical.
• Kernel creation, RecordMoeIndexedMatmul branches, descriptor-cache invalidation and disposal in VulkanTransformerModel.

Verification — #344's standard

1. Real GGUF bytes are the primary oracle. RealGgufMoeIndexedRoutedBankParityTests reads real llama.cpp-quantised tensors and assembles expert banks from real contiguous tensor rows (GGUF stores [K, M] as M packed rows, so the first E*Me rows ARE a valid E-expert bank byte-for-byte). No self-authored blocks anywhere in the oracle path. Reference is the CPU scalar dequant (DotLLM.Cpu.Kernels.Dequantize) — the only valid cross-backend reference for a packed layout; CUDA cannot serve as one (#330).

Positive results, worst relative error across 6 tensors each:

type	worst rel
Q5_0	5.09e-4
IQ4_NL	3.72e-3

2. Fixture asserts the type is present. Both tests Assert.Fail with the full type census if the file carries no usable tensor of the type — the SmolLM-135M i1-Q3_K_M lesson.

3. Negative controls — demonstrated, both directions.

Automated and permanent: ..._Discriminates_... siblings recompute the reference with the exact bug class under suspicion — qh bit 2j instead of j for Q5_0, swapped nibble planes for IQ4_NL — and assert the kernel does not match. Observed divergence: Q5_0 worst rel 2.97e+3, IQ4_NL worst rel 7.75e+1, against a 2e-2 bar.

Demonstrated against deliberately broken SPIR-V: each shader was recompiled with the transposed mapping and installed over the real .spv; the positive parity tests went red on 48/48 cells for both types (Q5_0 worst rel 1.83e+1, IQ4_NL worst rel 1.58e+2). Restored and re-verified green afterwards.

4. Routing proven by observation, not capability flags. Three levels:

• ResolveMoeBankResidency_KeepsQ5_0AndIq4NlRoutedBanksResident — the same predicate the upload uses; also asserts PlanMoeF32HostDequant now reports CanSkip with zero host F32 bytes.
• Reflected W1/W2/W3DeviceQuantType off the loaded model.
• End-to-end CPU-vs-Vulkan forward parity: Forward_RoutedKQuant_MatchesCpuReference gained Q5_0 and IQ4_NL cases (CPU decodes via MoeQuantSwiGluMlp's GemvDequantRows fallback, i.e. the trusted scalar oracle; Vulkan via the new kernels). Q5_0 fixture bytes come from the production encoder DotLLM.Cpu.Kernels.Quantize, not a test-local one.

5. Full builds only — no --no-build anywhere (#341).

Test results

suite	result
CPU-side unit (Category!=GPU) — the #364 item 7 gate	2683 passed, 0 failed
Unit, ~Moe filter (this change's blast radius)	342 passed, 12 failed
Integration, ~Vulkan	22 passed, 6 failed
New parity + negative controls	4/4 pass
New/updated MoE routed tests	18/18 pass

The full unit ~Vulkan filter was attempted and killed at the harness's 60-minute cap before producing results. Coverage was substituted with the ~Moe filter (this diff's entire reachable surface — VulkanWeights routed-bank resolution and VulkanTransformerModel MoE dispatch), the complete integration ~Vulkan suite, and the full CPU-side suite. Note also that a literally green Vulkan suite is unattainable on this box regardless of this change: origin/dev itself carries the pre-existing failures listed below.

Every failure is pre-existing and was baselined on origin/dev in this session, not merely asserted:

• Unit: the same failure classes on origin/dev produce 14 failures vs 12 on this branch. The counts differ because the baseline ran those five classes in isolation while the branch numbers come from the broader ~Moe run, and the individual InlineData instances that failed differed between runs — which is itself evidence these are flaky, wave-size-dependent failures rather than deterministic ones. Classes: (VulkanMoeIndexedMatmulQ4K/Q5_1/Q8_0MmvqKernelTests, VulkanMoeGroupByExpertF32KernelTests, CudaMoeFfnBitNetI2SBatchedGemmTests).
• Integration: origin/dev produces exactly the same 6 (BitNetVulkanAccuracyTests x3, VulkanCrossBackendTimingDemoTests, RealGgufVulkanParityTests.Llama32_1B_Q8_0_..., VulkanMmvqSharedQuantParityTests).

All are subgroup/coopmat-dependent kernels or CUDA, and none are touched by this diff. Values like 7.76e+23 in the MMVQ failures are wave-size garbage — the local Arc iGPU is not the wave32 target those shaders assume.

Deviations from the issue's scope — stated explicitly

• MMVQ not added, despite the issue's checkbox. The only MoE MMVQ dispatch in the tree lives inside the Gemma-4 forward (gated on config.Gemma4DualFfn, consumes g4.DownExpertScale); the generic routed path has no MMVQ dispatch at all — even Q8_0's MoE MMVQ kernel is unreachable there. Landing Q5_0/IQ4_NL MMVQ kernels without that infrastructure would be precisely the unreachable-kernel failure this issue exists to fix. Building the generic-path dispatch is separate work.
• MMQ not added — measurement-gated per Vulkan expands 5 quantization types to F32 (5.3x-7.5x) for want of kernels — umbrella #344, and Checkpoint-gated test classes hard-FAIL with IOException on partial/locked checkpoint files instead of skipping #384-ContinuousBatchScheduler disposes per-sequence CUDA state caches directly without context binding — silent VRAM leak on preemption path #391 found all four tiling variants regressed on this hardware. No A/B showed a gain, so none was added.
• Not verified on a real nemotron_h_moe GGUF. No such fixture is available locally and feat(vulkan): nemotron_h_moe MoE forward — sigmoid/bias router, ungated relu2 experts, shared expert #402's forward pass does not exist yet. Zero-F32-expansion is verified on synthetic MoE fixtures through the same predicate the upload path uses, plus real-GGUF block-level parity — equivalent evidence at fixture scale, but it is a proxy and is named as one.
• The full unit ~Vulkan suite did not complete — killed at the 60-minute harness cap, as described above. Substituted with the ~Moe filter plus the full integration ~Vulkan and CPU-side suites.
• VulkanQwen3MoeMoeUpload.s_ResidentQuantTypes deliberately untouched — a different architecture path (Qwen3.6 hybrid), not on Nemotron's route, and extending it needs its own kernel wiring and evidence.

Fixtures used

~/.dotllm/quant-ladder/SmolLM2-135M/SmolLM2-135M-pure-Q5_0.gguf and ...-pure-IQ4_NL.gguf — real llama.cpp quantizations. The resolver accepts DOTLLM_SMOLLM135M_Q5_0_GGUF / DOTLLM_SMOLLM135M_IQ4_NL_GGUF (and DOTLLM_QUANT_FIXTURE_* aliases) and probes the quant-ladder directory, since the conventional QuantFactory paths do not exist locally.

Side effect worth noting

DeepSeek-V2-Lite Q4_K_M — the #326/#327 worked example — had 14 of its 26 ffn_down_exps banks stored as Q5_0 at K=1408, costing ~9.6 GiB of host F32. 1408 is 32-aligned, so those banks are now device-resident too. MoeF32HostDequantPreflightTests builds its plans by hand rather than resolving them from a GGUF, so it still tests the accounting and refusal message unchanged; a note records why the premise in its remarks is now historical.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DzekWWxE4d52Hpa31WBYfX

[Copilot]

Pull request overview

Adds Vulkan support for routed MoE expert-bank matmul (moe_indexed) for Q5_0 and IQ4_NL, closing the gap where these routed banks would otherwise fall back to host F32 dequantization (large memory expansion / potential OOM). This fits into the Vulkan backend’s quant-residency work by making the new shader kernels reachable via resolver wiring and by adding real-GGUF parity coverage.

Changes:

• Add new Vulkan moe_indexed_matmul_* compute shaders and corresponding C# kernel wrappers for Q5_0 and IQ4_NL routed expert banks.
• Wire the new routed-bank quant types into VulkanWeights.MoeRoutedRawDeviceQuantType, and split routed-bank byte sizing into RoutedBankUploadBytes to avoid overlay-path size hazards.
• Add/extend unit + integration tests, including real-GGUF-bytes parity tests with negative controls to ensure the assertions discriminate against suspected layout bugs.

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/DotLLM.Tests.Unit/Vulkan/VulkanTransformerModelMoeKQuantRoutedForwardTests.cs	Extends routed MoE forward parity and residency/preflight tests to cover Q5_0 and IQ4_NL reachability and updated fallback scenarios.
tests/DotLLM.Tests.Unit/Vulkan/MoeF32HostDequantPreflightTests.cs	Updates documentation to reflect that Q5_0/IQ4_NL routed banks are now expected to be fully resident post-#407 (historical note).
tests/DotLLM.Tests.Integration/Vulkan/RealGgufMoeIndexedRoutedBankParityTests.cs	New real-GGUF parity + negative-control suite for the new moe_indexed kernels (Q5_0 and IQ4_NL).
src/DotLLM.Vulkan/VulkanWeights.cs	Adds routed-bank resolver support for Q5_0/IQ4_NL and introduces RoutedBankUploadBytes to size routed banks based on resolved type.
src/DotLLM.Vulkan/VulkanTransformerModel.cs	Creates/dispatches/disposes the new kernels and adds descriptor-cache invalidation plumbing for them.
src/DotLLM.Vulkan/Kernels/MoeIndexedMatmulQ5_0F32Kernel.cs	New Vulkan kernel wrapper for Q5_0 routed expert-bank indexed matmul (SPIR-V-backed).
src/DotLLM.Vulkan/Kernels/MoeIndexedMatmulIq4NlF32Kernel.cs	New Vulkan kernel wrapper for IQ4_NL routed expert-bank indexed matmul (SPIR-V-backed).
native/vulkan/shaders/moe_indexed_matmul_q5_0_f32.comp	New compute shader implementing indexed matmul for packed Q5_0 expert banks.
native/vulkan/shaders/moe_indexed_matmul_iq4_nl_f32.comp	New compute shader implementing indexed matmul for packed IQ4_NL expert banks.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.