Nightly 2026-07-24: deps fresh · burn.toml kernel validation · chat-template byte gate (caught + fixed a real divergence) · pre-existing GPU test failure fixed

[physics515]

Nightly 2026-07-24 — deps fresh · burn.toml kernel validation · the chat-template byte gate is real (and it caught a divergence) · a pre-existing GPU test failure fixed

Four increments + research folded, each verified green before landing.

1 · Dependency freshness (092c8db)

cargo update: 10 patch-level bumps (clap 4.6.4, glob 0.3.4, libc 0.2.189, rustls-pki-types 1.15.1, syn 3.0.3, tokio 1.53.1, tokio-util 0.7.19, xxhash-rust 0.8.18, zerocopy 0.8.55 ×2). cargo upgrade --incompatible offered only wgpu 29→30 — held per the intentional pin (burn 0.21 resolves wgpu 29 transitively; it unblocks with a burn bump, not a cargo upgrade). tokenizers stays current at 0.23.1; no Burn 0.22 exists yet.

2 · burn.toml — CubeCL kernel-OOB validation armed for the correctness suites (e9bb46d, perf-section item ticked)

Read the discovery logic out of cubecl-runtime 0.10's source (RuntimeConfig::from_current_dir): it walks up from the process CWD, checking cubecl.toml before burn.toml at each level, first hit wins — and cargo runs every crate's tests/benches with CWD = the package dir. Hence two files:

• burn.toml (repo root): [cubecl.compilation] check_mode = "validate" — bounds-check every kernel launch AND validate explicitly-unchecked kernels for OOB (the failure class behind silent wrong-logits drift). Arms all crates/mummu parity/real-model suites.
• crates/mummu-bench/cubecl.toml: check_mode = "auto" — the budget/bench crate finds this first, so recorded perf numbers never include validation overhead.

Consumers run from their own CWDs — untouched. Proofs: a poisoned root burn.toml fails a GPU test at config load naming the exact bad key (unknown variant 'not-a-real-mode', expected enforce/validate/auto); a poisoned bench cubecl.toml does the same from the bench crate (so the opt-out file is genuinely the one discovered there); the whole real-model battery below ran with validation armed — no OOB surfaced; and the budget gate is unmoved (12.4 tok/s with the files vs 12.1 in a no-file A/B).

3 · Pre-existing GPU failure found, root-caused, fixed (7c10026)

real_qwen3_gguf_loads_and_agrees_with_safetensors fails on unmodified origin/main (serial AND parallel — verified from the untouched main checkout, which exonerated this run's changes): instantiating GpuF16 flips Burn's per-device default dtype policy process-wide, so the f32 GGUF cross-check running after the f16 leg in the same test binary panics reading its logits back (TypeMismatch: expected F16, got F32). Fix = the repo's own convention: every GpuF16 leg lives in real_f16.rs as its own process; qwen3's f16 test moved there. After the move: real_qwen3 2/2 with the recorded numbers back (GGUF-vs-bf16 cosine 0.98910, top-1 151667 both builds), real_f16 2/2. The library-level hazard (a consumer switching precision at runtime in one process) is now a P6 roadmap item.

4 · The template byte gate — renders now byte-identical to transformers.apply_chat_template (e9b56b8, P3 item ticked)

The "template-vs-renderer consistency" marker check is now a true byte gate, and building it caught a real divergence:

The bug it found (fixed): our renders embedded tool JSON compact ({"a":1}) where every checkpoint template runs Jinja tojson = Python json.dumps separators ({"a": 1}) — and the models emit that spacing back (the 07-11 real-GPU proof literally logged {"name": "get_weather", "arguments": {"city": "Paris"}}). New public chat::py_json (a json.dumps-separator serde Formatter) feeds all template-embedded JSON in both conventions.

The gate: tools/template-probe (out-of-workspace bin, the candle-probe pattern) renders a template via hf-chat-template 0.2.1 (minijinja + transformers compat layer). Deliberately NOT a dev-dependency: hf-chat-template enables serde_json's preserve_order, which — features being additive — would silently flip our own test builds from sorted to insertion-ordered maps, diverging the very renders the tests verify from what plain-feature consumers build.

tests/template_gate.rs (ignored, env-keyed) pulls each checkpoint's OWN template through the real import path (TokenizerConfig::from_dir, incl. LFM2.5's standalone chat_template.jinja — fetched into the local cache this run, exercising the 07-20 fallback on a real checkpoint) and byte-compares. 9/9 pass: Qwen2.5 plain/tools/tool-call-history · Qwen3 plain/tools · LFM2.5 plain±system/tools±system/think-stripping/pythonic-call+tool-turns — all byte-identical. The only family divergences are pinned to their exact deltas so any other drift fails loudly: Qwen2.5's no-system branding preamble ("You are Qwen, …" vs our neutral one), Qwen3's no-system no-preamble, Qwen3's history think-stripping (→ new [ ]: ChatMl::qwen3() with the strip).

Verification battery (RTX 4070 Ti SUPER, quiet card; real-model suites ran with kernel validation armed)

Gate	Result
cargo fmt / clippy --all-targets	clean, 0 warnings
Unit + load-gate suites	177 + 6 green
Template byte gate	9/9 byte-identical
real_qwen3 (safetensors + GGUF cross-check)	2/2 — cosine 0.98910, top-1 151667 both builds
real_f16 (qwen2 + qwen3 f16 legs)	2/2 — coherent, no NaN
real_toolcall ×3 (Qwen2.5 / LFM2.5 / Qwen3, new spaced prompts)	3/3 — each emitted a clean parseable call, in exactly the spaced spelling
real_inference	4/4
Parity (Qwen2.5 vs Candle + Ollama fp16 greedy)	bit-identical — max |Δlogit| 2.670288e-5, greedy leg exact
GPU budget gate (bench crate opted out of validation)	TTFT 115 ms (≤150) · 12.4 / 12.4 tok/s across two runs (≥10) — matches the 12.1 no-config A/B, budgets unmoved
CPU budget gate	13.28 tok/s (≥6; baseline 11.7)

Research folded

• KV-cache FP8/e4m3 quantization → new P9 [ ] with vLLM's shipped numbers + transferable lessons (two-level f32 accumulation, layer-selective beats uniform, head_dim sensitivity). https://vllm.ai/blog/2026-04-22-fp8-kvcache
• burn.toml semantics source-verified (cubecl-runtime 0.10) and folded into the ticked perf item.
• Pin watch unchanged: wgpu 30 still held (burn-bump-gated); tokenizers 0.23.1 current; no Burn 0.22 yet.
• New [ ]s from this run's findings: ChatMl::qwen3() think-stripping; general template-fallback renderer (payoff 2, preserve_order caveat documented); the P6 one-process Gpu/GpuF16 dtype-policy hazard.

What's next

• SentencePiece tokenizer.model import (needs a fixture fetch).
• Config-driven EOS/BOS surfacing (TokenizerConfig on the Loaded* structs).
• Qwen3.5-4B/9B FC catalog run (download + the proven FC decode).
• P9 keep-quantized VRAM (now with the KV-cache FP8 lever researched).

🤖 Generated with Claude Code

[physics515]

Merged origin/main (PR #12 — the parallel 2026-07-22 nightly) into this branch and reconciled the duplicate work; see the merge commit message for the full resolution. Highlights: main's in-process gate + workspace preserve_order win over this branch's out-of-workspace probe (which is dropped); this branch's extra gate coverage is ported (now 10 byte-gate cases across Qwen2.5/Qwen3/LFM2.5, all passing); burn.toml kernel validation, the P6 heterogeneous-precision design, and the P9 KV-cache FP8 item are kept. Merged tree re-verified end to end: 176 unit tests, gate 10/10, real-model GPU suites green with validation armed, budgets hold (CPU 13.56 tok/s, GPU 12.3–13.4 tok/s, TTFT ~104 ms).