Nightly 2026-07-22/23: template byte gate (prompts byte-identical to transformers), dtype-pollution fix, config-driven EOS/BOS salvage, dep refresh

[physics515]

Nightly run report for 2026-07-22/23 (the run crossed midnight). Four increments on top of the salvaged 2026-07-21 dep bump, all gates green.

What shipped

1. Salvaged the crashed 2026-07-21 run's work — config-driven EOS/BOS (feefac8).
The previous nightly crashed after finishing (but not committing) its increment. Re-verified from scratch, then committed: all three safetensors loaders surface the parsed tokenizer_config.json as a public tokenizer_config: Option<TokenizerConfig> field on LoadedQwen2/LoadedQwen3/LoadedLfm2, so consumers read config-driven EOS/BOS/PAD ids off the loaded model. Additive — is_eos still rides config.json's cross-checked eos_token_id, decode/parity byte-unchanged, debug-asserted EOS agreement, GGUF loads surface None. REAL-GPU: safetensors leg asserts surfaced eos_id() == 151645; GGUF leg asserts None.

2. Root-caused and fixed a live dtype-pollution flake (cbe4846).
While verifying #1, the f32 GGUF leg died TypeMismatch("expected F16, got F32") whenever the f16 test ran first in the same process — A/B-confirmed pre-existing at f1e547a, not the salvage. Cause: Burn 0.21 resolves unspecified-dtype tensor creation against a per-DEVICE settings policy, so GpuF16 and Gpu clients sharing the device flip each other's ambient float dtype; the loaders' zeros-probed target_float inherited it, silently loading the "f32" model in f16. Fix: every loader derives target_float from the TYPE (<B::FloatElem as Element>::dtype()), and all GpuF16 legs moved to real_f16.rs (own process). The remaining planner-level policy work is a new P6 roadmap item. real_qwen3 is now 2/2 green sequential AND parallel.

3. Template byte gate — prompts now byte-identical to transformers (ed310f0).
Landed the P3 hf-chat-template evaluation as tests/template_gate.rs: renders the checkpoint's own imported chat_template through hf-chat-template 0.2.1 and byte-compares with ChatMl. All four legs byte-identical on Qwen3-0.6B: plain 142 B, multi-turn 201 B, full Hermes tools block 748 B, FC history 324 B. The gate caught two real divergences, both fixed:

• serde_json/preserve_order is now a first-class workspace feature — insertion-order keys are what Python/transformers renders; without it the dev-dep would have silently flipped TEST builds to insertion order while production builds stayed alphabetical (tests verifying different prompt bytes than consumers ship).
• Prompt JSON now serializes with Python json.dumps separators via a small custom Formatter (chat::python_json) — the exact spacing the reference stack renders and models emit back.

Live FC re-proof with the changed prompt bytes on ALL THREE families (4070 Ti SUPER): Qwen3-0.6B and Qwen2.5-1.5B emitted clean parseable Hermes calls; LFM2.5-1.2B emitted exactly <|tool_call_start|>[get_weather(city="Paris")]<|tool_call_end|>.

4. Dependency freshness (9f64ed7).
cargo update: clap 4.6.4, glob 0.3.4, libc 0.2.189, syn 3.0.3, tokio-util 0.7.19. cargo upgrade --incompatible offered exactly one major — the direct wgpu probe dep 29→30 — reverted per the documented pin (burn 0.21 resolves wgpu 29 transitively; two wgpu majors in one binary = the probe inspecting different state than the runtime; unblocks with a burn bump). Upstream check: burn 0.21 and tokenizers 0.23.1 are still current.

Gates (all green)

Gate	Result
fmt / clippy --all-targets / build	clean, zero warnings, every increment
Unit + integration tests	176 lib + all integration suites green
Qwen3 strict parity vs llama.cpp (same Q4_K_M)	top-5 ids exact in order, max |Δlogprob| 4.0156e-1 — unchanged, re-run after every code increment
Qwen2 parity, both legs	max |Δlogit| vs Candle 2.670288e-5 (unchanged); Ollama fp16 greedy byte-identical
Template byte gate (new)	4/4 legs byte-identical to transformers.apply_chat_template
Real FC decodes (3 families)	Qwen3-0.6B, Qwen2.5-1.5B, LFM2.5-1.2B all emit parseable calls from the new prompt bytes
Perf budget GPU (budget.rs)	TTFT 126.9 ms (≤ 150) · decode 13.2 tok/s (≥ 10) on a quiet card
Perf budget CPU (budget_cpu.rs)	10.44 tok/s (≥ 6)

Operational note: the first GPU budget attempt read 246 ms / 5.0 tok/s because another agent's qwen3.5:9b (6.7 GB) was resident in Ollama on the card — re-ran clean after it unloaded. Corroborates the ROADMAP "quiet machine" warning; check ollama ps before trusting a budget number.

Fixtures fetched this run

Qwen2.5-1.5B-Instruct (2.9 GB), Qwen2.5-0.5B-Instruct (1.0 GB), LFM2.5-1.2B-Instruct (2.2 GB) — the local cache now covers both parity_qwen2 legs, all three FC suites, and both budget gates offline.

Research folded into ROADMAP

• Pin watch re-checked: burn still 0.21, tokenizers still 0.23.1. wgpu 30's changelog lifts SHADER_F16 to ALL shader kinds (WGSL included) + the Vulkan f16 IO polyfill — when burn bumps to wgpu 30, f16 stops being SPIR-V-only.
• Qwen3.5-9B/4B remain the ≤8 GB FC picks per mid-2026 field guides; Qwen3.6 (27B dense / 35B-A3B) ships a new qwen3_coder tool-call parser — check template markers before assuming Hermes covers a 3.6-class port.
• hf-chat-template 0.2.1 current; adopted (dev-dep) with a new [ ] split for a general fallback renderer.
• New P6 item: per-device default-dtype policy for in-process mixed precision (today's flake, evidence + mitigation plan recorded).

What's next

• P2: Qwen3.5-4B/9B FC catalog run (download + decode; arch already parity-verified).
• P3: SentencePiece tokenizer.model import (route documented; needs a fixture fetch).
• Perf: burn.toml check_mode="validate" bench A/B (weights for it are now cached).
• P6: the mixed-precision dtype policy item this run opened.

🤖 Generated with Claude Code