Nightly 2026-07-09: P0 scaffold → P1 backends → P2 model zoo (Qwen2 + LFM2.5 + MiniLM) running on real weights - #1
Merged
Conversation
…n stub Workspace with the parity-pinned combo (burn 0.21 wgpu/ndarray/fusion/autotune/store, wgpu 29, tokenizers 0.22), criterion 0.7 harness wired and verified green (fmt / clippy --all-targets / test / build / bench all pass). Cargo.lock committed for reproducible builds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…uF16/Cpu aliases One binary compiles both Wgpu (Fusion, via fusion+autotune) and NdArray. A cheap adapter-enumeration probe (no device creation), cached in a OnceCell, picks GPU else CPU; the inventory records per-adapter/per-API SHADER_F16 for the P6 planner. Verified on hardware: RTX 4070 Ti SUPER (Vulkan f16=true, DX12 f16=false) + an integrated AMD adapter. 5 unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…LIV short-conv Extracted from laurelane's parity-validated ports, generic over B: Backend. GqaAttention covers both proven variants (Qwen2 bias / LFM2 per-head q-k RMSNorm) with a per-layer KV cache; ShortConv is LFM2's double-gated causal conv with the rolling K-1 decode state. 18 unit tests, including prefill+decode == full-forward equivalence for both cache kinds, strict causality, and RoPE offset consistency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…decode utils, disk accounting Two decoders on the shared nn blocks, config-driven and generic over B: Backend: - models::qwen2 — verified on real weights: Qwen2.5-1.5B (3.1 GB bf16 safetensors) loads with zero missing keys and greedy-decodes '2+2 equals 4.' on the wgpu GPU (top-5 probe led by id 9707 'Hello'); toy cache-equivalence test. - models::lfm2 — the hybrid conv+attention 1.2B; real GPU decode produced a correct primes list; greedy parity test vs Ollama written but the local lfm2.5 tag now resolves to the 8.5B MoE (different weights), so the P7 gate stays open. - import — CastFloatAdapter (bf16→backend float) + load_checked (fail-loud, never silently zero-init) + typed ImportError. - decode — on-device argmax + top-k probe. manage — P8 disk usage + traversal-safe removal (ported with tests). 42 unit tests green; clippy clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cargo update + criterion major bump (dev-only; bench harness re-verified green). tokenizers 0.23 and wgpu 30 exist but stay pinned — the burn 0.21 / wgpu 29 / tokenizers 0.22 combination is the parity-validated set (see ROADMAP Stack). Thin LTO cuts the release verify-loop link from ~13 min to a fraction; inference is GPU-kernel-bound so fat LTO bought nothing measurable — revisit once bench/BASELINE.md exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ol, L2 normalize Third model in the zoo, generic over B: Backend; ids+mask in, unit-norm embedding out (tokenization stays caller-side, keeping the lib app-agnostic). Unit tests incl. padding-invisibility; real-weights semantic check on the actual checkpoint: paraphrase cosine 0.556 vs cross-topic -0.02/-0.00, unit norm holds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…o the test cwd) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR converts the repository from docs-only into an initial working Rust/Burn workspace that can import real safetensors weights, run inference on CPU/WGPU backends, and includes early model ports (Qwen2/2.5, LFM2/2.5 hybrid, MiniLM sentence embeddings) plus supporting shared blocks and smoke/parity tests.
Changes:
- Added a Cargo workspace with
mummu(library) andmummu-bench(Criterion harness), plus pinned dependency stack and thin-LTO release profile. - Implemented shared NN building blocks (RoPE, GQA+KV cache, SwiGLU, LFM2 short-conv with rolling state), model loaders with checked safetensors import + dtype casting, and basic decode primitives.
- Added ignored “real weights” smoke tests and an Ollama-based parity test harness scaffold; updated README/ROADMAP to reflect current status.
Reviewed changes
Copilot reviewed 24 out of 26 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| ROADMAP.md | Updates phase checkboxes and adds nightly run notes/links. |
| README.md | Adds “Status — what runs today” section describing current capabilities. |
| crates/mummu/tests/real_minilm.rs | Ignored real-weights MiniLM semantic smoke test (CPU). |
| crates/mummu/tests/real_inference.rs | Ignored real-weights Qwen2 greedy decode + top-5 probe smoke tests. |
| crates/mummu/tests/parity_lfm2.rs | Ignored parity scaffold comparing greedy output vs local Ollama. |
| crates/mummu/src/nn/rope.rs | RoPE table generation + apply/rotate utilities with unit tests. |
| crates/mummu/src/nn/mod.rs | Exposes shared NN modules + defines MAX_CONTEXT_TOKENS. |
| crates/mummu/src/nn/mlp.rs | SwiGLU MLP block (bias-free) with unit tests. |
| crates/mummu/src/nn/conv.rs | LFM2 short-conv (“LIV”) operator with rolling decode state + tests. |
| crates/mummu/src/nn/attention.rs | Cache-aware GQA attention with causal mask, RoPE, KV cache tests. |
| crates/mummu/src/models/qwen2.rs | Qwen2/Qwen2.5 decoder, config parsing/validation, checked load, greedy decode. |
| crates/mummu/src/models/mod.rs | Exposes model zoo modules (lfm2/minilm/qwen2). |
| crates/mummu/src/models/minilm.rs | MiniLM/BERT embedder with checked load + embedding API + unit tests. |
| crates/mummu/src/models/lfm2.rs | LFM2 hybrid decoder (conv/attn by layer_types), checked load, greedy decode. |
| crates/mummu/src/manage.rs | Model cache disk accounting + traversal-safe removal resolution + tests. |
| crates/mummu/src/lib.rs | Library entrypoint + recursion_limit for Burn fusion generics. |
| crates/mummu/src/import.rs | Shared import utilities: bf16→target float adapter + fail-loud load wrapper. |
| crates/mummu/src/decode.rs | On-device argmax + top-k probe utilities + tests. |
| crates/mummu/src/backend.rs | Runtime GPU inventory/probe and backend type aliases. |
| crates/mummu/Cargo.toml | Defines mummu crate dependencies via workspace pins. |
| crates/mummu-bench/src/lib.rs | Empty bench support crate (benches live under benches/). |
| crates/mummu-bench/Cargo.toml | Criterion bench crate wiring (runner bench, harness=false). |
| crates/mummu-bench/benches/runner.rs | Criterion smoke benchmark to verify harness wiring. |
| Cargo.toml | Workspace definition, pinned deps, and thin-LTO release profile. |
| .gitignore | Adjusts target ignore pattern. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+75
to
+79
| // Reference first: skip cleanly (not red) when Ollama isn't up. | ||
| let reference = match ollama_greedy(&raw, MAX_TOKENS) { | ||
| Ok(r) => r, | ||
| Err(e) => panic!("Ollama reference unavailable: {e}"), | ||
| }; |
Comment on lines
+261
to
+265
| .clone() | ||
| .reshape([1, 1, 1, n]) | ||
| .neg() | ||
| .add_scalar(1.0) | ||
| .mul_scalar(-1e30); |
Comment on lines
+77
to
+92
| fn validate(&self) -> Result<(), String> { | ||
| if self.num_key_value_heads == 0 | ||
| || !self | ||
| .num_attention_heads | ||
| .is_multiple_of(self.num_key_value_heads) | ||
| { | ||
| return Err(format!( | ||
| "num_attention_heads ({}) must be a positive multiple of num_key_value_heads ({})", | ||
| self.num_attention_heads, self.num_key_value_heads | ||
| )); | ||
| } | ||
| if self.num_hidden_layers == 0 || self.vocab_size == 0 { | ||
| return Err("num_hidden_layers and vocab_size must be positive".into()); | ||
| } | ||
| Ok(()) | ||
| } |
| - [ ] `cargo build` / `test` / `clippy --all-targets` green baseline; `mummu-bench` (criterion) crate stub. | ||
| - [x] Cargo workspace: `crates/mummu` (the library), model code generic over `B: Backend`. `.gitignore` | ||
| (Rust); commit `Cargo.lock` for reproducible builds/benchmarks. *(2026-07-09) Workspace + both | ||
| crates; pinned combo burn 0.21 / wgpu 29 / tokenizers 0.22 / criterion 0.7; release profile fat-LTO.* |
Comment on lines
+82
to
+106
| fn validate(&self) -> Result<(), String> { | ||
| if self.layer_types.len() != self.num_hidden_layers { | ||
| return Err(format!( | ||
| "layer_types has {} entries but num_hidden_layers is {}", | ||
| self.layer_types.len(), | ||
| self.num_hidden_layers | ||
| )); | ||
| } | ||
| if self.num_key_value_heads == 0 | ||
| || !self | ||
| .num_attention_heads | ||
| .is_multiple_of(self.num_key_value_heads) | ||
| { | ||
| return Err(format!( | ||
| "num_attention_heads ({}) must be a positive multiple of num_key_value_heads ({})", | ||
| self.num_attention_heads, self.num_key_value_heads | ||
| )); | ||
| } | ||
| if self.conv_l_cache < 2 { | ||
| return Err(format!( | ||
| "conv_L_cache must be >= 2, got {}", | ||
| self.conv_l_cache | ||
| )); | ||
| } | ||
| Ok(()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nightly run report — 2026-07-09
First implementation run: the repo went from docs-only to a working library with three models running on real weights. 8 commits, each independently green (
cargo fmt/clippy --all-targets(0 warnings) /test/build).Dependency freshness
cargo updateclean; tokenizers 0.23 and wgpu 30 exist but stay pinned per the roadmap's pin rationale.Shipped (roadmap items ticked)
crates/mummu+crates/mummu-bench), Cargo.lock committed, criterion harness wired and run.Fusion<Wgpu>+NdArray; cached runtime GPU probe; device inventory with per-adapter/per-APISHADER_F16(dev box: 4070 Ti SUPER f16=true on Vulkan, false on DX12, + an integrated AMD adapter — a real second device for P6 multi-GPU).GqaAttention(one block covers Qwen2-bias and LFM2 per-head-q/k-RMSNorm variants), manual RoPE, SwiGLU, LFM2's double-gated causalShortConvwith rolling decode state.CastFloatAdapterbf16→backend float,load_checkedfail-loud, typedImportError), per-arch key-remap tables, config-driven models (serde configs w/ validation).manage: per-model disk usage + traversal-safe removal validation.Model ports (stay
[ ]pending the P7 parity gate, per the parity rule)models::qwen2), LFM2/2.5 hybrid (models::lfm2), all-MiniLM (models::minilm).Tests + real-inference verification
kv_cache_decode_matches_full_forward,rolling_state_decode_matches_full_prefill, whole-model equivalences for both decoders, causality, RoPE offset-consistency, padding-invisibility, import/manage/backend suites) — all green.tests/parity_lfm2.rs, Ollama raw-mode temperature-0). It failed by design: localollama lfm2.5:latestnow resolves to the 8.5B MoE Q4 w/ thinking — different weights, invalid reference (verified viaollama show; no 1.2B tag exists). New P7 item: stand up a Candle logits reference + pullqwen2.5:1.5b-instruct-fp16for the greedy leg. Ports stay unticked until the gate passes.Benchmarks
bench/BASELINE.mdland with P5's engine (next run's top perf item). No budgets existed, none regressed.Research folded into ROADMAP (with links)
What's next (top of the roadmap)
mummu-benchreal benches +bench/BASELINE.mdbudgets.Modeltrait; P3 downloads (resumable, shard-aware) + manifest registry.Run wall-clock ≈ 4 h. Worktree used:
../mummu-nightly; main checkout untouched; not merging this PR per the routine's rules.🤖 Generated with Claude Code