MTP: separate sidecar draft model, llama.cpp b10435 (Qwen 3.8)

[nyo16]

Why

Qwen 3.8 ships MTP as a sidecar pair. ggml-org/Qwen3.8-27B-GGUF publishes
Qwen3.8-27B-Q4_K_M.gguf with zero nextn layers, and the head alone in
mtp-Qwen3.8-27B-Q4_0.gguf:

file	n_layer_nextn	n_embd_out	size
Qwen3.8-27B-Q4_K_M.gguf	0	5120	17.7 GiB
mtp-Qwen3.8-27B-Q4_0.gguf	1	5120	1.55 GiB

MTP.init/2 built both contexts from the one model, so it refused that pair
outright with "this GGUF contains no MTP head". The NIF already accepted two
independent contexts, and upstream's draft-mtp impl only requires the hidden
widths to agree — the restriction was purely on the Elixir side.

No new architecture work was needed: Qwen 3.8 loads under the existing qwen35
arch, and llama-model.cpp already lists LLM_ARCH_QWEN35 in mtp_on_hybrid_qwen.

What changed

MTP.init/2 takes :draft_model — the equivalent of upstream's
-hf <target> -hfd <draft> --spec-type draft-mtp:

{:ok, target} = LlamaCppEx.load_model("Qwen3.8-27B-Q4_K_M.gguf", n_gpu_layers: 999, load_mtp: true)
{:ok, head}   = LlamaCppEx.load_model("mtp-Qwen3.8-27B-Q4_0.gguf", n_gpu_layers: 999, load_mtp: true)

{:ok, mtp} = LlamaCppEx.MTP.init(target, draft_model: head, n_draft: 1)

Mismatched pairings are refused before any context is built: a sidecar without
load_mtp: true, an ordinary model passed as :draft_model, and a target/draft
hidden-width mismatch. That last one matters — upstream compares the widths with a
GGML_ASSERT, an unconditional ggml_abort that would take the VM down rather than
return an error — so Model.n_embd_out/1 is exposed to check it first.

stats/1 gains timing_us.ckpt. A hybrid target cannot roll back part of a
sequence, so every speculative iteration snapshots the whole recurrent state
(48 SSM layers beside 16 attention ones, ~150 MiB). That cost was folded into
:other, whose documented cause is Metal GPU-sync waits. Splitting it out dropped
:other for one M1 Max run from 8.7 s to 0.17 s and showed ckpt was
6.9 s of the 16.3 s total.

Also: Model.n_layer_nextn/1 (wraps an existing NIF that was only reachable via
LlamaCppEx.NIF), vendor/llama.cpp a94d563 (b10423) → 9e40df6 (b10435)
with LLAMA_COMMIT bumped to match.

Measured

Q4_K_M target + Q4_0 head, 120-token greedy generations:

n_draft	acceptance	M1 Max (Metal)	GB10 (DGX Spark)
1	75.0%	0.89×	1.24×
2	54–59%	0.66×	1.17×
3	40–44%	0.68×	1.09×
4	32%	—	0.96×
5	30%	0.56×	—

MTP is a net loss on Metal for this model, and a win on GB10 at n_draft: 1 —
unlike the 35B MoE already documented in the README, whose optimum was 2. Check
ckpt against total before trusting speculation on any hybrid model.

Deliberately not changed

• --spec-default stays unreachable. It stacks n-gram speculation on top of a
  model-based drafter; the NIF pins COMMON_SPECULATIVE_TYPE_DRAFT_MTP. Untested here.
• At n_draft >= 4 greedy output diverges from baseline — deterministically,
  always the same token. This is not a rollback bug. Plain greedy decode with
  speculation switched off entirely flips at the same position purely from batch row
  count (1–3 rows → " computational", 4+ → " latency"): ggml-metal-ops.cpp
  takes the mul_mv_ext path for Q4_K only at ne11 >= 4 and picks r1ptg by
  ne11, and the verify batch is 1 + n_draft rows. A near-tie decided by
  reduction order.

Verification

• macOS (Metal): 428 passed, 149 excluded, plus 6 passed with
  --include mtp_sidecar against the Qwen 3.8 pair
• DGX Spark (CUDA 13.0, sm_121a): 428 passed
• mix format, mix compile --warnings-as-errors, mix credo --strict clean

New gate, since the pair needs two files that :mtp's single-file model cannot satisfy:

GGML_METAL_NO_RESIDENCY=1 LLAMA_BACKEND=metal \
LLAMA_SMOKE_MTP_MODEL=/path/to/Qwen3.8-27B-Q4_K_M.gguf \
LLAMA_SMOKE_MTP_DRAFT_MODEL=/path/to/mtp-Qwen3.8-27B-Q4_0.gguf \
  mix test --include mtp_sidecar

Review notes

• The GB10 column is weaker evidence than the README's existing GB10 note. It is
  one warm sweep, not the interleaved n=11 methodology used for the 35B MoE figures.
  Worth re-running interleaved before those numbers are leaned on.
• Both GB10 arms are warm-cache. A cold first run read ~19 GB and reported a
  4.18 tok/s baseline against 10.83 warm, which inverts the comparison entirely.
• The near-tie explanation is confirmed on Metal only; the CUDA divergence point
  was never probed.