Add MTP for GLM-5.3-Flash

[timkhronos]

Overview

Adds the NextN draft head for GLM-5.3-Flash

Builds on #27773, which does convert the Nextn tensors, but leaves them unused.

Additional information

Enable --spec-type draft-mtp for glm5-next.

Also implements index_share_for_mtp_iteration which is true in the GLM 5.3 Flash config. The draft head computes its indexer selection once, then reuses it across each call, instead of rebuilding.

Measured draft acceptance rate with the layer quantized to Q4 (I will also test with Q8):

draft acceptance = 0.74485 ( 3474 accepted /  4664 generated), mean len =  4.13
acc per pos = (0.924, 0.813, 0.600, 0.441, 0.348)

On my heavily CPU offloaded setup, with draft max 3 I am getting roughly 15-30% speedup with MTP enabled.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES, AI was used in an assistive capacity.

[borakilo]

Spent a few hours benchmarking this branch on a 4-GPU box and figured the numbers might be useful. Thanks for the MTP work — the long-context decode fix in d2fc716 made a big difference.

Benchmarks

4× 48GB Ampere, -sm layer -fa on, UD-IQ4_XS:

depth	pp512	tg128
0	505	25.0
32k	311	22.3
128k	146	19.6

Cold prefill on an actual 45k prompt comes out around 492 t/s. The -d numbers are marginal cost at full depth, so they read much lower than what you see prefilling a growing context.

For comparison, the same benchmark on #27773 and on Unsloth's glm5next/upstream gives 13.8–14.0 t/s at 32k versus your 22.3. Prefill is identical across all three, so d2fc716 looks decode-specific.

MTP came out slower for me, which surprised me. With --spec-type draft-mtp: 19.0 t/s at --spec-draft-n-max 1 (77.6% acceptance) and 16.4 at 3 (58.5%), against 25.0 with no speculation. Everything was in VRAM, so I assume the draft overhead only pays for itself when each target pass is expensive enough — which would match your CPU-offload setup.

One other thing worth knowing: --override-kv glm5next.attention.indexer.top_k=int:1024 gets about +7.5% decode with no measurable prefill change. Fits the idea that prefill is dominated by the indexer scoring pass rather than attention over the selected keys.

Investigation and write-up done with Claude Opus 5.