fix(gguf/gqa): correct Qwen3.5/3.8 hybrid export (MTP blocks, M-RoPE interleave, partial-RoPE dim)

[justinchuby]

Summary

Three GGUF→ONNX export bugs surfaced while converting the hybrid Qwen3.8-27B (Gated DeltaNet + GQA, partial RoPE 64/256) 4-bit GGUF to an int4 model. Each produced a silently-wrong model (garbage tokens) rather than a hard failure. All three also affect the sibling Qwen3.5 family.

1. MTP / nextn block-count (_config_mapping.py)

GGUF block_count includes the trailing Multi-Token-Prediction ("nextn") block, so num_hidden_layers was one too high. The base decode model doesn't build the MTP head and its weights are skipped during mapping, leaving an extra decoder layer whose linear-attn/GQA initializers have no backing GGUF weights → _check_weights save invariant fails. Fix: subtract <arch>.nextn_predict_layers.

2. rope_interleave from M-RoPE sections (_config_mapping.py)

rope.dimension_sections encodes Qwen-VL M-RoPE section sizes (e.g. [11,11,10,0]); it does not select GPT-J adjacent-pair rotation. Deriving rope_interleave from section presence set rotary_interleaved=1 on the exported GQA/RotaryEmbedding, corrupting RoPE (Qwen uses split-half / NEOX rotate_half). Only deepseek4 genuinely needs the flat flag.

3. rotary_embedding_dim dropped in GQA fusion (_group_query_attention.py)

RotaryAttentionToGQA fuses external RotaryEmbedding + Attention into a GroupQueryAttention with do_rotary=1, but never copied the source rotary_embedding_dim. Partial-RoPE models (Qwen3.5/3.8 rotate only 64 of 256 head elements) defaulted the fused GQA to the full head_dim, reading past the partial cos/sin cache → garbage from every full-attention layer. Fix: read it off the q RotaryEmbedding node and forward it when nonzero.

Verification

• Verified on Qwen3.8-27B-Q4_0 GGUF export: exported GQA nodes now carry rotary_interleaved=0 and rotary_embedding_dim=64 (were 1 / absent before), and decoder layer count is correct (65→64, MTP block excluded).
• Added regression tests: TestQwen35MtpBlockExclusion, TestQwen35RopeInterleave, test_partial_rotary_embedding_dim_propagated, test_full_rotary_omits_rotary_embedding_dim.
• pytest src/mobius/integrations/gguf/_config_mapping_test.py src/mobius/rewrite_rules/_group_query_attention_test.py → 53 passed.

Note (out of scope)

Even with all three fixes the 27B GGUF int4 model still decodes incoherently, which points to a separate remaining gap in the GGUF Gated-DeltaNet in_proj handling (the direct attn_qkv→in_proj_qkv / ssm_alpha,ssm_beta→in_proj_a,in_proj_b map is shape-correct but the fused-projection ordering vs the HF in_proj_qkvz/in_proj_ba split path is unverified). The only coherence-validated DeltaNet path today is the safetensors builder. Tracking separately; these three fixes are correct and independently testable.

[Copilot]

Pull request overview

Fixes three GGUF→ONNX export correctness bugs affecting hybrid Qwen3.5/3.8 (and related) models by ensuring decoder layer counts, RoPE interleave semantics, and partial-RoPE dimensions are preserved through config mapping and GQA fusion.

Changes:

• Adjust GGUF block_count → decoder num_hidden_layers by subtracting nextn_predict_layers (MTP/“nextn” blocks) during config mapping.
• Stop deriving rope_interleave from rope.dimension_sections (M-RoPE sections); keep the flat interleave flag only for truly adjacent-pair (GPT-J style) layouts (currently deepseek4).
• Propagate rotary_embedding_dim from RotaryEmbedding into fused GroupQueryAttention nodes when nonzero (partial RoPE), with regression tests covering both partial and full RoPE behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
src/mobius/rewrite_rules/_group_query_attention.py	Preserve partial-RoPE rotary_embedding_dim when fusing RotaryEmbedding+Attention into GroupQueryAttention.
src/mobius/rewrite_rules/_group_query_attention_test.py	Adds regression coverage for partial-RoPE propagation and full-RoPE omission of rotary_embedding_dim.
src/mobius/integrations/gguf/_config_mapping.py	Fixes Qwen “nextn” block layer-count handling and corrects rope_interleave derivation logic.
src/mobius/integrations/gguf/_config_mapping_test.py	Adds regression tests for Qwen3.5/3.8 MTP block exclusion and M-RoPE section handling not forcing interleave.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[justinchuby]

Update: Gated-DeltaNet weight-transform fixes → coherent Qwen3.8-27B decode

Pushed b541508, which fixes the remaining GGUF Gated-DeltaNet incoherence for qwen35/qwen35moe (Qwen3.5 / Qwen3.8 hybrid linear-attention). Three converter transforms were not being undone when importing into mobius's GatedDeltaNet (whose forward expects raw HF-style params):

1. A_log double-exp — GGUF stores ssm_a = -exp(A_log); GatedDeltaNet re-applies -exp(A_log), collapsing every head's decay to ≈-1. Recover via A_log = log(-ssm_a).
2. Zero-centered RMSNorm +1 — converter bakes +1 into all *norm.weight except linear_attn.norm.weight; OffsetRMSNorm re-adds it → double count. Subtract 1 (arch-scoped).
3. V-head grouped→tiled reorder — for grouped linear attention (num_value_heads != num_key_heads) the converter tiles every V-indexed linear_attn tensor; mobius consumes grouped order. _reorder_deltanet_v_heads undoes it for in_proj_qkv(V rows), in_proj_z, in_proj_a/b, A_log, dt_bias, conv1d(V channels), out_proj(quantized K-block columns), handling MatMulNBits triplets losslessly. Derived from config — no hardcoded 16/48/128.

Result (unsloth/Qwen3.8-27B-Q4_0 → int4 CUDA ONNX, greedy):

• The capital of France is → Paris. The capital of Germany is Berlin. The capital of Italy is Rome. ...
• Once upon a time → , in a land far away, there was a group of young adventurers ...
• The largest planet in the solar system is → Jupiter, which has a diameter of 142,800 km. ...

Adds regression tests asserting _reorder_deltanet_v_heads is the exact inverse of the converter's _reorder_v_heads (round-trip on float row tensors + quantized out_proj block/zero-point columns), plus A_log / norm-offset unit tests. _builder_test.py: 55 passed.

[justinchuby]

@copilot update from main.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ justinchuby
❌ Copilot
_{You have signed the CLA already but the status is still pending? Let us recheck it.}