Skip to content

fix(qwen): resolve GGUF export failure when MTP config is nested under text_config - #101

Open
IshantDere wants to merge 1 commit into
unslothai:masterfrom
IshantDere:fix/qwen3-mtp-text-config-nesting
Open

fix(qwen): resolve GGUF export failure when MTP config is nested under text_config#101
IshantDere wants to merge 1 commit into
unslothai:masterfrom
IshantDere:fix/qwen3-mtp-text-config-nesting

Conversation

@IshantDere

Copy link
Copy Markdown

Closes unslothai/unsloth#8443

Problem

_QwenMtpMixin.__init__ reads num_hidden_layers and mtp_num_hidden_layers
from the top level of self.hparams only. For Qwen3.5/3.6 models these keys
are nested under text_config, so __init__ always read 0 MTP layers. It then
hit assert self.opt_num_mtp_layers != 0, but opt_num_mtp_layers is only
recovered from tensor names during filter_tensors/index_tensors, which run
after __init__. The first GGUF conversion in a fresh process always
crashed with a cryptic AssertionError:

assert self.opt_num_mtp_layers != 0

Fix

  1. Flatten config — merge text_config into hparams the same way
    index_tensors already does, so nested mtp_num_hidden_layers /
    num_hidden_layers are found.
  2. Replace the bare assert with an actionable ValueError explaining the
    MTP count could not be found and how to work around it
    (--no-mtp, or add mtp_num_hidden_layers to config.json).

Verification

Reproduced against the three relevant config shapes (MTP nested under
text_config, flat top-level config, and a config missing MTP entirely):

Scenario Before After
mtp_num_hidden_layers under text_config AssertionError 25 blocks
flat top-level config 25 blocks 25 blocks
no MTP info anywhere AssertionError actionable ValueError

Files changed

  • conversion/qwen.py_QwenMtpMixin.__init__

…r text_config

Qwen3.5/3.6 models nest `num_hidden_layers` and `mtp_num_hidden_layers`
under `text_config` in config.json. _QwenMtpMixin.__init__ was reading
only the top-level hparams, finding 0 MTP layers, then hitting:

    assert self.opt_num_mtp_layers != 0

This assert failed because opt_num_mtp_layers is only populated later
during filter_tensors (called from index_tensors). The first conversion
in a fresh process always crashed.

Two fixes:
1. Merge text_config into hparams in __init__, matching what
   index_tensors already does, so nested keys are found.
2. Replace the bare assert with a ValueError that tells the user what
   went wrong and how to work around it.

Fixes unslothai/unsloth#8443

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GGUF export of Qwen3.5 fails on assert self.opt_num_mtp_layers != 0 when MTP config is nested under text_config

1 participant