fix(qwen): resolve GGUF export failure when MTP config is nested under text_config - #101
Open
IshantDere wants to merge 1 commit into
Open
fix(qwen): resolve GGUF export failure when MTP config is nested under text_config#101IshantDere wants to merge 1 commit into
IshantDere wants to merge 1 commit into
Conversation
…r text_config
Qwen3.5/3.6 models nest `num_hidden_layers` and `mtp_num_hidden_layers`
under `text_config` in config.json. _QwenMtpMixin.__init__ was reading
only the top-level hparams, finding 0 MTP layers, then hitting:
assert self.opt_num_mtp_layers != 0
This assert failed because opt_num_mtp_layers is only populated later
during filter_tensors (called from index_tensors). The first conversion
in a fresh process always crashed.
Two fixes:
1. Merge text_config into hparams in __init__, matching what
index_tensors already does, so nested keys are found.
2. Replace the bare assert with a ValueError that tells the user what
went wrong and how to work around it.
Fixes unslothai/unsloth#8443
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes unslothai/unsloth#8443
Problem
_QwenMtpMixin.__init__readsnum_hidden_layersandmtp_num_hidden_layersfrom the top level of
self.hparamsonly. For Qwen3.5/3.6 models these keysare nested under
text_config, so__init__always read 0 MTP layers. It thenhit
assert self.opt_num_mtp_layers != 0, butopt_num_mtp_layersis onlyrecovered from tensor names during
filter_tensors/index_tensors, which runafter
__init__. The first GGUF conversion in a fresh process alwayscrashed with a cryptic
AssertionError:Fix
text_configintohparamsthe same wayindex_tensorsalready does, so nestedmtp_num_hidden_layers/num_hidden_layersare found.ValueErrorexplaining theMTP count could not be found and how to work around it
(
--no-mtp, or addmtp_num_hidden_layerstoconfig.json).Verification
Reproduced against the three relevant config shapes (MTP nested under
text_config, flat top-level config, and a config missing MTP entirely):mtp_num_hidden_layersundertext_configAssertionErrorAssertionErrorValueErrorFiles changed
conversion/qwen.py—_QwenMtpMixin.__init__