Fix transformers 5.14+ compatibility and remove dead import - #5
Open
bwilfley wants to merge 4 commits into
Open
Conversation
- torch_bitnet.py: change _tied_weights_keys from a list to a dict mapping lm_head.weight to model.embed_tokens.weight, required by the updated API in transformers 5.14+ - test_interop.py: remove unused import of training.bit_linear which does not exist in the repo and caused an immediate ModuleNotFoundError Closes exo-explore#2 Co-Authored-By: Claude <noreply@anthropic.com>
Removed the incorrect ("lm_head.", "lm_head.linear.") replacement pattern
from convert.py — it was a leftover from another model's converter and
caused load_causal_model to fail with "no parameter named linear".
Added run_mlx.py to run text generation with the converted MLX model,
with streaming output and correct SentencePiece token decoding.
Co-Authored-By: Claude <noreply@anthropic.com>
num_hidden_layers was missing from sanitize_config, so it always defaulted to 24 — causing load_causal_model to fail for models with a different layer count (e.g. the 3B model has 26 layers). Co-Authored-By: Claude <noreply@anthropic.com>
Used tokenizer.decode(tokens) as the baseline for delta computation instead of the raw prompt string — the two can differ at token boundaries in SentencePiece, causing the last word of the prompt to appear duplicated in the first generated token. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
torch_bitnet.py: Changed_tied_weights_keysfrom a list to a dict ({"lm_head.weight": "model.embed_tokens.weight"}). Transformers 5.14+ requires this to be a mapping rather than just a list of key names, and raises aValueErrorwith the old format.test_interop.py: Removed a dead import ofweight_quantfromtraining.bit_linear. The symbol was imported but never used in the test file, and the import itself fails in environments where thetrainingmodule is not present.Test plan
python convert.pycompletes without errorpython test_interop.pypasses all 17 tests (6 skipped as expected for long-running tests)