Fix Olive-quantized Qwen3 MoE export - #525
Open
titaiwangms wants to merge 2 commits into
Open
Conversation
Preserve packed fused expert sidecars until QMoE packing and emit a supported Qwen3 ORT GenAI runtime type with reasoning-token metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e10674c1-6909-4b09-9f5a-d41b28c89d2d
Performance Comparison
|
Contributor
Author
|
The current all-files Lint failure is inherited from |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes export of Olive/GPTQ/AWQ quantized Qwen3-MoE by preserving packed fused-expert sidecar tensors during HF→ONNX key renaming, ensuring the QMoE packer receives the correct expert-major layout. It also updates ORT GenAI auto-export to emit a supported runtime model.type for qwen3_moe, and adds regression coverage across weight preprocessing, graph construction, and runtime config generation.
Changes:
- Preserve packed quantized MoE expert sidecar keys during generic MoE renaming via a shared
is_packed_quant_key(...)predicate. - Map
qwen3_moe→qwen3for ORT GenAIgenai_config.jsonto avoid unsupported type errors while keeping Qwen3 reasoning-token metadata behavior. - Add targeted regression tests for the Qwen3-MoE quantized graph ABI and weight-preprocess behavior.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/build_graph_test.py | Adds a graph/initializer ABI regression test for Olive-int4 Qwen3-MoE emitting QMoE + MatMulNBits with expected shapes. |
| src/mobius/models/moe.py | Prevents splitting packed quantized expert sidecars during _rename_moe_expert_weights. |
| src/mobius/models/moe_test.py | Adds end-to-end tests ensuring packed fused expert sidecars survive renaming and bind to QMoE parameters. |
| src/mobius/integrations/ort_genai/auto_export.py | Maps qwen3_moe to ORT GenAI-supported qwen3 type. |
| src/mobius/integrations/ort_genai/auto_export_test.py | Adds tests covering qwen3_moe model type resolution in both hf-id and config export modes. |
| src/mobius/_weight_utils.py | Introduces is_packed_quant_key and centralizes packed-sidecar suffixes; reuses predicate in quantized preprocessing validation. |
| src/mobius/_weight_utils_test.py | Adds unit tests for is_packed_quant_key. |
| CHANGELOG.md | Documents the packed-sidecar preservation fix and the ORT GenAI qwen3_moe type resolution fix. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinchuby
approved these changes
Aug 20, 2026
| """Whether ``key`` is a packed sidecar of the specific ``float_key``. | ||
|
|
||
| Unlike the module-level :func:`is_packed_quant_key` suffix predicate, | ||
| this matches the *exact* sidecar keys of one named float parameter. |
| # (``<pname>_qweight``, see Olive's ``olive/common/quant/state_dict.py``), while | ||
| # GPTQ/AWQ store dotted sibling buffers on the owning module | ||
| # (``<module>.qweight``). Both conventions occur in raw HF checkpoints. | ||
| OLIVE_PACKED_QUANT_SUFFIXES = ("_qweight", "_scales", "_qzeros") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
qwen3runtime type forqwen3_moe, preserving Qwen3 reasoning token metadataThis completes the generic Qwen3-MoE KQuant path built on #517. Before this change, packed
gate_up_proj_qweightand_scalessidecars collided after being split into the same per-expert.weightkeys, causing a deterministic weight-shape mismatch during export.Validation
220 passed— focused MoE, weight utility, Qwen3.5, and QMoE fusion tests94 passed, 8 skipped— quantized/MoE graph tests192 passed— ORT GenAI integration testsQwen/Qwen3-30B-A3BOlive KQuant + Mobius export completed successfullybor=151667,eor=151668)17 * 23 = 391