Split ONNX opsets for legacy and dynamo export - #2
Closed
quic-amitraj wants to merge 2 commits into
Closed
Conversation
Signed-off-by: Amit Raj <amitraj@qti.qualcomm.com>
ochougul
reviewed
Jul 21, 2026
| module.__dict__["ops"] = previous_ops | ||
|
|
||
|
|
||
| def qeff_custom_op(domain: str, version: int): |
There was a problem hiding this comment.
Add a comment saying get rid of the decorator once, legacy export is deprecated.
There was a problem hiding this comment.
do you think is this the right way to do this?
ochougul
approved these changes
Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked On
This PR is stacked on quic#1194 by targeting
smedhe:smedhe_enable_dynamo_for_causallm.Problem
Legacy TorchScript export and dynamo export need different ONNX opsets. The previous code used a single
ONNX_EXPORT_OPSETvalue everywhere, so both export paths, custom-op symbolic registration, ONNXScript function protos, and the dynamo custom translation table all effectively shared the same opset-backed custom op definitions.For this branch we need:
dynamo=False) to stay on ONNX opset 17dynamo=True) to use ONNX opset 18CtxScatter,CtxGather,CustomRMSNorm, etc.) to emit function bodies built against the matching default-domain opsetApproach
Instead of duplicating
ctx_scatter_gather.py,ctx_scatter_gather_cb.py, andrms_norm.pyinto separate opset-17/opset-18 files, this PR keeps one source definition for each custom op and compiles two ONNXScript variants from it.The helper in
QEfficient/customop/onnxscript_utils.pytemporarily binds the module-levelopssymbol to the requested default ONNX opset while the ONNXScript decorator compiles the function. The default returned function is the legacy/opset-17 variant, and the dynamo/opset-18 variant is attached to it for explicit lookup. This avoids code duplication while ensuring both export modes get different ONNXScript function protos.Key Changes
ONNX_LEGACY_EXPORT_OPSET = 17ONNX_DYNAMO_EXPORT_OPSET = 18get_onnx_export_opset(dynamo)opset_version=17.opset_version=18.CustomOpTransform.apply(..., onnx_export_opset=...)mode-aware so it registers custom symbolics and appends ONNXScript function protos for the selected opset.DYNAMO_CUSTOM_OP_TABLEto point at the opset-18 ONNXScript variants, because dynamo uses that table during export before post-export transforms run.CustomRMSNormsource can produce opset-17 and opset-18 function protos.Why This Design
Duplicating the custom-op files would work, but it creates long-term maintenance risk: every custom-op behavior fix would need to be made in two places. This PR centralizes only the opset selection while keeping the custom-op implementation itself single-source. The mode boundary is explicit at export time and transform time.