Only push a transpose through a shared output when it can cancel. - #32868
Open
xiaohanAMD wants to merge 3 commits into
Open
xiaohanAMD wants to merge 3 commits into
xiaohanAMD wants to merge 3 commits into
Conversation
The cost check treated any output that leads to a transpose as a benefit, so a shared QDQ value was pushed even when another branch could not cancel the permutation. Count that benefit only for a single consumer or a path that reaches the inverse permutation. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Path feasibility and non-node consumers can still cause false cancellation benefits.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Refines transpose cost estimation so shared outputs are pushed only when a downstream transpose can cancel the permutation.
Changes:
- Adds downstream cancellation-path analysis.
- Adds regression coverage for branched QDQ graphs.
| File | Description |
|---|---|
onnxruntime/core/optimizer/transpose_optimization/onnx_transpose_optimization.cc |
Refines transpose benefit calculation. |
onnxruntime/test/optimizer/transpose_optimizer_test.cc |
Tests shared QDQ output behavior. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Author
|
Scott McKay (@skottmckay) mirounga |
GetValueConsumers only lists node consumers, so a graph output or subgraph input was treated as a single consumer and the shared transpose was still pushed. Co-authored-by: Cursor <cursoragent@cursor.com>
The cost check already ignored a non-comprehensive consumer on the node output. A later value on the path could still be a graph output and was credited when a node consumer reached the inverse permutation. Co-authored-by: Cursor <cursoragent@cursor.com>
mirounga
self-requested a review
September 28, 2026 14:49
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The transpose optimizer now pushes a transpose through a shared output only when that permutation can be canceled.
The cost check used to treat any output that leads to a downstream Transpose as a benefit. A shared quantized value was pushed even when another branch could not cancel the permutation, which left an extra Transpose on that branch.
The benefit is counted only when:
the output has a single node consumer, or
a walk through nodes that can carry the transpose reaches a Transpose with the inverse permutation.
A graph output or subgraph input is not listed in GetValueConsumers::nodes (comprehensive == false). That use cannot cancel the permutation, so it blocks the benefit both on the value being pushed through and on any later value the walk would cross.
The tests cover a shared QDQ output, a graph output that shares the quantized value, and an intermediate graph output that sits on the only canceling path.
Motivation and Context
Pushing a transpose through a shared output duplicates it onto every consumer. When one consumer cannot cancel it, the optimized graph keeps a Transpose that the cost check expected to disappear. A graph output or subgraph input is the same kind of extra consumer, because the consumer list only includes nodes.