Support Conv+QuickGelu fusion in the WebGPU EP - #2
Open
4n4ny4 wants to merge 1 commit into
Open
Conversation
Gate QuickGelu fusion to WebGPU and parse it in WebGPU fused Conv. Preserve an explicit alpha and materialize 1.702 when omitted so the fused node retains the schema default. An EP must not be authorized without its parser: conv.h hard-fails with ORT_ENFORCE(GetFusedActivationAttr(info, activation_).IsOK()). Any future backend authorization must therefore ship with its parser. Integrated browser measurements eliminated 87 standalone QuickGelu dispatches and reduced total steady dispatches from 288 to 201. The GPU path was performance-neutral, while host-side T0-T2 improved by about 0.100 ms. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The WebGPU EP could not fuse
Convwithcom.microsoft.QuickGelu, so SiLU-style activations ran as standalone dispatches. On YOLO26n that is 87 extra dispatches, each reading and rewriting a full tensor to apply one cheap elementwise operation.The activation appears as
QuickGelurather thanSigmoid+MulbecauseQuickGeluFusionmerges that pair earlier in the pipeline.ConvActivationFusionthen declined it, andGetFusedActivationAttrhad no parser for it.Change
Conv→com.microsoft.QuickGelufusion forkWebGpuExecutionProvider, preserving thealphaattribute.onnxruntime/core/providers/webgpu/nn/fuse_utils.{h,cc}.Important invariant
An execution provider must never be authorized in the optimizer allow-list without its matching activation parser. Without the native parser,
conv.h'sORT_ENFORCE(GetFusedActivationAttr(info, activation_).IsOK())hard-fails rather than falling back. The path is reachable on ordinary models: partition, Level 2,QuickGeluFusion, then a Level 4 graph mutation re-runs the optimization loop and Level 2 seesConv→QuickGelu.This PR therefore gates authorization to the native WebGPU EP only. Extending it to the JSEP backend is future work and must ship in the same change as the JSEP parser plus positive tests for both providers.
Validation
graph_transform_test.ccverifies the fusion including default alpha. On YOLO26n, dispatches drop 288 → 201 with 0 standalone QuickGelu remaining, and output is unchanged.Performance
Dispatch-count reduction is real, but end-to-end GPU time on this path is unchanged — those dispatches were already overlapped and off the critical path. The measurable benefit is roughly 0.100 ms of host-side encode work. Presented as correctness and enablement, not a performance win.