[MIGraphX EP] Add support for user-provided HIP streams - #32870
Open
Andrea Bocci (fwyzard) wants to merge 3 commits into
Open
Andrea Bocci (fwyzard) wants to merge 3 commits into
Andrea Bocci (fwyzard) wants to merge 3 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The new user-stream behavior lacks regression tests for parsing, synchronization, ownership, and device validation.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Adds user-provided HIP stream support to the MIGraphX execution provider.
Changes:
- Adds and parses user stream provider options.
- Validates stream device ownership.
- Integrates external streams with execution, synchronization, and lifecycle handling.
| File | Description |
|---|---|
migraphx_stream_handle.h |
Adds configurable stream ownership. |
migraphx_stream_handle.cc |
Prevents destruction or flushing of external streams. |
migraphx_execution_provider.h |
Stores and reports external-stream state. |
migraphx_execution_provider.cc |
Uses, validates, and synchronizes user streams. |
migraphx_execution_provider_info.h |
Defines and hashes the new options. |
migraphx_execution_provider_info.cc |
Parses and serializes the new options. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
… options Add the "has_user_compute_stream" and "user_compute_stream" provider options to the MIGraphX execution provider, with the same names and semantics as in the CUDA execution provider: the address of a HIP stream provided by the user, that the execution provider should use instead of creating its own streams. Signed-off-by: Andrea Bocci <andrea.bocci@cern.ch>
Andrea Bocci (fwyzard)
force-pushed
the
migraphx-user-compute-stream
branch
2 times, most recently
from
September 28, 2026 08:35
7ced9b8 to
c486891
Compare
Andrea Bocci (fwyzard)
added a commit
to fwyzard/cmsdist
that referenced
this pull request
Sep 28, 2026
Add support for user-provided HIP streams to the ONNXRuntime MIGraphX execution provider, using the same syntax and sematic as the CUDA EP. See microsoft/onnxruntime#32850 for more details and microsoft/onnxruntime#32870 for the upstream implementation.
When the "user_compute_stream" option is set, use the HIP stream provided by the user for all the
work submitted by the MIGraphX execution provider, instead of creating a new stream for each
DeviceStreamCollection:
- pass the user stream to RegisterMIGraphXStreamHandles(), resolving the TODO in
RegisterStreamHandlers();
- mark the MIGraphXStream that wraps it as not owned, so that the user stream is neither
synchronised by Flush() nor destroyed together with the session;
- check that the user stream belongs to the device used by the execution provider at creation;
- select the device of the stream on every execution thread;
- synchronise the user stream in OnRunEnd() only if sync_stream is set, so that the
"disable_synchronize_execution_providers" run option lets the user keep submitting work
asynchronously;
- synchronise the user stream in Sync(), because a non-blocking stream is not synchronised
together with the null stream.
Fix the behaviour without a user compute stream, and select the device of the internal stream on
every execution thread.
Signed-off-by: Andrea Bocci <andrea.bocci@cern.ch>
Test that the "has_user_compute_stream" and "user_compute_stream" options are parsed and reported
back by the execution provider, and that the reported options round-trip.
Test that, with the "user_compute_stream" option, the MIGraphX execution provider
- runs the inference in the user stream, ordered after the work already queued in it;
- does not synchronise the host with the user stream when the synchronisation of the execution
providers is disabled, and does synchronise it otherwise;
- does not take ownership of the user stream, which stays valid after the session is destroyed;
- supports several sessions, each with its own user stream, running concurrently;
- handles a missing, null or invalid stream address;
- rejects a user stream that belongs to a different device;
- checks for a session running from a thread with a different current device;
- validates the behaviour of the input/output synchronisation.
Link the HIP runtime to the unit tests when the MIGraphX execution provider is enabled, because
the tests create and manage the HIP streams and buffers themselves.
Signed-off-by: Andrea Bocci <andrea.bocci@cern.ch>
Andrea Bocci (fwyzard)
force-pushed
the
migraphx-user-compute-stream
branch
from
September 28, 2026 13:46
c486891 to
9d28cec
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add the
"has_user_compute_stream"and"user_compute_stream"provider options to the MIGraphX execution provider, with the same names and semantics as in the CUDA execution provider: the address of a HIP stream provided by the user, that the execution provider should use instead of creating its own streams.When the
"user_compute_stream"option is set, use the HIP stream provided by the user for all the work submitted by the MIGraphX execution provider, instead of creating a new stream for eachDeviceStreamCollection:RegisterMIGraphXStreamHandles(), resolving the TODO inRegisterStreamHandlers();MIGraphXStreamthat wraps it as not owned, so that the user stream is neithersynchronised by
Flush()nor destroyed together with the session;OnRunEnd()only ifsync_streamis set, so that the"disable_synchronize_execution_providers"run option lets the user keep submitting work asynchronously;Sync(), because a non-blocking stream is not synchronisedtogether with the null stream.
The behaviour without a user compute stream is unchanged.
Implement a new unit test for the
"has_user_compute_stream"and"user_compute_stream"options and user-provided HIP streams.Motivation and Context
Implements #32850 .