Problem cache follow-on: pluggable backends, layered cache paths, compile-options API, and offline tooling - #5117
Draft
danieyan-amd wants to merge 9 commits into
Draft
Conversation
added 7 commits
August 5, 2026 16:48
Introduce a type-erased problem_cache_backend so cache storage is interchangeable without inheritance. Add json_problem_cache as the default concrete backend and route the runtime problem_cache through it; a static_assert confirms the backend satisfies the concept. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Add sqlite_problem_cache as a second concrete backend using system SQLite via migraphx::sqlite (no vendored amalgamation). It persists the device-keyed cache to a solutions table and satisfies the problem_cache_backend concept. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Load a list of problem caches as a read-only priority list: search them in order and return the first hit (highest priority first, e.g. application-provided, then shipped). Shipped caches are immutable. compile_ops and the gemm paths look up through find_in_problem_caches so the priority applies across the pipeline. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Add problem_cache_path and the ordered problem_cache_paths to compile_options, wired into the gpu target so callers can supply cache locations without the env var. Expose problem_cache_paths through the C and C++ APIs for the ONNX Runtime EP. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Add an offline aggregator that merges per-GPU caches into a master, validates a cache, and converts between backends. Storage is selected through the type-erased backend via a make_backend(json/sqlite) factory; reads enumerate the concrete backend. Supports conflict policies and legacy empty-device handling. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Add aggregate_cache, validate_cache, and convert_cache driver subcommands wrapping the offline aggregator, reporting errors cleanly instead of crashing. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
Contributor
|
Thank you for your contribution! Since this is an external pull request, a maintainer must review PR and add the "ok-to-test" label if it is approved for testing. |
pfultz2
reviewed
Aug 5, 2026
| MIGRAPHX_C_EXPORT migraphx_status migraphx_compile_options_set_advance_backend_options( | ||
| migraphx_compile_options_t compile_options, const char* options_json, ...); | ||
|
|
||
| MIGRAPHX_C_EXPORT migraphx_status migraphx_compile_options_set_problem_cache_paths( |
Collaborator
There was a problem hiding this comment.
The problem cache should be set as a backend option since this is a GPU-only option. So there shouldn't be an API changes.
| } | ||
| }; | ||
|
|
||
| #ifdef HAVE_GPU |
Collaborator
There was a problem hiding this comment.
I dont think we should be putting a lot of GPU-only code into the driver. The offline aggregation could be done as a python script.
|
|
||
| /// Search all caches in priority order (read-only first, then writable). | ||
| /// Returns the first hit. This is what compile_ops should call. | ||
| optional<value> find_in_problem_caches(const std::string& name, const value& problem) const |
Collaborator
There was a problem hiding this comment.
This shouldn't be in the context method. This should be handled by problem_cache::has.
| // | ||
| // te.py DSL input for migraphx::gpu::problem_cache_backend. | ||
| // | ||
| // REGENERATION (until tools/generate.py learns gpu/ subdir routing): |
Collaborator
There was a problem hiding this comment.
The tools/generate.py needs to be updated to handle the gpu/subdir. This should not be done manual as this will not be checked by CI.
| void insert(const cache_device_key& dk, const value& key, const value& solution); | ||
|
|
||
| /// Insert a sentinel (null value) for `key` to record "we've tried | ||
| /// this problem and there is no solution". Distinguished from a hit |
Collaborator
There was a problem hiding this comment.
This is not what mark is for. Its for a pending solution. This comment is completely wrong.
| * new solutions are written to the last (writable) path. Empty falls back | ||
| * to the MIGRAPHX_PROBLEM_CACHE environment variable. | ||
| */ | ||
| std::vector<std::string> problem_cache_paths; |
Collaborator
There was a problem hiding this comment.
There should be only one path. The path for the system level db or user level db should be compiled into migraphx.
| /** | ||
| * Problem cache file paths, searched in priority order (first hit wins); | ||
| * new solutions are written to the last (writable) path. Empty falls back | ||
| * to the MIGRAPHX_PROBLEM_CACHE environment variable. |
Collaborator
There was a problem hiding this comment.
With the backend option the MIGRAPHX_PROBLEM_CACHE variable should be removed.
Route include/gpu te.py inputs through generate.py (moving the DSL into tools/include/gpu/), and fix the mark() comment to describe a pending solution rather than a missing one. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
…one API and env var Problem cache file paths now arrive through the GPU backend_options (problem_cache_files) instead of a dedicated compile_options API, and the MIGRAPHX_PROBLEM_CACHE environment variable is removed. A single file is writable; multiple are a read-only priority list. Signed-off-by: danieyan-amd <daniel.anieyan@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-on to #4835 (hardware-provenance problem cache). This adds the deployment and tooling layer around the problem cache so pre-tuned caches can be shipped, discovered, and combined without environment variables.
problem_cache_backendwith a JSON backend and an optional SQLite backend (built on system SQLite), selectable throughcompile_options.compile_optionsgains an ordered list of problem-cache paths (problem_cache_paths, with a single-path convenience). Paths are searched in priority order (first hit wins) and loaded read-only, so an application-provided cache can take precedence over a shipped one without mutating either.migraphx_compile_options_set_problem_cache_pathsand the matching C++/Python wrappers, so the paths can be set programmatically rather than via an environment variable.error_on_conflict/first_wins/last_wins).Testing
New GPU unit tests cover the backend abstraction, the SQLite backend round-trip, the layered path override, and the aggregator (merge / validate / convert plus conflict and legacy-device policies), along with an API-level compile-options test.
Notes
developand for review.