feat: add dynamic shapes kernel specialization strategy for TRT-RTX#4184
Draft
tp5uiuc wants to merge 4 commits intopytorch:mainfrom
Draft
feat: add dynamic shapes kernel specialization strategy for TRT-RTX#4184tp5uiuc wants to merge 4 commits intopytorch:mainfrom
tp5uiuc wants to merge 4 commits intopytorch:mainfrom
Conversation
Add runtime cache support for TensorRT-RTX JIT compilation results, replacing the timing cache which is not used by RTX (no autotuning). Changes: - Skip timing cache creation/saving for TensorRT-RTX in _TRTInterpreter - Add RUNTIME_CACHE_PATH default and runtime_cache_path setting - Wire up IRuntimeCache in PythonTorchTensorRTModule (setup, load, save) - Persist runtime cache to disk with filelock for concurrent access safety - Thread runtime_cache_path through all compile functions - Add unit tests (12 tests) and E2E model tests (6 tests) - Update docstrings and RST documentation Fixes pytorch#3817 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Version provided by upstream torch; no pin needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Expose IRuntimeConfig.setDynamicShapesKernelSpecializationStrategy()
through the Torch-TensorRT Python API. Users can now control how
shape-specialized kernels are compiled at runtime for dynamic shapes
on TensorRT-RTX via the new `dynamic_shapes_kernel_specialization_strategy`
compilation setting ("lazy", "eager", or "none").
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
github-actions
bot
added
documentation
tp5uiuc
commented
Apr 12, 2026
tp5uiuc
commented
Apr 12, 2026
Address review feedback: compile with torchtrt.Input min/opt/max ranges so dynamic shapes are actually exercised. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Expose
IRuntimeConfig.setDynamicShapesKernelSpecializationStrategy()through the Torch-TensorRT Python API for TensorRT-RTX builds.Users can now control how shape-specialized kernels are compiled at runtime for dynamic shapes via the new
dynamic_shapes_kernel_specialization_strategycompilation setting:"lazy"(default): Compile shape-specialized kernels in the background, use fallback until ready"eager": Compile immediately (blocking)"none": Always use fallback kernels, never specializeDepends on: #4180 (runtime cache API — provides the
IRuntimeConfiginfrastructure)Type of change
Checklist: