Repository navigation
Avoid load-order invalidation of the compilation pipeline - #1052
Merged
Merged
Conversation
maleadt
added
the
latency
This was referenced Oct 21, 2026
`llvm_datalayout` is called on an abstractly-typed target, so its result was inferred as `Any`, and the string concatenation in `julia_datalayout` was compiled with an edge to `*(::Any, ::String)`. Packages that add methods to `*`, like ChainRulesCore, invalidated it and the `compile_method_instance` closure that calls it. Part of #1048.
Code that calls Base functions on abstractly-typed values is compiled with edges to their abstract signatures, e.g. `Colon(::Int64, ::Real)` for `x[2:end]` on a value of unknown type. Packages loaded before GPUCompiler that extend those functions (`promote_rule` by ChainRulesCore, `Colon` by Static) then make Julia discard GPUCompiler's precompiled code when it is loaded, and recompile it for the first kernel. Assert types that are known, narrow before indexing, and use typed array literals.
The generated boxing functions use `|` from Base. Left implicit, its binding is only created when a downstream package (e.g. KernelAbstractions) precompiles code that calls them, and Julia 1.13 then reports the edges to that binding as invalidations every time that package is loaded. The code is not actually recompiled, but the spurious entries obscure real invalidations in SnoopCompile.
maleadt
force-pushed
the
tb/invalidations
branch
from
October 22, 2026 20:02
b11db0e to
8edcfd1
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1052 +/- ##
=======================================
Coverage 92.41% 92.41%
=======================================
Files 31 31
Lines 7261 7266 +5
=======================================
+ Hits 6710 6715 +5
Misses 551 551 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#1048 reports that loading ChainRulesCore before KernelAbstractions makes the first CPU kernel launch about 6× slower. The cause is load-order invalidation: ChainRulesCore adds methods like
promote_rule(::Type{<:Number}, ::Type{<:AbstractZero})and*(::NotImplemented, ::Any), which invalidate Base method instances with abstract signatures (Colon(::Int64, ::Real),*(::Any, ::String), ...). Any precompiled GPUCompiler code with an edge to one of those fails verification when GPUCompiler is loaded, and so does everything that calls it, so most of the compilation pipeline (emit_llvm,compile,finish_ir!,check_ir!,build_runtime) gets recompiled for the first kernel.precompile_blockersonly shows the part that's invalidated when ChainRulesCore is loaded after GPUCompiler, which is why the issue only foundjulia_datalayout.This PR keeps the GPUCompiler code involved concretely typed: it asserts the result of
llvm_datalayout(the suggestion in the issue), the method instance's signature incheck_invocationand theVarargcount in the mangler, narrows statements before slicing their arguments, and uses typed array literals for rewritten calls. The rest of the chains are fixed in the dependencies: JuliaLLVM/LLVM.jl#702 and maleadt/CompilerCaching.jl#33 (the latter is the biggest single contributor). A separate commit imports|explicitly in the runtime module, which only removes spurious invalidation entries that Julia 1.13 reports for lazily created bindings (JuliaLang/julia#59135).JuliaGPU/KernelAbstractions.jl#914 and JuliaGPU/CUDA.jl#3440 fix similar chains in those packages. Other packages invalidate the same pipeline through different methods, e.g. DataFrames' dependencies through LLVM.jl's
@nospecializeC API layer. That needs further work in LLVM.jl, so this doesn't eliminate the problem for every package, but ChainRulesCore (loaded by most of the AD and ML ecosystem) is covered.Tested with the full test suite on Julia 1.13. With all three PRs, first-kernel times on Julia 1.13.2 with ChainRulesCore loaded first:
@cuda+ broadcast +sum)Fixes #1048