[vchuravy]

Draft for CI only, not for merging. Like #1026, this depends on unreleased work, all taken from branches until it is released:

• JuliaCodegen.jl (unregistered; its stacked draft PRs up to Native code's conventions and IR where GPUCompiler depends on them JuliaLLVM/JuliaCodegen.jl#25);
• JuliaIR.jl, which JuliaCodegen.jl depends on;
• the LLVM.jl branch they need (tb/inline-api, which needs an LLVMExtra build from source).

The commit titled [temp] sets up CI for that and will be dropped.

GPUCompiler lowers Julia IR to LLVM IR with Julia's own code generator, through jl_emit_native on the (CodeInstance, CodeInfo) pairs its interpreter inferred. This PR lets a compilation hand the same pairs to JuliaCodegen.jl instead, a code generator written in Julia. It's a first step towards a middle end that GPUCompiler controls: device policies, Enzyme hooks, and no dependence on the C++ codegen's ABI.

• Configuration: CompilerConfig gets a codegen field, :julia (as today) or :juliacodegen (Julia 1.12 and later). Its default comes from the codegen preference. It is part of the configuration, so cached artifacts are keyed on it; inference caches are not.
• Extension: JuliaCodegen.jl is a weak dependency. Its extension registers a code generator with the contract of compile_method_instance's code generation. It returns the module; for each method instance its func (jlcall) and specfunc (specsig) entries; and gv_to_value for the relocation machinery.
  • It uses JuliaCodegen.emit_into! with the job's target, data layout and debug info (llvm_debug_info), gcstack_arg=false, and can_safepoint(job), as the CodegenParams GPUCompiler gives Julia's code generator. For NoDebug, it still asks for locations, as Julia's code generator emits them unless Julia runs with -g0: validation's backtraces read them, then they are stripped.
  • The job's entry is named as native code names it (julia_f_N).
  • Constructs JuliaCodegen.jl doesn't support yet call native code, which validation reports on targets without Julia's runtime. With JULIACODEGEN_UNSUPPORTED=error, they throw instead, which says what they are.
• Loading: loading the extension alone changes nothing. Selecting :juliacodegen without it is an error before compilation starts. When the preference selects it, loading GPUCompiler also loads JuliaCodegen.jl if the environment provides it.
• Precompilation: workloads of packages that don't load JuliaCodegen.jl keep using Julia's code generator. Since the code generator is part of the configuration, a session with the preference set doesn't reuse their cached artifacts. test/native/precompile.jl now expects that when the preference selects JuliaCodegen.jl.

Tests (test/juliacodegen.jl)

The file runs where JuliaCodegen.jl is available; runtests.jl loads it if the environment has it. It covers:

• the configuration, the requirements, loading through the preference, and a precompilation workload without JuliaCodegen.jl;
• that JuliaCodegen.jl actually ran, checked through the extension's ledger (JULIACODEGEN_LEDGER);
• [Draft, for CI] Take Julia's LLVM passes from JuliaIR.jl, as an opt-in #1026's corpus of kernels:
  • for PTX, GCN, SPIR-V, Metal, native and BPF (with kernel both ways), the entry has the same function type and calling convention with either code generator;
  • on the native target, the kernels give the same results (compiled to an object, loaded, run).

All 204 tests pass locally on 1.12.

The whole suite with the preference set is the real differential. Locally on 1.12:

codegen	Pass	Fail	Error	Broken
julia	4456	0	0	4 (unchanged)
juliacodegen	4446	10	0	4

The juliacodegen row started at 149 failures and 15 errors. Getting it down produced the JuliaCodegen.jl fixes in JuliaLLVM/JuliaCodegen.jl#16 to #25: native code's conventions (specsig for varargs and unions, aggregates by reference, jlcall entries calling specsig entries), error paths through the runtime, getfield with dynamic indices, svec, the names and small-tag slots of objects' globals, llvmcall details, and locations for backtraces. The 10 that remain test the shape of native code's IR, which JuliaCodegen.jl doesn't produce:

• nested insertvalue (SPIR-V, 4) and replace i128 allocas (SPIR-V, 4): native code builds those aggregates, or returns them, through stack memory; JuliaCodegen.jl keeps them in SSA values.
• merged relocation slots (Metal): native code loads object addresses where they are used, and LLVM merges the loads; JuliaCodegen.jl loads each once per function.
• smalltag boxed union constant demotion (Metal): the kernel reads the union's payload, never the constant's box, so there is nothing to demote.

CI on this PR (temporary)

• GitHub Actions: every job builds LLVMExtra from the LLVM.jl branch (.ci/juliacodegen.jl build, in MSYS2 on Windows) and sets up test/ with that LLVM.jl checkout. From Julia 1.12 on, it also adds JuliaCodegen.jl and JuliaIR.jl. Coverage is not collected meanwhile.
• codegen axis: julia everywhere, juliacodegen from 1.12 on. The juliacodegen jobs check that the preference was applied and that JuliaCodegen.jl compiled code (the ledger). They are continue-on-error while JuliaCodegen.jl is experimental, so they report without failing the workflow.
• Buildkite: not changed yet; the back-end steps don't run on drafts. A juliacodegen variant per back end, like [Draft, for CI] Take Julia's LLVM passes from JuliaIR.jl, as an opt-in #1026's ports variants, can follow.

🤖 Generated with Claude Code