Build: Switch from frontend-PGO to IR-PGO by kinke · Pull Request #23870 · dlang/dmd

kinke · 2026-09-17T01:37:59Z

I.e., from LDC's own AST-based implementation (-fprofile-instr-…) to LLVM-builtin IR-based PGO (-fprofile-…). With the downside of having to enable optimizations for the instrumented compiler (=> longer build time).

github-actions · 2026-09-17T01:51:23Z

DMD perf check

Metric	Base	PR	Δ
compile hello.d (instr)	213.0 M	211.6 M	-0.689%
compile hello.d -O -release (instr)	231.1 M	229.1 M	-0.827%
compile Phobos (instr)	5,081.2 M	4,903.0 M	-3.508%
compile Phobos codegen (instr)	1,442.3 M	1,429.4 M	-0.896%
compile vibe.d (instr)	15,019.0 M	14,495.8 M	-3.484%
dmd binary size (stripped)	6.91 MB	8.13 MB	+17.76%
peak RSS (compile hello.d)	43.12 MB	44.16 MB	+2.40%

Breakdown — compile hello.d

Phase (wall, self time)	Base	PR	Δ
sema1	10.4 ms	9.2 ms	-11.40%
parse	32.8 ms	31.7 ms	-3.49%
sema3	5.7 ms	5.4 ms	-6.44%
codegen	1.9 ms	1.9 ms	-2.34%
sema_other	13.5 ms	13.5 ms	-0.06%

Breakdown — compile Phobos

-178.3 M instructions: frontend -165.3 M (-4.54%), codegen -12.9 M (-0.90%)

Phase (wall, self time)	Base	PR	Δ
sema3	598 ms	540 ms	-9.66%
sema1	228 ms	202 ms	-11.31%
codegen	402 ms	389 ms	-3.30%
sema_other	205 ms	195 ms	-4.55%
ctfe	14.2 ms	11.8 ms	-17.26%
parse	108 ms	110 ms	+1.47%
sema2	1.5 ms	0.8 ms	-43.73%
inline	4.1 ms	4.2 ms	+1.42%

All measurements

Metric	Base	PR	Δ
compile hello.d (instr)	213.0 M	211.6 M	-0.689%
compile hello.d -O -release (instr)	231.1 M	229.1 M	-0.827%
compile Phobos (instr)	5,081.2 M	4,903.0 M	-3.508%
compile Phobos codegen (instr)	1,442.3 M	1,429.4 M	-0.896%
compile vibe.d (instr)	15,019.0 M	14,495.8 M	-3.484%
dmd binary size (stripped)	6.91 MB	8.13 MB	+17.76%
hello binary size (stripped)	0.72 MB	0.72 MB	0.00%
peak RSS (compile hello.d)	43.12 MB	44.16 MB	+2.40%
peak RSS (compile Phobos)	618.8 MB	615.4 MB	-0.56%
peak RSS (compile vibe.d)	1916 MB	1908 MB	-0.42%
compile dmd itself (wall)	12.1 s	12.0 s	-0.77%
compile hello.d (wall)	64.3 ms	61.6 ms	-4.27%
compile Phobos (wall)	1,560 ms	1,452 ms	-6.90%

_{2c9ff5f vs merge-base 278dce2 · about these metrics}

I.e., from LDC's own AST-based implementation (-fprofile-instr-…) to LLVM-builtin IR-based PGO (-fprofile-…). With the downside of having to enable optimizations for the instrumented compiler (=> longer build time).

kinke · 2026-09-17T02:06:48Z

Oh wow, pretty significant change in metrics! Note that due to optimizing the instrumented compiler, the runtime of the perf CI job (to gather the performance metrics) would be prolonged by ~2.5 minutes. At least according to https://github.com/dlang/dmd/actions/runs/35171311430/job/105043379050?pr=23870, where the non-optimized build for master took 13 secs, and the optimized one 1:27 mins.

FYI @abulgit @0xEAB (testing this on Windows would be nice :])

abulgit · 2026-09-17T08:01:17Z

Nice numbers! I'd re-run the perf workflow once or may be twice, just to make sure it's as deterministic as it was with the frontend profile. Otherwise pretty good result!

kinke · 2026-09-17T16:19:31Z

Okay, I've re-run it 2 more times - and the 3 latest runs are really extremely stable (as can be seen in the edit history of the bot comment), only peak RSS (compile hello.d) fluctuating a bit. [For the very first run yesterday, #23869 hadn't landed yet, so the profile did not include druntime and Phobos, explaining the different metric there.]

0xEAB · 2026-09-17T20:07:12Z

Still failing on Windows.

FAILED targets:
- compilable\test21331.sh
- compilable\test21359.sh
make: *** [Makefile:168: dmd-pgo] Error 1

  ...
  ... compilable\test9672.d
       compilable\test9613.d  -o- -m32 ()   ()     ...  ... compilable\test9692a.d         (compilable\test9692.d )()        (  )  ... compilable\test9766.d          () ... compilable\test9701.d          -verrors=simple ()
  ... compilable\test9818.d
    ( -o- ()()
) ... compilable\test9919.d          -o- ()
  ...
     ... compilable\testAliasLookup.d
   -preview=fixAliasThis () ... compilable\testcheckimports.d  ()

 (
compilable\testcdefines.i

)
  ... compilable\testcomplex.c
 ... compilable\testcomplex.i       ... compilable\testCpCtor.d        () ()   ...   ...  ...  compilable\testcorrectthis.d

 compilable\testcov1.dcompilable\testcontracts.d
 ...     ()-vcg-ast -o-     ...  ...  ... compilable\testcstuff1.c    ... compilable/testclidflags.sh
    compilable\testcstuff2.ccompilable\testcstuff3.d       ()
compilable\testcstuff4.i       ()
     (
) ... compilable/test9680.sh
()()      () ( ... compilable\testdip1008.d       -preview=dip1008 ())      -cov -Icompilable/imports ...  ... compilable/testcolor.sh

  ... compilable\testDIP37.d         -Icompilable/extra-files ()()compilable\testdefines.d       ()
 ... compilable\testDIP37a.d        -Icompilable/extra-files ()


 ... compilable\testDIP37_10302.d   -Icompilable/extra-files () ... compilable\testDIP37_10421.d   -Icompilable/extra-files ()
 ... compilable\testDIP37_10354.d   -o- -Icompilable/extra-files ()
 ... compilable\testDIP42.d         -o- ()



 ... compilable\testexpression.d    () ... compilable\testfptr.d    ... compilable\testfwdref.d        ()  ...
 ... compilable\testheader1.d      ...   -o- -unittest -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader1.di -ignore   compilable\testheader12567a.d (-d -dw)

compilable\testgotoskips.d     () -o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader12567a.di ()    ( ...  ...
 ... compilable\testheader1i.d      -o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader1i.di -inline -ignore (-d -dw))compilable\testheader12567b.dcompilable\testheader17125.d


 ... compilable\testheader2.d       -o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader2.di ()-o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader12567b.di () ... compilable\testheader3.d       -o- -unittest -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader3.di (-d -dw) ...  -o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader17125.di ()compilable\testheaderudamodule.d -o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheaderudamodule.di
 ... (
compilable\testheader2i.d    )
 ...   -o- -H -HfC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/testheader2i.di -inline ( ... )compilable\testhelp.d    compilable\testimport12242.d
   ()


  ...  ... compilable\testlambdacomp.d    ()     -conf=compilable/extra-files/empty.conf --help () ... compilable\testnoinline.ccompilable\testInference.d
   ... compilable\testlibmain.d       -lib ()     ()  ...    () ... compilable\testparse.d         -o- ()compilable\testos.d
    ...  ... compilable\testprofile.d
compilable\testpostblit.d  -profile  (-os=host -os=linux -os=osx -os=freebsd -os=solaris)!!! [DISABLED on win64]
 ()



 ... compilable\testsctreturn.d     -preview=dip1000 ()   ... compilable\testpull1810.d   ...  ... ()compilable\testtempl2.d
compilable\test_issue23367.d ... compilable\testVRP.d           (-O -inline) ... compilable\test_isOverlapped.d ()   ()        ()



 ... compilable\test_nosharedaccess_ctor_nested_new.d -preview=nosharedaccess
  ... compilable\test_uda_tuple_propagation.d () ... compilable\test__ctfeWrite.d   () ... compilable\test_nosharedaccess_shared_struct_literal.d -preview=nosharedaccess ()   ...
 compilable\traits.d            -extern-std=c++98
 ...  ()compilable\traits_getFunctionAttributes.d () ( ... compilable\transitionhelp.d    () )

 ... compilable\transition_in.d     -transition=in ()-c -w

( ... compilable\ucn.d  ... )

   ...  ... compilable\uda.d   compilable\typeid_name.d

 ... compilable\udamodule1.d            -Icompilable/extra-files ()   compilable\udamodule2.d       ...

 ...  -verrors=simple ()compilable\union_initialization.d
            ()( ... compilable\unittest_roots.d) ... compilable\user_defined_attributes.d compilable\uda_lambda.d
      ... compilable\valist.c            ()() -i -unittest-roots ()
       (  ...   ( ... compilable\vararg.dcompilable\vcg-ast-arraylength.d ))       ()

            () ... compilable\vastartend.c        ()-vcg-ast -o-  ... compilable\vcg-ast-lowering.d(


 ... compilable\vcg-ast.d           -vcg-ast -o- ())
 -vcg-ast -o- ... compilable\vcg_ast_compilable.d -vcg-ast -o- ()
  ...  ... compilable\version_d_optimized.d ()compilable\verrors_spec.d( ...  ... compilable\version.d)
compilable\vgc1.d              -vgc -o- ()

    ... compilable\vgc2.d
 ...  ... compilable\vgc3.d              -vgc -o- ()     ...    -vgc -o- ()
compilable\vtemplates.d        ()
compilable\vtemplates_list.d

   -vtemplates=list-instances ()      -vtemplates   ...
( compilable\warn12809.d     ...
)  ... compilable\warn3882.d -verrors=simple -verrors=spec  ... compilable\zerosize2.d         () ... compilable\__fastdfa_escape_test.d -preview=fastdfa
compilable\zerosize.d          () ()

 ()       (-w -wi -debug)   -o- -w ()









FAILED targets:
- compilable\test21331.sh
- compilable\test21359.sh
make: *** [Makefile:168: dmd-pgo] Error 1

The build log looks kinda weird in general; might be related to Git Bash or not — dunno.

0xEAB · 2026-09-17T20:11:51Z

Update

I’ve been able to extract an actual error message from the mosaic:

compilable\test21495.d ... compilable/test21359.sh
      ... compilable\test21501a.d
       ( ...      )()
   ()compilable\test21504a.d  ...  compilable\test21504b.d   ()        ()

        ... compilable\test21514.d         -verrors=simple ()!!! [DISABLED on win64]
 ()    ... compilable\test21543.d

    ()       ...  ... () ... compilable\test21659.d  compilable\test21591.dcompilable\test21637.d     ... compilable\test21661.d         ()




  ...    ... compilable\test21668.d (  ==============================
 compilable\test21667.d ... Test compilable/test21331.sh failed. The logged output:
 ) ... compilable\test21672.d
  () ...   compilable\test21680.d
 compilable\test21707.d   ()         -vcg-ast -o- ()        ()    ... compilable\test21719.d compilable/test21331.sh: line 6: nm: command not found

  (
 ... compilable\test21740.i         ()) ==============================

Test compilable/test21331.sh failed. The xtrace output:

 ...     () ... compilable\test21743.d         ()     -mcpu=avx2 -O () ()
compilable\test21742.d + source compilable/test21331.sh
++ objfile='C:\Users\pi\dev\dmd\compiler\test\test_results/compilable/test21331.obj'
++ ../../generated/windows/release/64/dmd.exe -c -m64 '-ofC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/test21331.obj' compilable/extra-files/test21331.d
++ nm 'C:\Users\pi\dev\dmd\compiler\test\test_results/compilable/test21331.obj'
++ grep -E 16__lambda_L13_C17.+15__lambda_L5_C19
+ finish 1
+ set +x
 ...

 ==============================
Test compilable/test21359.sh failed. The logged output:
 compilable\test21753.d ...  ... compilable\test21806.d
compilable\test21794.d  ...      (   compilable\test21802.dcompilable/test21359.sh: line 9: nm: command not found
 )




  ==============================

 Test compilable/test21359.sh failed. The xtrace output:
   ...     ... compilable\test21812.d   ()  ()     ...     ... (compilable\test21839.d compilable\test21828.d              (   (  ...  ))) ...  compilable\test21832.d  ...  compilable\test21835.d+ source compilable/test21359.sh
++ objfile='C:\Users\pi\dev\dmd\compiler\test\test_results/compilable/test21359.obj'
++ ../../generated/windows/release/64/dmd.exe -c -m64 -allinst '-ofC:\Users\pi\dev\dmd\compiler\test\test_results/compilable/test21359.obj' compilable/extra-files/test21359a.d -Icompilable/extra-files
++ mangled_sym=_D14TypeInfo_HiAya6__initZ
+++ nm --defined-only 'C:\Users\pi\dev\dmd\compiler\test\test_results/compilable/test21359.obj'
+++ grep -F _D14TypeInfo_HiAya6__initZ
+++ wc -l
++ num_typeinfo_syms=0
+ finish 1
+ set +x
 ... compilable\test21830.d         -de -unittest ()  ()        -de ()     >>> TARGET FAILED:
compilable\test21850.d      compilable\test21331.sh

0xEAB

If the primary advantage of this change is to make it work on Windows, too, I’d prefer if that would be resolved first.

0xEAB · 2026-09-17T22:18:18Z

The test that currently uses nm should probably use dumpbin.exe instead.

kinke · 2026-09-17T22:22:46Z

No, the primary aim is to make the dmd-pgo compiler faster. (And eventually use it for the official DMD releases too.)

Your newest Windows fails are interesting, because they are 100% unrelated compilable test suite problems. These 2 *.sh tests assume the nm tool is available; I was sure these *.sh tests are skipped on Windows in general, but it looks like they might accidentally run and work for Windows CI here, with some apparently compatible nm tool in PATH in the GitHub Actions Windows image. [Other .sh tests use something like if [[ $OS = *"win"* ]]; then exit 0; fi to be skipped on Windows.]

So at least you don't get a failure for the instrumented-compiler build anymore (#23858 (comment)). Edit: So this is progress.

a reason

0xEAB · 2026-09-18T00:51:05Z

@abulgit

I'd re-run the perf workflow once or may be twice, just to make sure it's as deterministic as it was with the frontend profile. Otherwise pretty good result!

Done. 2nd re-run is still active, but the first one suggests that the performance changes are quite deterministic.

kinke · 2026-09-18T13:34:12Z

The dmd-pgo target now works (with LDC v1.43 at least) for all DMD-supported platforms, except for 32-bit Linux (only because the official LDC Linux package doesn't have any 32-bit compiler-rt libraries, required for PGO): dlang/installer#748

On Mac, an extra HOST_DFLAGS=-wi might be required to ignore a few unexpected profile-mismatch warnings. At least is for the CI run on macOS 14.

abulgit · 2026-09-21T19:10:05Z

The Benchmark Dashboard is Live now at - https://perf.dlang.org
Please Have a Look @kinke @0xEAB

kinke added the Infrastructure:Build System label Sep 17, 2026

Build: Switch from frontend-PGO to IR-PGO

2c9ff5f

I.e., from LDC's own AST-based implementation (-fprofile-instr-…) to LLVM-builtin IR-based PGO (-fprofile-…). With the downside of having to enable optimizations for the instrumented compiler (=> longer build time).

kinke force-pushed the pgo_ir_based branch from 41a797d to 2c9ff5f Compare September 17, 2026 02:05

kinke marked this pull request as ready for review September 17, 2026 16:22

abulgit approved these changes Sep 17, 2026

View reviewed changes

0xEAB previously requested changes Sep 17, 2026

View reviewed changes

0xEAB approved these changes Sep 17, 2026

View reviewed changes

kinke merged commit a374bfc into dlang:master Sep 17, 2026
45 checks passed

kinke deleted the pgo_ir_based branch September 17, 2026 22:54

kinke mentioned this pull request Sep 17, 2026

Windows: Skip 3 .sh 'compilable' tests depending on nm, if nm isn't in PATH #23873

Merged

Navigation Menu

Repository navigation

Build: Switch from frontend-PGO to IR-PGO - #23870

Build: Switch from frontend-PGO to IR-PGO#23870
kinke merged 1 commit into
dlang:masterfrom
kinke:pgo_ir_based

kinke commented Sep 17, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 17, 2026 •

edited

Loading

Uh oh!

kinke commented Sep 17, 2026

Uh oh!

abulgit commented Sep 17, 2026

Uh oh!

kinke commented Sep 17, 2026

Uh oh!

0xEAB commented Sep 17, 2026 •

edited

Loading

Uh oh!

0xEAB commented Sep 17, 2026

Uh oh!

0xEAB left a comment

Uh oh!

0xEAB commented Sep 17, 2026

Uh oh!

kinke commented Sep 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

0xEAB commented Sep 18, 2026

Uh oh!

kinke commented Sep 18, 2026

Uh oh!

abulgit commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kinke commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DMD perf check

Uh oh!

kinke commented Sep 17, 2026

Uh oh!

abulgit commented Sep 17, 2026

Uh oh!

kinke commented Sep 17, 2026

Uh oh!

0xEAB commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xEAB commented Sep 17, 2026

Update

Uh oh!

0xEAB left a comment

Choose a reason for hiding this comment

Uh oh!

0xEAB commented Sep 17, 2026

Uh oh!

kinke commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

0xEAB commented Sep 18, 2026

Uh oh!

kinke commented Sep 18, 2026

Uh oh!

abulgit commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kinke commented Sep 17, 2026 •

edited

Loading

github-actions Bot commented Sep 17, 2026 •

edited

Loading

0xEAB commented Sep 17, 2026 •

edited

Loading

kinke commented Sep 17, 2026 •

edited

Loading