Benchmark kernel launch overhead and partitioning by giordano · Pull Request #774 · JuliaGPU/KernelAbstractions.jl

giordano · 2026-09-11T22:42:00Z

The benchmark suite only measured saxpy throughput, so changes to the host side of a launch went unnoticed: a warm CPU launch allocated about 50 times and the per-PR benchmark comment could not show it. Add a `launch` group timing a single-workgroup launch for the static and dynamic workgroup-size and ndrange combinations in one and three dimensions, with the kernel constructed in the setup and the backend synchronized in the teardown so that only the launch is measured, and a `partition` group timing the host-side partitioning of the iteration space on its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VHciC8x39gm97sABrSvBkt

The launch benchmarks measure a launch that the CPU backend executes synchronously, so their time is dominated by the OpenCL runtime and has a spread of about 30%, while the host-side work of a launch shows up as allocations: a regression there changes the count by tens of allocations per launch and is invisible in the time table. `benchpkgtable` can print the allocation and memory table next to the time table. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VHciC8x39gm97sABrSvBkt

github-actions · 2026-09-11T23:40:21Z

Benchmark Results

Show table

	main	`00c47a3`...	main / `00c47a3`...
saxpy/default/Float16/1024	0.0828 ± 0.012 ms	0.0836 ± 0.013 ms	0.991 ± 0.21
saxpy/default/Float16/1048576	0.781 ± 0.029 ms	0.784 ± 0.029 ms	0.996 ± 0.053
saxpy/default/Float16/16384	0.0779 ± 0.033 ms	0.0728 ± 0.033 ms	1.07 ± 0.66
saxpy/default/Float16/2048	0.0841 ± 0.022 ms	0.0847 ± 0.025 ms	0.993 ± 0.39
saxpy/default/Float16/256	0.082 ± 0.015 ms	0.0828 ± 0.013 ms	0.99 ± 0.24
saxpy/default/Float16/262144	0.245 ± 0.032 ms	0.245 ± 0.031 ms	1 ± 0.18
saxpy/default/Float16/32768	0.0813 ± 0.032 ms	0.0815 ± 0.032 ms	0.997 ± 0.55
saxpy/default/Float16/4096	0.0863 ± 0.029 ms	0.0872 ± 0.029 ms	0.99 ± 0.47
saxpy/default/Float16/512	0.0823 ± 0.017 ms	0.0831 ± 0.014 ms	0.991 ± 0.26
saxpy/default/Float16/64	0.0822 ± 0.015 ms	0.0823 ± 0.027 ms	0.999 ± 0.38
saxpy/default/Float16/65536	0.102 ± 0.028 ms	0.102 ± 0.028 ms	0.996 ± 0.39
saxpy/default/Float32/1024	0.0814 ± 0.01 ms	0.0824 ± 0.0095 ms	0.987 ± 0.17
saxpy/default/Float32/1048576	0.485 ± 0.024 ms	0.49 ± 0.023 ms	0.99 ± 0.069
saxpy/default/Float32/16384	0.0629 ± 0.032 ms	0.0638 ± 0.031 ms	0.986 ± 0.7
saxpy/default/Float32/2048	0.0813 ± 0.026 ms	0.0825 ± 0.026 ms	0.986 ± 0.45
saxpy/default/Float32/256	0.0819 ± 0.013 ms	0.0827 ± 0.018 ms	0.989 ± 0.26
saxpy/default/Float32/262144	0.172 ± 0.029 ms	0.173 ± 0.029 ms	0.994 ± 0.24
saxpy/default/Float32/32768	0.0703 ± 0.03 ms	0.0709 ± 0.03 ms	0.992 ± 0.6
saxpy/default/Float32/4096	0.0777 ± 0.031 ms	0.0781 ± 0.031 ms	0.994 ± 0.55
saxpy/default/Float32/512	0.0812 ± 0.018 ms	0.0823 ± 0.011 ms	0.986 ± 0.25
saxpy/default/Float32/64	0.0819 ± 0.019 ms	0.0833 ± 0.012 ms	0.984 ± 0.28
saxpy/default/Float32/65536	0.0829 ± 0.029 ms	0.0839 ± 0.029 ms	0.988 ± 0.49
saxpy/default/Float64/1024	0.0815 ± 0.013 ms	0.0825 ± 0.0097 ms	0.987 ± 0.19
saxpy/default/Float64/1048576	0.55 ± 0.079 ms	0.583 ± 0.089 ms	0.945 ± 0.2
saxpy/default/Float64/16384	0.0661 ± 0.03 ms	0.066 ± 0.029 ms	1 ± 0.64
saxpy/default/Float64/2048	0.0812 ± 0.029 ms	0.0831 ± 0.028 ms	0.976 ± 0.48
saxpy/default/Float64/256	0.0817 ± 0.015 ms	0.0825 ± 0.01 ms	0.99 ± 0.22
saxpy/default/Float64/262144	0.189 ± 0.035 ms	0.191 ± 0.035 ms	0.989 ± 0.26
saxpy/default/Float64/32768	0.0731 ± 0.029 ms	0.0738 ± 0.028 ms	0.991 ± 0.54
saxpy/default/Float64/4096	0.0768 ± 0.029 ms	0.077 ± 0.031 ms	0.997 ± 0.55
saxpy/default/Float64/512	0.0816 ± 0.014 ms	0.0825 ± 0.013 ms	0.989 ± 0.23
saxpy/default/Float64/64	0.0824 ± 0.015 ms	0.0833 ± 0.015 ms	0.989 ± 0.25
saxpy/default/Float64/65536	0.0914 ± 0.029 ms	0.0925 ± 0.03 ms	0.987 ± 0.45
saxpy/static workgroup=(1024,)/Float16/1024	0.0804 ± 0.018 ms	0.0814 ± 0.013 ms	0.989 ± 0.27
saxpy/static workgroup=(1024,)/Float16/1048576	0.779 ± 0.029 ms	0.781 ± 0.03 ms	0.998 ± 0.053
saxpy/static workgroup=(1024,)/Float16/16384	0.0691 ± 0.032 ms	0.069 ± 0.033 ms	1 ± 0.67
saxpy/static workgroup=(1024,)/Float16/2048	0.0815 ± 0.027 ms	0.0826 ± 0.028 ms	0.987 ± 0.47
saxpy/static workgroup=(1024,)/Float16/256	0.0805 ± 0.015 ms	0.0807 ± 0.013 ms	0.998 ± 0.25
saxpy/static workgroup=(1024,)/Float16/262144	0.24 ± 0.03 ms	0.239 ± 0.031 ms	1 ± 0.18
saxpy/static workgroup=(1024,)/Float16/32768	0.0772 ± 0.03 ms	0.0776 ± 0.03 ms	0.995 ± 0.55
saxpy/static workgroup=(1024,)/Float16/4096	0.0837 ± 0.031 ms	0.0848 ± 0.031 ms	0.987 ± 0.51
saxpy/static workgroup=(1024,)/Float16/512	0.08 ± 0.016 ms	0.0808 ± 0.018 ms	0.991 ± 0.3
saxpy/static workgroup=(1024,)/Float16/64	0.08 ± 0.015 ms	0.0806 ± 0.015 ms	0.992 ± 0.26
saxpy/static workgroup=(1024,)/Float16/65536	0.098 ± 0.027 ms	0.0988 ± 0.027 ms	0.992 ± 0.39
saxpy/static workgroup=(1024,)/Float32/1024	0.0792 ± 0.012 ms	0.08 ± 0.013 ms	0.99 ± 0.22
saxpy/static workgroup=(1024,)/Float32/1048576	0.488 ± 0.024 ms	0.487 ± 0.025 ms	1 ± 0.072
saxpy/static workgroup=(1024,)/Float32/16384	0.0607 ± 0.029 ms	0.0614 ± 0.029 ms	0.988 ± 0.67
saxpy/static workgroup=(1024,)/Float32/2048	0.0788 ± 0.028 ms	0.0791 ± 0.029 ms	0.996 ± 0.5
saxpy/static workgroup=(1024,)/Float32/256	0.0795 ± 0.022 ms	0.0806 ± 0.021 ms	0.987 ± 0.37
saxpy/static workgroup=(1024,)/Float32/262144	0.17 ± 0.028 ms	0.17 ± 0.029 ms	0.999 ± 0.24
saxpy/static workgroup=(1024,)/Float32/32768	0.067 ± 0.028 ms	0.0681 ± 0.028 ms	0.983 ± 0.58
saxpy/static workgroup=(1024,)/Float32/4096	0.075 ± 0.032 ms	0.0749 ± 0.031 ms	1 ± 0.59
saxpy/static workgroup=(1024,)/Float32/512	0.0792 ± 0.014 ms	0.0802 ± 0.011 ms	0.987 ± 0.22
saxpy/static workgroup=(1024,)/Float32/64	0.0801 ± 0.014 ms	0.0808 ± 0.015 ms	0.991 ± 0.25
saxpy/static workgroup=(1024,)/Float32/65536	0.0814 ± 0.028 ms	0.0812 ± 0.028 ms	1 ± 0.49
saxpy/static workgroup=(1024,)/Float64/1024	0.0792 ± 0.021 ms	0.0801 ± 0.012 ms	0.989 ± 0.3
saxpy/static workgroup=(1024,)/Float64/1048576	0.564 ± 0.076 ms	0.552 ± 0.057 ms	1.02 ± 0.17
saxpy/static workgroup=(1024,)/Float64/16384	0.0613 ± 0.026 ms	0.063 ± 0.028 ms	0.974 ± 0.6
saxpy/static workgroup=(1024,)/Float64/2048	0.0791 ± 0.029 ms	0.0781 ± 0.031 ms	1.01 ± 0.55
saxpy/static workgroup=(1024,)/Float64/256	0.0803 ± 0.015 ms	0.0807 ± 0.013 ms	0.995 ± 0.24
saxpy/static workgroup=(1024,)/Float64/262144	0.188 ± 0.033 ms	0.189 ± 0.032 ms	0.994 ± 0.24
saxpy/static workgroup=(1024,)/Float64/32768	0.0719 ± 0.027 ms	0.071 ± 0.027 ms	1.01 ± 0.54
saxpy/static workgroup=(1024,)/Float64/4096	0.0723 ± 0.03 ms	0.0728 ± 0.032 ms	0.993 ± 0.6
saxpy/static workgroup=(1024,)/Float64/512	0.0798 ± 0.013 ms	0.08 ± 0.011 ms	0.997 ± 0.21
saxpy/static workgroup=(1024,)/Float64/64	0.0804 ± 0.015 ms	0.0807 ± 0.015 ms	0.996 ± 0.26
saxpy/static workgroup=(1024,)/Float64/65536	0.0901 ± 0.028 ms	0.09 ± 0.029 ms	1 ± 0.45
time_to_load	0.756 ± 0.0045 s	0.767 ± 0.0072 s	0.985 ± 0.011

	main	`00c47a3`...	main / `00c47a3`...
saxpy/default/Float16/1024	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/1048576	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/16384	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/2048	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/256	0.064 k allocs: 3.22 kB	0.064 k allocs: 3.22 kB	1
saxpy/default/Float16/262144	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/32768	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/4096	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/512	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float16/64	0.064 k allocs: 3.22 kB	0.064 k allocs: 3.22 kB	1
saxpy/default/Float16/65536	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/1024	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/1048576	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/16384	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/2048	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/256	0.064 k allocs: 3.22 kB	0.064 k allocs: 3.22 kB	1
saxpy/default/Float32/262144	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/32768	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/4096	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/512	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float32/64	0.064 k allocs: 3.22 kB	0.064 k allocs: 3.22 kB	1
saxpy/default/Float32/65536	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/1024	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/1048576	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/16384	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/2048	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/256	0.064 k allocs: 3.22 kB	0.064 k allocs: 3.22 kB	1
saxpy/default/Float64/262144	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/32768	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/4096	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/512	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/default/Float64/64	0.064 k allocs: 3.22 kB	0.064 k allocs: 3.22 kB	1
saxpy/default/Float64/65536	0.074 k allocs: 3.38 kB	0.074 k allocs: 3.38 kB	1
saxpy/static workgroup=(1024,)/Float16/1024	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/1048576	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/16384	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/2048	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/256	0.056 k allocs: 3.02 kB	0.056 k allocs: 3.02 kB	1
saxpy/static workgroup=(1024,)/Float16/262144	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/32768	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/4096	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/512	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float16/64	0.056 k allocs: 3.02 kB	0.056 k allocs: 3.02 kB	1
saxpy/static workgroup=(1024,)/Float16/65536	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/1024	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/1048576	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/16384	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/2048	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/256	0.056 k allocs: 3.02 kB	0.056 k allocs: 3.02 kB	1
saxpy/static workgroup=(1024,)/Float32/262144	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/32768	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/4096	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/512	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float32/64	0.056 k allocs: 3.02 kB	0.056 k allocs: 3.02 kB	1
saxpy/static workgroup=(1024,)/Float32/65536	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/1024	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/1048576	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/16384	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/2048	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/256	0.056 k allocs: 3.02 kB	0.056 k allocs: 3.02 kB	1
saxpy/static workgroup=(1024,)/Float64/262144	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/32768	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/4096	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/512	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
saxpy/static workgroup=(1024,)/Float64/64	0.056 k allocs: 3.02 kB	0.056 k allocs: 3.02 kB	1
saxpy/static workgroup=(1024,)/Float64/65536	0.059 k allocs: 3.06 kB	0.059 k allocs: 3.06 kB	1
time_to_load	0.201 k allocs: 11.8 kB	0.201 k allocs: 11.8 kB	1

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

giordano added continuous integration (ci) github_actions Pull requests that update GitHub Actions code labels Sep 11, 2026

giordano and others added 2 commits September 11, 2026 23:42

giordano force-pushed the mg/launch-benchmarks branch from b2927d4 to 00c47a3 Compare September 11, 2026 22:42

giordano mentioned this pull request Sep 11, 2026

Cut allocations when launching CPU kernels #768

Merged

vchuravy merged commit 7aabc96 into main Sep 12, 2026
57 of 82 checks passed

vchuravy deleted the mg/launch-benchmarks branch September 12, 2026 17:24

Navigation Menu

Benchmark kernel launch overhead and partitioning - #774

Benchmark kernel launch overhead and partitioning#774
vchuravy merged 2 commits into
mainfrom
mg/launch-benchmarks

giordano commented Sep 11, 2026

Uh oh!

github-actions Bot commented Sep 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

giordano commented Sep 11, 2026

Uh oh!

github-actions Bot commented Sep 11, 2026

Benchmark Results

Benchmark Plots

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants