Skip to content

[pocl] Enable float add and min/max atomics via cl_ext_float_atomics - #737

Draft
vchuravy wants to merge 2 commits into
mainfrom
vc/pocl-float-atomics
Draft

[pocl] Enable float add and min/max atomics via cl_ext_float_atomics#737
vchuravy wants to merge 2 commits into
mainfrom
vc/pocl-float-atomics

Conversation

@vchuravy

@vchuravy vchuravy commented Jul 27, 2026

Copy link
Copy Markdown
Member

Part of #600 (the last open item: float atomics) — fixes #671.

Problem

Atomix lowers @atomic A[i] += x and @atomic max(A[i], x) on floats to LLVM atomicrmw fadd / fmin / fmax. The SPIR-V backend refuses to translate a module containing them unless the guarding extension has been permitted, so on the CPU backend every float atomic fails to compile:

LLVM ERROR: The atomic float instruction requires the following SPIR-V extension: SPV_EXT_shader_atomic_float_add
LLVM ERROR: The atomic float instruction requires the following SPIR-V extension: SPV_EXT_shader_atomic_float_min_max

Enzyme's reverse mode hits the first one too, because it accumulates gradients with atomic fadd.

Change

  • nanoOpenCL: add the cl_ext_float_atomics constants and expose the per-precision capability bitfields as dev.single_fp_atomic_capabilities, dev.double_fp_atomic_capabilities, dev.half_fp_atomic_capabilities (zero when the extension is absent).
  • default_spirv_extensions(dev) permits SPV_EXT_shader_atomic_float_add and SPV_EXT_shader_atomic_float_min_max when the device natively supports the operation for Float32 or Float64, in both global and local memory (same rule as OpenCL.jl's has_fp_atomics). Listing an extension only permits it, so kernels without float atomics are unaffected. An explicit extensions= keyword still wins.
  • Testsuite: new Atomics entry running @atomic add/max/min for Int32, UInt32, Float32 and, where supports_float64, Float64 on every backend (skipped when supports_atomics is false). runtests.jl additionally checks that pocl's CPU device gets both extensions and that the override works.

pocl's CPU device reports native add and min/max atomics for single and double precision (global and local), and none for half.

Not in this PR

🤖 Generated with Claude Code

https://claude.ai/code/session_01DnYMbgSmZQr7KtRUwei63a

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main 0d0a4ad... main / 0d0a4ad...
saxpy/default/Float32/1024 0.0861 ± 0.008 ms 0.085 ± 0.0097 ms 1.01 ± 0.15
saxpy/default/Float32/1048576 0.5 ± 0.025 ms 0.489 ± 0.027 ms 1.02 ± 0.076
saxpy/default/Float32/16384 0.0685 ± 0.035 ms 0.066 ± 0.035 ms 1.04 ± 0.76
saxpy/default/Float32/2048 0.0834 ± 0.027 ms 0.0835 ± 0.027 ms 0.999 ± 0.46
saxpy/default/Float32/256 0.0863 ± 0.0097 ms 0.0857 ± 0.012 ms 1.01 ± 0.18
saxpy/default/Float32/262144 0.178 ± 0.031 ms 0.173 ± 0.03 ms 1.03 ± 0.25
saxpy/default/Float32/32768 0.0739 ± 0.031 ms 0.0721 ± 0.031 ms 1.02 ± 0.61
saxpy/default/Float32/4096 0.0831 ± 0.031 ms 0.085 ± 0.031 ms 0.978 ± 0.51
saxpy/default/Float32/512 0.0853 ± 0.012 ms 0.0852 ± 0.011 ms 1 ± 0.2
saxpy/default/Float32/64 0.087 ± 0.01 ms 0.0859 ± 0.013 ms 1.01 ± 0.2
saxpy/default/Float32/65536 0.09 ± 0.031 ms 0.0858 ± 0.031 ms 1.05 ± 0.53
saxpy/default/Float64/1024 0.085 ± 0.017 ms 0.0853 ± 0.014 ms 0.996 ± 0.26
saxpy/default/Float64/1048576 0.546 ± 0.08 ms 0.557 ± 0.098 ms 0.98 ± 0.22
saxpy/default/Float64/16384 0.0715 ± 0.031 ms 0.0687 ± 0.03 ms 1.04 ± 0.64
saxpy/default/Float64/2048 0.0841 ± 0.029 ms 0.0844 ± 0.029 ms 0.996 ± 0.48
saxpy/default/Float64/256 0.0855 ± 0.0079 ms 0.0859 ± 0.0093 ms 0.995 ± 0.14
saxpy/default/Float64/262144 0.194 ± 0.036 ms 0.192 ± 0.037 ms 1.01 ± 0.27
saxpy/default/Float64/32768 0.079 ± 0.03 ms 0.0763 ± 0.029 ms 1.03 ± 0.55
saxpy/default/Float64/4096 0.0811 ± 0.028 ms 0.0805 ± 0.03 ms 1.01 ± 0.51
saxpy/default/Float64/512 0.0853 ± 0.01 ms 0.0854 ± 0.013 ms 0.999 ± 0.19
saxpy/default/Float64/64 0.0863 ± 0.011 ms 0.0864 ± 0.011 ms 0.999 ± 0.18
saxpy/default/Float64/65536 0.0964 ± 0.03 ms 0.0954 ± 0.03 ms 1.01 ± 0.45
saxpy/static workgroup=(1024,)/Float32/1024 0.0831 ± 0.0099 ms 0.0827 ± 0.012 ms 1 ± 0.19
saxpy/static workgroup=(1024,)/Float32/1048576 0.425 ± 0.03 ms 0.422 ± 0.034 ms 1.01 ± 0.11
saxpy/static workgroup=(1024,)/Float32/16384 0.065 ± 0.032 ms 0.0631 ± 0.032 ms 1.03 ± 0.73
saxpy/static workgroup=(1024,)/Float32/2048 0.0814 ± 0.028 ms 0.083 ± 0.029 ms 0.981 ± 0.48
saxpy/static workgroup=(1024,)/Float32/256 0.0838 ± 0.012 ms 0.0837 ± 0.015 ms 1 ± 0.23
saxpy/static workgroup=(1024,)/Float32/262144 0.156 ± 0.032 ms 0.152 ± 0.033 ms 1.03 ± 0.31
saxpy/static workgroup=(1024,)/Float32/32768 0.0686 ± 0.03 ms 0.0681 ± 0.029 ms 1.01 ± 0.62
saxpy/static workgroup=(1024,)/Float32/4096 0.0818 ± 0.029 ms 0.0822 ± 0.03 ms 0.995 ± 0.51
saxpy/static workgroup=(1024,)/Float32/512 0.0833 ± 0.011 ms 0.0835 ± 0.0021 ms 0.997 ± 0.13
saxpy/static workgroup=(1024,)/Float32/64 0.0837 ± 0.012 ms 0.0842 ± 0.0048 ms 0.994 ± 0.16
saxpy/static workgroup=(1024,)/Float32/65536 0.0819 ± 0.032 ms 0.081 ± 0.032 ms 1.01 ± 0.56
saxpy/static workgroup=(1024,)/Float64/1024 0.0821 ± 0.018 ms 0.0827 ± 0.0036 ms 0.993 ± 0.22
saxpy/static workgroup=(1024,)/Float64/1048576 0.496 ± 0.084 ms 0.495 ± 0.093 ms 1 ± 0.25
saxpy/static workgroup=(1024,)/Float64/16384 0.0662 ± 0.029 ms 0.0635 ± 0.028 ms 1.04 ± 0.64
saxpy/static workgroup=(1024,)/Float64/2048 0.08 ± 0.03 ms 0.0829 ± 0.03 ms 0.964 ± 0.51
saxpy/static workgroup=(1024,)/Float64/256 0.0829 ± 0.011 ms 0.0832 ± 0.0025 ms 0.996 ± 0.13
saxpy/static workgroup=(1024,)/Float64/262144 0.185 ± 0.033 ms 0.186 ± 0.038 ms 0.991 ± 0.27
saxpy/static workgroup=(1024,)/Float64/32768 0.0745 ± 0.027 ms 0.0736 ± 0.028 ms 1.01 ± 0.53
saxpy/static workgroup=(1024,)/Float64/4096 0.0787 ± 0.029 ms 0.0744 ± 0.031 ms 1.06 ± 0.58
saxpy/static workgroup=(1024,)/Float64/512 0.0832 ± 0.0091 ms 0.083 ± 0.0023 ms 1 ± 0.11
saxpy/static workgroup=(1024,)/Float64/64 0.0835 ± 0.015 ms 0.0837 ± 0.005 ms 0.997 ± 0.19
saxpy/static workgroup=(1024,)/Float64/65536 0.0934 ± 0.027 ms 0.0913 ± 0.028 ms 1.02 ± 0.43
time_to_load 0.876 ± 0.00089 s 0.896 ± 0.025 s 0.979 ± 0.027

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@simeonschaub

Copy link
Copy Markdown
Member

Have you seen JuliaGPU/OpenCL.jl#453? Might want to copy some of the logic from there. SPV_EXT_shader_atomic_float_min_max might be useful too

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.89%. Comparing base (65690fd) to head (4cc0e6e).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #737      +/-   ##
==========================================
+ Coverage   64.77%   64.89%   +0.12%     
==========================================
  Files          24       24              
  Lines        2024     2031       +7     
==========================================
+ Hits         1311     1318       +7     
  Misses        713      713              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vchuravy vchuravy added this to the 0.10.0 milestone Sep 7, 2026
vchuravy and others added 2 commits September 7, 2026 21:16
The SPIR-V backend refuses to translate a module containing an atomic fadd
unless the extension guarding it has been listed:

    LLVM ERROR: The atomic float instruction requires the following SPIR-V
    extension: SPV_EXT_shader_atomic_float_add

Enzyme's reverse mode runs into this because it accumulates gradients with
atomic fadd, so reverse-mode AD over a POCL kernel fails to compile outright.

Derive the extension list from the device the way `supports_fp16`/`supports_fp64`
already are, keyed off the corresponding OpenCL extension (`cl_ext_float_atomics`,
which PoCL advertises). Listing an extension only permits it -- nothing is
emitted unless a module needs those instructions -- so kernels that do not use
float atomics are unaffected. An explicit `extensions=` keyword still wins, and
`default_spirv_extensions` gives later extensions an obvious home.

Verified end-to-end: with this change, reverse-mode Enzyme over a POCL kernel
compiles, runs and produces correct gradients.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…pabilities

Atomix lowers `@atomic A[i] += x` and `@atomic max(A[i], x)` on floats to
LLVM `atomicrmw fadd`/`fmin`/`fmax`, which the SPIR-V backend only
translates when SPV_EXT_shader_atomic_float_add resp.
SPV_EXT_shader_atomic_float_min_max is permitted. Derive both from the
per-precision cl_ext_float_atomics capability bitfields instead of the
bare extension string, requiring the global and local memory bits like
OpenCL.jl does, and expose the bitfields as device properties.

Adds an `Atomics` testsuite entry exercising add/min/max for Int32,
UInt32, Float32 and (where supported) Float64 on all backends, and a
POCL-specific check that both extensions are permitted on pocl's CPU
device while an explicit `extensions=` still wins.

Assisted-by: Claude Code (Fable 5.1)
@vchuravy
vchuravy force-pushed the vc/pocl-float-atomics branch from 4cc0e6e to 0d0a4ad Compare September 7, 2026 19:22
@vchuravy vchuravy changed the title [pocl] Enable SPV_EXT_shader_atomic_float_add where supported [pocl] Enable float add and min/max atomics via cl_ext_float_atomics Sep 7, 2026
@christiangnrd

Copy link
Copy Markdown
Member

CUDA failure on 1.10/1.11 seem real?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

POCL backend atomix

3 participants