Skip to content

Define neutral elements for the @fastmath reduction operators - #840

Open
unsound-mint wants to merge 2 commits into
JuliaGPU:mainfrom
unsound-mint:perf-833-fastmath-neutral-element
Open

unsound-mint wants to merge 2 commits into
JuliaGPU:mainfrom
unsound-mint:perf-833-fastmath-neutral-element

Conversation

@unsound-mint

@unsound-mint unsound-mint commented Oct 26, 2026 •

Copy link
Copy Markdown

Fixes #833.

@fastmath replaces max, min, + and * with max_fast, min_fast, add_fast and mul_fast. Reductions with these were slower than with the plain operators, for two reasons:

  • neutral_element had no methods for them, so AcceleratedKernels seeded every partial result with an empty lane instead. The first commit adds the four methods to GPUArraysCore. That makes this a GPUArraysCore 0.2.2 release, with the pin in the root Project.toml updated as in Move neutral_element to GPUArraysCore #789.
  • Since Reduce with operators not known to commute in element order #822, only operators known to commute use AcceleratedKernels' unordered kernels. The second commit adds the @fastmath operators to _commutative, on the same types as the operators they replace. Strictly they commute only up to NaNs and the sign of zero, which @fastmath may ignore anyway.

Times in µs on an RTX 5080, @fastmath reduce(op, x; dims, init) against the plain reduce(op, ...), L2 cache flushed, minimum of 20 runs:

case max max_fast on main max_fast with this PR
Float32 1024×1024×96, dims=1 471 940 471
BFloat16 1024×1024×96, dims=1 442 926 442
Float32 768×8192, dims=2 37 72 37

add_fast and scalar (dims=:) reductions behaved the same way. The main column is from before #822, when BFloat16 max took 242 µs. BFloat16 isn't an IEEEFloat, so #822 sends its plain reductions down the ordered path too. That's a separate issue.

One visible difference: a @fastmath sum of -0.0 values now returns 0.0, because add_fast is seeded with +0.0. @fastmath permits this.

Tests:

  • the neutral elements of the four operators (these fail on main);
  • _commutative for the @fastmath operators;
  • one @fastmath reduction per operator and real element type, compared against the CPU.

@maleadt
maleadt force-pushed the perf-833-fastmath-neutral-element branch from a327728 to 119ba7c Compare October 27, 2026 20:07
unsound-mint and others added 2 commits October 27, 2026 21:07
`@fastmath reduce(max, x; dims, init)` reduces with `Base.FastMath.max_fast`,
which had no `neutral_element` method. AcceleratedKernels then falls back to
its slower empty-lane path even though `init` is given. Add methods for
`add_fast`, `mul_fast`, `min_fast` and `max_fast`, matching the operators
they replace.

On an RTX 4070 Laptop GPU, `@fastmath reduce(max, x; dims=1, init=-Inf)`
on a BFloat16 1024x1024x96 array goes from 2.4x to 1.0x the time of
`reduce(max, ...)`, and on a 50304x8192 array from 1.7x to 1.0x.

Bump GPUArraysCore to 0.2.2, as GPUArrays pins its exact version.
Since JuliaGPU#822, only operators known to commute use AcceleratedKernels'
unordered kernels, and `add_fast`, `mul_fast`, `min_fast` and `max_fast`
were not among them, so `@fastmath` reductions took the slower ordered
path. They commute on the same types as the operators they replace, up
to NaNs and the sign of zero, which `@fastmath` may ignore anyway.
@maleadt
maleadt force-pushed the perf-833-fastmath-neutral-element branch from 119ba7c to f36a90f Compare October 27, 2026 20:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@fastmath reductions are 2–5× slower: no neutral_element for Base.FastMath.max_fast and co.

2 participants