Skip to content

Launch kernels over a list of indices - #765

Closed
giordano wants to merge 2 commits into
JuliaGPU:mainfrom
giordano:mg/ndrange-index-map
Closed

giordano wants to merge 2 commits into
JuliaGPU:mainfrom
giordano:mg/ndrange-index-map

Conversation

@giordano

@giordano giordano commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Part 2/2 of upstreaming Oceananigans piracy of KA (part 1 is #764). Ref: #757. CC @simone-silvestri.

ndrange may now be a device vector of CartesianIndex{N} or
NTuple{N, <:Integer} elements. Each work item handles one listed index:
@index(Global, Cartesian) and @index(Global, NTuple) return it,
@index(Global, Linear) its position in the vector, @ndrange() the
vector's length. The kernel must have a dynamic ndrange and a 1-D
workgroup size, static or given at launch; the partial last workgroup is
bounds-checked against the vector's length.

This is how Oceananigans iterates over the active cells of an immersed
boundary grid. It currently wraps the kernel function, pirates
Base.getproperty on Kernel, Adapt.adapt_structure on NDRange
and CompilerMetadata, partition, expand, and adds one
@device_override __validindex per GPU backend, which the compiled CPU
backend's own ::Any override shadows.

Implementation:

  • IndexMap{N} wraps the vector and yields CartesianIndex{N} on
    indexing; it is the mapping of a 1-D NDRange (MappedNDRange).
    expand and linear_index on a MappedNDRange look the index up.
  • partition dispatches vectors to mapped_partition, which validates
    the kernel and workgroup size and partitions (length(map),); the
    dynamic-check flag is computed as for any other ndrange.
  • Index validity moves into the generic __validindex(ctx, groupidx, idx), which dispatches on the iteration space (I in ndrange for
    regular spaces, position <= length(map) for index maps). The 1-arg
    __validindex(ctx) is now a plain method built on
    KernelInterface.get_group_id/get_local_id, so the POCL override is
    removed; __index_Global_Linear goes through __global_linear in the
    same way. Backends that still override __validindex(ctx) keep
    working for regular ndranges and should forward to the 3-arg method
    to support index maps.
  • Adapt.adapt_structure for NDRange, IndexMap and
    CompilerMetadata moves the vector to the device with the kernel
    arguments; CompilerMetadata treats a vector ndrange as
    CartesianIndices((length(v),)).

giordano and others added 2 commits September 10, 2026 11:48
Kernels can now iterate over a region whose indices do not start at 1.
Each entry of `ndrange` may be a range instead of an extent, and the
whole `ndrange` may be a single range or a `CartesianIndices`, both
statically (`kernel(backend, workgroupsize, (-2:N+3, 0:M+1))`) and at
launch (`ndrange=(-2:N+3, 0:M+1)`). `@index(Global, Cartesian)` and
`@index(Global, NTuple)` return the shifted indices, `@index(Global,
Linear)` counts the region from 1 in column-major order, `@ndrange()`
returns the extents.

Downstream packages such as Oceananigans implement this today by
pirating `partition`, `expand`, `__ndrange` and `__groupsize` with a
custom `_Size` subtype smuggled into `NDRange`'s dynamic-workitems type
parameter, which is fragile and broke with the compiled CPU backend.

Implementation:
- `StaticSize` stores `UnitRange{Int}` axes next to `Int` extents;
  `StaticSize(ranges)` and `StaticSize(::CartesianIndices)` normalise
  their input, `Base.OneTo` axes become plain extents.
- `NDRange` gains a `mapping` field holding a `StaticOffset` or
  `DynamicOffset` (or `nothing`), which `expand` adds to the blocked
  index. Plain-size launches keep `mapping === nothing`, so their
  `NDRange` types are unchanged.
- `partition` normalises `ndrange`/`workgroupsize` (integer, range,
  tuple, `CartesianIndices`) and compares static and launch ndranges by
  extents and offsets, so `(1:128,)` matches a static `(128,)`.
- `CompilerMetadata` keeps an offset `CartesianIndices` as `ndrange`
  (`CartesianIndices(::CartesianIndices)` would drop the offsets), so
  the existing `I in __ndrange(ctx)` bounds check is correct.
- The global linear index is computed by `linear_index`, since
  `LinearIndices` only supports 1-based axes.
- The POCL autotune path uses `extents(ndrange)`.

Offsets need no backend changes as long as the workgroup size is static
or given at launch; autotuning a dynamic workgroup size from a range
`ndrange` requires backends to call `extents` where they use
`prod(ndrange)`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHciC8x39gm97sABrSvBkt
`ndrange` may now be a device vector of `CartesianIndex{N}` or
`NTuple{N, <:Integer}` elements. Each work item handles one listed index:
`@index(Global, Cartesian)` and `@index(Global, NTuple)` return it,
`@index(Global, Linear)` its position in the vector, `@ndrange()` the
vector's length. The kernel must have a dynamic `ndrange` and a 1-D
workgroup size, static or given at launch; the partial last workgroup is
bounds-checked against the vector's length.

This is how Oceananigans iterates over the active cells of an immersed
boundary grid. It currently wraps the kernel function, pirates
`Base.getproperty` on `Kernel`, `Adapt.adapt_structure` on `NDRange`
and `CompilerMetadata`, `partition`, `expand`, and adds one
`@device_override __validindex` per GPU backend, which the compiled CPU
backend's own `::Any` override shadows.

Implementation:
- `IndexMap{N}` wraps the vector and yields `CartesianIndex{N}` on
  indexing; it is the `mapping` of a 1-D `NDRange` (`MappedNDRange`).
  `expand` and `linear_index` on a `MappedNDRange` look the index up.
- `partition` dispatches vectors to `mapped_partition`, which validates
  the kernel and workgroup size and partitions `(length(map),)`; the
  dynamic-check flag is computed as for any other ndrange.
- Index validity moves into the generic `__validindex(ctx, groupidx,
  idx)`, which dispatches on the iteration space (`I in ndrange` for
  regular spaces, position `<= length(map)` for index maps). The 1-arg
  `__validindex(ctx)` is now a plain method built on
  `KernelInterface.get_group_id`/`get_local_id`, so the POCL override is
  removed; `__index_Global_Linear` goes through `__global_linear` in the
  same way. Backends that still override `__validindex(ctx)` keep
  working for regular ndranges and should forward to the 3-arg method
  to support index maps.
- `Adapt.adapt_structure` for `NDRange`, `IndexMap` and
  `CompilerMetadata` moves the vector to the device with the kernel
  arguments; `CompilerMetadata` treats a vector `ndrange` as
  `CartesianIndices((length(v),))`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHciC8x39gm97sABrSvBkt
@vchuravy

Copy link
Copy Markdown
Member
  • ndrange may be a device vector of CartesianIndex/NTuple indices, running one work item per
    listed index (kernel(A, ndrange=active_cells)).

I think that's a step too far.

The user could have simply written:

ndrange=length(active_cells) and then done. idx = active_cells(@index Global)

@giordano giordano changed the title Accept index ranges in ndrange Launch kernels over a list of indices Sep 11, 2026
@giordano

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #772.

@giordano giordano closed this Sep 11, 2026
@giordano
giordano deleted the mg/ndrange-index-map branch September 11, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants