Implement an abstract `AbstractOffsetArray` type by simone-silvestri · Pull Request #371 · JuliaArrays/OffsetArrays.jl

simone-silvestri · 2025-03-12T19:16:31Z

Addresses #370 by providing an extension point for packages that need to store offsets differently.

Adds OffsetArrays.AbstractOffsetArray{T,N}, an abstract supertype of OffsetArray, so that packages can store offsets differently (narrower integers, type parameters) while reusing OffsetArrays' indexing, axes and printing machinery.

Interface. A subtype defines Base.parent, OffsetArrays.offsets (an NTuple{N,<:Integer}) and OffsetArrays.parenttype; IndexStyle, elsize, pointer, similar, broadcasting, Adapt, printing, etc. follow. Base.parent throws for a subtype that forgets to define it, to avoid recursing through Base's identity fallback. Subtypes are responsible for calling OffsetArrays.overflow_check in their constructors, and may extend OffsetArrays.unwrap to keep their own type through copy, zero, map(T, A), all-Colon indexing and adapt; other operations producing offset axes return an OffsetArray. The range specialisations (OffsetRange) remain specific to OffsetVector.

Public API. AbstractOffsetArray, AbstractOffsetVector, AbstractOffsetMatrix, offsets, parenttype, unwrap and overflow_check are documented and declared public on Julia ≥ 1.11. overflow_check is specified for axes with Int endpoints.

Compatibility. OffsetArray behaves as before, with two exceptions: its supertype is now AbstractOffsetArray rather than AbstractArray directly, and fill!(A, x) now returns A itself instead of a new wrapper around the same parent. Minimum Julia version is 1.0 (the 0.7 compat entry is dropped). Bumps the version to 1.18.0.

codecov · 2026-01-29T10:19:38Z

Codecov Report

❌ Patch coverage is 97.26027% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.86%. Comparing base (b27ac81) to head (4cd3751).

Files with missing lines	Patch %	Lines
src/OffsetArrays.jl	97.14%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #371      +/-   ##
==========================================
+ Coverage   96.63%   96.86%   +0.23%     
==========================================
  Files           6        6              
  Lines         475      479       +4     
==========================================
+ Hits          459      464       +5     
+ Misses         16       15       -1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

giordano

I'll defer the decision about adding a new type parameter to the maintainers of the package

Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com>

simone-silvestri · 2026-09-08T16:18:38Z

Would it be ok if I find a way to implement this functionality without introducing a new parameter?

simone-silvestri · 2026-09-08T16:53:24Z

Actually, thinking a bit about it, it seems to be a bit to difficult to achieve without an extra parameter. Another possibility would be to just allow this feature in downstream packages, for example by defining an abstract type AbstractOffsetArray{T, N} which could be subtyped and move most of the methods to target AbstractOffsetArray. Would that be an option?

jishnub · 2026-09-08T16:55:46Z

Yes, that makes sense

giordano · 2026-10-17T13:02:17Z

@jishnub can you please have another look at this PR? It's good to go from my point of view, but I'd appreciate your review before merging this and tagging a new release.

Drop the field-reading defaults for `parent` and `offsets` on `AbstractOffsetArray` and make the three accessors (`Base.parent`, `OffsetArrays.offsets`, `OffsetArrays.parenttype`) the documented subtype interface, defined for `OffsetArray` only. `parenttype` moves next to them and gains a generic instance method. `Base.parent(::AbstractOffsetArray)` now throws an `ArgumentError` naming the interface: without that guard Base's `parent(a::AbstractArray) = a` makes a subtype that forgets `parent` recurse through `axes` into a `StackOverflowError`. Document `AbstractOffsetVector`, add `AbstractOffsetMatrix` for parity with `OffsetMatrix`, and list both plus `parenttype` in the reference. The manual example and the test subtypes define the three methods; tests cover the aliases, `parenttype`, and the error for incomplete subtypes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

`Base.IndexStyle`, `Base.elsize`, `Adapt.parent_type` and `Adapt.unwrap_type` were the only methods still restricted to the concrete `OffsetArray`, because they need the parent type without an instance. Now that `parenttype` is part of the subtype interface they dispatch on `AbstractOffsetArray`. Without `elsize`, `strides` advertised subtypes as strided but `pointer(B, i)` and pointers into views of `B` threw a `MethodError`; without `IndexStyle`, subtypes silently fell back to Cartesian indexing. The test subtypes drop their hand-written `IndexStyle` methods, and the tests cover `IndexStyle`, `elsize`, `pointer(B, i)`, a pointer into a column view, and `adapt`/`Adapt.parent_type`/`unwrap_type` on a subtype. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

`fill!` went through `parent_call`, which fills the parent and then rebuilds the wrapper as an `OffsetArray`. For `OffsetArray` itself the rebuilt wrapper is `===` to the argument, but for any other subtype the data was mutated in place while a different wrapper type was returned, breaking the `fill!` contract. Fill the parent and return `A` directly, and stop listing `fill!` among the operations `unwrap` controls. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

Only the `OffsetArray` constructor runs `overflow_check`; `axes` on an `AbstractOffsetArray` adds the stored offsets to the parent's axes unchecked, so a subtype that stores offsets without validating them can produce wrapped-around axes silently. Give `overflow_check` a docstring (it is specified for `Int` axes and accepts any `Integer` offset), state in the `AbstractOffsetArray` docstring and the manual that a subtype's constructor is responsible for calling it, and have the manual example and the test subtype do so. The test constructs a parent whose axis ends at `typemax(Int)` and checks that the subtype rejects an offset that would overflow. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

`AbstractOffsetArray`, `AbstractOffsetVector`, `AbstractOffsetMatrix`, `offsets`, `parenttype`, `unwrap` and `overflow_check` are documented extension points, so declare them `public` on Julia >= 1.11. The declaration is parsed from a string because the keyword does not parse on older versions, which the package still supports. The test uses the same version bound so the file still loads there. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

State precisely which operations go through `unwrap` (`copy`, `zero`, `map(T, A)`, all-`Colon` indexing, `Adapt.adapt`) and which always return an `OffsetArray` (`similar`, broadcasting, range indexing, `AbstractArray{T}(A)`, `convert`, `reshape` to offset axes). Note that the range specialisations are keyed on `OffsetVector` wrapping an `AbstractRange`, so a subtype wrapping a range uses the generic `AbstractArray` fallbacks, and test that those give the same values. Comment in `showarg` that `nameof` yields the struct name, so the `OffsetVector`/`OffsetMatrix` aliases keep printing as `OffsetArray`, and pin that. Add the `sizeof` comparison that states the motivation of issue 370: narrow offsets make the wrapper smaller than an `OffsetArray`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

`similar(::Type{<:OffsetArray}, dims)` allocates an `Array{eltype}`; the same default serves any subtype, so widen the signature. Without it, `similar(S, dims)` for a subtype type `S` hits Base's `T(undef, dims)` fallback and throws, and so does `similar(S, offset_axes)`, which bottoms out in this method. `Array{eltype(A)}` rather than `parenttype(A)` keeps partially specified types such as `S{Float64}` working, as for `OffsetVector{Float64}`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

giordano · 2026-10-19T23:34:46Z

Mumble mumble mumble...tests haven't been running in this repo since last year for some reason, sigh

The testset resized the parent vector after `permutedims` had reshaped it. On Julia < 1.11 `reshape` marks an `Array` as shared and Base then refuses to resize it, so `pop!` and `push!` failed on 1.10 with "cannot resize array with shared data". Resize a vector that has not been reshaped instead; the `permutedims` checks are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

Compare the pointer of a view with offset axes against the OffsetArray reference rather than the parent: on old Julia versions Base computes the pointer of such a view one element off, for OffsetArray and subtypes alike, so the previous assertion could not hold there. Use `adapt(Array, N)` and check for `OffsetArray{Int,2}`: converting the eltype through `adapt(Array{Float64}, ...)` is Adapt 4 behaviour, and the test is about the wrapper being rebuilt, not about the eltype. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VuNog1u33YxkfTsqyeDB8F

giordano · 2026-10-20T00:33:04Z

I got some help for a review from a couple of bots and they flagged some gaps in the interface (I'm not very familiar with this package, so it's not surprising I had missed them), which should now be fixed. After separately fixing the CI infrastructure in #396 (it was in a fairly bad shape), tests are now passing in Julia v1.0 through v1.15.

@jishnub I'm somewhat inclined to merge this next week if I don't hear objections in the meantime.

simone-silvestri added 10 commits March 12, 2025 11:01

check it out

ddea88f

add

f136b40

this change

2133df4

continue it

dbdc028

A couple of tests do not pass

b47cdaa

some changes

0fbdc28

restore a test

9b7d56a

Merge branch 'JuliaArrays:master' into master

050ad77

Merge branch 'master' into master

6d0389b

fix tests and constructors

49487b8

simone-silvestri added 3 commits January 29, 2026 11:25

Fix reshape ambiguity on older julia

ba88cfb

move back _indexlength to return Int

1e9a796

add more test coverage

ed6b86f

simone-silvestri marked this pull request as ready for review January 29, 2026 11:25

simone-silvestri mentioned this pull request Jan 29, 2026

Try using 8-bit integers for reducing parameter space CliMA/Oceananigans.jl#5230

Open

giordano reviewed Jan 30, 2026

View reviewed changes

Comment thread ext/OffsetArraysAdaptExt.jl Outdated

Comment thread src/OffsetArrays.jl Outdated

Comment thread src/OffsetArrays.jl Outdated

Comment thread src/OffsetArrays.jl Outdated

simone-silvestri and others added 6 commits May 1, 2026 17:59

Update ext/OffsetArraysAdaptExt.jl

2976fe3

Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com>

Update src/OffsetArrays.jl

f702460

Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com>

Update src/OffsetArrays.jl

ff73384

Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com>

Update src/OffsetArrays.jl

40a36fa

Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com>

fix tests

288462a

a couple of more fixes

b645e13

implement AbstractOffsetArrays

51d7de8

simone-silvestri changed the title ~~Trying to allow Int8 as indices~~ Implement an abstract AbstractOffsetArray type Sep 8, 2026

simone-silvestri added 2 commits September 8, 2026 20:46

Delete test/abstractoffsetarray.jl

4111997

Merge branch 'JuliaArrays:master' into master

f20b5b9

giordano requested a review from jishnub September 27, 2026 08:42

Merge branch 'master' into master

d6d6e65

This was referenced Oct 17, 2026

[Draft, benchmarking] Pass CUDA field data to kernels with static dimensions CliMA/Oceananigans.jl#6211

Draft

Static dimensions and offsets CliMA/Oceananigans.jl#6270

Open

giordano and others added 8 commits October 17, 2026 14:03

Bump version to v1.18.0

c755cb8

giordano closed this Oct 19, 2026

giordano reopened this Oct 19, 2026

giordano and others added 4 commits October 19, 2026 19:02

Merge remote-tracking branch 'origin/master' into ss/abstractoffsetarray

2fe612a

Bump minimum Julia version to v1.0

4cd3751

Navigation Menu

Repository navigation

Implement an abstract `AbstractOffsetArray` type - #371

Implement an abstract `AbstractOffsetArray` type#371
simone-silvestri wants to merge 35 commits into
JuliaArrays:masterfrom
simone-silvestri:master

simone-silvestri commented Mar 12, 2025 •

edited by giordano

Loading

Uh oh!

codecov Bot commented Jan 29, 2026 •

edited

Loading

Uh oh!

giordano left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

simone-silvestri commented Sep 8, 2026

Uh oh!

simone-silvestri commented Sep 8, 2026 •

edited

Loading

Uh oh!

jishnub commented Sep 8, 2026

Uh oh!

giordano commented Oct 17, 2026

Uh oh!

giordano commented Oct 19, 2026

Uh oh!

giordano commented Oct 20, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

simone-silvestri commented Mar 12, 2025 • edited by giordano Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

giordano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

simone-silvestri commented Sep 8, 2026

Uh oh!

simone-silvestri commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jishnub commented Sep 8, 2026

Uh oh!

giordano commented Oct 17, 2026

Uh oh!

giordano commented Oct 19, 2026

Uh oh!

giordano commented Oct 20, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

simone-silvestri commented Mar 12, 2025 •

edited by giordano

Loading

codecov Bot commented Jan 29, 2026 •

edited

Loading

simone-silvestri commented Sep 8, 2026 •

edited

Loading