Skip to content

fix(arrow): support zero-length fixed-size lists - #1308

Merged
zeroshade merged 1 commit into
apache:mainfrom
jakthom:fix/zero-size-fixed-size-list
Sep 15, 2026
Merged

zeroshade merged 1 commit into
apache:mainfrom
jakthom:fix/zero-size-fixed-size-list

Conversation

@jakthom

@jakthom jakthom commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

DataFusion can return FixedSizeList(0, Null) for an empty list. Arrow Go currently rejects the C data schema +w:0, and its fixed-size list constructors also panic for size zero. This prevents importing results such as:

select arrow_cast(a, 'FixedSizeList(0, Null)')
from values ([]), (NULL) t(a);

A zero-size list has no child values, but its parent array still has rows and a validity bitmap that distinguishes empty lists from null lists. This fixes the two failing cases in DataFusion's arrow_typeof.slt.

What changes are included in this PR?

  • Accept zero in the three fixed-size list type constructors and in C data schema imports.
  • Allow zero-size fixed-size list arrays to validate without dividing by zero. Negative sizes, invalid child schemas, and overflowing C schema sizes remain rejected.
  • Add constructor, array, C data, and IPC regression tests covering empty and null rows, bulk appends, nonzero slice offsets, and checked-allocator cleanup with null, integer, and nested-list element types.

Are these changes tested?

Local verification on macOS ARM64 with Go 1.25.2:

  • pre-commit run --all-files --show-diff-on-failure
  • ci/scripts/build.sh "$PWD"
  • ci/scripts/test.sh "$PWD" (race detector, assert,test, and noasm variants)
  • go test -race -short -tags=assert,test ./internal/...

The datafusion-go SQLLogicTest corpus also passes 24,958/24,958 assertions across 365 executable SQL files on macOS when an isolated Go module file replaces Arrow Go with this checkout. That includes all 62 records in arrow_typeof.slt; the released Arrow Go dependency fails two of those records. This downstream run used DataFusion 55.0.0 and datafusion-go commit 8fbd0218b6c5f3d9d64ec798ae466f66725be325.

Are there any user-facing changes?

Zero-size fixed-size lists can now be constructed, validated, and imported through the C data interface. Existing constructor and import checks for negative sizes are preserved.

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zero-length fixed-size-list construction, validation, slicing, IPC, and C Data handling look consistent. Focused zero-size tests passed, including go test -p 2 -count=1 -tags test ./arrow/cdata -run TestZeroSizeFixedSizeListRoundTrip. The recorded Conda integration failure is the unrelated nocgo package selection issue fixed by apache/arrow#51287.

@zeroshade
zeroshade merged commit df6bddc into apache:main Sep 15, 2026
104 of 109 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants