fix: Export a zero offset for empty arrays with an empty offsets buffer

[rustyconover]

An empty variable-size array (string, binary, their large variants, list, large list, map) needs one offset, offsets[0], even though it has no elements. The C data interface doesn't carry buffer sizes, so consumers read offsets[0] without checking. Arrow C++'s ImportArray() does this, for example.

IPC allows a zero-length offsets buffer for an empty array, and nanoarrow's own encoder writes one, since ArrowArrayViewSetArray() sizes the offsets buffer of a length-0 array as 0 bytes. When such an array is decoded, ArrowArrayFinishBuilding() leaves the empty buffer as it is. If the buffer is NULL, it reserves one byte but never initializes it. Either way, the exported ArrowArray has an offsets pointer with no valid offset behind it. With shared buffers, that pointer points into whatever follows in the message body.

I found this moving a DuckDB extension's IPC from Arrow C++ to nanoarrow. A MAP column holding a single NULL round-trips through a worker as an empty key/value child. After nanoarrow decoded it and Arrow C++ imported it, about one run in four failed ValidateFull() with array starts at negative offset -2099303496. The value depends on whatever bytes happen to sit in the message body after the empty buffer.

With this PR, ArrowArrayFinalizeBuffers() gives a DATA_OFFSET buffer of an empty array, if it holds no bytes, a single zero offset of the right width (4 or 8 bytes). This works for both owned and shared buffers: the shared reference is released, and the buffer is replaced with an owned one. Arrays with elements are left alone, so validation still reports a truncated offsets buffer as before (ArrayTestExplicitValidationLevel checks this).

Things worth a look in review:

• ArrowArrayFinalizeBuffers() only runs at NANOARROW_VALIDATION_LEVEL_DEFAULT or higher, because it assumes CPU data access. Decoding at MINIMAL or NONE still exports the empty buffer unchanged. I've left that gate alone. Putting a host buffer behind a device array would be worse.
• An alternative would be for the encoder to always write offsets[0], as Arrow C++ does. That only covers streams nanoarrow wrote itself, though. Other writers can emit zero-length offsets buffers too, so the fix belongs on the decode side.

Tests: NanoarrowIpcWriter.RoundtripEmptyArraysHaveZeroOffset writes a zero-row batch with string, large_binary and list<int32> columns. It reads the batch back with and without shared buffers, then checks that each exported offsets buffer has a size of 4 or 8 bytes and holds 0. The test fails on main, where the buffers are 0 bytes. ctest passes locally (macOS arm64, Debug, 389 tests). The touched suites (array, IPC writer/reader/decoder) pass under ASan/UBSan. array_test needs allocator_may_return_null=1 there, as it does on main.

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.28%. Comparing base (e5f3d0a) to head (31f9949).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #966      +/-   ##
==========================================
+ Coverage   78.24%   78.28%   +0.04%     
==========================================
  Files         106      106              
  Lines       16901    16906       +5     
  Branches     2012     2012              
==========================================
+ Hits        13224    13235      +11     
+ Misses       2435     2429       -6     
  Partials     1242     1242              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[paleolimbot]

I think we should test this differently, but the change itself is great...thank you!

It may be a bit of a gotcha that reducing the verification level may produce arrays that will crash Arrow C++. It came up a while back that we should possibly avoid reading any buffers of arrays with length 0 and I thought Arrow C++ fixed it (I believe we fixed it here); however, this is still a good change to make.

[paleolimbot]

Probably this is more appropriate as a test for ArrowArrayFinalizeBuffers in array_test.cc, although I get the motivating use case was IPC reads.

The IPC read case is I think better tested with a record batch bytes fixture (you can construct the schema programmatically to save space).

[rustyconover]

Moved. ArrayTestFinishBuildingEmptyArrayOffsets in array_test.cc checks ArrowArrayFinishBuildingDefault() on empty string, large binary and list arrays, both when the offsets buffer was never allocated and when it has 0 bytes over storage filled with 0xff. The IPC case is now NanoarrowIpcDecodeEmptyArrayOffsets in decoder_test.cc. It uses a record batch bytes fixture (one row of struct<l: list<string>, i: int32>, with the schema built in code) where the empty string array's 0-byte offsets buffer sits at the same body offset as i's data, 0x55555555. It decodes with both a borrowed and a shared body. Both tests fail on main. I removed the round-trip test from writer_test.cc.

[rustyconover]

Thanks! Agreed on the gotcha with lower validation levels. I ran into exactly that: decoding at NANOARROW_VALIDATION_LEVEL_MINIMAL (to match what Arrow C++'s reader checks) skips ArrowArrayFinalizeBuffers(), so the empty offsets buffer is exported unchanged. I worked around it on my side. It might be worth decoupling the zero offset (and the existing non-NULL data pointer) from the validation level for arrays we know are on the CPU, which the IPC decoder's output always is. I'm happy to do that as a follow-up if you think it's worthwhile.

[paleolimbot]

Thank you!

Not sure what the best solution is for minimal validation here...default validation is probably not much slower and would be what I'd recommend here (if it is a lot slower for some reason we could separate CPU access from validation level in a follow up as you noted).