feat(ipc): Read and write DictionaryEncoding.custom_metadata for dictionary values

[rustyconover]

Draft until the Arrow format vote. This implements the proposed Arrow format 1.6 change from apache/arrow#49704. The companion drafts are apache/arrow#53994 (format and C++), apache/arrow-go#1381 (Go) and apache/arrow-rs#11622 (Rust).

DictionaryEncoding gains an optional custom_metadata that holds the metadata of the dictionary value type. It is the IPC counterpart of ArrowSchema.dictionary->metadata, while Field.custom_metadata corresponds to the parent ArrowSchema.metadata. Because nanoarrow works on ArrowSchema directly, the mapping is lossless. The member has the same semantics as Field.custom_metadata, applied to the dictionary values, so this PR writes and reads all keys of ArrowSchema.dictionary->metadata.

• Regenerated header: both commits regenerate flatcc_generated.h with dev/update_fbs.sh (flatcc e3e44836).
  • The first commit regenerates from apache/arrow main as committed, so it only changes comments: upstream docstring fixes and the Tensor/SparseTensor deprecation notes.
  • The second commit regenerates from apache/arrow main plus the draft Schema.fbs change. Its header diff is only the new member.
• Encoder: writes schema->dictionary->metadata (all keys) to DictionaryEncoding.custom_metadata, omitted when NULL or empty, and keeps writing schema->metadata to Field.custom_metadata. Previously the value metadata was silently dropped.
• Decoder: keeps Field.custom_metadata on the parent schema, as before, and now sets schema->dictionary->metadata from DictionaryEncoding.custom_metadata.
  • Messages without the member decode exactly as before. There is no fallback for older files, per the spec.
  • This covers dictionaries nested in struct/list children and footer decoding.
• Footer encoding: carries the member when the caller fills footer.dictionaries. nanoarrow's file writer doesn't fill it (Support dictionary-encoded types in the IPC writer #622), so nanoarrow doesn't write IPC files with dictionaries today.
• Integration JSON: the reader and writer support the optional "metadata" of a field's "dictionary" object. TestingJSONComparison now compares dictionary value metadata at any depth, treating empty as absent.

Tests:

• nanoarrow-only (run in CI):
  • encode/decode round trips that check the flatbuffer directly: extension and arbitrary keys, non-UTF-8 values, nesting in struct and list, alongside field metadata, empty metadata omitted;
  • a footer encode/decode round trip with the dictionary encodings filled in;
  • decoding hand-built messages with and without the member;
  • a writer/reader stream round trip;
  • JSON reader/writer/comparison tests.
• Arrow C++ interop: with NANOARROW_BUILD_TESTS_WITH_ARROW, dictionary(int32, uuid) cases are enabled, including Arrow C++ reading nanoarrow-encoded schemas.
  • They skip at runtime unless the linked Arrow C++ writes the new member, so they skip on released Arrow C++.
  • I ran them locally against an Arrow C++ build of GH-49704: [Format][C++] Add custom_metadata to DictionaryEncoding for the dictionary value type arrow#53994: 888/888 (886 tests plus the two integration tests), also under ASan/UBSan.

Archery: with the dictionary_extension case enabled for nanoarrow, C Data passes between C++ and nanoarrow in both directions. On main, nanoarrow→C++ fails because the value metadata is lost. IPC integration for dictionary cases is still blocked by existing writer limitations tracked in #622, which also affect the existing dictionary case.

This PR's code and description were written by AI (Claude Code) and reviewed by a human maintainer and an AI reviewer.

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

❌ Patch coverage is 27.02703% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.15%. Comparing base (e5f3d0a) to head (a3edfdd).

Files with missing lines	Patch %	Lines
src/nanoarrow/testing/testing.cc	36.00%	1 Missing and 15 partials ⚠️
src/nanoarrow/ipc/encoder.c	9.09%	0 Missing and 10 partials ⚠️
src/nanoarrow/ipc/decoder.c	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #972      +/-   ##
==========================================
- Coverage   78.24%   78.15%   -0.10%     
==========================================
  Files         106      106              
  Lines       16901    16932      +31     
  Branches     2012     2040      +28     
==========================================
+ Hits        13224    13233       +9     
  Misses       2435     2435              
- Partials     1242     1264      +22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.