GH-50915: [FORMAT] Allow TIMESTAMP logical type to annotate FIXED_LEN_BYTE_ARRAY(12)

[divjotarora]

Rationale for this change

See apache/parquet-format#600 for rationale.

What changes are included in this PR?

This PR adds support for using TimestampType to annotate FIXED_LEN_BYTE_ARRAY(12) values. It also adds functionality to convert FLBA(12) values to Arrow INT64 timestamps with the following flags/logic to handle overflow:

1. The conversion is guarded behind the convert_flba_timestamps property (default true). If false, conversion fails regardless of value.
2. If the FLBA(12) value overflows max int64 or underflows min int64, the flba_timestamp_clamp_on_overflow property (default false) is consulted. If false, conversion fails. If true, the value is clamped to min/max int64.

Are these changes tested?

Yes, via unit tests and an e2e test that reads the file added in parquet-testing (apache/parquet-testing#123).

Are there any user-facing changes?

No

• GitHub Issue: [Format] Allow TimestampType to annotate FLBA(12) #50915

[github-actions]

⚠️ GitHub issue #50915 has been automatically assigned in GitHub to PR creator.

[emkornfield]

Some questions and comments I think the biggest one is scope and whether we should have an option to convert this value to a proper arrow type. Wemight also want to make it configurable the target of the arrow type

[emkornfield]

I think the conversion code always assumes a 1:1 mapping between arrow timestamp granularity and parquet granularity. I think in the common path when schema is inferred this is workable, but IIRC users can also supply there own schema (we should add test coverage for this path).

[divjotarora]

I think the conversion code always assumes a 1:1 mapping between arrow timestamp granularity and parquet granularity. I think in the common path when schema is inferred this is workable, but IIRC users can also supply there own schema (we should add test coverage for this path).

There doesn't seem to be an API at this level to supply a custom schema. The Arrow timestamp unit is derived from the Parquet logical type during schema conversion and that's passed down to the data converters. Based on this, I don't think any scaling is needed in this PR. To ensure correctness, I added a defensive check that fails the conversion if the Arrow and Parquet units differ.

[emkornfield]

@github-actions crossbow submit

[github-actions]

no tasks were provided for the job
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/35890072900

[emkornfield]

CI is green and changes look reasonale to me. I'll plan to merge Monday unless there are additional concerns raised.

[divjotarora]

Thanks for the feedback @pitrou, this is ready for another look

[pitrou]

LGTM now, thank you @divjotarora !

[pitrou]

@github-actions crossbow submit -g cpp

[github-actions]

Revision: ad03173

Submitted crossbow builds: ursacomputing/crossbow @ actions-183bfb69cc

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

[divjotarora]

@emkornfield @pitrou The failure in C GLib & Ruby / ARM64 macOS GLib & Ruby (pull_request) seem to be unrelated to my changes, is this expected?

/Users/runner/work/arrow/arrow/build/cpp/_deps/google_cloud_cpp-src/google/cloud/internal/openssl/parse_service_account_p12_file.cc
/Users/runner/work/arrow/arrow/build/cpp/_deps/google_cloud_cpp-src/google/cloud/internal/openssl/parse_service_account_p12_file.cc:88:14: error: cannot initialize a variable of type 'X509_NAME *' (aka 'X509_name_st *') with an rvalue of type 'const X509_NAME *' (aka 'const X509_name_st *')
   88 |   X509_NAME* name = X509_get_subject_name(cert.get());
      |              ^      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[113/662] Building CXX object _deps/google_cloud_cpp-build/google/cloud/storage/CMakeFiles/google_cloud_cpp_storage.dir/auto_finalize.cc.o

[pitrou]

@emkornfield @pitrou The failure in C GLib & Ruby / ARM64 macOS GLib & Ruby (pull_request) seem to be unrelated to my changes, is this expected?

I would not say it's "expected" but you are certainly not responsible for it :) No worries!

[pitrou]

The 32-bit build above is green (test-debian-13-cpp-i386), which is what I wanted to check. I'll merge now.

[pitrou]

Gasp, I forgot to replace the [Format] in the PR title :(