Binary search optimization for Parquet Variant's metadata lookup

[abigalekim]

Description

find_key_in_metadata maps an object field name (e.g. "name") to its integer dictionary ID in a Parquet Variant metadata blob. This function previously did a linear scan over all dictionary keys to find the integer ID for a path component. This PR implements binary search for find_key_in_metadata when the keys are sorted lexicographically.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 869e5dde-c8d1-45b9-8067-8860cf66e749

📥 Commits

Reviewing files that changed from the base of the PR and between d314b74 and 5c3ad9d.

📒 Files selected for processing (1)

• cpp/src/io/parquet/experimental/variant_extract.cu

🚧 Files skipped from review as they are similar to previous changes (1)

• cpp/src/io/parquet/experimental/variant_extract.cu

---

📝 Walkthrough

Summary by CodeRabbit

• Performance Improvements
  • Improved metadata key lookups with faster searching when dictionary entries are sorted.
  • Preserved compatibility with dictionaries that are not sorted.
• Bug Fixes
  • Improved resilience when processing malformed metadata offsets, preventing invalid results from being returned.

Walkthrough

Changes

Variant dictionary lookup

Layer / File(s)	Summary
Validated dictionary search cpp/src/io/parquet/experimental/variant_extract.cu	find_key_in_metadata reads the sorted metadata flag. Shared decoding validates offsets and bounds. Sorted dictionaries use binary search; unsorted dictionaries retain linear search. Malformed entries return nullopt.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: mhaseeb123, vuule

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the binary search optimization for Parquet Variant metadata lookup.
Description check	✅ Passed	The description explains the metadata lookup optimization and its use of binary search for sorted keys.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[PointKernel]

Looks great. Only some non-blocking nits.

[abigalekim]

/ok to test 5b6c9a7

[vuule]

I would like us to add benchmarks that show the improvement before merging this.
#22434 has benchmarks that can be used as a starting point.