Skip to content

TST: Add test for writing UUIDs to parquet with pyarrow #61602 - #75

Open
xirafa wants to merge 28 commits into
mainfrom
GiTaDi-CrEaTe/tests/io-parquet-uuid-61602
Open

xirafa wants to merge 28 commits into
mainfrom
GiTaDi-CrEaTe/tests/io-parquet-uuid-61602

Conversation

@xirafa

@xirafa xirafa commented Jul 29, 2026 •

Copy link
Copy Markdown

Mirror of pandas-dev/pandas#65647. Runs the pandas test suite through dagz.

Summary by CodeRabbit

  • Tests
    • Added coverage for writing and reading UUID data through the Parquet format using PyArrow.
    • The test runs only with supported PyArrow versions and is marked as an expected failure while an upstream compatibility issue remains unresolved.

GiTaDi-CrEaTe and others added 27 commits May 15, 2026 15:20
@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@xirafa, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 59 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d608721a-e405-434a-82af-a5c3b85ff742

📥 Commits

Reviewing files that changed from the base of the PR and between aa6d566 and e0d6174.

📒 Files selected for processing (1)
  • pandas/tests/io/test_parquet.py
📝 Walkthrough

Walkthrough

Adds a PyArrow parquet test that writes and reads UUID-valued data, asserts exact DataFrame equality, skips unsupported PyArrow versions, and marks the test as an expected failure for the upstream casting issue.

Changes

Parquet UUID support

Layer / File(s) Summary
UUID parquet round-trip validation
pandas/tests/io/test_parquet.py
Adds UUID and test-decorator imports, then tests PyArrow parquet write/read round-tripping with version-based skipping and an expected-failure marker.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: jbrockmendel

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: adding a PyArrow parquet test for UUID values.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch GiTaDi-CrEaTe/tests/io-parquet-uuid-61602

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pandas/tests/io/test_parquet.py`:
- Line 1529: Update the xfail reason near the existing GH 61602 reference to
also cite Apache Arrow issue `#50312`, preserving the explanation about PyArrow’s
FIXED_LEN_BYTE_ARRAY-to-UUID casting failure and both issue references.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 51c9bd28-64a1-48ae-8751-7931be138a70

📥 Commits

Reviewing files that changed from the base of the PR and between b182e90 and aa6d566.

📒 Files selected for processing (1)
  • pandas/tests/io/test_parquet.py



@pytest.mark.xfail(
reason="Upstream PyArrow fails to cast FIXED_LEN_BYTE_ARRAY to UUID - GH 61602"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file and target lines =="
wc -l pandas/tests/io/test_parquet.py 2>/dev/null || true
sed -n '1515,1545p' pandas/tests/io/test_parquet.py 2>/dev/null || true

echo "== search related references =="
rg -n "GH 61602|61602|FIXED_LEN_BYTE_ARRAY|UUID|50312|apache/arrow/issues" pandas/tests io pandas -S 2>/dev/null | head -n 120

Repository: autodag-dev/pandas

Length of output: 8295


🌐 Web query:

Apache Arrow FIXED_LEN_BYTE_ARRAY UUID Arrow issue 50312

💡 Result:

Apache Arrow issue 50312 refers to a bug involving the improper handling of UUID extension types during conversion between PyArrow and pandas [1]. Specifically, when a PyArrow Table containing a UUID column is converted to a pandas DataFrame, the data is returned as raw bytes instead of Python uuid.UUID objects [1][2]. Key details regarding this issue include: Root Cause: The issue stems from the UuidType in PyArrow not implementing to_pandas_dtype, which causes Table.to_pandas to fall back to the underlying storage type, fixed_size_binary(16), resulting in bytes [2]. Status: As of July 29, 2026, the issue is open and actively being addressed [1]. A pull request (GH-50325) was initiated to fix this by adding the missing method [2]. However, project maintainers have suggested a more robust, performance-optimized implementation in C++ rather than the initial Python-level fix [3]. Broader Context: This is part of ongoing work to improve UUID and extension type support in PyArrow [4][5]. Other recent related issues have included inconsistencies in Parquet schema reading (where extension types were dropped) [4] and difficulties in constructing UuidArrays from scalars [5]. Users encountering this behavior should follow the progress on GitHub issue 50312 for updates and the final resolution [1].

Citations:


Add the upstream Arrow issue to the xfail reason.

GH 61602 tracks the pandas test, but this PyArrow FIXED_LEN_BYTE_ARRAY → UUID regression is tracked separately as Apache Arrow issue #50312. Include both references so maintainers can determine when this unconditional xfail can be removed.

Proposed fix
 `@pytest.mark.xfail`(
-    reason="Upstream PyArrow fails to cast FIXED_LEN_BYTE_ARRAY to UUID - GH 61602"
+    reason=(
+        "Upstream PyArrow fails to cast FIXED_LEN_BYTE_ARRAY back to UUID - "
+        "Pandas GH 61602 / Arrow issue: "
+        "https://github.com/apache/arrow/issues/50312"
+    )
 )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
reason="Upstream PyArrow fails to cast FIXED_LEN_BYTE_ARRAY to UUID - GH 61602"
reason=(
"Upstream PyArrow fails to cast FIXED_LEN_BYTE_ARRAY back to UUID - "
"Pandas GH 61602 / Arrow issue: "
"https://github.com/apache/arrow/issues/50312"
)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pandas/tests/io/test_parquet.py` at line 1529, Update the xfail reason near
the existing GH 61602 reference to also cite Apache Arrow issue `#50312`,
preserving the explanation about PyArrow’s FIXED_LEN_BYTE_ARRAY-to-UUID casting
failure and both issue references.

Source: MCP tools

@xirafa
xirafa force-pushed the main branch 30 times, most recently from 1096988 to 7d70807 Compare August 11, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants