Repository navigation
GH-51145: [Python] Accept Arrow integer scalars in slice methods - #51160
Conversation
AlenkaF
left a comment
There was a problem hiding this comment.
Thanks for the PR! I was thinking of suggesting the use of Py_ssize_t-typed Cython parameter but operator.index() does solve everything: accepts Python ints and Arrow Scalars if they are integer type. cc @raulcd what do you think?
In any case, we need to update docstrings for offset and length as they do not need to be int but can also be Arrow Scalars.
Update offset and length documentation for all four slice methods. Arrow scalars must be non-null integers. Prepared and validated with OpenAI Codex assistance.
raulcd
left a comment
There was a problem hiding this comment.
Not an expert on this area but this seems reasonable to me.
|
Hi @AlenkaF and @raulcd, thanks for reviewing this. The offset and length docstrings have been updated across all four slice methods to cover non-null Arrow integer scalars. Following up after the September 14 approval: is there anything else needed from me before this can proceed? Codex-assisted follow-up, posted with my approval. |
|
thanks for the ping @YusefSyed . @AlenkaF I think this is ok to merge, I'll wait in case you can take a quick look. I'll merge next week otherwise. |
AlenkaF
left a comment
There was a problem hiding this comment.
Thanks for the ping and the update!
I have no follow-up comments, will merge.
Rationale for this change
Array.slice,ChunkedArray.slice,RecordBatch.slice, andTable.sliceaccept Python and NumPy integer-like values but reject Arrow integer scalars,
even though those scalars implement Python's integer-index protocol. This makes
values produced by Arrow APIs unnecessarily unusable as slice offsets and
lengths.
What changes are included in this PR?
Normalize non-
Noneoffsets and lengths withoperator.index()at the fourpublic Python wrapper boundaries before the existing validation and C++ calls.
Add regression coverage for all signed and unsigned Arrow integer scalar types
across the four wrappers, plus NumPy compatibility, invalid and null scalars,
negative values, offset clamping, and int64 overflow behavior.
Are these changes tested?
Yes. Current Arrow C++ and editable PyArrow were built from source in an
isolated Python 3.13 environment. The focused new tests and existing slice
controls passed: 48 passed.
Cython translation, Python test-file compilation, Flake8, and
git diff --checkalso passed.Are there any user-facing changes?
Yes. The four Python
.slice()methods now accept non-null Arrow integerscalars for offsets and lengths. Existing Python/NumPy integer behavior and
error behavior for unsupported values are preserved.
AI assistance
AI assisted analysis, implementation, test drafting, build work, and review.
The account holder reviewed and approved the final four-file diff. The account
holder did not personally run the commands reported above.
.slicemethods should accept Arrow scalar types #51145