Skip to content

[C++][Python] pairwise_diff ignores array offsets for sliced inputs #50524

Closed

Description

@hsusul

Describe the bug

pairwise_diff and pairwise_diff_checked produce incorrect results when the input array has a nonzero offset, such as an array created with slice().

The kernel reads values from the parent array before the logical slice instead of consistently indexing relative to the sliced input.

Reproduction

import pyarrow as pa
import pyarrow.compute as pc

base = pa.array([99, 1, 4, 9, 16, 88], type=pa.int64())
sliced = base.slice(1, 4)
rebuilt = pa.array(sliced.to_pylist(), type=sliced.type)

print("offset:", sliced.offset)
print("input:", sliced.to_pylist())
print("actual:", pc.pairwise_diff(sliced, period=1).to_pylist())
print("expected:", pc.pairwise_diff(rebuilt, period=1).to_pylist())

Output:

offset: 1
input: [1, 4, 9, 16]
actual: [None, -98, 3, 5]
expected: [None, 3, 5, 7]

The -98 result comes from subtracting the parent array's prefix value 99, which is outside the logical slice.

The same behavior is reproducible through the public C++ API.

Expected behavior

A sliced array should produce the same result as an equivalent zero-offset array containing the same logical values:

[None, 3, 5, 7]

Actual behavior

The first computed difference reads from the parent array before the slice:

[None, -98, 3, 5]

Root cause

In cpp/src/arrow/compute/kernels/vector_pairwise.cc, PairwiseExecImpl copies the input ArraySpan and then calls:

left.SetSlice(left_start, computed_length);
right.SetSlice(right_start, computed_length);

ArraySpan::SetSlice() assigns the provided value as an absolute offset rather than adding it to the existing input offset.

For a sliced input, the shifted spans therefore index from the parent buffers rather than from input.offset.

The validity calculation still uses the original input offset, so the value and validity spans may also refer to different logical elements.

Affected inputs

The issue affects sliced inputs where:

offset > 0
0 < abs(period) < length

It is reproducible for both positive and negative periods and affects numeric, decimal, and temporal kernel signatures that share this implementation.

Proposed direction

Preserve the original input offset when creating the shifted operand spans, and add focused C++ regression coverage for sliced inputs.

I searched existing open and closed issues, pull requests, commits, release notes, tests, and source history and did not find an existing report for this sliced-array offset bug.

Component(s)

C++, Python

Activity

  1. hsusul commented on Jul 16, 2026

    @hsusul
    Author

    I reproduced this in both PyArrow and the public C++ API and have a small C++ fix with focused regression coverage scoped out. I’d be happy to work on it if the proposed direction looks appropriate.

  2. Alb3e3 commented on Jul 28, 2026

    @Alb3e3
    Contributor

    Is anyone actively working on this? You mentioned on Jul 16 that you had a small C++ fix with focused regression coverage scoped out — if you're still planning to send it, please go ahead; I don't want to duplicate your work.

    If you'd rather hand it off, I'm happy to pick it up. I had a look at the kernel and the offset does need to be threaded through consistently: pairwise_diff reads the input values relative to the parent buffer instead of the sliced view, so the first output subtracts the pre-slice element (the -98 in your example). A focused fix plus a regression test over a sliced input covering both pairwise_diff and pairwise_diff_checked looks right. Just let me know which way you'd prefer.

  3. added a commit that references this issue on Aug 25, 2026
  4. added this to the 26.0.0 milestone on Aug 25, 2026
  5. pitrou commented on Aug 25, 2026

    @pitrou
    Member

    Issue resolved by pull request 50858
    #50858

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions