Skip to content

[C++][Compute] Move scatter function into compute core #47375

Closed

Description

@zanmato1984

In order to support special form (#47374), the kernels have to respect the selection vector. Currently none of the kernels does. And it's almost impossible for us to make all existing kernels to respect the selection vector at once (and we probably never will). Thus we need an incremental way to add selection-vector-aware kernels on demand, meanwhile accommodate legacy (selection-vector-non-aware) kernels to be executed "selection-vector-aware"-ly in a general manner - the idea is to first "gather" selected rows from the batch into a new batch, evaluate the expression on the new batch, then "scatter" the result rows into the positions where they belong in the original batch.

This makes the take and scatter functions dependencies of the exec facilities, which is in compute core (libarrow). And take is already in compute core. Now we need to move scatter.

Activity

  1. changed the title [-]Issue # placeholder: Move `scatter` function into compute core. As mentioned above, the `take` ("gather") and `scatter` functions play an essential role to support selective execution for arbitrary legacy (selection-vector-non-aware) kernels, thus they have to exist in libarrow. Function `take` already does. We need `scatter`.[/-] [+][C++][Compute] Move `scatter` function into compute core[/+] on Aug 20, 2025
  2. self-assigned this
    on Aug 20, 2025
  3. added a commit that references this issue on Aug 27, 2025
  4. added this to the 22.0.0 milestone on Aug 27, 2025
  5. zanmato1984 commented on Aug 27, 2025

    @zanmato1984
    ContributorAuthor

    Issue resolved by pull request 47378
    #47378

  6. added a commit that references this issue on May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions