Skip to content

Accept (i, name) -> Bool functions for select and drop - #1210

Open
quinnj wants to merge 2 commits into
mainfrom
jq/select-drop-functions
Open

quinnj wants to merge 2 commits into
mainfrom
jq/select-drop-functions

Conversation

@quinnj

@quinnj quinnj commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Fixes #1207.

CSV 0.10 accepted a selector function of the form (i, name) -> Bool for select and drop. 1.0 rejected it and pointed users at Tables.Scan. A Scan cannot express a computed rule over header names, such as normalizing each name before matching it against a list, which is the #1207 workflow.

Change

  • The function needs only header names, and CSV settles those before it reads any value. _selectpositions now turns the function into the Bool mask it already accepts.
  • Every reader that settles columns goes through that one path: CSV.File, CSV.lazy, CSV.File(lazyfile), CSV.Rows, CSV.Chunks, and multi-source vectors.
  • As in 0.10, the function gets the column position and the Symbol name (after normalizenames) and must return a Bool.
  • types and pool still reject functions.
  • Docs: reading guide, examples, migration guide, and the Rows docstring.
  • Release: bumps the version to 1.2.0 and adds the 1.2.0 release notes.

Tests

  • New select and drop functions testset: File from IO and from bytes, drop, all-false masks, lazy, Rows, Chunks, multi-source vectors, normalizenames, positions under CSV.File(lazyfile), and a non-Bool return.
  • The two assertions that the function form throws now check the new behavior instead.
  • Local: full suite 331561/331561 on Julia 1.12 (--check-bounds=yes -t4), test/api.jl on Julia 1.10, test/quality.jl 11/11, and the docs build.
  • The help migrating to CSV v1.0 #1207 reporter's function works with drop=_drop. It must call String(name) before Unicode.normalize, because the name is a Symbol, as in 0.10.

🤖 Generated with Claude Code

CSV 0.10 accepted a selector function for `select` and `drop`. 1.0 rejected
it and pointed users at `Tables.Scan`, but a Scan cannot express a computed
rule over header names, such as normalizing each name before matching it
against a list (#1207).

The function only needs header names, which are settled before any value is
read, so `_selectpositions` turns it into the Bool mask it already accepts.
Every reader that settles columns (`File`, `lazy`, `File(lazyfile)`, `Rows`,
`Chunks`, and multi-source vectors) gets it through that one path. As in 0.10,
the function receives the column position and the (normalized) `Symbol` name
and must return a `Bool`. `types` and `pool` still reject functions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.75%. Comparing base (a0335ff) to head (4c2e868).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1210   +/-   ##
=======================================
  Coverage   95.75%   95.75%           
=======================================
  Files           9        9           
  Lines        5489     5491    +2     
=======================================
+ Hits         5256     5258    +2     
  Misses        233      233           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

help migrating to CSV v1.0

1 participant