[Python] Restore ASV benchmark setup #52359

Activity

1. jorisvandenbossche

   added

   Component: Python

   Component: Benchmarking

   on Oct 8, 2026
2. jorisvandenbossche

   added a parent issue on Oct 8, 2026

   • [Python][Packaging] Support abi3 wheels for pyarrow #50398
3. rok commented on Oct 9, 2026

   

   rok

   on Oct 9, 2026

   Member

   More actions

   To be honest I thought we did not have PyArrow benchmarks, however with some (🤖) searching:

   arctosalliance/arrow-benchmarks-ci:

   • Machine filters: https://github.com/arctosalliance/arrow-benchmarks-ci/blob/main/config.py#L85
   • Benchmarkable repo pointer: https://github.com/arctosalliance/arrow-benchmarks-ci/blob/main/buildkite/benchmark/run.py#L32

   Group list with commands:

   • https://github.com/arctosalliance/benchmarks/blob/main/benchmarks.json

   Benchmark implementations (arctosalliance/benchmarks):

   • csv-read: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/csv_benchmark.py
   • dataframe-to-table: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/dataframe_to_table_benchmark.py
   • dataset-filter: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/dataset_filter_benchmark.py
   • dataset-read: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/dataset_read_benchmark.py
   • dataset-select: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/dataset_select_benchmark.py
   • dataset-selectivity: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/dataset_selectivity_benchmark.py
   • dataset-serialize: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/dataset_serialize_benchmark.py
   • file-read and file-write: https://github.com/arctosalliance/benchmarks/blob/main/benchmarks/file_benchmark.py

   Results can be seen here:

   • csv-read: https://conbench.arrow-dev.org/c-benchmarks/csv-read
   • dataframe-to-table: https://conbench.arrow-dev.org/c-benchmarks/dataframe-to-table
   • dataset-filter: https://conbench.arrow-dev.org/c-benchmarks/dataset-filter
   • dataset-read: https://conbench.arrow-dev.org/c-benchmarks/dataset-read
   • dataset-select: https://conbench.arrow-dev.org/c-benchmarks/dataset-select
   • dataset-selectivity: https://conbench.arrow-dev.org/c-benchmarks/dataset-selectivity
   • dataset-serialize: https://conbench.arrow-dev.org/c-benchmarks/dataset-serialize
   • file-read: https://conbench.arrow-dev.org/c-benchmarks/file-read
   • file-write: https://conbench.arrow-dev.org/c-benchmarks/file-write
   • recursive-get-file-info: https://conbench.arrow-dev.org/c-benchmarks/recursive-get-file-info
   • wide-dataframe: https://conbench.arrow-dev.org/c-benchmarks/wide-dataframe
4. rok commented on Oct 9, 2026

   

   rok

   on Oct 9, 2026

   Member

   More actions

   Please note - I did not design this system, I just maintain it :D
5. jorisvandenbossche commented on Oct 12, 2026

   

   jorisvandenbossche

   on Oct 12, 2026

   MemberAuthor

   More actions

   Yes, I know we have those benchmarks (see also our zulip chat about this in August), but those have a different (although somewhat overlapping) coverage.
   The conbench ones are mostly about Parquet/Dataset (and some pandas.DataFrame -> pyarrow.Table conversion), while the ASV benchmarks also include basic python <-> arrow conversions (which is much more likely to be impacted by CPython C API usage changes).
6. jorisvandenbossche

   self-assigned this

   on Oct 12, 2026
7. rok commented on Oct 13, 2026

   

   rok

   on Oct 13, 2026

   Last edited by rok

   Member

   More actions

   Oh, I see. Also sorry I missed you meant ASV benchmarks.
   I suppose you could reintroduce ASV or add new conbench benchmarks, whichever better serves your goals.

   Please note there is an effort in progress to rewrite conbench (https://github.com/rok/benchx). The goal is to make benchmark authoring easier as well as making maintenance of benchmark store realistic.