Parquet: read-path redesign — serve bytes as they arrive, bounded coalescing, lifetime memory pools, read-ahead controller, filesystem-cache fixes - #2275

Open

UnamedRus wants to merge 27 commits into

antalya-26.6from

parquet-v3-read-sizing

Commits on Aug 27, 2026

Parquet: design spec and implementation plan for the read-path redesign

UnamedRus
and
claude
committed
b43afbe
Parquet: derive the IO pool size from the query and make the read-task size a setting

UnamedRus
and
claude
committed
a88e21f
Parquet read-path spec: fold in the vig-test measurements and the cache cooperation phase

UnamedRus
and
claude
committed
c6583f1
Parquet read-path spec: rewrite §4.2c as alignment with the upstream `ReaderExecutor`

UnamedRus
and
claude
committed
9d28162
Parquet read-path plan: re-cut Tasks 4-11 for the patch-up and Phase 2, defer Phase 3

UnamedRus
and
claude
committed
6c84dba
Parquet: serve a coalesced read's bytes as they arrive instead of waiting for the whole task

UnamedRus
and
claude
committed
c30c6c1
Parquet read-path plan: state the readBigAt progress-callback contract correctly (true means stop)

UnamedRus
and
claude
committed
2b241cc
Parquet: fix zero-copy readiness race and use a waiters counter instead of a single threshold

UnamedRus
and
claude
committed
3143a37
Parquet: test that decoding starts on a coalesced read before it completes

UnamedRus
and
claude
committed
59a06a1
Filesystem cache: honour the per-query boundary alignment on the readBigAt path

UnamedRus
and
claude
committed
45c338a
Filesystem cache: let a reader opt out of background download of partially read segments

UnamedRus
and
claude
committed
54b68f7

Commits on Aug 28, 2026

Commits on Aug 29, 2026

Parquet: count only executing bytes against the read-ahead target

UnamedRus
and
claude
committed
2870a61
Parquet: bound read amplification by absolute waste, not only by ratio

UnamedRus
and
claude
committed
619efb8
Parquet: read through gaps the source already has cached

UnamedRus
and
claude
committed
747a6c7
Page cache: read through short islands of cached blocks

UnamedRus
and
claude
committed
d064c0b
Parquet: relax the read-amplification floor test's threshold

UnamedRus
and
claude
committed
6503da0