Repository navigation
Stat path-table rows only when a stat column is read - #1319
Merged
Merged
Conversation
2 tasks done
# Conflicts: # testing-conventions.toml
Speed up the path-table walk on large directories
This was referenced Oct 6, 2026
Closed
This was referenced Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stat a path-table row only when the statement reads
size,mtime,ctimeorcontent. Stacked on #1317.Fixes #1318
Red commit first:
build_rows_stats_no_fileasserts that building rows stats no file. CI on f51c5b9 failed on exactly that assertion (vtab.rs:713).Measurements (release build, warm cache)
SELECT path FROM './*', 1M flat filessizeandcontentThe output is byte-identical to the base in both cases.
An earlier version stat'ed lazily one row at a time. It made the join 7.8s, so it was replaced before push.
glob_speed_one_level_test(#1305) still fails: dirsql takes 1.30s against a 0.38s budget. The walk is the next fix (#1320).E2E Verification
uv run python -m pytest tests/e2e/ -q --deselect tests/e2e/glob_speed_one_level_test.py, 40 passed.just e2e-attest-ts(pnpm test:e2e) passed.cargo test --workspace --features clipassed apart from flaky server watcher tests.post_query_returns_rows_over_httpandserver_drops_the_rows_of_a_directory_moved_out_of_the_rootfail the same way on the base branch under load and pass alone.SELECT *returns the stat and content values.SELECT path, sizereturns correct sizes.SELECT pathreturns paths only.Changelog / Migrations
packages/rust/changelog.d/2026-10-05-path-table-lazy-stat.mdpackages/rust/migrations.d/2026-10-05-path-table-lazy-stat.md(nothing breaks)