Repository navigation
perf: skip the startup walk when no tables are declared - #997
Merged
Merged
Conversation
With no `-c`, dirsql declares zero named tables: filesystem access goes through path-tables, which mint their own per-statement scan. Yet `prepare_resolved` runs `scan_directory` unconditionally, and with no table globs to match against that walk returns nothing by construction — 78-85% of the runtime of a query that reads no files. The walk has no result a caller can tell apart from not walking, so whether it ran takes a seam: `prepare_resolved_with` takes the walk as an injected callable, and `prepare_resolved` passes `scan_directory`. Purely a seam — the walk still runs unconditionally, so the two zero-table tests are RED and the declared-table test (which guards against over-skipping) is green. Refs #986 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E88j3ZKfp1bsLA9yxqFxwG
thekevinscott
enabled auto-merge
August 20, 2026 20:34
Zero tables compiles to a `TableMatcher` with no entries, and `match_all`
filters those entries — so `scan_directory` is guaranteed to return
nothing. Short-circuit it.
That is the configless case, where filesystem access goes through
path-tables and `PathTabCursor::filter` runs its own per-statement
`scan_glob`. The startup walk contributed nothing there and cost 78-85%
of the runtime of a query that reads no files.
Only the walk is skipped. `reconcile_scan` still runs with an empty
scan, so `deleted` is derived exactly as before — and `ctx.cached` is
provably empty with zero tables anyway, since a non-empty cache needs a
prior run whose `compute_glob_config_hash` matched, and a prior
zero-table run wrote nothing.
Measured, debug build, 20k files across 10 dirs, best of 5:
SELECT 1, empty dir 7 ms -> 7 ms
SELECT 1, 20k files 45 ms -> 7 ms
Fixes #986
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E88j3ZKfp1bsLA9yxqFxwG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #986
Problem
Configless startup declares zero named tables — filesystem access goes through path-tables, which mint their own per-statement scan in
PathTabCursor::filter. Yetprepare_resolvedcalledscan_directoryunconditionally, and that walk returns nothing by construction:compile_matcherover zero tables builds aTableMatcherwith emptyentries, andmatch_allfiltersentries— so every entry the walk visits yields nothing.Fix
table_nameswas already in scope and discarded vialet _ = table_names;, now removed.Only the walk is skipped.
reconcile_scanstill runs with an empty scan, sodeletedis derived exactly as before, and path-table queries never consulted the startup scan to begin with.Red/green
c26e698Unit tests:678 passed; 2 failed, both on their own assertions —assertion left == right failed: zero tables can match nothing, so the startup walk is pure waste / left: 1 / right: 0.Lintgreen.34d8ba3680 passed; 0 failed.The walk has no result a caller can tell apart from not walking, so whether it ran needs a seam:
prepare_resolveddelegates to a privateprepare_resolved_with(resolved, scan: ScanFn<'_>)passing&scan_directory. Rust has no monkeypatching, andscan_directoryis naturally a callable the unit receives, so this is the DI case AGENTS.md carves out. The seam landed in the red commit and is behaviour-preserving on its own.Three unit tests, using a recording double that counts invocations and returns a canned result (hermetic — no real tree, so
unit lintisolation passes):c26e698prepare_does_not_walk_the_tree_when_no_tables_are_declaredprepare_still_opens_the_persist_cache_when_no_tables_are_declaredprepare_walks_the_tree_when_a_table_is_declaredscanned_filesUnit tier only, deliberately. The change alters no observable behaviour and no public API — the walk's contribution is provably empty — so integration/binding/e2e have nothing new to assert; existing coverage at those tiers proves the absence of regression. Per AGENTS.md's decision tree (no public-API change + adequate integration coverage → unit tests only) that is the right shape.
On the persist interaction
#986 flags
reconcile_scanas the one real interaction and asks for confirmation rather than assumption, suggesting a narrowing to the non-persist path as a conservative first cut. Confirmed — and the narrowing is not needed, in both directions:scan_directory.reconcile_scan(&root, scanned, ctx, &RealFs)still runs withscannedempty, sodeletedis derived exactly as today.ctx.cachedis provably empty whenevertablesis empty anyway: a non-empty cache requires a prior run whosecompute_glob_config_hash(tables, ignore)matched, and a prior zero-table run wrote nothing (its scan was empty too). A prior run with tables hashes differently, soprepare_persistwipes the cache and hands back an empty index.Manual verification
Debug build, 20k files across 10 dirs, warm cache, best of 5 runs of
dirsql "SELECT 1 AS x"(which touches no filesystem, so all variation is the startup walk):The walk is gone; the 20k-file case now matches the empty-dir baseline exactly.
Correctness, same binary:
Changelog / Migrations
<root>/<pkg>/changelog.d/for each changed package (packages/rust/changelog.d/2026-08-20-skip-configless-startup-walk.md)<root>/<pkg>/migrations.d/(or: not required — additive/bugfix only) — not required: no public API change, and no behaviour change (the skipped walk contributed nothing)PARITY.mdis untouched: no SDK's public API surface changes. No parity drift.E2E Verification
packages/pythonattestation refreshed (just e2e-attest-pythonequivalent)packages/tsattestation refreshed (just e2e-attest-tsequivalent)cd packages/python && uvx testing-conventions e2e attest 'uv run python -m pytest tests/e2e/ -x -q'cd packages/ts && uvx testing-conventions e2e attest 'pnpm test:e2e'packages/rust/srcchange stales both bindings, and the red-commit receipts were staled by the fix).Gates run locally on the green tree:
cargo fmt --check,cargo clippy --workspace -- -D warnings,cargo test -p dirsql --lib --features cli(680 pass),cargo test -p dirsql --features cli --tests,unit colocated-test,unit lint,integration lint,unit mutation --base origin/main(no surviving mutants).packagingneeds a built artifact, so CI covers it.One local-only note:
tests/vtab.rs::unreadable_file_yields_null_content_without_erroring_the_rowfails in this sandbox both with and without the change — itchmod 000s a file, and the sandbox runs as root, which bypasses that. Unrelated to this diff and green on CI's non-root runner.