Repository navigation
Test one-level glob listings at speed of light - #1317
Merged
Merged
Conversation
2 tasks done
# Conflicts: # testing-conventions.toml
# Conflicts: # testing-conventions.toml
The repeated-column test fails: a projection naming a column twice writes its key twice.
…on spare cores The sibling test fails: every directory is explored on the calling thread.
…aude/1325-parallel-walk
This was referenced Oct 6, 2026
test: glob speed of light across matcher forms (stacked on #1317)
test: glob speed of light with symlinks (stacked on #1317)
test: glob speed of light with .gitignore (stacked on #1317)
test: glob speed of light with node_modules (stacked on #1317)
…ects kind_of takes its stat as an argument so a followed and an unfollowed link are judged without a real symlink, and run_default writes to a given writer.
Drops the scanner.rs mutation exemption lines the kind_of, stat_of and gitignores_above unit tests now cover.
This was referenced Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Speed-of-light e2e test for one-level listings:
'./*','./'and'./docs'over a wide flat root anddocs/, beside subdirectories a one-level listing must not read, plus dotfiles it must hide. Native isfind -mindepth 1 -maxdepth 1 -type f ! -name '.*'; a bash glob loses by an order of magnitude here (11-15s vs 0.5s on a 1M-entry directory) because it must stat every entry to drop directories.Fixes #1305
Status: dirsql misses the bar
Rows match native for all three globs. Speed does not (local run, 2M-entry directories, startup 0.046s):
'./*''./''./docs'Profile on a 1M-entry directory (release CLI,
SELECT path FROM './*'): find 0.45s, dirsql 2.5-3.5s.statxper matched file (980k calls, 11-25s of system time across the stat workers) to fillsize/mtime/ctime, even when the statement reads onlypath. A serial stat of the same files costsfind6.8s on this filesystem, so the parallel stat is right when those columns are read and pure overhead when they are not.The fix lands in follow-up PRs stacked on this one.
E2E Verification
cargo test -p dirsql --features cli --lib)packages/python/e2e-attestations/claude-1305-glob-speed.jsonreceipt written (just e2e-attest-python)packages/ts/e2e-attestations/claude-1305-glob-speed.jsonreceipt written (just e2e-attest-ts)uv run maturin develop --releaseinpackages/python, thenjust e2e-attest-python,just e2e-attest-ts./docs0.753s vs 0.750s,./docs/0.616s vs 0.614s); both cases passed when rerun alone, and the third full run passed.