Skip to content

fix: start a prefixed relative glob's walk at its literal directory - #1236

Merged
thekevinscott merged 8 commits into
mainfrom
claude/1199-glob-prefix
Oct 5, 2026
Merged

thekevinscott merged 8 commits into
mainfrom
claude/1199-glob-prefix

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1199

Approach

path_table::resolve now treats a ./ table the way it already treated an absolute one: the literal directory chain ahead of the first glob component becomes PathTable::root (and path_prefix), and the glob is the remainder. './small/*.md' is therefore root <index>/small, glob *.md, and the walk never lists the index root. path is reported exactly as before.

With the walk starting at the named directory, the scanner no longer needs to know which part of a glob was literal, so scan_glob drops its ignore_base argument and path_table::ignore_base is removed (the ignore-walker machinery for "the base is itself under an ignored directory" went with it; naming a skipped directory outright still scans it, now by construction).

Because the scan root can sit below the index root, the dirsql_parsed module takes the index root as a sixth fixed argument and spawns the parser there, with {root} naming it. Rebased onto main after #1219: the parser is still spawned once per table through on_file::run with the absolute paths as trailing arguments; only its working directory and {root} change. That is what docs/reference/hooks.md documents for {root}; absolute path-tables previously got it wrong (the parser ran in the scan root), which this fixes as well.

Skip rules and .gitignore are judged from the directory the scan starts in. A .gitignore at or below that directory still applies beneath it (a_scoped_glob_still_honors_gitignore_rules_beneath_its_base). One above it inside the index root does not -- which is also what main does today: a root *.log rule never applied to './docs/*' before this change either (verified by probe against the main binary). That behavior is pinned by a_gitignore_above_the_named_base_does_not_reach_beneath_it rather than changed; the task brief assumed the opposite, so flagging it here.

Before / after

Best of 5 back-to-back runs, release build, warm cache. deep = big/ with 500,000 files in 500 subdirectories beside small/ with 10 files; wide = 200,000 top-level files beside small/ with 10 files.

tree command main this PR
deep dirsql "SELECT count(*) FROM './small/*.md'" 207 ms 2 ms
deep dirsql "SELECT count(*) FROM './small/'" 251 ms 3 ms
deep find ./small -name '*.md' 2 ms 0 ms
deep find . -type f 198 ms 146 ms
wide dirsql "SELECT count(*) FROM './small/*.md'" 153 ms 2 ms
wide dirsql "SELECT count(*) FROM './small/'" 143 ms 2 ms
wide find ./small -name '*.md' 0 ms 0 ms
wide find . -type f 92 ms 96 ms

Behavior changes

  • './.dirsql' scans the reserved directory (naming it outright, as every other skipped directory already behaves); './x/.dirsql/**' is pruned.
  • A symlinked directory named ahead of the first wildcard is entered, as it already was for an absolute table.
  • {a,b} now counts as a glob metacharacter when splitting (it always was one to the matcher).
  • --on-file over an absolute path-table runs the parser in the index root with {root} = the index root.

Public Rust API: scanner::scan_glob(root, glob, ignore, gitignore); path_table::ignore_base removed; dirsql_parsed(root, glob, parser, gitignore, cache, index root, ignores...). No SDK-facing change, so PARITY.md is untouched.

Manual run

Built the release CLI and ran it by hand: SELECT path FROM './docs/*.md' with the index root made unlistable returns docs/a.md, docs/b.md; --on-file './parse.sh {root}' over './docs/*.md' and over the absolute form both run parse.sh from the index root and substitute it for {root} (the main binary fails to spawn the parser for the absolute form). The issue's deep/wide trees produced the table above.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass
  • TypeScript SDK e2e: pass
  • Rust core e2e: pass (cargo test --workspace --features cli)
  • packages/python/e2e-attestations/claude-1199-glob-prefix.json receipt written (just e2e-attest-python)
  • packages/ts/e2e-attestations/claude-1199-glob-prefix.json receipt written (just e2e-attest-ts)
  • Command(s) run: cargo fmt --all, cargo clippy --workspace --features cli --all-targets -- -D warnings, cargo test --workspace --features cli, cargo bench -p dirsql, uv run maturin develop --release, pnpm build, just e2e-attest-python, just e2e-attest-ts, just preflight
  • Result summary: Rust suite green; Python e2e 40 passed, 2 xfailed; TypeScript e2e 18 passed; cargo bench ran clean (there is no scan_glob bench); just preflight 0 failing pairs.

Changelog / Migrations

  • Changelog fragment added under packages/rust/changelog.d/2026-10-02-glob-prefix-walk.md
  • Migration fragment added under packages/rust/migrations.d/2026-10-02-glob-prefix-walk.md

@thekevinbot
thekevinbot force-pushed the claude/1199-glob-prefix branch from 46f730c to f1f0669 Compare October 2, 2026 18:27
A './' path-table now resolves like an absolute one: the literal
directory chain ahead of the first wildcard becomes the scan root and
the rest the glob, with paths reported under that chain. The walker
no longer needs an ignore base, so that machinery goes.
The scan root of a prefixed relative table now sits below the index root,
so the parsed module takes the index root as its own argument and runs the
parser there, with {root} naming it, as the on-file hook documents. An
absolute table's parser ran in the scan root before; it runs in the index
root too now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Path-table glob walks the entire root instead of the glob's literal directory prefix

2 participants