[thekevinbot]

Fixes #1325

Stacked on #1327.

The directory walk, used by path-table scans and the startup scan, now hands its directories to a depth-first walk in tree_walk.rs. Workers on spare cores explore directories ahead of the visiting. Exploring a directory means:

• reading it
• judging each entry
• taking or dropping each file against the glob and the skip rules, which used to run serially afterwards

The calling thread hands the results out in walk order, so which files each glob returns, and their order, do not change. Workers start only after the walk has found 16 directories, so a small tree never spawns a thread.

Red commit 8b8306f: sibling_directories_are_explored_on_more_than_one_thread failed in CI, because every directory was explored on the calling thread.

tests/path_table_parallel_scan.rs checks that two path-tables in one statement walk their trees at the same time. Its trees were 50 sibling directories, and each walk now spreads those across the cores, so the two walks no longer had idle cores to overlap on. Each tree is now a chain of 50 nested directories. A single walk then never has two directories to read at once, so the test still measures exactly the overlap between the two walks.

Measurements

Wall time, best of 3, /usr/bin/time, release builds, #1327 compared with this PR:

Tree	Query	#1327	this PR	find
1.6M files in 2.3k directories	SELECT count(*) FROM './**/*.md'	0.72s	0.23s	0.57s
1.6M files in 2.3k directories	SELECT path FROM './**/*.md'	0.80s	0.33s	0.57s
1M files, flat	SELECT count(*) FROM './*'	0.38s	0.37s	0.43s
1M files, flat	SELECT path FROM './*'	0.44s	0.44s	0.43s
this repo	SELECT path FROM './**/*.rs' (mean of 20)	5ms	4ms
two-file directory	SELECT path FROM './*' (mean of 50)	2.9ms	2.6-3.0ms

Both speed-of-light e2e tests pass locally against this build:

• glob_speed_one_level_test.py (Glob speed: one-level listings ('./*', './docs', './') #1305)
• glob_speed_recursive_test.py (Glob speed: recursive '**' and a literal prefix #1306, from Test recursive and prefixed globs against find at speed of light #1323)

I compared output from the #1327 binary and this one on 15 queries:

• this repo, which has nested .gitignore files
• the 1M flat tree
• the 1.6M recursive tree
• cell types

All 15 are byte-identical.

E2E Verification

• Python: just e2e-attest-python (pytest tests/e2e/): 43 passed. I also ran Test recursive and prefixed globs against find at speed of light #1323's glob_speed_recursive_test.py: 2 passed.
• TypeScript: just e2e-attest-ts (pnpm test:e2e) passed.
• Rust: cargo test -p dirsql --features cli --no-fail-fast passed, except the watcher test server_drops_the_rows_of_a_directory_moved_out_of_the_root. That test also fails in a full run on Stream the CLI's JSON rows #1327, and it passes 3 times out of 3 when its binary runs on its own.
• By hand: the identical-output queries above and the migration's verification snippet.

Changelog / Migrations

• packages/rust/changelog.d/2026-10-05-path-table-parallel-walk.md
• packages/rust/migrations.d/2026-10-05-path-table-parallel-walk.md (nothing breaks)