[thekevinbot]

Fixes #1216
Fixes #1187

Status

Removes the per-file machinery the batched on-file contract left dead:

• The persisted row cache for a parsed path-table: parsed_cache.rs, collect_rows_cached and its fakes, the vtab's cache argument (FIXED_ARGS 5 -> 4), Db::set_path_table_cache, and the _dirsql_parsed_rows create/clear calls in persist.rs. The parser runs on every start.
• Per-table failure accumulation. A config-declared on-file command that fails now fails the build with DirSqlError::TableCommand { name, message }; OnFileFailure.path is a file path again. scan_failures() stays for the programmatic per-file hook (Rust/Python/TS parity surface, untouched).
• The CLI's PARTIAL_SCAN_EXIT = 23, report_scan_failures, and the dirsql: skipping ... lines. A failed command exits 1 with the table named on stderr.
• {path} from every fixture, doc and generic-runner test.

Kept, with the reason:

• {root} substitution and the {path} startup rejection. The issue allowed removing substitution only "if nothing else uses substitution"; {root} is the documented contract used by the embeddings plugin, docs/howto/write-a-plugin.md, and every config-driven test fixture. So rg '\{path\}' packages/rust/src docs still returns the rejection constant and its tests (on_file.rs, two lib.rs tests) and the one doc line that documents the rejection (hooks.md), beyond the changelog entries.

Tests rewritten to the batched contract: tests/on_file.rs (four command-failure tests assert the build error), tests/scan_failures.rs (clean scan only), tests/scan_failures_e2e.rs -> tests/on_file_failure_e2e.rs (exit 1, table named on stderr, nothing on stdout), tests/multi_config.rs, tests/on_file_e2e.rs (server stays up and answers 503 naming the table). Deleted: tests/persist_parsed_path_table.rs and the Python SDK's tests/e2e/persist_query_test.py (both asserted the row cache). The scan-failure binding test headers in Python and TS no longer describe exit 23. Fixtures in tests/extensions.rs and tests/no_implicit_files_table.rs fed cat non-JSON and relied on the silent empty table; they now emit rows.

Docs touched only where the old text became false: hooks.md, cli.md, path-tables.md, config.md, define-tables.md, persist.md, PARITY.md.

Source size (git diff --numstat origin/main..HEAD): 2143 lines removed, 426 added across 33 files; packages/rust/src alone 1235 removed, 108 added.

Rebase onto main (2026-10-04)

Rebased over #1248 (per-table hook rows as one rowid range), #1239 (parallel walk), #1236 (glob prefix walk), #1250 and #1247. Conflicts in db.rs, parsed_vtab.rs and persist.rs; resolution keeps main's new ingest and walk code and this PR's removals:

• dirsql_parsed module arity: main had added the index root as a sixth argument after the cache path; the cache slot is gone, so FIXED_ARGS is now 5 (root, glob, parser, gitignore switch, index root) and path_table_ddl emits exactly that. tests/schema_inference.rs was still passing the empty cache slot and is corrected.
• drop_user_tables keeps main's DELETE FROM _dirsql_internal_ranges and drops the parsed_cache::clear call.
• persist.rs doc comments that still described a second cache tier (_dirsql_parsed_rows) and the PARITY rows pointing at the deleted persist_parsed_path_table.rs are trimmed.

Hand check on the rebased release binary (cargo build --release --features cli), temp dir with a.md, b.md, c.md and a parse.sh that emits one row per file but exits 3 when handed b.md:

• dirsql query "SELECT name FROM './*.md'" --on-file 'sh parse.sh' -> exit 1, nothing on stdout, stderr dirsql query: SQLite error: on-file command failed: command sh parse.sh failed (exit 3): cannot parse /tmp/.../b.md.
• Same query after renaming b.md away -> [{"name":"a.md"},{"name":"c.md"}], exit 0.
• strings target/release/dirsql | grep -c 'skipping '->0`.

Hand verification (pre-rebase)

Release build (cargo build --release --features cli) over a temp dir with three *.md, two *.tag, and a counting wrapper hook:

• --on-file 'sh count.sh {path}' -> dirsql query: failed to build the index: on-file command ... uses {path} ..., exit 1.
• --on-file 'sh count.sh' over ./*.md -> one spawn with 3 paths, rows [a,b,c], exit 0.
• Config with notes and tags tables -> two spawns (3 paths, 2 paths), one per table, exit 0.
• Config whose command exits 3 -> dirsql query: failed to load config: table notes: on-file command failed: ... (exit 3): nope, exit 1.
• strings target/release/dirsql | grep -c 'skipping '` -> 0.

Speed of light

Five speed-of-light tests run sequentially on a quiet machine after the change (commands from #1205):

test	command	result
dossier	uv run python -m pytest tests/e2e/use_case_dossier_test.py -q -rxX -k speed_of_light	passed (44.93s)
join	... use_case_join_test.py ...	xfailed, strict: "two serial tree walks run over the speed-of-light bar" (48.40s)
notebook	... use_case_notebook_test.py ...	passed (10.83s)
transcripts	... use_case_transcripts_test.py ...	xfailed, strict: "ingesting the parser's JSON rows costs more than a tenth of the parse itself" (18.14s)
embeddings	uv run --with-editable ../../packages/python python -m pytest tests/e2e/speed_of_light_test.py -q -rxX (in plugins/dirsql-plugin-embeddings)	passed (9.91s)

The five verdicts are the ones origin/main's markers expect (join and transcripts are strict xfail there). An earlier run under a load average of 34 had dossier and notebook miss their bars (1.098s vs 0.751s; 1.701s vs 1.569s); both pass unloaded.

E2E Verification

Re-run on the rebased head (2026-10-04):

• Ran e2e suites locally for every affected SDK
• Python SDK e2e: pass (just e2e-attest-python: 39 passed, 1 xfailed in 144.88s)
• TypeScript SDK e2e: pass (just e2e-attest-ts: 6 files, 18 tests passed)
• Rust core e2e: pass (cargo test --workspace --features cli, exit 0; cargo clippy --workspace --features cli --all-targets -- -D warnings clean; cargo fmt --all -- --check clean)
• packages/python/e2e-attestations/claude-1216-on-file-cleanup.json receipt written (just e2e-attest-python)
• packages/ts/e2e-attestations/claude-1216-on-file-cleanup.json receipt written (just e2e-attest-ts)
• Command(s) run: cargo test --workspace --features cli, cargo clippy --workspace --features cli --all-targets -- -D warnings, cargo fmt --all -- --check, uv run maturin develop --release (packages/python), pnpm build (packages/ts), just e2e-attest-python, just e2e-attest-ts, just preflight
• Result summary: all green; just preflight reports 0 failing pair(s), 3 skipped (the three skips are the packaging gates, which need CI-built artifacts)

Changelog / Migrations

• Changelog fragment added under packages/rust/changelog.d/2026-10-02-on-file-no-partial-scan.md
• Migration fragment added under packages/rust/migrations.d/2026-10-02-on-file-no-partial-scan.md

Also closes #1187

With the per-file loop gone, a parser that fails to spawn fails the statement with that error instead of per-file skip lines followed by parser produced no rows ...; cannot infer a schema. Hand check on this branch's release binary, dirsql query "SELECT * FROM './*.md'" --on-file 'definitely-not-a-program':

dirsql query: SQLite error: on-file command failed: failed to spawn `definitely-not-a-program`: not found on $PATH

exit 1. Main prints two skipping lines and ends on the schema-inference error.