[thekevinbot]

Part of #957 (core half). The other half is #1001, implemented in #1002, stacked on this branch.

Problem

A cold scan over a large corpus is silent. The walk visits every file, then each matched file costs one on-file round trip plus whatever the table's ddl fires on insert. With #956's vector-index build that is an embed() round trip per ingested row — 41k rows on the reference corpus, minutes of wall time, nothing on the terminal. There is no way to tell a slow scan from a wedged one.

What this does

Both phases now draw a counter on stderr, rewritten in place:

dirsql: scanning 128413 files
dirsql: indexing 9204/41231 files (22%)

The walk reports a running count (it cannot know a total until it has found one); the ingest reports a fraction. When a phase ends its line is erased and replaced by one summary of what it cost:

dirsql: scanned 128413 files in 4.2s
dirsql: indexed 41231 files in 3m12s

stdout is untouched on every path.

Three constraints shaped it

• A pipe stays silent. The default consults stderr.is_terminal() before writing a byte, so | jq, 2> redirects and CI logs are byte-for-byte unchanged. Two of the six e2e tests exist only to pin that.
• Fast work stays invisible. Under the default a phase draws nothing until it has run 500 ms, so a quick scan leaves the terminal exactly as it found it.
• No new dependencies, per the issue. The line is a \r-rewritten counter rather than a drawn bar: a bar wants the terminal width and every portable way to ask for it is a crate. A counter reads the same at any width.

The knob

DIRSQL_PROGRESS, read by the core rather than by a launcher, so it governs an index built from any SDK:

Value	Effect
unset, auto	Terminal only, and only past the warmup. The default.
always, 1, true	Report regardless of terminal, from the first update.
never, 0, false	Silent, always. The embedder's opt-out.

Case-insensitive, whitespace-trimmed; anything unrecognized reads as auto, so a typo cannot stop a scan from running.

always skipping the warmup is deliberate and does double duty: it is the "watch my redirected run" affordance, and it is what makes the drawn output assertable from a subprocess e2e test without a pty.

Notes for review

• scan_directory keeps its signature and delegates to a new scan_directory_reporting(root, matcher, &mut on_file), so the benches and the ten call sites in tests/scanner.rs are untouched.
• impl Drop for Progress erases the live line, so an error mid-ingest cannot leave a half-drawn counter under the error message the caller is about to print.
• Two independent reporters rather than one shared across the split-phase build: prepare_resolved and finish_build may run on different threads for the async bindings.
• No SDK-level progress= argument. That would be exactly the per-SDK surface PARITY.md exists to prevent; the env var is the whole knob.

Red/green

• 6bc3b4c pushed the e2e first. CI went red on Integration tests (run 32417694440): test result: FAILED. 3 passed; 3 failed — the three progress assertions failing on the empty stderr they captured, the three silence assertions passing. That commit also reddened Lint, because the new test file had not been rustfmt'd; fixed in the implementation commit.
• eed9548 is the implementation, which flips those three green.

⚠️ The mutation gate reports 0 mutant(s) tested on this diff

Worth a maintainer's eye, because it is not specific to this PR.

rust / Unit mutation — changed lines passes here in 18 seconds with:

unit mutation: no surviving mutants — every mutation was caught (0 mutant(s) tested)

over a diff with 662 inserted lines under packages/rust/src, with the right base (BASE: 63d786008f…) and the right scan path (packages/rust). AGENTS.md says to treat exactly that signature as a bug rather than a pass. It reproduces locally with npx -y testing-conventions unit mutation --language rust --base origin/main packages/rust, so it is not a CI artifact.

Run directly against the same diff, cargo-mutants finds plenty:

$ cargo mutants --in-diff <this diff> --package dirsql --features cli
49 mutants tested in 16m: 1 missed, 35 caught, 13 unviable

And the one survivor was real: seen += 1 → seen *= 1 in scan_directory_reporting, i.e. the walk could report a frozen 0 files forever with every test still green, because the e2e only asserted the phase label. Fixed in 7864bae with an assertion, not a weaker test — the walk now has to summarize the four files it actually reaches. 0 missed after that.

I have not touched the gate; it is upstream (testing-conventions) and out of scope here. Flagging it because everything the rust mutation gate has passed since it landed may have been passing this way.

Manual run

AGENTS.md asks for the feature to be exercised, not just tested. Over 41 files with a 50 ms-per-file hook:

$ DIRSQL_PROGRESS=always dirsql query "SELECT count(*) AS n FROM items" -c .dirsql.toml
dirsql: scanned 43 files in 0.0s
dirsql: indexing 40/41 files (97%)     ← redrawn ~10×/s, then erased
dirsql: indexed 41 files in 2.2s
[{"n":41}]

Under a real pty (script -qec …, no env var set) the ingest phase starts reporting at 10/41 — half a second in, as designed — and the walk phase, which finishes instantly, stays completely silent. Piped with no env var, stderr is empty.

E2E Verification

• Ran e2e suites locally for every affected SDK
• Python SDK e2e: pass (32 passed)
• TypeScript SDK e2e: pass (16 passed)
• Rust core e2e: pass (--tests runs every suite under packages/rust/tests/, including the new scan_progress_e2e)
• packages/python/e2e-attestations/claude-tackle-957-lrm0z6.json receipt written
• packages/ts/e2e-attestations/claude-tackle-957-lrm0z6.json receipt written
• Command(s) run:
  • cargo test --manifest-path packages/rust/Cargo.toml --features cli --lib → 705 passed (25 new)
  • cargo test --manifest-path packages/rust/Cargo.toml --features cli --tests → all suites passed
  • cd packages/python && uvx testing-conventions e2e attest 'uv run python -m pytest tests/e2e/ -x -q'
  • cd packages/ts && uvx testing-conventions e2e attest 'pnpm test:e2e'
  • cargo clippy --workspace -- -D warnings, cargo fmt --all -- --check → clean
  • cargo mutants --in-diff … --package dirsql --features cli → 0 missed after 7864bae
• Result summary: green everywhere. Both receipts are committed, as a packages/rust/src change stales both bindings' attestations via [e2e].extra_scope.

Changelog / Migrations

• Changelog fragment: packages/rust/changelog.d/2026-08-20-scan-progress-reporting.md
• Migration fragment: packages/rust/migrations.d/2026-08-20-scan-progress-reporting.md — required, because this changes runtime behavior without an API change: a build that used to write nothing to stderr may now write to it. DIRSQL_PROGRESS=never is the documented restore.

Parity

PARITY.md updated. No drift: the reporter lives in the shared core and is driven from prepare_resolved / finish_build, so all three SDKs got it in the same commit and none carries its own copy. No follow-up issues needed for lagging SDKs.