Skip to content

refactor(rust): draw scan progress with indicatif - #1123

Merged
thekevinscott merged 5 commits into
mainfrom
claude/1081-indicatif-progress
Sep 17, 2026
Merged

thekevinscott merged 5 commits into
mainfrom
claude/1081-indicatif-progress

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1081.

Replaces the drawing and throttling internals of packages/rust/src/progress.rs
with indicatif, per Kevin's 2026-09-17 decision on the issue.

The public surface is unchanged: PROGRESS_ENV, the Mode enum and its four
modes, Progress::new's 7-arg signature, and the Progress::update /
Progress::finish call sites all stay exactly as they were. DIRSQL_PROGRESS
remains the user-facing knob with the same modes.

What moved

  • REDRAW_INTERVAL hand-throttling and the \r + drawn_width erase dance are
    gone; a ProgressBar on a term_like_with_hz(_, 10) draw target does both.
  • impl Drop for Progress is deleted. indicatif's BarState::drop already
    finishes with ProgressFinish::AndClear (its #[default]), so the manual
    erase-on-drop was redundant. Verified in indicatif's src/state.rs, and the
    mutation gate confirms no test was relying on our copy.
  • A bespoke SinkTerm: TermLike keeps DIRSQL_PROGRESS=always working when
    stderr is a pipe. indicatif has no override for drawing to a non-TTY
    (indicatif#87), so we own the terminal abstraction: clear_line() writes
    spaces rather than \x1b[2K, and cursor moves are no-ops at n == 0. The
    result is that the always byte stream carries zero ANSI escapes, same as
    before.
  • finish() writes the summary straight to the sink with writeln! rather than
    ProgressBar::println, which does not terminate its line.

indicatif = "0.18.6" and console = "0.16.6" are declared in
packages/rust/Cargo.toml. console was already transitive via indicatif and
adds no new crates; it is declared because line_width calls console::Term
directly and the declared-deps gate requires a real declaration. The module
header is updated — it documented a "No new dependencies" decision that this PR
reverses.

Invariant 1: a pipe stays silent — verified empirically

Captured piped stdout and redirected stderr for every mode against a 200-file
fixture, before (origin/main) and after (this branch's HEAD), and compared byte
for byte:

IDENTICAL  default.stdout   (12 bytes)
IDENTICAL  default.stderr   (0 bytes)
IDENTICAL  auto.stdout      (12 bytes)
IDENTICAL  auto.stderr      (0 bytes)
IDENTICAL  never.stdout     (12 bytes)
IDENTICAL  never.stderr     (0 bytes)
IDENTICAL  always.stdout    (12 bytes)
DIFFERS    always.stderr    (188 -> 6548 bytes)
IDENTICAL  bogus.stdout     (12 bytes)   # DIRSQL_PROGRESS=banana
IDENTICAL  bogus.stderr     (0 bytes)

Every non-TTY path that was silent before is still silent, byte for byte,
including the unrecognized-value fallback. The only stream that changed is
always on a pipe — the one mode that is supposed to draw there.

That stream is still escape-free and still ends in a whole line:

ESC bytes in always.stderr:  before 0,  after 0
tail (cat -A):  ^M<80 spaces>^Mdirsql: indexed 200 files in 0.1s$

It grew because each redraw now clears the full line width (80 columns on a
pipe) instead of only the characters it previously drew, and because indicatif's
leaky-bucket rate limiter allows a burst before settling to 10 Hz. Percentages
still floor (19/200 -> 9%). Harness: /tmp/claude/dirsql-1081/capture.sh.

Manual exercise

Ran the real CLI under a pty (script -qec) over a 300-file fixture with a
10 ms/file on-file command:

DIRSQL_PROGRESS=<mode> script -qec "dirsql query 'SELECT count(*) AS n FROM items' -c .dirsql.toml"

never:  332 bytes,  0 progress frames
always: 12066 bytes, 52 progress frames
auto:   7981 bytes, 47 progress frames

auto waits out the ~500 ms warmup, then throttles to ~10 Hz (counter steps
183 -> 192 -> 201, about 90 ms per frame), erases itself and leaves
dirsql: indexed 300 files in 3.3s followed by the query result. never draws
nothing. All three modes behave as documented.

Line delta

The issue estimated a ~200-line net deletion. That does not hold. Measuring
progress.rs above its #[cfg(test)] module:

before after delta
total lines 343 457 +114
non-blank 310 408 +98
code (non-blank, non-comment) 205 270 +65

The TermLike impl we have to own costs more than the throttling and erase
logic it replaces. The 2026-09-09 spike called this direction correctly (+44);
the issue's −200 premise was optimistic. The change is still worth making — the
throttling and redraw correctness now live upstream — but it is not a deletion.

Coverage

No floor moved. cargo llvm-cov --lib --features cli --branch on nightly, from
a clean profile:

progress.rs   regions 97.92%  functions 100.00%  lines 98.30%  branch 86.36%
TOTAL         regions 93.55%  functions  93.02%  lines 94.39%  branch 79.68%

Floors in root testing-conventions.toml are lines 92 / functions 91 /
regions 91 / branch 72. Every dimension clears with margin, so no floor was
re-pinned
. (An intermediate run reported progress.rs at ~80%; that was stale
merged profile data and did not reproduce after cargo llvm-cov clean --workspace.)

unit mutation (cargo-mutants, diff-scoped against origin/main): no surviving
mutants, 27 tested.

PARITY.md

Not updated, deliberately. No SDK-visible surface moved — progress is internal
to the Rust core and the Python and TypeScript SDKs expose nothing from it.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass
  • TypeScript SDK e2e: pass
  • Rust core e2e (if applicable): pass
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts)
  • Command(s) run: cargo test --workspace --features cli; just test-e2e;
    just e2e-attest-python; just e2e-attest-ts; just preflight;
    just test-packaging; cargo bench -p dirsql
  • Result summary: Rust workspace green (868 lib unit tests, 0 failures);
    Python e2e 37 passed; TS e2e 16 passed across 5 files; just preflight
    0 failing pairs, 3 skipped (packaging, which just test-packaging covers
    separately and which passed); benchmarks show no regression (progress drawing
    is not on any benched path). Receipts committed for both bindings — any
    packages/rust/src change puts both in scope via [e2e].extra_scope.

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment added under <root>/<pkg>/migrations.d/ (or: not required -- additive/bugfix only)

packages/rust/changelog.d/2026-09-17-indicatif-progress.md (Changed).

A migration fragment is required: this changes runtime behavior without an
API change. packages/rust/migrations.d/2026-09-17-indicatif-progress.md
records it — required changes _None._, behavior changes covering the
full-width padding and the extra frames in the always byte stream, plus the
list of things verified byte-identical.

RED for #1081. The hand-rolled reporter pads a shorter line only over the
tail of the line it drew before, so anything else sharing the row survives
a redraw. Clearing to the full width is what the indicatif swap brings.

skip-changelog: red test commit; the behavior change ships with the implementation
GREEN for #1081. The `\r`-rewritten counter, the 100 ms redraw throttle
and the drawn-width padding are indicatif's now. What stays ours is the
policy indicatif has no opinion about: the `DIRSQL_PROGRESS` modes, the
warmup before the first draw, the floored percentage and the summary line.

indicatif refuses to draw to a non-terminal and offers no override
(indicatif#87), so `Mode::Always` is served by a `TermLike` over the
reporter's own sink. It writes spaces rather than erase escapes, keeping a
redirected `always` capture free of ANSI, and the summary goes to the sink
directly because indicatif's `println` leaves the line unterminated once
the bar is cleared.

Invariant 1 verified empirically, not from the docs: piped stdout and
redirected stderr captured before and after across `auto`, `never`,
`always`, an unrecognized value and an unset variable. Every stream is
byte-identical but `always`'s stderr, which gains full-width padding.
Dropping a `ProgressBar` already clears its line (indicatif's default
`ProgressFinish::AndClear`), so the reporter's own `Drop` was a no-op
wrapped around one -- deleted, with the rationale moved onto the field it
belongs to. The test double's `flush` now counts instead of discarding.
@thekevinscott
thekevinscott merged commit c44a62e into main Sep 17, 2026
78 checks passed
@thekevinscott
thekevinscott deleted the claude/1081-indicatif-progress branch September 17, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

decide: replace the hand-rolled progress reporter with indicatif

2 participants