Progress on worker-backed function calls

PARITY.md

@@ -56,6 +56,15 @@ Python, Rust and TypeScript. No SDK exposes a knob for it: an SDK-level
 `progress=` argument would be exactly the kind of per-SDK surface this table
 exists to prevent.
 
+The same reporter covers **query time** (#1001): a `query` whose declared
+`[[dirsql.function]]` workers make round trips reports a running count of them
+(`dirsql: running 9204 worker calls`). It lives in `Db::query`, below every
+SDK's `query`/`await db.query`, so again all three change at once and none
+carries a copy. Notably this is the progress `dirsql-plugin-embeddings` gets
+for its `embed(content)`-per-row search: the plugin drives the Python SDK,
+which drives the same `Db::query`, so no plugin-side reporting exists or is
+needed.
+
 | API                        | Python                                         | Rust                                                 | TypeScript                                              |
 |----------------------------|------------------------------------------------|------------------------------------------------------|---------------------------------------------------------|
 | Constructor                | `DirSQL(root=None, *, tables=None, ignore=None, no_ignore=False, config=None, persist=False, persist_path=None, extensions=None)` | `DirSQL::builder().root(..).tables(..).ignore(..).config(..).persist(Option<path>).extensions(..).build()` (also `DirSQL::new`/`with_ignore` shortcuts) | `new DirSQL(configPath)` or `new DirSQL({ root?, tables?, ignore?, config?, persist?, persistPath?, extensions?, noIgnore? })` + `await db.ready` |

docs/reference/cli.md

@@ -483,6 +483,24 @@ dirsql: indexed 41231 files in 3m12s
 
 stdout is untouched — it carries the query result and nothing else.
 
+A query gets the same treatment when it calls a
+[worker-backed function](./config.md#dirsql-function). Those cost one round
+trip per row, so a query that calls one over a whole corpus is the slowest
+thing dirsql does:
+
+```
+dirsql: running 9204 worker calls
+```
+
+and, when the query ends:
+
+```
+dirsql: ran 41231 worker calls in 2m41s
+```
+
+A query that calls no worker prints nothing, whatever the setting — the
+reporting only ever speaks when there is something to report.
+
 By default this is **terminal-only, and only for work slow enough to wonder
 about**: a phase that finishes in under half a second prints nothing at all,
 and a run whose stderr is a pipe or a file prints nothing regardless of how

docs/reference/config.md

@@ -133,6 +133,11 @@ process alive for the rest of the invocation, sending it every subsequent
 call — one process total, never one per row or per file. The worker is torn
 down when the invocation ends.
 
+A query paying per-row round trips reports them on stderr while it runs
+(`dirsql: running 9204 worker calls`), on the same terms as every other
+[progress line](./cli.md#progress-reporting): terminal-only by default, silent
+on a pipe, and gone before the result is printed.
+
 The `timeout` bounds each **round-trip call**, not the query: a query that
 calls the function on 10 000 rows is 10 000 individually timed calls. A call
 that times out, or a worker that crashes or closes its pipes, fails the query

docs/reference/sdk.md

@@ -408,6 +408,10 @@ one summary of what each cost:
 dirsql: indexed 41231 files in 3m12s
 ```
 
+A `query` that calls a worker-backed function is reported the same way, as a
+running count of round trips (`dirsql: running 9204 worker calls`) — the shape
+`dirsql-plugin-embeddings` produces when it embeds a corpus one row at a time.
+
 This is terminal-only by default, and only for a phase that runs longer than
 half a second — a program whose stderr is a pipe, a file, or a log collector
 sees nothing, whatever the scan costs. Set `DIRSQL_PROGRESS=never` to

packages/python/e2e-attestations/claude-tackle-957-lrm0z6-worker-calls.json

@@ -0,0 +1,7 @@
+{
+  "command": "uv run python -m pytest tests/e2e/ -x -q",
+  "ran_at": 1787263769,
+  "exit_code": 0,
+  "commit": "bced5b72c85b9c50073fd6326c1528326a1bd9d3",
+  "branch": "claude/tackle-957-lrm0z6-worker-calls"
+}

packages/rust/changelog.d/2026-08-20-worker-call-progress.md

@@ -0,0 +1,3 @@
+**Added**
+
+- **A query reports the worker round trips it is paying for.** A `[[dirsql.function]]` costs one worker round trip per row, so a query calling one over a whole corpus — `dirsql-plugin-embeddings` runs `embed(content)` that way — can run for minutes inside a single `query()` with nothing on the terminal. Core now counts the round trips and draws them with the same reporter the startup scan uses (`dirsql: running 9204 worker calls`, erased at the end of the query and replaced by `dirsql: ran 41231 worker calls in 2m41s`), under the same `DIRSQL_PROGRESS` gate: terminal-only and slow-work-only by default, silent on a pipe. A query that calls no worker prints nothing at any setting. The count is drawn **and erased by dirsql itself** rather than by the worker, which is what keeps the result off the progress line: a worker's own bar cannot be cleaned up, since the worker is SIGKILLed and is torn down only after the result has been printed. (#1001, #957)

packages/rust/migrations.d/2026-08-20-worker-call-progress.md

@@ -0,0 +1,51 @@
+### A query reports its worker round trips on stderr (#1001)
+
+#### Summary
+
+`Db::query` now brackets each statement with a progress phase that counts the
+round trips its declared `[[dirsql.function]]` workers make, and reports them
+on stderr. No API changes; what changes is that a query that used to write
+nothing to stderr may now write to it. Gated exactly as the startup scan's
+reporting is: terminal-only, and only past a half-second warmup, unless
+`DIRSQL_PROGRESS` says otherwise.
+
+#### Required changes
+
+_None._ `DIRSQL_PROGRESS=never` is the opt-out for a program that insists on an
+empty stderr, and it already covers the scan's reporting too.
+
+#### Deprecations removed
+
+_None._
+
+#### Behavior changes without code changes
+
+- **stderr may carry a worker-call counter on a terminal.** A query whose
+  declared functions make round trips for longer than 500 ms draws
+  `dirsql: running <n> worker calls`, rewritten in place, then erases it and
+  prints `dirsql: ran <n> worker calls in <t>`. A query that calls no worker
+  draws nothing, at any setting.
+- **Non-terminal stderr is unchanged**, at any duration. Anything asserting on
+  captured stderr keeps passing.
+- **stdout is unchanged.** The counter is erased before `query()` returns, so
+  the result is never printed onto a leftover progress line.
+- **`functions::register_all` takes a third argument**, the shared
+  `CallReporter`. It is `pub(crate)`, so no crate consumer can be calling it.
+
+#### Verification
+
+```bash
+# A worker-backed function over a corpus, forced on so no terminal is needed:
+DIRSQL_PROGRESS=always dirsql query "SELECT fake(basename) AS v FROM './*.txt'" -c ./.dirsql.toml
+# dirsql: running 13 worker calls      (rewritten in place, then erased)
+# dirsql: ran 14 worker calls in 0.7s
+# [{"v":"f1.txt"}, …]
+
+# Piped: stderr is empty, and stdout is parseable JSON with nothing glued to it.
+dirsql query "SELECT fake(basename) AS v FROM './*.txt'" -c ./.dirsql.toml 2>err.txt | jq .
+wc -c err.txt   # -> 0
+
+# A query that calls no worker reports nothing even when forced on:
+DIRSQL_PROGRESS=always dirsql query "SELECT 1" -c ./.dirsql.toml
+# -> [{"1":1}]      (nothing on stderr)
+```

packages/rust/src/db.rs

@@ -1,8 +1,10 @@
 use rusqlite::Connection;
 use std::collections::{HashMap, HashSet};
 use std::path::{Path, PathBuf};
+use std::sync::Arc;
 use thiserror::Error;
 
+use crate::functions::CallReporter;
 use crate::parsed_vtab;
 use crate::path_table::{self, PathTable, Resolution};
 use crate::scanner;
@@ -264,6 +266,10 @@ pub struct Db {
     /// The persistent cache a parsed path-table reuses across runs. `None` for
     /// an ephemeral index, which has nowhere to cache to.
     path_table_cache: Option<PathBuf>,
+    /// Counts the worker round trips a query pays for. Shared with every
+    /// declared function registered on `conn`; inert until a function is both
+    /// declared and called.
+    calls: Arc<CallReporter>,
 }
 
 /// The skip rules a fresh `Db` starts with.
@@ -299,6 +305,7 @@ impl Db {
             path_table_gitignore: true,
             path_table_parser: None,
             path_table_cache: None,
+            calls: CallReporter::new(),
         })
     }
 
@@ -321,6 +328,7 @@ impl Db {
             path_table_gitignore: true,
             path_table_parser: None,
             path_table_cache: None,
+            calls: CallReporter::new(),
         })
     }
 
@@ -365,6 +373,12 @@ impl Db {
         self.path_table_cache = Some(path);
     }
 
+    /// The reporter every declared function increments. Handed to
+    /// `functions::register_all` at build time.
+    pub(crate) fn call_reporter(&self) -> &Arc<CallReporter> {
+        &self.calls
+    }
+
     /// Borrow the underlying SQLite connection. Internal use only — exposed
     /// to the `persist` module so it can manage the sidecar tables.
     #[doc(hidden)]
@@ -687,6 +701,12 @@ impl Db {
     /// internal writes (`insert_row`, delete-by-file, persist), which never go
     /// through `query()`, are unaffected.
     pub fn query(&self, sql: &str) -> Result<Vec<HashMap<String, Value>>> {
+        // Bracket the whole statement, prepare included: a worker-backed
+        // function called on every row makes this the slowest thing dirsql
+        // does, and the guard is what erases the counter before the caller
+        // prints the rows.
+        let _calls = self.calls.phase();
+
         // Each iteration must register a table no earlier iteration did; a
         // repeat means the fallback is not making progress, so the SQLite
         // error stands. That is what bounds the loop.