Skip to content
7 changes: 7 additions & 0 deletions PARITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,13 @@ for its `embed(content)`-per-row search: the plugin drives the Python SDK,
which drives the same `Db::query`, so no plugin-side reporting exists or is
needed.

That line splits out cache hits (#1034): a worker may flag a response it served
from a cache of its own with `{"meta": {"cached": true}}`, and the count reports
those separately (`dirsql: ran 41231 worker calls in 2m41s (38104 cached)`). The
flag is part of the **worker protocol**, not any SDK's API surface — no SDK
gained or lost a symbol, and the parsing and the line both live in the shared
core, so all three SDKs change at once. **No drift.**

| API | Python | Rust | TypeScript |
|----------------------------|------------------------------------------------|------------------------------------------------------|---------------------------------------------------------|
| Constructor | `DirSQL(root=None, *, tables=None, ignore=None, no_ignore=False, config=None, persist=False, persist_path=None, extensions=None)` | `DirSQL::builder().root(..).tables(..).ignore(..).config(..).persist(Option<path>).extensions(..).build()` (also `DirSQL::new`/`with_ignore` shortcuts) | `new DirSQL(configPath)` or `new DirSQL({ root?, tables?, ignore?, config?, persist?, persistPath?, extensions?, noIgnore? })` + `await db.ready` |
Expand Down
18 changes: 18 additions & 0 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -498,6 +498,24 @@ and, when the query ends:
dirsql: ran 41231 worker calls in 2m41s
```

A worker that keeps a cache of its own can say so per response
(`{"meta": {"cached": true}}` — see [Worker
protocol](./config.md#worker-protocol)), and those round trips are then split
out, because the number that answers *"will this be fast next time"* is not
the total:

```
dirsql: running 9204 worker calls (8811 cached)
dirsql: ran 41231 worker calls in 2m41s (38104 cached)
```

Both figures count **worker round trips, not rows**. A function declared
`deterministic = true` lets SQLite reuse one answer for identical arguments
within a query, so repeated content never reaches the worker at all and both
numbers sit below the row count on a corpus with duplicates. The split covers
only what the worker itself reports serving from cache. A worker that sends no
`meta` gets the unsplit line above, not a `(0 cached)`.

A query that calls no worker prints nothing, whatever the setting — the
reporting only ever speaks when there is something to report.

Expand Down
9 changes: 9 additions & 0 deletions docs/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,15 @@ request line in, one response line out, per call:
JSON-text vectors) — or `{"err": "message"}`, which **fails the query**
with that message. An `{"err": ...}` response leaves the healthy worker
running; only transport failures (timeout, crash) recycle it.
- **Optional `"meta"`**, alongside `"ok"`: a second top-level object carrying
facts about how the answer was produced rather than what it is. One key is
read today — `{"ok": [...], "meta": {"cached": true}}` says the worker
answered from a cache of its own, which dirsql reports as the `(N cached)`
split on its [worker-call progress line](./cli.md#progress-reporting). The
field is **optional and advisory**: a worker that omits it is a worker that
reports no cache hits, keys dirsql does not recognize are ignored (they
always have been), and a `meta` of an unexpected shape is ignored too rather
than failing a query over a progress counter.
- **stderr passes through** to `dirsql`'s stderr, so a worker's progress
bars and download logs reach the terminal.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"command": "just test-e2e",
"ran_at": 1788557530,
"exit_code": 0,
"commit": "97fdc1dfbdf490f71bd2067d8be66a50dce22bcb",
"branch": "claude/tackle-1034-cachesplit"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
**Added**

- **The worker-call progress line splits out cache hits.** A worker may now flag a response it served from a cache of its own — `{"ok": <value>, "meta": {"cached": true}}` — and dirsql reports those separately: `dirsql: running 9204 worker calls (8811 cached)`, and `dirsql: ran 41231 worker calls in 2m41s (38104 cached)` when the query ends. A raw round-trip count reads identically whether every call ran a model or every call was a dictionary lookup, so it cannot answer the question a user watching a slow first embedding build is actually asking: will the second one cost this too. Core cannot derive the split — a cache hit *is* a round trip — so only the worker can report it, and `meta` is the generic channel for that: any `[[dirsql.function]]` may set it, and core stays ignorant of what any particular worker caches. The field is optional and advisory; a worker that omits it reports no cache hits and gets exactly the line it got before, with no `(0 cached)`. An unrecognized `meta` shape is ignored rather than failing the query. (#1034, #1001)
44 changes: 44 additions & 0 deletions packages/rust/migrations.d/2026-09-04-worker-call-cache-split.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
## Summary

The `[[dirsql.function]]` worker response contract gains an optional top-level
`"meta"` object next to `"ok"`. dirsql reads one key from it today,
`{"meta": {"cached": true}}`, and reports those round trips as the `(N cached)`
split on the worker-call progress line.

## Required changes

_None._ The change is additive on the wire and in the parser: the response
parser has always read `err`, else `ok`, and ignored every other key, so a
worker written before this release keeps working byte-for-byte unchanged. The
break is in the *documented* contract — `docs/reference/config.md` enumerated
the response shape exhaustively — not in what any existing worker sends.

## Deprecations removed

_None._

## Behavior changes without code changes

- A worker that already emitted a top-level `"meta"` key for its own purposes
now has `meta.cached` interpreted by dirsql. The only effect is on the
progress line's `(N cached)` figure; the value bound into the query is still
the `"ok"` field alone, and no query result changes.
- The worker-call summary line grows a parenthetical when a worker reports
cache hits: `dirsql: ran 41231 worker calls in 2m41s (38104 cached)`. A
worker that reports none gets the unchanged line — there is no `(0 cached)`.
The line is stderr-only and still gated on `DIRSQL_PROGRESS` (terminal-only
and slow-work-only by default), so piped and redirected runs are unaffected.
- Both figures count **worker round trips, not rows**: a `deterministic = true`
function lets SQLite reuse an answer for identical arguments within a query,
and those repeats never reach the worker.

## Verification

Declare a worker that flags its responses and run a query that calls it:

```console
$ DIRSQL_PROGRESS=always dirsql query "SELECT embed(basename) FROM './*.txt'"
dirsql: ran 4 worker calls in 0.0s (4 cached)
```

A worker that sends no `meta` prints `dirsql: ran 4 worker calls in 0.0s`.
Loading
Loading