Skip to content

REPL slice 3: table output when rows are read, JSON when they are piped - #1006

Merged
thekevinscott merged 4 commits into
claude/epic-953-slice2-reedlinefrom
claude/epic-953-slice3-format
Aug 21, 2026
Merged

thekevinscott merged 4 commits into
claude/epic-953-slice2-reedlinefrom
claude/epic-953-slice3-format

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Aug 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #989

Slice 3 of 3 under #953 — the last one. Stacked on #998, which is stacked on #991, so this PR's diff is slice 3's own. Stack: #991 ← #998 ← this PR.

A JSON array is right for dirsql "<sql>" | jq and wrong for a person: SELECT * FROM './' over a few thousand files puts a few thousand JSON objects on one line. Rows now follow their destination — a table when stdout is a terminal, the same JSON array when it is not.

Piped output is unchanged, byte for byte. That is the regression that mattered most here, and it is asserted directly rather than by eye: auto in a pipe and --format json must produce identical bytes.

The open question, resolved

#989 asked whether the user gets a say once two formats exist, and ruled out .mode (dot-commands were closed in #953). The answer is the issue's own preferred option: --format {auto,table,json}, defaulting to auto, valid in the REPL and dirsql query alike. It stays in flag-space rather than reopening a meta-command namespace, and it serves the one-shot query too.

auto reads stdout, not stdin, as #989 asked explicitly. dirsql > rows.json typed at a terminal is still headed for a file, and the file wants JSON.

The flag is declared on Cli and QueryArgs — the same places --on-file lives — and deliberately not on ConfigArgs, which is flattened into ServerArgs too. The server speaks JSON over HTTP and could not honour a rendering flag; a test pins that it still rejects one.

Also settled

  • Column widths: cells truncate past 60 characters with …. One content column holding a whole file would otherwise set the width of every row.
  • NULL: spelled out, so it cannot be read as an empty string. An empty string stays empty — both halves of that distinction are tested.
  • Row-count footer: yes, and singular at one row. "1 rows" is the kind of detail that makes output look unfinished.
  • Paging: explicitly out of scope, said so in the docs rather than silently omitted. Pipe to less. Terminal-width-aware layout is out of scope for the same reason — the per-cell cap solves the problem the issue actually named.
  • Ragged results: every column any row carries is shown, and a row missing one renders NULL rather than dropping the column.

What the manual run caught that the tests had not

AGENTS.md asks for a manual run because tests prove what they assert and nothing else. This is what the table looked like the first time:

basename  content
--------  ----------
a.md      alpha

b.md      beta beta

content values end in a newline, so one row spanned two lines and the table stopped being a table. Every unit test passed — none of them had put a newline in a cell. Fixed by escaping control characters on the way into a cell (\n, \t, \r, and \u{…} for the rest); a bare \r is worse than a newline, since it rewinds the cursor over a row already drawn. --format json returns values unaltered.

After:

basename  content
--------  -----------
a.md      alpha\n
b.md      beta beta\n

2 rows

Manual run (AGENTS.md)

Against a directory holding a.md (6 B) and b.md (10 B):

dirsql "SELECT basename, size FROM './' ORDER BY basename" --format table
# basename  size
# --------  ----
# a.md      6
# b.md      10
#
# 2 rows

dirsql "SELECT basename, size FROM './' ORDER BY basename" | cat
# [{"basename":"a.md","size":6},{"basename":"b.md","size":10}]

The second is the one that matters: a pipe is not a terminal, so auto picks JSON and the bytes are what they always were.

The terminal half is confirmed from the other side too — a_terminal_session_renders_a_table_by_default drives a real pty with no --format and sees a table rather than a JSON array.

PARITY

No drift. The CLI is one shared implementation in the Rust core that every SDK's launcher reaches through run_cli; no SDK public API changed. PARITY.md needs no edit — stated here rather than skipped silently, per AGENTS.md.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass (37 passed)
  • TypeScript SDK e2e: pass (16 passed)
  • Rust core e2e: pass — cargo test --features cli --tests, 45 binaries green, including the new repl_format_e2e (12) and the updated repl_tty_e2e (8)
  • packages/python e2e attestation refreshed (a packages/rust/src change stales both bindings)
  • packages/ts e2e attestation refreshed
  • Command(s) run:
    cargo test --manifest-path packages/rust/Cargo.toml --features cli --lib      # 755 passed
    cargo test --manifest-path packages/rust/Cargo.toml --features cli --tests    # 45 binaries green
    cd packages/python && uv run python -m pytest tests/e2e/ -q                   # 37 passed
    cd packages/ts && pnpm test:e2e                                               # 16 passed
    cd packages/python && uvx testing-conventions e2e attest 'uv run python -m pytest tests/e2e/ -x -q'
    cd packages/ts     && uvx testing-conventions e2e attest 'pnpm test:e2e'
    
  • Result summary: all green. The one pre-existing, environment-only failure is tests/vtab.rs::unreadable_file_yields_null_content_without_erroring_the_row — this sandbox runs as uid 0, so a chmod 000 file is still readable and the NULL-content assertion cannot hold. Green in CI, unrelated to this change.

Changelog / Migrations

  • Changelog fragment: packages/rust/changelog.d/2026-08-20-format-flag.md
  • Migration fragment: packages/rust/migrations.d/2026-08-20-format-flag.md — required: rows printed to a terminal change shape. Piped output is untouched, but a script capturing output through a pty sees a table now and needs --format json.

Red/green

6f693fd pushed the failing suite alone — nine red, three passing as regression guards (piped JSON unchanged; the server still rejects the flag) — and CI was observed red on Integration tests. The implementation follows in 060576f.

Other gates run locally

unit colocated-test and unit lint (rust) exit 0 · cargo clippy --workspace -- -D warnings and cargo fmt --all --check clean · unit coverage (rust, nightly) exit 0, with actuals above every floor: lines 93.55 (floor 92), regions 92.81 (91), functions 92.52 (91), branch 78.07 (72) · unit mutation against this slice's diff, full-suite scope: 44 mutants, 43 caught, 1 unviable, zero survivors.

One unit lint note worth recording for whoever writes the next renderer: a unit test may import only super:: and pure std, so serde_json::json! is out of bounds. serialize.rs gets away with it because it imports json! at module level for production use; table.rs does not, so its tests build rows with Value::from_iter([(String, Value)]) — an object constructor reachable through super::* alone.

claude added 4 commits August 20, 2026 22:18
#989 asks for a table when rows are read with eyes and JSON when they are
piped to `jq`, and leaves open whether the user can choose. These tests
answer that with a `--format {auto,table,json}` flag -- the issue's own
preferred option, and the one that stays in flag-space rather than
reopening the dot-command question #953 closed.

Nine fail today; three pass as regression guards. The guards are the point
of the shape: `dirsql "<sql>" | jq` must keep printing exactly the bytes it
prints now, and the server must keep rejecting a rendering flag its
transport cannot honour.

Every case forces the format or runs through a pipe, so none of it needs a
terminal.

Refs #989

skip-changelog: test-only commit; the fragment lands with the implementation

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GU6WPGs4bbkad8QhQfyXDD
A JSON array is right for `dirsql "<sql>" | jq` and wrong for a person:
`SELECT * FROM './'` over a few thousand files puts a few thousand JSON
objects on one line. Rows now follow their destination -- a table when
stdout is a terminal, the same JSON array when it is not.

#989 left open whether the user gets a say. They do, through
`--format {auto,table,json}`, which is the issue's own preferred option and
the one that stays in flag-space: `.mode` would mean reintroducing the
dot-command namespace #953 closed, and a flag serves `dirsql query` too.
`auto` is the default and reads **stdout**, not stdin -- `dirsql >
rows.json` typed at a terminal is still headed for a file, and the file
wants JSON. `server` does not take the flag; its transport is JSON over
HTTP and could not honour it.

Piped output is unchanged byte for byte, which is the regression that
mattered most here and is asserted directly rather than by eye.

The renderer is deliberately modest, in `sqlite3`'s `.mode column` style:
aligned columns, a rule under the header, a row count, `NULL` spelled out
so it cannot be read as an empty string, and every column any row carries
so a ragged result drops nothing.

Two things happen to a value on its way into a cell, and a `content` column
is the reason for both. Its newlines are escaped, because the manual run
showed a single unescaped one collapsing the whole layout -- one row spans
as many lines as its longest value and no column lines up with any other.
And it is truncated past 60 characters, because a cell holding a whole file
otherwise sets the width of every row. Tabs, carriage returns and other
control characters are escaped alongside the newlines; a bare `\r` would
rewind the cursor over a row already drawn. `--format json` returns values
unaltered. Terminal-width-aware layout and paging are out of scope, said so
in the docs rather than silently omitted.

Slice 2's pty cases now ask for `--format json`: on a terminal they would
otherwise read rows through a table layout, making every one of them a
rendering test as well as a termination test. One new case covers the
default they gave up.

Fixes #989

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GU6WPGs4bbkad8QhQfyXDD
@thekevinscott
thekevinscott merged commit 6f9e34a into main Aug 21, 2026
61 checks passed
@thekevinscott
thekevinscott deleted the claude/epic-953-slice3-format branch August 21, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

REPL slice 3: table output in TTY, JSON in pipes

3 participants