REPL slice 2: line editing, history, and multi-line entry via reedline

docs/reference/cli.md

@@ -55,12 +55,12 @@ pipeline, same flags, same output. See that section for config discovery,
 dirsql
 # dirsql 0.2.7 — this directory is a database.
 #
-#   SELECT basename, size FROM './' ORDER BY size DESC LIMIT 5
-#   SELECT path FROM './**/*.md' WHERE content LIKE '%TODO%'
+#   SELECT basename, size FROM './' ORDER BY size DESC LIMIT 5;
+#   SELECT path FROM './**/*.md' WHERE content LIKE '%TODO%';
 #
 # `exit`, `quit`, or Ctrl-D to leave.
 #
-# dirsql> SELECT count(*) AS files FROM './'
+# dirsql> SELECT count(*) AS files FROM './';
 # [{"files":128}]
 # dirsql>
 ```
@@ -79,10 +79,52 @@ rather than re-walking the directory each time, and the live watcher keeps it
 fresh between them. Files the scan had to skip are named on stderr once, up
 front.
 
+### Where a statement ends
+
+At its semicolon — the same rule `sqlite3` uses, and **SQLite's own tokenizer**
+decides where that semicolon is. So a statement can be laid out over as many
+lines as it needs, and a `;` inside a string literal, a comment, or a
+`BEGIN … END` body is not mistaken for the end of one:
+
+```
+dirsql> SELECT basename, size
+   ...> FROM './'
+   ...> ORDER BY size DESC
+   ...> LIMIT 5;
+```
+
+The `...>` prompt says the statement is still open. `exit`, `quit`, and a blank
+line are not SQL, so they are taken as typed rather than waiting for a
+terminator.
+
+### Editing and history
+
+The prompt is a full line editor ([reedline](https://github.com/nushell/reedline)),
+with the emacs bindings a shell prompt has:
+
+| Key | Does |
+|---|---|
+| ↑ / ↓ | Walk back and forth through history. |
+| Ctrl-R | Reverse-search history; type to narrow, Enter to accept. |
+| Ctrl-A / Ctrl-E | Jump to the start / end of the line. |
+| Alt-B / Alt-F | Move back / forward a word. |
+| Ctrl-W, Ctrl-K, Ctrl-Y | Kill the previous word, kill to end of line, yank it back. |
+| Ctrl-C | Abandon the line and return to a fresh prompt. **Does not exit.** |
+| Ctrl-D | Leave. |
+
+History is kept in one file for every directory — a query worked out in one
+project is worth recalling in the next, the same way `sqlite3` keeps a single
+`~/.sqlite_history`. It holds the last 1000 statements, at
+`$XDG_DATA_HOME/dirsql/history`, falling back to
+`~/.local/share/dirsql/history` (`%APPDATA%\dirsql\history` on Windows). If
+none of those resolve, history is kept in memory for the session only.
+
 ### Terminal vs. pipe
 
-The prompt and banner appear only when **stdin is a terminal**. From a pipe or
-a redirect there is neither, so results arrive alone:
+The prompt, banner, editor, and history exist only when **stdin is a terminal**.
+From a pipe or a redirect there is none of that, and the terminator rule does
+not apply either: a redirected script is not being typed, so there is no
+continuation prompt to hang it off. **One statement per line, no `;` needed:**
 
 ```bash
 printf "SELECT 1 AS n\nSELECT 2 AS n\n" | dirsql
@@ -92,7 +134,7 @@ printf "SELECT 1 AS n\nSELECT 2 AS n\n" | dirsql
 dirsql < queries.sql > rows.jsonl
 ```
 
-One statement per line, in both modes. Blank lines do nothing.
+Blank lines do nothing in either mode.
 
 ### Leaving
 
@@ -112,9 +154,9 @@ the one behavioral difference from `dirsql query`, which exits `1` on the first
 failure:
 
 ```
-dirsql> SELECT nope FROM missing
+dirsql> SELECT nope FROM missing;
 dirsql: SQLite error: no such table: missing
-dirsql> SELECT 1 AS n
+dirsql> SELECT 1 AS n;
 [{"n":1}]
 ```
 

packages/python/changelog.d/2026-08-20-curtaincall-dev-dependency.md

@@ -0,0 +1,3 @@
+**Changed**
+
+- Added [curtaincall](https://github.com/thekevinscott/curtaincall) to the dev dependency group, for the REPL's interactive e2e cases: they need a real PTY behind a VT100 emulator, since the line editor repaints by moving the cursor and its behavior cannot be read off an output stream. **Nothing about the published package changes** — the dev group is excluded from the wheel, so no installed dependency, import, or runtime behavior is affected. It carries a `python_version >= "3.12"` marker against this package's `>=3.10` floor (curtaincall declares `requires-python >=3.12`), and the suite skips itself below that. (#988)

packages/python/e2e-attestations/claude-epic-953-slice2-reedline.json

@@ -0,0 +1,7 @@
+{
+  "command": "uv run python -m pytest tests/e2e/ -x -q",
+  "ran_at": 1787263606,
+  "exit_code": 0,
+  "commit": "f51e578dbee1f6645d0dd3a3323acf60e0083313",
+  "branch": "claude/epic-953-slice2-reedline"
+}

packages/python/pyproject.toml

@@ -44,6 +44,11 @@ dev = [
     "maturin>=1.0",
     # ty is pre-1.0 -- pin tightly. Re-pin in lockstep with CI.
     "ty==0.0.42",
+    # Real PTY + VT100 for the REPL's interactive e2e cases, which need a
+    # screen rather than an output stream. Marked because curtaincall declares
+    # `requires-python >=3.12` against this package's >=3.10 floor; the suite
+    # skips itself below that (thekevinscott/curtaincall#26).
+    'curtaincall>=0.3; python_version >= "3.12"',
 ]
 
 [project.scripts]

packages/python/tests/e2e/repl_interactive_test.py

@@ -0,0 +1,164 @@
+"""E2E tests for the REPL behaviors that need a terminal, not an output stream.
+
+The `dirsql` REPL's line editor repaints by moving the cursor, so what a pipe
+sees is a byte stream rather than a screen: a harness reading it can only guess
+when a redraw has finished, and a keystroke sent into the gap after a statement
+runs is read by nothing. These tests therefore drive the real console script
+through [curtaincall](https://github.com/thekevinscott/curtaincall) -- a real
+PTY behind a VT100 emulator -- and wait on the *screen* before each keystroke.
+
+Nothing is mocked: real console script, real binary, real pty, real editor,
+real filesystem, real SQLite. `HOME` and `XDG_DATA_HOME` point into the test's
+own tmp tree, so the suite never touches the developer's history file.
+
+What lives here is what only a screen can answer -- history recall with the up
+arrow, history surviving the process, and `Ctrl+C` abandoning a line without
+ending the session. The rest of the interactive contract (a statement ending
+at its semicolon, the continuation prompt) is asserted from the byte stream by
+`packages/rust/tests/repl_tty_e2e.rs`, which runs in CI.
+
+Skipped below Python 3.12: curtaincall declares `requires-python >=3.12` while
+this package's floor is 3.10 (thekevinscott/curtaincall#26).
+"""
+
+from __future__ import annotations
+
+import os
+import shutil
+
+import pytest
+
+pytest.importorskip(
+    "curtaincall",
+    minversion="0.3",
+    reason="curtaincall requires Python >= 3.12; this package supports >= 3.10",
+)
+
+from curtaincall import expect
+
+# curtaincall registers its `terminal` fixture through an entry point, so
+# there is nothing to add to `pytest_plugins`.
+
+_PROMPT = "dirsql>"
+
+
+def _answer_cursor_queries(term) -> None:
+    """Let the terminal answer the editor's cursor-position queries.
+
+    The line editor asks where the cursor is (a `ESC [ 6 n` Device Status
+    Report) and gives up when nothing answers, so without this every session
+    dies with "the cursor position could not be read". pyte already knows the
+    answer -- `report_device_status` formats the reply -- but its
+    `write_process_input` hook is an inert stub, so the reply never reaches
+    the process. Pointing that hook at the pty is the whole of the fix.
+    """
+    term._screen.write_process_input = term.write
+
+
+def _cli() -> str:
+    """Resolve the `dirsql` console script for this test env."""
+    dirsql = shutil.which("dirsql")
+    assert dirsql is not None, (
+        "`dirsql` console script not on PATH -- run `uv run maturin develop`"
+    )
+    return dirsql
+
+
+def describe_the_interactive_repl():
+    @pytest.fixture
+    def home(tmp_path):
+        """A tmp HOME so history never lands in the developer's real one."""
+        data = tmp_path / "data"
+        data.mkdir()
+        return data
+
+    @pytest.fixture
+    def open_repl(terminal, tmp_path, home):
+        """Open a REPL over an empty directory and wait for its first prompt."""
+
+        def _open():
+            workdir = tmp_path / "tree"
+            workdir.mkdir(exist_ok=True)
+            term = terminal(
+                f"sh -c 'cd {workdir} && exec {_cli()}'",
+                env={
+                    **os.environ,
+                    "XDG_DATA_HOME": str(home),
+                    "TERM": "xterm",
+                },
+            )
+            _answer_cursor_queries(term)
+            expect(term.get_by_text(_PROMPT)).to_be_visible(timeout=30)
+            return term
+
+        return _open
+
+    def it_recalls_the_previous_statement_with_the_up_arrow(open_repl):
+        term = open_repl()
+        term.submit("SELECT 41 AS n;")
+        expect(term.get_by_text('[{"n":41}]')).to_be_visible(timeout=30)
+
+        term.key_up()
+        # The recalled text is on the line before it is submitted: without
+        # this the arrow key and the enter could race the editor's redraw.
+        expect(term.get_by_text("SELECT 41 AS n;")).to_be_visible(timeout=10)
+        term.key_enter()
+
+        expect(term.get_by_text('[{"n":41}]')).to_be_visible(timeout=30)
+        term.submit("quit")
+        expect(term).to_have_exited(timeout=30)
+        assert term.exit_code == 0
+
+    def it_keeps_history_across_sessions(open_repl):
+        # The point of a history *file*: a query worked out yesterday is still
+        # one up-arrow away today.
+        first = open_repl()
+        first.submit("SELECT 43 AS n;")
+        expect(first.get_by_text('[{"n":43}]')).to_be_visible(timeout=30)
+        first.key_ctrl_d()
+        expect(first).to_have_exited(timeout=30)
+
+        second = open_repl()
+        second.key_up()
+
+        expect(second.get_by_text("SELECT 43 AS n;")).to_be_visible(timeout=30)
+
+    def it_writes_history_where_xdg_says(open_repl, home):
+        term = open_repl()
+        term.submit("SELECT 44 AS n;")
+        expect(term.get_by_text('[{"n":44}]')).to_be_visible(timeout=30)
+        term.key_ctrl_d()
+        expect(term).to_have_exited(timeout=30)
+
+        history = home / "dirsql" / "history"
+        assert history.exists(), f"no history file at {history}"
+        assert "SELECT 44 AS n;" in history.read_text()
+
+    def it_abandons_a_line_on_ctrl_c_without_ending_the_session(open_repl):
+        # #988's reason for routing the interrupt through the editor rather
+        # than a signal handler: the process must survive it.
+        term = open_repl()
+        term.write("SELECT 45 AS n")
+        expect(term.get_by_text("SELECT 45 AS n")).to_be_visible(timeout=10)
+
+        term.key_ctrl_c()
+
+        term.submit("SELECT 46 AS n;")
+        expect(term.get_by_text('[{"n":46}]')).to_be_visible(timeout=30)
+        assert term.is_alive, "Ctrl+C must not kill the session"
+        expect(term.get_by_text('[{"n":45}]')).not_to_be_visible(timeout=5)
+
+        term.submit("quit")
+        expect(term).to_have_exited(timeout=30)
+        assert term.exit_code == 0
+
+    def it_shows_a_continuation_prompt_until_the_terminator(open_repl):
+        term = open_repl()
+        term.submit("SELECT")
+
+        expect(term.get_by_text("...>")).to_be_visible(timeout=10)
+
+        term.submit("47 AS n;")
+        expect(term.get_by_text('[{"n":47}]')).to_be_visible(timeout=30)
+        term.key_ctrl_d()
+        expect(term).to_have_exited(timeout=30)

packages/rust/Cargo.toml

@@ -46,6 +46,7 @@ cli = [
     "dep:axum",
     "dep:clap",
     "dep:futures",
+    "dep:reedline",
     "dep:tokio-stream",
     "tokio/rt-multi-thread",
     "tokio/macros",
@@ -64,6 +65,9 @@ globset = "0.4"
 # semantics), without adopting its walker: the scan keeps `walkdir`.
 ignore = "0.4"
 notify = "7"
+# The REPL's interactive line editor (history, Ctrl+R, multi-line entry).
+# CLI-only: a library consumer never pulls crossterm.
+reedline = { version = "0.49", optional = true }
 rusqlite = { version = "0.34", features = ["bundled", "load_extension", "hooks", "vtab", "functions"] }
 serde.workspace = true
 serde_json.workspace = true

packages/rust/changelog.d/2026-08-20-repl-line-editor.md

@@ -0,0 +1,3 @@
+**Added**
+
+- **The REPL's interactive prompt is a full line editor.** On a terminal, bare `dirsql` now runs through [reedline](https://github.com/nushell/reedline): ↑/↓ history recall, Ctrl-R reverse search, emacs editing (word motions, kill/yank), and Ctrl-C to abandon a line and return to a fresh prompt without ending the session. History persists across sessions in one file for every directory — `$XDG_DATA_HOME/dirsql/history`, falling back to `~/.local/share/dirsql/history` (`%APPDATA%\dirsql\history` on Windows), the last 1000 statements — mirroring `sqlite3`'s single `~/.sqlite_history`; with no location resolvable it stays in memory for the session. Statements may now span lines: a typed statement ends at its semicolon, and **SQLite's own tokenizer** (`sqlite3_complete`) decides where that is, so a `;` inside a string literal, a comment, or a `BEGIN … END` body does not terminate it. A `...>` prompt marks a statement still being typed. The piped path is unchanged — no editor, no history, and still one statement per line with no terminator needed. (#988)

packages/rust/migrations.d/2026-08-20-repl-line-editor.md

@@ -0,0 +1,67 @@
+### A typed REPL statement ends at its semicolon (#988)
+
+#### Summary
+
+The REPL read one line per statement. On a terminal it now reads until the
+statement is **terminated**, exactly as `sqlite3` does, so a statement can span
+lines. No API changed and no flag was added or removed; what a keystroke means
+at the interactive prompt did.
+
+The piped path is untouched: `dirsql < script.sql` and
+`echo "SELECT 1" | dirsql` still run one statement per line with no terminator.
+
+#### Required changes
+
+| Before | After | Fix |
+| ------ | ----- | --- |
+| Typing `SELECT 1` ⏎ at the prompt ran it | The prompt becomes `...>` and waits | Terminate it: `SELECT 1;`. |
+| Ctrl-C at the prompt killed the process | Abandons the current line, session continues | Use Ctrl-D, `exit`, or `quit` to leave. |
+| No history file was written | `$XDG_DATA_HOME/dirsql/history` (or `~/.local/share/dirsql/history`) is created and appended to | Nothing, unless you want it elsewhere — point `XDG_DATA_HOME` at another directory, or unset both `XDG_DATA_HOME` and `HOME` for an in-memory session. |
+
+#### Deprecations removed
+
+_None._
+
+#### Behavior changes without code changes
+
+- **Statements typed at a terminal now require a `;`.** An unterminated one is
+  never executed: reaching EOF mid-statement discards the fragment rather than
+  running half of it.
+- `exit`, `quit`, and a blank line are exempt from the terminator rule — none
+  is SQL, and waiting for a semicolon on the one input meant to end the session
+  would trap the user.
+- **Ctrl-C no longer terminates the process** at the prompt. It is handled by
+  the editor rather than a signal handler, so it clears the line and re-prompts.
+  A `SIGINT` delivered from outside (`kill -INT`) is unaffected.
+- The REPL writes a history file where it previously wrote nothing. It holds
+  the last 1000 statements and is shared across directories, so statements from
+  one project are recallable in another.
+- On a terminal the prompt is now drawn by the editor rather than written to
+  stdout directly. It is still `dirsql> `, with `   ...> ` for a continuation.
+
+#### Verification
+
+```bash
+# A statement over several lines runs as one:
+dirsql
+# dirsql> SELECT basename, size
+#    ...> FROM './'
+#    ...> LIMIT 1;
+# [{"basename":"a.md","size":6}]
+
+# A semicolon inside a literal is data, not a terminator:
+# dirsql> SELECT ';' AS s;
+# [{"s":";"}]
+
+# Ctrl-C clears the line; the session stays up. Ctrl-D leaves:
+echo $?
+# 0
+
+# The piped path is unchanged -- no terminator needed:
+printf 'SELECT 1 AS n\nSELECT 2 AS n\n' | dirsql
+# [{"n":1}]
+# [{"n":2}]
+
+# History lands where XDG says:
+ls "${XDG_DATA_HOME:-$HOME/.local/share}/dirsql/history"
+```