Skip to content

Declared tables: require an explicit name key, stop deriving it from ddl - #968

Merged
thekevinscott merged 5 commits into
mainfrom
claude/issue-962-0jjrfv
Aug 20, 2026
Merged

thekevinscott merged 5 commits into
mainfrom
claude/issue-962-0jjrfv

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Aug 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #962

A [[table]] entry gains a required name. dirsql stops tokenizing the
CREATE TABLE head of ddl to learn the table name — that was bespoke
SQL-as-text parsing, and SQLite's own catalog answers the question natively.

[[table]]
name    = "messages"
glob    = "sessions/*/messages/*.json"
on-file = "..."
ddl     = "CREATE TABLE messages (session TEXT, idx INT, role TEXT, text TEXT)"

What changed

  • Config: name is required on [[table]] (missing → Missing required field 'name' in [[table]] entry; blank → EmptyField). Duplicate-table detection across composed configs compares name keys.
  • Every call site that keyed on the parsed name now reads table.name: the build path, compile_matcher, compute_glob_config_hash, and the config merge.
  • db::parse_table_name is deleted, along with the strip_keyword_ci / parse_identifier / skip_ws_comments tokenizer behind it and its ~165 lines of unit tests. Db::create_table now takes (name, ddl). validate_identifier stays — name is spliced into format!()-built INSERT/DELETE SQL, so it remains the injection guard.
  • Existence check via the catalog: after create_table, table_exists asks pragma_table_list whether name is in main. No row → DirSqlError::TableNotCreated, raised before any file is ingested and carrying the config-entry prefix.
  • SDK builders take the name in all three SDKs: Table::new(name, ddl, glob, on_file) (and try_new / strict), Table(name=..., …), TableDef.name. PARITY.md updated.
  • TypeScript parseTableName(ddl) is removed — it wrapped the core parser that no longer exists. Its napi export and colocated tests go with it.

Quoted (CREATE TABLE "messages"), schema-qualified (main.messages) and IF NOT EXISTS DDL keep working with a plain name = "messages": SQLite records the bare identifier in its catalog, so nothing has to interpret the DDL text.

Parity: restoring/holding, not drifting — all three SDKs gain the name argument in this PR, so no follow-up issues are needed.

Manual verification

Against the release binary over a real temp dir (data/a.json holding [{"id":"one"},{"id":"two"}]):

$ dirsql query "SELECT id FROM messages ORDER BY id" -c .dirsql.toml
[{"id":"one"},{"id":"two"}]                                          # exit 0

$ # with the `name` key removed
dirsql query: failed to load config: config error: Missing required field 'name' in [[table]] entry   # exit 1

$ # name = "messages" over ddl = "CREATE TABLE notes (id TEXT)"
dirsql query: failed to load config: table 'messages': its `ddl` ran but created no
table called 'messages'. Set `name` to the table the `ddl` creates.   # exit 1

$ # all three forms, each with a plain name = "messages"
CREATE TABLE \"messages\" (id TEXT)        -> [{"n":2}]
CREATE TABLE main.messages (id TEXT)       -> [{"n":2}]
CREATE TABLE IF NOT EXISTS messages (id TEXT) -> [{"n":2}]

Red/green

2c9c3f3 pushed the failing suites alone; Rust CI run 32398345357 went red on exactly their assertions (unknown field \name`, and a nameless entry loading fine), not on anything incidental. 706ed09` turns them green.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass
  • TypeScript SDK e2e: pass
  • Rust core e2e (if applicable): pass
  • packages/python/e2e-attestation.json refreshed if packages/python changed
  • packages/ts/e2e-attestation.json refreshed if packages/ts changed
  • Command(s) run:
    • cargo test --manifest-path packages/rust/Cargo.toml --features cli --lib --tests
    • cd packages/python && uv run python -m pytest tests/e2e/ -q
    • cd packages/ts && pnpm test:e2e
  • Result summary: Rust 667 lib + all integration/e2e targets pass; Python 108 unit / 26 hermetic / 105 binding / 32 e2e pass; TypeScript 105 unit / 17 hermetic / 95 binding / 16 e2e pass.

Changelog / Migrations

  • Changelog fragment added under packages/<pkg>/changelog.d/ for each changed package
  • Migration fragment added under packages/<pkg>/migrations.d/ — breaking config + API change, all three packages

claude added 2 commits August 20, 2026 17:33
A table's name is declared, never derived from `ddl`. These suites pin the
three observable rules across all three SDKs, at the binding/integration and
e2e tiers:

- a `[[table]]` entry's declared `name` is what the table registers as
- an entry without `name` is a load error naming the key
- a `name` the entry's `ddl` never creates is a load error carrying the
  config-entry prefix (`table 'messages'`), caught against SQLite's own
  catalog before any ingestion

Quoted, schema-qualified and `IF NOT EXISTS` DDL forms are covered too: those
are SQL delimiter forms, so the catalog records the bare name and the declared
`name` matches without dirsql reading the DDL text.

RED: all four Rust integration cases and all three Rust e2e cases fail today
(`name` is an unknown key; a nameless entry loads fine).

Refs #962

skip-changelog: test-only commit; the behavior change ships with the
implementation commit

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hvg3vjPNZPa6EyqvFoNb9X
A table's SQL name was derived by tokenizing the `CREATE TABLE` head of `ddl`
(`db::parse_table_name`) -- bespoke SQL-as-text parsing that had grown cases
for quoted identifiers, `schema.table`, `IF NOT EXISTS` and leading comments.
Never reinvent what SQLite expresses natively: the name is now declared, and
SQLite's own catalog settles whether the DDL produced it.

- `[[table]]` gains a required `name` key; `Table::new` / `try_new` / `strict`
  take it first; Python `Table(name=...)`; TypeScript `TableDef.name`.
- Every call site that keyed on the parsed name reads `table.name` instead:
  the build path, `compile_matcher`, `compute_glob_config_hash`, and
  duplicate-table detection across composed configs.
- `Db::create_table` takes `(name, ddl)`. `validate_identifier` stays -- `name`
  is spliced into `format!()`-built INSERT/DELETE SQL, so it remains the
  injection guard.
- After the DDL runs, `table_exists` asks `pragma_table_list` whether `name` is
  in the catalog. No row is `DirSqlError::TableNotCreated`, raised before any
  file is ingested and carrying the config entry's name.
- `db::parse_table_name` is deleted, with the `strip_keyword_ci` /
  `parse_identifier` / `skip_ws_comments` tokenizer behind it, and so is the
  TypeScript `parseTableName` export that wrapped it.

Quoted, schema-qualified and `IF NOT EXISTS` DDL keep working without any of
that: SQLite records the bare identifier, so a plain `name` matches.

Manually exercised against the release binary over a real temp dir: the happy
path returns rows; a nameless entry exits 1 with `Missing required field
'name' in [[table]] entry`; `name = "messages"` over a `ddl` creating `notes`
exits 1 with `table 'messages': its `ddl` ran but created no table called
'messages'.`; and all three of `CREATE TABLE "messages"`,
`CREATE TABLE main.messages` and `CREATE TABLE IF NOT EXISTS messages` load
and query under `name = "messages"`.

GREEN: the red suites from the previous commit now pass in all three SDKs.

Fixes #962

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hvg3vjPNZPa6EyqvFoNb9X
@thekevinbot
thekevinbot marked this pull request as ready for review August 20, 2026 18:24
claude added 3 commits August 20, 2026 18:44
`mutants.out/` was already ignored, but cargo-mutants rotates the previous
run's directory to `mutants.out.old/` on every re-run — so the second local
run of the mutation gate leaves an untracked directory behind. Same rule,
both forms.

skip-changelog: repo tooling only; no packaged code changes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hvg3vjPNZPa6EyqvFoNb9X
@thekevinscott
thekevinscott merged commit 36981d8 into main Aug 20, 2026
78 checks passed
@thekevinscott
thekevinscott deleted the claude/issue-962-0jjrfv branch August 20, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Declared tables: require an explicit name key, stop deriving it from ddl

3 participants