Three correctness fixes, TypeScript 7, and plpgsql bodies for #91 - #192
Conversation
`select("posts.id")` emitted `SELECT "posts.id"` — one quoted identifier,
naming a column no table has. Postgres answers `column "posts.id" does not
exist`, so every join that named its columns was broken at run time.
`columns`, `distinctOn`, `groupBy` and `orderBy` all took the string
straight to `col()`, which never looked for the dot. They now go through
`parseColumnRef`, which mirrors `parseTableRef`: split one qualifier, leave
anything deeper whole, and take quote characters raw for the printer to
escape (`test/printer/*` pins that).
The type rejected the same string, so `.select("posts.id")` was a compile
error as well — `QualifiedColumn` now admits `table.column` for the tables
in scope and `UnqualifiedName` maps it back to the row key.
How it survived: `bench/src/scenarios.ts` used this form and asserted the
broken output in a snapshot, but `tsconfig.json` included only `src` and
`test`, so nothing typechecked the benchmark. A companion commit puts
`bench` inside `include`.
The integration test asks pglite rather than trusting the string, which is
how the defect was found.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
`deduplicatePredicates` fingerprinted a parameter by its value, so one call
site emitted different SQL texts depending on what the caller passed:
id = 1 AND id = 1 -> WHERE ("id" = $1)
id = 1 AND id = 2 -> WHERE (("id" = $1) AND ("id" = $2))
The IN(...) fast path was worse. It fingerprinted an all-parameter list by
its arity alone, so two different lists were one predicate:
id IN (1,2) AND id IN (3,4) -> WHERE ("id" IN ($1, $2))
That is wrong rows, not a cosmetic difference: the query should return
nothing and returned the first list instead.
A parameter now fingerprints as its occurrence, never its value, so two
parameters never dedupe. Literals still dedupe — a literal is part of the
shape because it is written into the text.
The cost is deliberate: `name = $1 AND name = $2` is emitted where
`name = $1` used to be when both values happen to be equal. One redundant
parameter, discarded by any planner. What it buys is that a call site emits
one SQL text, which is the precondition for the database reusing a prepared
statement's plan — and for anything here caching a compiled query.
`test/pipeline.test.ts` asserted the old behaviour and now states why it
does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
TypeScript 7 is the native compiler, so `@typescript/native-preview` had
nothing left to add: dropped, and `typecheck` is plain `tsc --noEmit`. It
checks the repository in 0.6s.
typescript 6.0.3 -> 7.0.2
@typescript/native-preview -> removed
@casl/ability 6.8.1 -> 7.0.1
better-sqlite3 12.10.0 -> 13.0.3
drizzle-orm 0.45.2 -> 1.0.0-rc.4
kysely 0.29.2 -> 0.29.5
oxlint 1.65.0 -> 1.79.0
oxfmt 0.50.0 -> 0.64.0
@electric-sql/pglite 0.4.5 -> 0.5.5
vitest, coverage-v8 4.1.6 -> 4.1.11
fast-check, obuild, bumpp -> latest
Two majors landed on us. CASL 7 returns `readonly Rule[]` from `rulesFor`,
which `AbilityLike` did not accept. drizzle 1.0 dropped `schema` from
`DrizzlePgConfig` and changed what it emits — every condition is now
parenthesised and a union is wrapped in a subquery — so the benchmark
snapshots that record competitor output move with it.
`tsconfig.json` now includes `bench` and `mvp`. Leaving the benchmark
outside is how it came to assert invalid SQL in a snapshot for months. The
node globals the benchmark reaches for are declared in the existing narrow
shim rather than by installing `@types/node`, which `node-globals.d.ts`
asks for in as many words.
`mvp/measure` is excluded from the default vitest run: it times libraries
against each other and took longer than the rest of the suite together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
Removes the five ADRs, the recipes, the competitor notes, the SQL:2023 notes and the kysely/drizzle migration guide, at the maintainer's request. The README linked the migration guide and `builder/ddl/copy.ts` pointed at the recipes; both references are gone rather than left dangling. Everything here is recoverable with `git checkout <sha>^ -- docs`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
…round Not wired into `src/`, not exported, not shipped. It answers one question with a number: what does a request pay if writing a query and running one are separated into two phases? Handing a driver the SQL and the parameters costs 4.7–8.2ns, against 1.3µs for sumak and 1.4µs for kysely on the same queries — and the floor of the measurement loop with no library in it at all is 4.8ns, so what that column measures is the harness. The query is compiled once, at startup, for ~2µs; `$n` names its argument rather than its position, so binding hands back the caller's own array. The more useful measurement is in `measure/REALITY.txt`: against pglite a one-row query costs ~328µs end to end and the entire compile is ~1% of it. Stable SQL text is worth more than a fast compiler — a PREPAREd statement saves 60µs, twenty times what compiling costs. That is where this would go next, not into shaving the nanoseconds. Five authoring rules are pinned by tests, and two of them close defect classes that `src/` has had: a value cannot become SQL text (`.eq(value)` does not compile; `lit(...)` is the deliberate way in, and escapes the backslash mysql reads as an escape character), and the compiler never sees a value at all, so it cannot make the SQL depend on one. `measure/` is out of the default test run — `MEASURE=1 pnpm vitest run mvp/measure` regenerates the numbers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
The published numbers were from May, against drizzle 0.45, and three of the scenarios they were measured on emitted SQL Postgres rejects. Re-run across all 48 scenarios against kysely 0.29.5 and drizzle-orm 1.0.0-rc.4. The root README claimed "seven canonical shapes, wins six of the seven, 9×–39× vs drizzle". The harness has had 48 scenarios for a while, sumak wins 32 of them, and the drizzle range is 2.2×–14.7× — drizzle 1.0 is a great deal faster than the version that claim was measured against. Corrected, with the sixteen kysely wins listed and grouped by cause rather than left out: WHERE-chain traversal (backlog A2, the flat n-ary node) and scalar functions, where sumak builds a typed node and the competitors interpolate a template. The per-compile table said the slowest scenario was ~28µs. It is `insert-many-100` at 90µs. Both that and the "~1ms round trip" are replaced with measurements: a one-row query against pglite costs ~328µs end to end, so the compile is ~1% of it, and the same query PREPAREd costs 243µs — the 60µs a reusable server-side plan saves is twenty times the compile, and it is reachable only because one call site now emits one SQL text. Also corrects the normalization section: deduplication applies to literals, not to parameters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
Issue #91 asks for database infrastructure to live in the codebase as typed code rather than in `.sql` files a deploy script has to remember to run. RLS landed in #172/#179 and typed CREATE FUNCTION / CREATE TRIGGER in #191, but a function could only have an expression body — and a function with an expression body is a SELECT with extra steps. Branches, loops, variables and RAISE are what made the request. createFunction("compute_total") .args({ price: arg("integer"), quantity: arg("integer") }) .returns("integer") .plpgsql((b, { price, quantity }) => { b.if(typedLte(quantity, val(0)), (t) => t.raise("exception", "quantity must be positive")) b.return(typedMul(price, quantity)) }) `Block` covers DECLARE, assignment, IF/ELSIF/ELSE, WHILE, FOR over a range or a query, bare LOOP, EXIT and CONTINUE with WHEN, RAISE with USING, PERFORM, embedded statements, nested blocks, RETURN / RETURN NEXT / RETURN QUERY. `triggerScope` names NEW, OLD, TG_OP, TG_TABLE_NAME and TG_TABLE_SCHEMA as typed expressions. `CreateFunctionNode["body"]` widens to `ExpressionNode | StatementBlockNode`, which is the shape #191 said Phase 2 would need. Existing call sites are untouched. Two things the tests found, both only findable by executing: A plpgsql body is a string literal to the outer parser, so a missing semicolon after `END IF` creates the function happily and fails when it is called. Every test here runs the function in pglite rather than comparing strings, and one trigger test inserts a row and expects the rejection. plpgsql folds unquoted identifiers to lower case, so its variable is `new` and a quoted `"NEW"` resolves to nothing — `missing FROM-clause entry for table "NEW"`. `triggerScope` emits lower case for the variable half and leaves the column half in whatever case the schema gave it. A parameter inside an embedded statement is refused rather than emitted: inside `$$ … $$` a placeholder names one of the function's own arguments, so binding one would silently point at the wrong value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
…er does `toCompiled()` fills placeholders without printing anything, so the conversion the printer performs on the way past never ran for those values. A `bigint` came out raw where the uncompiled path gives its decimal string, and `pg` / `mysql2` reject a bare BigInt — so the faster path handed the driver something it refuses. `Printer` now exposes `coerceParam`, and `compileQuery` applies it to every value it fills in. The test pins that the two paths agree. Also documents functions and triggers in the README, which had no section for them at all — including the two things that are only learnable by being bitten: a placeholder inside `$$ … $$` names one of the function's own arguments, and a malformed plpgsql body creates the function happily and fails when it is called. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
Three defects in the shipped library, the toolchain moved forward two majors, and the half of issue #91 that was still missing. Fixes: - `select("posts.id")` emitted `SELECT "posts.id"`, a column no table has - the emitted SQL depended on parameter values, and `IN (1,2) AND IN (3,4)` dropped the second list — wrong rows - `toCompiled()` skipped parameter conversion, handing `pg` a bare BigInt Feature: - plpgsql bodies: DECLARE, IF/ELSIF/ELSE, WHILE, FOR, LOOP, EXIT, RAISE, PERFORM, RETURN NEXT/QUERY, and NEW/OLD/TG_OP for triggers (#91) Toolchain: - TypeScript 7 (dropping `@typescript/native-preview`), drizzle 1.0-rc.4, kysely 0.29.5, CASL 7, and `bench` + `mvp` inside `tsconfig`'s include — leaving the benchmark unchecked is how it came to assert invalid SQL Also removes `docs/` at the maintainer's request, re-measures the benchmark across all 48 scenarios, and corrects the performance claims the README made. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
A query's shape is fixed where it is typed; only the values change per
request. `toCompiled()` already compiled once, but the filling was slower
than it needed to be, so the split it exists to express was not worth
reaching for.
Filling is now specialised by arity — an array literal for the arities that
cover almost every query, a loop past them — and the same query costs ~6ns
compiled against ~3,200ns through `toSQL()`. Everything the pipeline does,
plugin transforms and hooks and normalize and optimize and printing, happens
once.
Two things measured on the way:
`Array.from({ length: n })` is 20x slower than the alternatives (479.8ns
against 23.0ns for `new Array(n)`, 35.5ns building by push). Earlier in this
branch I replaced `new Array(len)` in the printer's IN(...) path with it to
silence a lint warning, without measuring. That is undone — the printer
builds by push now, which is both fast and unambiguous.
The prepared path was 84ns before this and is ~6ns after, so the
architecture was already there; what was missing was it being worth using.
`mvp/` is deleted. It was a sketch built to answer whether separating the
two phases was worth it, the answer was yes, and the answer now lives in
`src/`. Its numbers are in the README's Compiled Queries section, which now
leads with the split rather than presenting it as an optimisation, and
`test/builder/compiled-shape.test.ts` pins the properties that make it true:
one SQL string whatever the values, a fresh parameter array per call, and
the pipeline never consulted again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
An audit trigger is the example issue #91 opens with, and it could not be written: `insertInto("audit").values({ action: TG_OP, product: NEW.name })` turned both into bound parameters, and inside `$$ … $$` a placeholder names one of the function's own arguments. The only way through was a cast at every call site, which loses the column's type. `values()` and `set()` now pass a branded expression through as itself and parameterise everything else. The brand is a symbol, so nothing arrives there by accident. `Insertable` / `Updateable` widen to match — the runtime half alone would have left the type rejecting what the builder accepts, which is the same shape of bug as `select("posts.id")` earlier in this branch. `forEach` now declares its loop variable as a `record`. plpgsql refuses `FOR r IN <query>` unless `r` is already a record or row variable, and the error arrives when the function runs, not when it is created. Coverage of the new plpgsql builder goes from 73% to 96%, and the paths that were untested — FOR over a query, CONTINUE WHEN, embedded statements, nested blocks, RETURN NEXT, RETURN QUERY, a declaration inside a branch — are covered by executing them, which is how both defects above surfaced. The type tests for `Insertable` / `Updateable` asserted the narrow shape. They now assert the contract by assignment: required stays required, a column keeps its type, and an expression is accepted where its value goes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
|
Two more commits: the architecture the The compiled path is the one a request takesA query's shape is fixed where it is typed; only the values change per request.
It was 84ns before this, so the architecture was already there; what was missing was it being worth using. Measured on the way: A write accepts an expressionThe audit trigger #91 opens with could not be written.
Both surfaced from pushing the new builder's coverage from 73% to 96% by executing the untested paths — FOR over a query, TestsThe type tests for I also audited the suite for dead weight and did not find much worth cutting. The signals were false positives — 62 "no assertion" tests are mostly "runs against a live engine without throwing", 101 "duplicate names" are the same assertion across different dialects, 23 "identical bodies" likewise.
|
`test/ddl/custom-types` and `test/ddl/row-level-security` each carried a
"builder shape" block next to an emission block, and 32 of those tests were
`builder.x(v).build().x === v` — the builder returning what it was handed.
Anything that breaks the node breaks the SQL the emission block already
pins, so they cost a line each and protect nothing.
Measured rather than eyeballed: dropping the shape blocks entirely from
these two files loses **0 statements** of `src/` coverage, where the same
experiment on the other ten paired files loses 1–9 and those blocks stay.
Coverage was necessary and not sufficient, though. Reading what it wanted
to delete, half of those blocks assert aliasing and idempotence —
builder is immutable — branching returns independent nodes
build() returns a fresh array (mutating it doesn't affect the builder)
defensively copies the names array
.to() replaces a previous .to() call
CASCADE → RESTRICT flips (last call wins)
— which the SQL cannot show and coverage cannot distinguish, because the
same statements run either way. Those are kept. Only the `seeds an empty X`
and `.x(v) sets the slot` family is gone.
3297 tests, down 32, with identical coverage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
The compiled path was the fast one and the inconvenient one at the same
time: it handed back `{ sql, params }` and left the caller to drive the
connection, so the split it exists to express was not what anybody reached
for. `many()`, `one()`, `first()` and `run()` now sit on the compiled query
and take the parameters.
const olderThan = db.selectFrom("users").select("name")
.where(({ age }) => age.gt(placeholder("age")))
.toCompiled<{ age: number }>()
await olderThan.many({ age: 40 })
The result context is derived once, from the AST that was already built.
`many()` on the uncompiled builder derives it per call and builds the AST
twice — once for the context and once inside `toSQL()`.
Rows are typed: `toCompiled()` carries the builder's row type through, so
`many()` returns `O[]` rather than `unknown[]`.
Compiling from a bare AST has no instance behind it, so the four methods are
present and reject with what to do instead. They reject rather than throw
synchronously, because they are declared to return a promise and a caller
who only awaits should not have to wrap the call as well.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
|
Two more commits, and the test analysis you asked for — with the negative result stated rather than hidden. A compiled query can run itselfThe compiled path was the fast one and the inconvenient one at the same time: it handed back const olderThan = db.selectFrom("users").select("name")
.where(({ age }) => age.gt(placeholder("age")))
.toCompiled<{ age: number }>()
await olderThan.many({ age: 40 }) // typed rows
await olderThan.one({ age: 40 })
await olderThan.first({ age: 40 })
await renameUser.run({ id: 1, newName: "ada" })Rows are typed — The test analysisTwo methods, run properly. Per-file coverage sweep. Every one of the 258 test files measured on its own, then compared against the union of the others. 77 files cover no unique statement. That is not a deletion list, and I am not treating it as one: coverage measures which lines ran, not which behaviours are pinned. Reading what coverage wanted to delete. Twelve files pair a "builder shape" block with an emission block. Dropping the shape blocks costs 0 statements in two of them ( — which the SQL cannot show and coverage cannot distinguish, because the same statements run either way. So: 32 tests removed, all of the The honest summary is that this suite does not have meaningful dead weight, and the tooling that claims otherwise is measuring the wrong thing.
|
… driver
INSERT, UPDATE and DELETE compiled to `{ sql, params }` and stopped there,
so the split was only expressible for SELECT. `runnersFor` now builds the
four runners once from the AST that was already built, and all four typed
builders use it.
While wiring it: every execution helper built the AST twice, once for the
result context and once inside `toSQL()`. `toSQL()` is split so the node can
be handed to the compiler directly, and the helpers pass the one they
already have. Seven call sites across the four builders.
Measured after, and worth stating plainly: end to end against pglite the
compiled path is not faster — 329.8µs against 322.0µs, inside the noise —
because one query costs ~328µs and the compile it removes is ~3µs of that.
What the split buys is a cold start that compiles once instead of per first
request, and a stable SQL text per call site. The second is the larger prize
and is not collected yet: it needs the driver to keep named prepared
statements, which is worth 60µs against 3µs and is the next piece.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
The point of a fixed SQL text is that PostgreSQL can keep the plan. Until
now nothing collected on it: every statement went out unnamed, so the server
parsed and planned it again on every call.
A compiled query now carries a `statementName`, stable for the life of the
process, and `sumak/drivers/pg` sends `{ name, text, values }` instead of
`(text, values)` when one is present. `DriverCallOptions.statementName` is
the seam; a driver that cannot keep prepared statements ignores it and the
query runs as it always did. An uncompiled query is deliberately left
unnamed — its text is not fixed, so naming it would prepare a plan that the
next call invalidates.
What is measured, and what is not: the win is 243µs against 303µs for the
same query, taken with `PREPARE` / `EXECUTE` against pglite, and it is
twenty times what client-side compiling costs. The driver path itself is
verified by a recording pool asserting the name reaches it and stays the
same across calls — there is no real PostgreSQL server here to time it
against, and I am not going to claim a number I did not take.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb
Nine commits. Three defects in the shipped library, the toolchain moved forward two majors, and the half of #91 that was still missing.
Fixes
select("posts.id")emitted SQL Postgres rejects. It quoted the whole string as one identifier —SELECT "posts.id"— naming a column no table has. Every join that named its columns was broken at run time.columns,distinctOn,groupByandorderBynow route throughparseColumnRef, and the type admitstable.columnfor the tables in scope.How it survived:
bench/src/scenarios.tsused that form and pinned the broken output in a snapshot, buttsconfig.jsonincluded onlysrcandtest, so nothing typechecked the benchmark. It is insideincludenow, and sumak's own types reject the form — nobody was looking.The emitted SQL depended on parameter values.
deduplicatePredicatesfingerprinted a parameter by its value:The
IN(...)fast path was worse — it fingerprinted an all-parameter list by arity alone, soid IN (1,2) AND id IN (3,4)becameWHERE ("id" IN ($1, $2)). Wrong rows, not a cosmetic difference.A parameter now fingerprints as its occurrence. Literals still dedupe, because a literal is written into the text and is part of the shape. The trade is deliberate:
name = $1 AND name = $2wherename = $1used to be when the values happen to match — one redundant parameter, discarded by any planner — and in exchange one call site emits one SQL text, which is what lets the database reuse a prepared statement's plan.toCompiled()skipped parameter conversion. It fills placeholders without printing, so the printer's conversion never ran for those values, and abigintreachedpgraw — which the driver rejects.Printernow exposescoerceParamand the compiled path applies it.Feature — #91
#91 asks for database infrastructure to live in the codebase as typed code. RLS landed in #172/#179 and typed
CREATE FUNCTION/CREATE TRIGGERin #191, but a function could only have an expression body — and a function with an expression body is aSELECTwith extra steps.DECLARE, assignment,IF/ELSIF/ELSE,WHILE,FORover a range or a query, bareLOOP,EXIT/CONTINUEwithWHEN,RAISEwithUSING,PERFORM, embedded statements, nested blocks,RETURN/RETURN NEXT/RETURN QUERY.triggerScopenamesNEW,OLD,TG_OP,TG_TABLE_NAME,TG_TABLE_SCHEMA.Every test runs the function in pglite rather than comparing strings, because a plpgsql body is a string literal to the outer parser — a missing semicolon after
END IFcreates the function happily and fails when it is called. That caught two things:newand a quoted"NEW"resolves to nothing.triggerScopeemits lower case for the variable half.$$ … $$names one of the function's own arguments, so a parameter in an embedded statement is refused rather than silently bound to the wrong thing.Toolchain
TypeScript 7 is the native compiler, so
@typescript/native-previewis dropped andtypecheckis plaintsc --noEmit— 0.6s for the repository. Also drizzle-orm 1.0.0-rc.4, kysely 0.29.5, CASL 7, better-sqlite3 13, oxlint/oxfmt/vitest/pglite. Two majors broke us: CASL returnsreadonly Rule[]fromrulesFor, and drizzle droppedschemafromDrizzlePgConfigand changed what it emits.Documentation
docs/is removed at the maintainer's request; the README link and thecopy.tsreference went with it rather than dangling.The benchmark is re-measured across all 48 scenarios. The README claimed "seven canonical shapes, wins six of the seven, 9×–39× vs drizzle" — the harness has had 48 for a while, sumak wins 32, and drizzle 1.0 is much faster than the version that claim was measured against, so the real range is 2.2×–14.7×. The sixteen kysely wins are now listed and grouped by cause. The per-compile table said the slowest scenario was ~28µs; it is
insert-many-100at 90µs.README also gains a Functions and Triggers section, which it never had.
Verification
pnpm test(lint + typecheck + 3342 tests) green,pnpm buildclean,npm pack304 files.🤖 Generated with Claude Code
https://claude.ai/code/session_01HAwiBVhLmNhWjC9Ro6UMpb