diff --git a/.changeset/connection-independent-queries.md b/.changeset/connection-independent-queries.md new file mode 100644 index 00000000..9a25aba1 --- /dev/null +++ b/.changeset/connection-independent-queries.md @@ -0,0 +1,14 @@ +--- +'@cleverbrush/knex-schema': minor +'@cleverbrush/orm': minor +--- + +Add immutable connection-independent query definitions with `query(Schema)`. +Define typed reads once and supply a Knex connection or transaction through +`definition(knex, ...values)`, `.query(knex, ...values)` or `.toSQL(knex, ...values)`. + +Preserve inferred parameters, projections, relation/variant schemas and existing +connection-first APIs. Capture selectors/scopes/customizers once, compile SELECTs +lazily per definition and actual Knex instance, and bind ordinary readers before +native SQL composition, pagination or supported writes. Include runtime, +declaration and PostgreSQL integration coverage and multi-file usage guidance. diff --git a/libs/knex-schema/README.md b/libs/knex-schema/README.md index a4e8f112..aaf424b3 100644 --- a/libs/knex-schema/README.md +++ b/libs/knex-schema/README.md @@ -202,6 +202,102 @@ Available WHERE methods: `where`, `andWhere`, `orWhere`, `whereNot`, `whereIn`, --- +## Connection-independent query definitions + +Define reads once at module scope with `query(Schema)`, then supply an injected +Knex connection or caller-owned transaction when using them. Definitions are +immutable and **not thenable**: constructing them, accessing `rowSchema`, or +awaiting the definition itself never runs a query. No Knex client is created +behind the scenes. + +```ts +// data/user-queries.ts +import { parameter, query } from '@cleverbrush/knex-schema'; +import { UserSchema } from './user-schema.js'; + +export const findUser = query(UserSchema) + .where(user => user.id, parameter('id')); + +export const userNames = query(UserSchema) + .select(user => ({ id: user.id, name: user.name })) + .orderBy(user => user.name); + +export const userNameRow = userNames.rowSchema; // available without Knex +``` + +```ts +// api/handlers/get-user.ts +import type { Knex } from 'knex'; +import { findUser, userNames } from '../../data/user-queries.js'; + +// The controller's dependency-injection layer supplies knex. +export async function getUser(knex: Knex, id: number) { + return findUser.query(knex, id).first(); // inferred row | undefined +} + +export async function listUserNames(knex: Knex) { + return userNames(knex); // inferred { id: number; name: string }[] +} + +export async function renameUser(knex: Knex, id: number, name: string) { + return knex.transaction(async trx => { + await findUser.query(trx, id).update({ name }); + return findUser(trx, id); // SELECT on the same transaction + }); +} +``` + +The connection is always the first argument; parameter values follow in +first-appearance order. Parameterless definitions still require a connection: +`userNames(knex)`, `userNames.query(knex)`, and `userNames.toSQL(knex)`. +`findUser.toSQL(knex, id)` returns SQL with placeholders and independent bindings, +without executing it. Direct compiled execution currently requires PostgreSQL. + +Framework HTTP handlers can stay in separate files with the same contract-bound +types. Here `GetUserEndpoint` declares the `id` parameter and injects a Knex token +under `knex`: + +```ts +// api/handlers/get-user.ts +import { NotFoundError, type Handler } from '@cleverbrush/server'; +import type { GetUserEndpoint } from '../endpoints.js'; +import { findUser } from '../../data/user-queries.js'; + +export const getUserHandler: Handler = async ( + { params }, + { knex } +) => { + const user = await findUser.query(knex, params.id).first(); + if (!user) throw new NotFoundError('User not found'); + return user; +}; +``` + +Definitions support table and named projections, typed aggregates, predicate +groups, scopes, alias joins, relation includes, and STI/CTI variants. Their types +retain positional parameter inference and projection/variant row schemas. +Selectors, scopes and customizers run during definition, **not** when binding +or executing. Captured dates, arrays and JSON values are snapshotted. + +Each immutable definition lazily caches its compiled SELECT separately for each +actual Knex instance, using weak references. Different instances never share a +plan, even when their connection settings match: identifier formatting and +other client configuration may differ. Directly supplied transactions have +their own cache entries. Explicit `.transacting(trx)` derivatives of a bound +parameterized reader retain the existing safe compilation-sharing behavior. +Each invocation has fresh bindings and executes against the database; this is +not a result cache. The caller owns connection disposal and transaction lifetime. + +`.query(knex, ...values)` creates an independent ordinary reader for additional +filters, `first()`, pagination, native SQL composition, or allowed writes. Writes +still require an unprojected, writable table shape. Bind before using `ref()`, +native Knex subqueries/raw objects, `apply()`, or raw projections. String raw +predicates/orderings with concrete, captured bindings remain available during +definition; they cannot introduce parameter placeholders. + +`query(knex, Schema)`, `createQuery(knex)`, and ORM DbSet APIs remain supported. +Use them when a connection is already available while configuring the query. + ## Parameterized compiled queries Use `parameter('name')` in a typed predicate to make a query callable. Call it @@ -1061,10 +1157,13 @@ See [Composable read queries](#composable-read-queries) for typed flat joins wit ordering, and multi-column cursor pagination. These APIs preserve existing calls and include runtime, type, and PostgreSQL integration coverage. -### `query(knex, schema, baseQuery?)` +### `query(schema)` / `query(knex, schema)` -Creates a `SchemaQueryBuilder`. `schema` must have `.hasTableName()` set. -Optionally pass a `baseQuery` (e.g. a scoped `knex('users').where('deleted_at', null)`) as the starting point. +`query(schema)` creates a connection-independent callable definition; +`query(knex, schema)` creates an ordinary connection-bound reader. Both infer +table, alias, or polymorphic query types. Table schemas need `.hasTableName()`. +Raw source arguments are not supported; bind first and use +`apply(configure, { output })` when changing the SQL output shape. ### `SchemaQueryBuilder` diff --git a/libs/knex-schema/integration/query-definitions.test.ts b/libs/knex-schema/integration/query-definitions.test.ts new file mode 100644 index 00000000..aaaf0181 --- /dev/null +++ b/libs/knex-schema/integration/query-definitions.test.ts @@ -0,0 +1,278 @@ +import { randomUUID } from 'node:crypto'; +import { + aggregate, + alias, + date, + defineEntity, + eq, + number, + object, + parameter, + query, + string +} from '@cleverbrush/orm'; +import Knex from 'knex'; +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; + +const prefix = `cb_definition_${randomUUID().replaceAll('-', '')}`; +const users = `${prefix}_users`; +const tasks = `${prefix}_tasks`; +const assets = `${prefix}_assets`; +const photos = `${prefix}_photos`; +const User = object({ + id: number().primaryKey(), + name: string().hasColumnName('display_name'), + birthday: date().optional(), + balance: number().decimal(24, 6), + profile: object({ score: number() }).optional() +}).hasTableName(users); +const Task = defineEntity( + object({ + id: number().primaryKey(), + ownerId: number().hasColumnName('owner_id'), + owner: User.optional() + }).hasTableName(tasks) +).belongsTo( + t => t.owner, + t => t.ownerId, + t => t.id +); +const Asset = defineEntity( + object({ id: number().primaryKey(), kind: string() }).hasTableName(assets) +) + .discriminator('kind') + .stiVariant('note', object({ text: string() })) + .ctiVariant( + 'photo', + defineEntity( + object({ + assetId: number().hasColumnName('asset_id'), + width: number() + }).hasTableName(photos) + ), + t => t.assetId + ); + +// Definitions genuinely precede connection construction, as in separate modules. +const findUser = query(User).where(t => t.id, parameter('id')); +const listUsers = query(User).orderBy('id'); +const connection = process.env.QUERY_TEST_DATABASE_URL; +if (!connection) throw new Error('QUERY_TEST_DATABASE_URL is required'); +const knex = Knex({ client: 'pg', connection }); +const second = Knex({ client: 'pg', connection }); + +beforeAll(async () => { + await knex.schema.createTable(users, t => { + t.integer('id').primary(); + t.text('display_name'); + t.timestamp('birthday', { useTz: true }); + t.decimal('balance', 24, 6); + t.jsonb('profile'); + }); + await knex.schema.createTable(tasks, t => { + t.integer('id').primary(); + t.integer('owner_id'); + }); + await knex.schema.createTable(assets, t => { + t.integer('id').primary(); + t.text('kind'); + t.text('text'); + }); + await knex.schema.createTable(photos, t => { + t.integer('asset_id').primary(); + t.integer('width'); + }); + await knex(users).insert([ + { + id: 1, + display_name: 'Jane', + birthday: '2000-01-01T00:00:00Z', + balance: '9007199254740993.000001', + profile: { score: 3 } + }, + { + id: 2, + display_name: 'John', + birthday: null, + balance: '2.500000', + profile: null + } + ]); + await knex(tasks).insert([ + { id: 10, owner_id: 1 }, + { id: 20, owner_id: 2 } + ]); + await knex(assets).insert([ + { id: 1, kind: 'note', text: 'hello' }, + { id: 2, kind: 'photo', text: null } + ]); + await knex(photos).insert({ asset_id: 2, width: 100 }); +}); +afterAll(async () => { + for (const table of [photos, assets, tasks, users]) + await knex.schema.dropTableIfExists(table); + await Promise.all([knex.destroy(), second.destroy()]); +}); + +describe('query definitions against PostgreSQL', () => { + it('executes parameterless and concurrent parameterized reads with lossless decoding', async () => { + const [one, two, anotherClient] = await Promise.all([ + findUser(knex, 1), + findUser(knex, 2), + findUser(second, 1) + ]); + expect(one).toEqual(anotherClient); + expect(one[0]).toMatchObject({ + name: 'Jane', + balance: '9007199254740993.000001', + birthday: new Date('2000-01-01'), + profile: { score: 3 } + }); + expect(two[0]).toMatchObject({ + name: 'John', + birthday: null, + profile: null + }); + expect(await listUsers(knex)).toEqual(await listUsers.query(knex)); + expect(await findUser.query(knex, 1).first()).toEqual(one[0]); + expect(await findUser.select('name')(knex, 2)).toEqual([ + { name: 'John' } + ]); + expect( + await query(User).where(t => t.profile.score, parameter('score'))( + knex, + 3 + ) + ).toEqual(one); + }); + + it('uses the supplied transaction for reads and bound writes, without taking ownership', async () => { + const trx = await knex.transaction(); + try { + const updated = await findUser + .query(trx, 1) + .update({ name: 'Pending' }); + expect(updated[0].name).toBe('Pending'); + expect((await findUser(trx, 1))[0].name).toBe('Pending'); + expect((await findUser(second, 1))[0].name).toBe('Jane'); + expect( + await findUser.query(knex, 1).transacting(trx).first() + ).toMatchObject({ name: 'Pending' }); + const nested = query(User) + .where(t => t.profile.score, parameter('score')) + .query(knex, 3) + .transacting(trx); + expect((await nested.first())?.name).toBe('Pending'); + expect(trx.isCompleted()).toBe(false); + } finally { + await trx.rollback(); + } + expect((await findUser(knex, 1))[0].name).toBe('Jane'); + }); + + it('keeps transaction-only compiled derivatives safe and cached', async () => { + const boundTemplate = query(User) + .query(knex) + .where(t => t.id, parameter('id')); + const sql = boundTemplate.toSQL(1); + const trx = await knex.transaction(); + try { + const derivative = boundTemplate.transacting(trx); + const compiler = vi.spyOn(trx.client, 'queryCompiler'); + try { + expect(derivative.toSQL(1).sql).toBe(sql.sql); + expect((await derivative(1))[0].id).toBe(1); + expect(compiler).not.toHaveBeenCalled(); + } finally { + compiler.mockRestore(); + } + } finally { + await trx.rollback(); + } + }); + + it('supports bound pagination, native composition and independent immutable branches', async () => { + const bound = listUsers.query(knex); + expect( + (await bound.paginate({ page: 1, pageSize: 1 })).data + ).toHaveLength(1); + expect(await bound.countValue()).toBe(2); + const firstPage = await bound.paginateAfter({ + limit: 1, + orderBy: [{ column: 'id', direction: 'asc' }] + }); + expect(firstPage.data.map(row => row.id)).toEqual([1]); + expect( + ( + await bound.paginateAfter({ + limit: 1, + cursor: firstPage.nextCursor, + orderBy: [{ column: 'id', direction: 'asc' }] + }) + ).data.map(row => row.id) + ).toEqual([2]); + expect( + await bound.whereExists( + knex(tasks).select('id').where('owner_id', bound.ref('id')) + ) + ).toHaveLength(2); + expect( + await bound.apply( + q => q.clearSelect().select({ label: 'display_name' }), + { output: object({ label: string() }) } + ) + ).toEqual([{ label: 'Jane' }, { label: 'John' }]); + const first = listUsers.limit(1); + expect(await first(knex)).toHaveLength(1); + expect(await listUsers(knex)).toHaveLength(2); + }); + + it('executes captured scopes, relation parameters, joins and aggregates on each client', async () => { + const defaultScope = vi.fn((q: any) => + q.where('id', '>', 0).orderBy('id') + ); + const customizer = vi.fn((q: any) => + q.where('name', parameter('name')).select('name') + ); + const definition = query( + Task.schema.defaultScope(defaultScope) + ).include('owner', customizer); + for (const client of [knex, second]) { + expect(await (definition as any)(client, 'Jane')).toEqual([ + { id: 10, ownerId: 1, owner: { name: 'Jane' } } + ]); + expect(await (definition as any).query(client, 'John')).toEqual([ + { id: 20, ownerId: 2, owner: { name: 'John' } } + ]); + } + expect(defaultScope).toHaveBeenCalledTimes(1); + expect(customizer).toHaveBeenCalledTimes(1); + const joined = query(alias(User, 'u')) + .leftJoin(alias(Task.schema, 't'), t => eq(t.u.id, t.t.ownerId)) + .groupBy(t => t.u.name) + .select(t => ({ name: t.u.name, count: aggregate.count(t.t.id) })) + .orderByRaw('?? asc', ['u.display_name']); + expect(await joined(knex)).toEqual([ + { name: 'Jane', count: 1 }, + { name: 'John', count: 1 } + ]); + }); + + it('executes STI/CTI definitions and removes parameters of discarded variants', async () => { + const definition = query(Asset.schema) + .forVariant('note', q => q.where(t => t.text, parameter('text'))) + .forVariant('photo', q => + q.where(t => t.width, '>=', parameter('width')) + ) + .orderBy('id'); + const first = await definition(knex, 'hello', 50); + expect(first.map(row => row.id)).toEqual([1, 2]); + expect(await definition.query(second, 'hello', 50)).toEqual(first); + expect(await definition.selectVariants(['photo'])(knex, 200)).toEqual( + [] + ); + expect( + await definition.selectVariants(['note'])(knex, 'hello') + ).toEqual([{ id: 1, kind: 'note', text: 'hello' }]); + }); +}); diff --git a/libs/knex-schema/src/AliasedQueryBuilder.ts b/libs/knex-schema/src/AliasedQueryBuilder.ts index 9071ef29..22ca4cd3 100644 --- a/libs/knex-schema/src/AliasedQueryBuilder.ts +++ b/libs/knex-schema/src/AliasedQueryBuilder.ts @@ -47,6 +47,7 @@ import { type SchemaForValue } from './read-schema.js'; import type { ReadColumn, ReadProjection } from './SchemaQueryBuilder.js'; +import { nativeSql } from './sql-description.js'; /** Schema-backed aliases whose exact numeric and outer-join values match decoded rows. */ export type ReadAliasTables = { @@ -65,8 +66,13 @@ type Selector = (tables: ReadAliasTables) => AliasedColumn; export class AliasedQueryBuilder< T, Row extends ReadObject = never, - P extends ParameterState = [] -> extends ReadPredicates, P, AliasParameterReader> { + P extends ParameterState = [], + Connected extends boolean = true +> extends ReadPredicates< + ReadAliasTables, + P, + AliasParameterReader +> { private fields?: Record; private schema?: Row; private predicates: readonly ReadPredicate[] = []; @@ -98,7 +104,10 @@ export class AliasedQueryBuilder< join( table: N extends keyof T ? never : TableAlias, on: (tables: ReadAliasTables>) => JoinPredicate - ): QueryView, Row, P>> { + ): QueryView< + AliasedQueryBuilder, Row, P, Connected>, + Connected + > { const copy = this.copy(); copy.planner = copy.planner.join(table, on as any) as any; return finishParameterizedQuery(copy) as any; @@ -109,7 +118,10 @@ export class AliasedQueryBuilder< on: ( tables: ReadAliasTables> ) => JoinPredicate - ): QueryView, Row, P>> { + ): QueryView< + AliasedQueryBuilder, Row, P, Connected>, + Connected + > { const copy = this.copy(); copy.planner = copy.planner.leftJoin(table, on as any) as any; return finishParameterizedQuery(copy) as any; @@ -117,7 +129,10 @@ export class AliasedQueryBuilder< /** Select exact columns and aggregates; opaque raw expressions are deliberately unsupported. */ select( select: (tables: ReadAliasTables) => Selected - ): QueryView, P>> { + ): QueryView< + AliasedQueryBuilder, P, Connected>, + Connected + > { const { knex, columns } = this.planner.readContext(); const entries = Object.values( columns as Record>> @@ -181,7 +196,7 @@ export class AliasedQueryBuilder< orderBy( column: Selector, direction: 'asc' | 'desc' = 'asc' - ): QueryView { + ): QueryView { const copy = this.copy(); copy.planner.orderBy(column as any, direction); return finishParameterizedQuery(copy) as any; @@ -190,15 +205,15 @@ export class AliasedQueryBuilder< orderByRaw( sql: string, bindings: A & WithoutParameters> = [] as any - ): QueryView { + ): QueryView { const { knex } = this.planner.readContext(); - const captured = captureReadRaw(knex, sql, bindings)().toSQL(); + const captured = captureReadRaw(knex, sql, bindings)(); const copy = this.copy(); - copy.planner.orderByRaw(captured.sql, captured.bindings); + copy.planner.orderByRaw(captured); return finishParameterizedQuery(copy) as any; } /** Group native columns for an aggregate projection. */ - groupBy(...columns: Selector[]): QueryView { + groupBy(...columns: Selector[]): QueryView { const copy = this.copy(); copy.planner.groupBy(...(columns as any)); return finishParameterizedQuery(copy) as any; @@ -210,7 +225,7 @@ export class AliasedQueryBuilder< ) => AggregateExpression | AliasedColumn, operator: string, right: V & WithoutParameters> - ): QueryView { + ): QueryView { const copy = this.copy(); copy.planner.having( value as any, @@ -220,7 +235,7 @@ export class AliasedQueryBuilder< return finishParameterizedQuery(copy) as any; } /** Limit the flat row count, including repeated parents produced by joins. */ - limit(count: number): QueryView { + limit(count: number): QueryView { if (!Number.isInteger(count) || count < 0) throw new ReadSchemaError('Limit must be a non-negative integer'); const copy = this.copy(); @@ -228,7 +243,7 @@ export class AliasedQueryBuilder< return finishParameterizedQuery(copy) as any; } /** Offset flat rows using caller-supplied deterministic ordering. */ - offset(count: number): QueryView { + offset(count: number): QueryView { if (!Number.isInteger(count) || count < 0) throw new ReadSchemaError('Offset must be a non-negative integer'); const copy = this.copy(); @@ -236,7 +251,7 @@ export class AliasedQueryBuilder< return finishParameterizedQuery(copy) as any; } /** Use a caller-owned transaction without mutating the original read query. */ - transacting(trx: Knex.Transaction): QueryView { + transacting(trx: Knex.Transaction): QueryView { const copy = this.copy(); copy.planner = copy.planner.transacting(trx); shareParameterCompilation(this, copy); @@ -265,7 +280,7 @@ export class AliasedQueryBuilder< } /** Return an independent mutable Knex snapshot. */ toKnexQuery(): Knex.QueryBuilder { - return this.compile(); + return nativeSql(this.compile()); } /** Configure raw SQL once and declare its complete output contract. */ apply( @@ -274,9 +289,10 @@ export class AliasedQueryBuilder< ): OpaqueQuery { assertParametersBound(this); const { knex, sql: source } = this.planner.readContext(); - const sql = this.fields ? this.compile() : source; + const planned = this.fields ? this.compile() : source; if (!this.fields) - for (const predicate of this.predicates) predicate(sql); + for (const predicate of this.predicates) predicate(planned); + const sql = nativeSql(planned); const result = configure(sql); if (result !== undefined && result !== sql) { if (result instanceof Promise) void result.catch(() => {}); @@ -321,6 +337,11 @@ export class AliasedQueryBuilder< ); return { knex: this.planner.readConnection(), + connect: knex => { + const copy = this.copy(); + copy.planner = this.planner.bindConnection(knex); + return copy; + }, uses: predicateParameters(this.predicates), compile: () => this.compile(COMPILE_PARAMETERS), decode: row => decodeObject(nodes, row, 'row'), @@ -350,15 +371,28 @@ export class AliasedQueryBuilder< } /** @internal Fluent return constructor for aliased SELECTs. */ -export interface AliasParameterReader - extends ParameterReader { +export interface AliasParameterReader< + T, + Row extends ReadObject, + Connected extends boolean = true +> extends ParameterReader { readonly result: QueryView< AliasedQueryBuilder< T, Row, this['parameters'] extends ParameterState ? this['parameters'] - : never - > + : never, + Connected extends true + ? true + : this['connection'] extends boolean + ? this['connection'] + : false + >, + Connected extends true + ? true + : this['connection'] extends boolean + ? this['connection'] + : false >; } diff --git a/libs/knex-schema/src/OpaqueQuery.ts b/libs/knex-schema/src/OpaqueQuery.ts index 72b1a842..d5e4ea2c 100644 --- a/libs/knex-schema/src/OpaqueQuery.ts +++ b/libs/knex-schema/src/OpaqueQuery.ts @@ -2,6 +2,7 @@ import type { InferType } from '@cleverbrush/schema'; import type { Knex } from 'knex'; import { captureReadRaw } from './read-predicates.js'; import { type ReadObject, ReadSchemaError } from './read-schema.js'; +import { actualConnection } from './sql-description.js'; /** Explicit output contract required when Framework cannot infer the SQL row shape. */ export interface QueryOutput { @@ -38,6 +39,7 @@ export class OpaqueQuery { sql: Knex.QueryBuilder, options: QueryOutput ): OpaqueQuery { + knex = actualConnection(knex); const compiled = sql.toSQL(); if (Array.isArray(compiled) || compiled.method !== 'select') throw new ReadSchemaError( diff --git a/libs/knex-schema/src/PolymorphicQueryBuilder.ts b/libs/knex-schema/src/PolymorphicQueryBuilder.ts index 80ec5002..8277160c 100644 --- a/libs/knex-schema/src/PolymorphicQueryBuilder.ts +++ b/libs/knex-schema/src/PolymorphicQueryBuilder.ts @@ -72,6 +72,12 @@ import { type SchemaAwareQuery, SchemaQueryBuilder } from './SchemaQueryBuilder.js'; +import { + bindDescriptionConnection, + bindSql, + nativeSql, + transactionConnection +} from './sql-description.js'; import type { PaginationResult } from './types.js'; type VariantMap = @@ -156,15 +162,17 @@ type BranchSource< type VariantRelationQuery< S extends ReadObject, K extends keyof VariantMap & string, - R extends string + R extends string, + Connected extends boolean = true > = R extends keyof ReadRelations> - ? SchemaAwareQuery>[R]>> + ? SchemaAwareQuery>[R]>, Connected> : SchemaQueryBuilder; type BranchQueries< S extends ReadObject, B extends Record, - P extends ParameterState = [] + P extends ParameterState = [], + Connected extends boolean = true > = { [K in keyof B & keyof VariantMap & string]: QueryView< SchemaQueryBuilder< @@ -172,8 +180,10 @@ type BranchQueries< B[K], ReadRelations & ReadRelations>, true, - ScopedParameters - > + ScopedParameters, + Connected + >, + Connected >; }; type Selector = ( @@ -192,11 +202,12 @@ type PolymorphicOrder = export class PolymorphicQueryBuilder< S extends ReadObject, B extends Record = VariantReadSchemas, - P extends ParameterState = [] + P extends ParameterState = [], + Connected extends boolean = true > extends ReadPredicates< ReadColumns>, P, - PolymorphicParameterReader + PolymorphicParameterReader > { /** @internal Nominal identity for typed child-query customizers. */ declare readonly [READ_QUERY]: true; @@ -206,9 +217,9 @@ export class PolymorphicQueryBuilder< readonly variantRowSchemas: Readonly; private branches: Record< string, - SchemaQueryBuilder + SchemaQueryBuilder >; - private fallback: SchemaQueryBuilder; + private fallback: SchemaQueryBuilder; private orders: PolymorphicOrder[] = []; private rowLimit?: number; private rowOffset?: number; @@ -268,7 +279,14 @@ export class PolymorphicQueryBuilder< buildColumnMap(source).propToCol.get(config.discriminatorKey) ?? config.discriminatorKey; // Invert only the discriminator guard, not caller/default-scope filters. - this.fallback = new SchemaQueryBuilder( + this.fallback = new SchemaQueryBuilder< + any, + any, + any, + any, + any, + Connected + >( knex, common, knex @@ -372,7 +390,7 @@ export class PolymorphicQueryBuilder< private branch( key: string, body: boolean - ): SchemaQueryBuilder { + ): SchemaQueryBuilder { const config = getVariants(this.source)!; const variant = config.variants[key]; const baseInfo = this.source.introspect(); @@ -488,7 +506,7 @@ export class PolymorphicQueryBuilder< '__read_cti_present' ); } - return new SchemaQueryBuilder( + return new SchemaQueryBuilder( this.knex, schema, query.select(columns), @@ -560,25 +578,25 @@ export class PolymorphicQueryBuilder< return finishParameterizedQuery(copy); } /** Remove the default scope while preserving explicit predicates. */ - unscoped(): QueryView { + unscoped(): QueryView { const copy = this.copy(); copy.skipDefaults = true; return finishParameterizedQuery(copy) as any; } /** Include soft-deleted entities in every branch. */ - withDeleted(): QueryView { + withDeleted(): QueryView { const copy = this.copy(); copy.deleted = 'include'; return finishParameterizedQuery(copy) as any; } /** Match only soft-deleted entities in every branch. */ - onlyDeleted(): QueryView { + onlyDeleted(): QueryView { const copy = this.copy(); copy.deleted = 'only'; return finishParameterizedQuery(copy) as any; } /** Apply a named immutable scope once. */ - scoped(name: string): QueryView { + scoped(name: string): QueryView { const scope = ( this.source.introspect().extensions?.scopes as | Record @@ -618,12 +636,12 @@ export class PolymorphicQueryBuilder< includeVariant< K extends keyof B & keyof VariantMap & string, R extends string, - Child extends ReadQueryShape = VariantRelationQuery + Child extends ReadQueryShape = VariantRelationQuery >( key: K, relation: R, customize?: ( - query: VariantRelationQuery + query: VariantRelationQuery ) => Child & CheckParameterState< MergeParameters>> @@ -655,8 +673,10 @@ export class PolymorphicQueryBuilder< ParametersOf >, `variant:${K}` - > - > + >, + Connected + >, + Connected > { return this.forVariant( key, @@ -668,12 +688,13 @@ export class PolymorphicQueryBuilder< include< K extends keyof ReadRelations & string, Child extends ReadQueryShape = SchemaAwareQuery< - Related[K]> + Related[K]>, + Connected > >( selector: K | ((relations: { [P in keyof ReadRelations]: P }) => K), customize?: ( - query: SchemaAwareQuery[K]>> + query: SchemaAwareQuery[K]>, Connected> ) => Child & CheckParameterState< AttachParameters< @@ -698,8 +719,10 @@ export class PolymorphicQueryBuilder< } >; }, - AttachParameters, `relation:${K}`> - > + AttachParameters, `relation:${K}`>, + Connected + >, + Connected > { const relations = (this.source.introspect().extensions?.relations ?? []) as { name: string }[]; @@ -775,8 +798,10 @@ export class PolymorphicQueryBuilder< PredicateValue>, Sel> >, `variant:${K}` - > - > + >, + Connected + >, + Connected > { return this.forVariant(key, query => (query as any).where(selector, operator, value) @@ -788,7 +813,7 @@ export class PolymorphicQueryBuilder< | Selector | (keyof ReadColumns> & string), direction: 'asc' | 'desc' = 'asc' - ): QueryView { + ): QueryView { if (direction !== 'asc' && direction !== 'desc') throw new ReadSchemaError('Invalid ordering direction'); const column = @@ -808,13 +833,13 @@ export class PolymorphicQueryBuilder< orderByRaw( sql: string, bindings: A & WithoutParameters> = [] as any - ): QueryView { + ): QueryView { const copy = this.copy(); copy.orders.push({ raw: captureReadRaw(this.knex, sql, bindings) }); return finishParameterizedQuery(copy) as any; } /** Limit the combined result across all variants. */ - limit(count: number): QueryView { + limit(count: number): QueryView { if (!Number.isInteger(count) || count < 0) throw new ReadSchemaError('Limit must be a non-negative integer'); const copy = this.copy(); @@ -828,8 +853,10 @@ export class PolymorphicQueryBuilder< PolymorphicQueryBuilder< S, Pick, - SelectParameterVariants - > + SelectParameterVariants, + Connected + >, + Connected > { if ( !keys.length || @@ -848,7 +875,7 @@ export class PolymorphicQueryBuilder< return finishParameterizedQuery(copy) as any; } /** Skip rows of the combined result, using a stable explicit ordering. */ - offset(count: number): QueryView { + offset(count: number): QueryView { if (!Number.isInteger(count) || count < 0) throw new ReadSchemaError('Offset must be a non-negative integer'); const copy = this.copy(); @@ -866,7 +893,7 @@ export class PolymorphicQueryBuilder< >( key: K, configure: ( - query: BranchQueries[K] + query: BranchQueries[K] ) => Q & CheckParameterState< AttachParameters>, `variant:${K}`> @@ -875,8 +902,10 @@ export class PolymorphicQueryBuilder< PolymorphicQueryBuilder< S, Omit & Record, - AttachParameters, `variant:${K}`> - > + AttachParameters, `variant:${K}`>, + Connected + >, + Connected > { const current = this.branches[key]; if (!current) throw new ReadSchemaError(`Unknown variant: ${key}`); @@ -902,7 +931,10 @@ export class PolymorphicQueryBuilder< copy.branches[key] = configured as unknown as SchemaQueryBuilder< any, any, - any + any, + any, + any, + Connected >; copy.refresh(); return finishParameterizedQuery(copy) as any; @@ -1069,14 +1101,14 @@ export class PolymorphicQueryBuilder< } /** Return a separately mutable Knex snapshot of the union statement. */ toKnexQuery(): Knex.QueryBuilder { - return this.compile(); + return nativeSql(this.compile()); } /** Configure a captured union SELECT and declare its complete raw output shape. */ apply( configure: (query: Knex.QueryBuilder) => Knex.QueryBuilder | undefined, options: QueryOutput ): OpaqueQuery { - const sql = this.compile(); + const sql = this.toKnexQuery(); const result = configure(sql); if (result !== undefined && result !== sql) { if (result instanceof Promise) void result.catch(() => {}); @@ -1176,9 +1208,9 @@ export class PolymorphicQueryBuilder< return this.execute().then(resolve, reject); } /** Bind independent branch queries to a caller-owned transaction. */ - transacting(trx: Knex.Transaction): QueryView { + transacting(trx: Knex.Transaction): QueryView { const copy = this.copy(); - Object.assign(copy, { knex: trx }); + Object.assign(copy, { knex: transactionConnection(this.knex, trx) }); copy.branches = Object.fromEntries( Object.entries(this.branches).map(([key, q]) => [ key, @@ -1208,6 +1240,24 @@ export class PolymorphicQueryBuilder< ...[...branches.values()].flatMap(branch => branch.uses), ...(this.includeUnknown ? readerParameters(this.fallback) : []) ], + connect: knex => { + const copy = this.copy(); + const bound = bindDescriptionConnection(knex); + Object.assign(copy, { + knex: bound, + base: bindSql(this.base, bound) + }); + copy.branches = Object.fromEntries( + Object.entries(this.branches).map(([key, branch]) => [ + key, + branch[COMPILED_READER]().connect!(knex) + ]) + ) as typeof this.branches; + copy.fallback = this.fallback[COMPILED_READER]().connect!( + knex + ) as typeof this.fallback; + return copy; + }, compile: () => this.compile(undefined, COMPILE_PARAMETERS), decode: row => { const value = (row as any).__read_poly; @@ -1241,7 +1291,8 @@ export class PolymorphicQueryBuilder< /** @internal Fluent return constructor for polymorphic SELECTs. */ export interface PolymorphicParameterReader< S extends ReadObject, - B extends Record + B extends Record, + Connected extends boolean = true > extends ParameterReader { readonly result: QueryView< PolymorphicQueryBuilder< @@ -1249,7 +1300,17 @@ export interface PolymorphicParameterReader< B, this['parameters'] extends ParameterState ? this['parameters'] - : never - > + : never, + Connected extends true + ? true + : this['connection'] extends boolean + ? this['connection'] + : false + >, + Connected extends true + ? true + : this['connection'] extends boolean + ? this['connection'] + : false >; } diff --git a/libs/knex-schema/src/SchemaQueryBuilder.ts b/libs/knex-schema/src/SchemaQueryBuilder.ts index 5f35e207..155b104d 100644 --- a/libs/knex-schema/src/SchemaQueryBuilder.ts +++ b/libs/knex-schema/src/SchemaQueryBuilder.ts @@ -81,6 +81,15 @@ import { readExpression, type SchemaForValue } from './read-schema.js'; +import { + bindDescriptionConnection, + bindSql, + configureSql, + connectionForSql, + isConnectionDescription, + nativeSql, + transactionConnection +} from './sql-description.js'; export { type BoundQuery, createQuery, query } from './query.js'; @@ -212,13 +221,40 @@ type Loaded = { required: boolean; }; /** Query factory result: a table query or a declared polymorphic union. */ -export type SchemaAwareQuery = - ReadVariantMetadata extends { - discriminator: string; - variants: Record; - } +export type SchemaAwareQuery< + S extends ReadObject, + Connected extends boolean = true +> = Connected extends true + ? ReadVariantMetadata extends { + discriminator: string; + variants: Record; + } ? PolymorphicQueryBuilder - : SchemaQueryBuilder; + : SchemaQueryBuilder + : ReadVariantMetadata extends { + discriminator: string; + variants: Record; + } + ? QueryView< + PolymorphicQueryBuilder< + S, + import('./PolymorphicQueryBuilder.js').VariantReadSchemas, + [], + Connected + >, + Connected + > + : QueryView< + SchemaQueryBuilder< + S, + ObjectReadSchema>, + ReadRelations, + true, + [], + Connected + >, + Connected + >; /** @internal Apply parent correlation before child selection, ordering and pagination. */ export type ReadCorrelation = ( query: Knex.QueryBuilder, @@ -231,7 +267,8 @@ export interface TableParameterReader< S extends ReadObject, Row extends ReadObject, Relations extends Record, - Writable extends boolean + Writable extends boolean, + Connected extends boolean = true > extends ParameterReader { readonly result: QueryView< SchemaQueryBuilder< @@ -241,8 +278,18 @@ export interface TableParameterReader< Writable, this['parameters'] extends ParameterState ? this['parameters'] - : never - > + : never, + Connected extends true + ? true + : this['connection'] extends boolean + ? this['connection'] + : false + >, + Connected extends true + ? true + : this['connection'] extends boolean + ? this['connection'] + : false >; } @@ -256,11 +303,12 @@ export class SchemaQueryBuilder< Row extends ReadObject = ObjectReadSchema>, Relations extends Record = ReadRelations, Writable extends boolean = true, - P extends ParameterState = [] + P extends ParameterState = [], + Connected extends boolean = true > extends ReadPredicates< ReadColumns, P, - TableParameterReader + TableParameterReader > { /** @internal Nominal identity for typed child-query customizers. */ declare readonly [READ_QUERY]: true; @@ -453,7 +501,7 @@ export class SchemaQueryBuilder< } /** Apply a named, synchronous shape-preserving scope once to an independent query. */ - scoped(name: ScopesOf): QueryView { + scoped(name: ScopesOf): QueryView { const scope = ( this.source.introspect().extensions?.scopes as | Record @@ -467,21 +515,21 @@ export class SchemaQueryBuilder< } /** Exclude only the default scope; keep explicitly configured predicates. */ - unscoped(): QueryView { + unscoped(): QueryView { const copy = this.copy(); copy.skipDefaults = true; return finishParameterizedQuery(copy) as any; } /** Include soft-deleted rows without changing the source query. */ - withDeleted(): QueryView { + withDeleted(): QueryView { const copy = this.copy(); copy.deleted = 'include'; return finishParameterizedQuery(copy) as any; } /** Match only soft-deleted rows. */ - onlyDeleted(): QueryView { + onlyDeleted(): QueryView { const copy = this.copy(); copy.deleted = 'only'; return finishParameterizedQuery(copy) as any; @@ -505,51 +553,58 @@ export class SchemaQueryBuilder< private filtered(mode?: typeof COMPILE_PARAMETERS): Knex.QueryBuilder { assertParametersBound(this, mode); const query = this.base.clone(); - const explicitWhere = (query as any)._statements.filter( - (statement: any) => statement.grouping === 'where' - ); - (query as any)._statements = (query as any)._statements.filter( - (statement: any) => statement.grouping !== 'where' - ); - if (this.defaults && !this.skipDefaults) { - const defaults = this.defaults.clone() as any; - const where = defaults._statements.filter( + return configureSql(query, query => { + const explicitWhere = (query as any)._statements.filter( (statement: any) => statement.grouping === 'where' ); - (query as any)._statements = [ - ...defaults._statements.filter( - (statement: any) => statement.grouping !== 'where' - ), - ...(query as any)._statements - ]; - (query as any)._single = { - ...defaults._single, - ...(query as any)._single - }; - if (where.length || this.defaultPredicates.length) + (query as any)._statements = (query as any)._statements.filter( + (statement: any) => statement.grouping !== 'where' + ); + if (this.defaults && !this.skipDefaults) { + const defaults = ( + isConnectionDescription(this.knex) + ? bindSql( + this.defaults.clone(), + connectionForSql(query) + ) + : this.defaults.clone() + ) as any; + const where = defaults._statements.filter( + (statement: any) => statement.grouping === 'where' + ); + (query as any)._statements = [ + ...defaults._statements.filter( + (statement: any) => statement.grouping !== 'where' + ), + ...(query as any)._statements + ]; + (query as any)._single = { + ...defaults._single, + ...(query as any)._single + }; + if (where.length || this.defaultPredicates.length) + query.where(nested => { + (nested as any)._statements = [...where]; + for (const predicate of this.defaultPredicates) + predicate(nested); + }); + } + if (explicitWhere.length) query.where(nested => { - (nested as any)._statements = [...where]; - for (const predicate of this.defaultPredicates) - predicate(nested); + (nested as any)._statements = [...explicitWhere]; }); - } - if (explicitWhere.length) - query.where(nested => { - (nested as any)._statements = [...explicitWhere]; - }); - if (this.predicates.length) - query.where(nested => { - for (const predicate of this.predicates) predicate(nested); - }); - const softDelete = this.source.introspect().extensions?.softDelete as - | { column: string } - | undefined; - if (softDelete && this.deleted !== 'include') { - query[this.deleted === 'only' ? 'whereNotNull' : 'whereNull']( - `${this.alias}.${softDelete.column}` - ); - } - return query; + if (this.predicates.length) + query.where(nested => { + for (const predicate of this.predicates) predicate(nested); + }); + const softDelete = this.source.introspect().extensions + ?.softDelete as { column: string } | undefined; + if (softDelete && this.deleted !== 'include') { + query[this.deleted === 'only' ? 'whereNotNull' : 'whereNull']( + `${this.alias}.${softDelete.column}` + ); + } + }); } private async scalar( @@ -721,8 +776,10 @@ export class SchemaQueryBuilder< ObjectSchemaBuilder>, K>>, Relations, false, - P - > + P, + Connected + >, + Connected >; /** Select named output fields and aggregates, replacing the previous scalar projection. */ select( @@ -741,8 +798,10 @@ export class SchemaQueryBuilder< >, Relations, false, - P - > + P, + Connected + >, + Connected >; select(...selectors: any[]): any { const selections = selectors.map(selector => @@ -801,8 +860,10 @@ export class SchemaQueryBuilder< ReadProjection<{ count: AggregateExpression }>, Relations, false, - P - > + P, + Connected + >, + Connected > { return this.select(() => ({ count: createAggregate( @@ -820,8 +881,10 @@ export class SchemaQueryBuilder< ReadProjection<{ countDistinct: AggregateExpression }>, Relations, false, - P - > + P, + Connected + >, + Connected > { return this.select(() => ({ countDistinct: createAggregate( @@ -839,8 +902,10 @@ export class SchemaQueryBuilder< ReadProjection<{ sum: AggregateExpression }>, Relations, false, - P - > + P, + Connected + >, + Connected > { return this.select(() => ({ sum: createAggregate( @@ -858,8 +923,10 @@ export class SchemaQueryBuilder< ReadProjection<{ avg: AggregateExpression }>, Relations, false, - P - > + P, + Connected + >, + Connected > { return this.select(() => ({ avg: createAggregate( @@ -881,8 +948,10 @@ export class SchemaQueryBuilder< }>, Relations, false, - P - > + P, + Connected + >, + Connected > { return this.select(() => ({ min: createAggregate< @@ -903,8 +972,10 @@ export class SchemaQueryBuilder< }>, Relations, false, - P - > + P, + Connected + >, + Connected > { return this.select(() => ({ max: createAggregate< @@ -914,7 +985,10 @@ export class SchemaQueryBuilder< } /** Keep only distinct selected rows, preserving the row schema. */ - distinct(): QueryView>; + distinct(): QueryView< + SchemaQueryBuilder, + Connected + >; /** Select a property subset and eliminate duplicate rows on that immutable projection. */ distinct & string>( ...columns: Array< @@ -927,8 +1001,10 @@ export class SchemaQueryBuilder< ObjectSchemaBuilder>, K>>, Relations, false, - P - > + P, + Connected + >, + Connected >; distinct(...columns: any[]): any { const copy = ( @@ -943,7 +1019,10 @@ export class SchemaQueryBuilder< havingRaw( sql: string, bindings: readonly Knex.RawBinding[] = [] - ): QueryView> { + ): QueryView< + SchemaQueryBuilder, + Connected + > { const copy = this.copy(); copy.base.havingRaw(captureReadRaw(this.knex, sql, bindings)()); copy.grouped = true; @@ -954,7 +1033,10 @@ export class SchemaQueryBuilder< column: ReadPredicateSelector>, operator: string, value: V & WithoutParameters> - ): QueryView> { + ): QueryView< + SchemaQueryBuilder, + Connected + > { const copy = this.copy(); copy.base.having( this.name(this.column(column)), @@ -968,7 +1050,10 @@ export class SchemaQueryBuilder< groupByRaw( sql: string, bindings: A & WithoutParameters> = [] as any - ): QueryView> { + ): QueryView< + SchemaQueryBuilder, + Connected + > { const copy = this.copy(); copy.base.groupByRaw(captureReadRaw(this.knex, sql, bindings)()); copy.grouped = true; @@ -996,8 +1081,10 @@ export class SchemaQueryBuilder< >, Relations, false, - P - > + P, + Connected + >, + Connected > { const definition = getProjections(this.source)[name]; if (!definition) @@ -1033,7 +1120,7 @@ export class SchemaQueryBuilder< return finishParameterizedQuery(copy); } /** @internal Apply an already captured Framework predicate to an independent query. */ - withPredicate(predicate: ReadPredicate): QueryView { + withPredicate(predicate: ReadPredicate): QueryView { return this.addReadPredicate(predicate) as any; } /** @internal Native storage columns for composing CTI table sources. */ @@ -1044,7 +1131,7 @@ export class SchemaQueryBuilder< orderBy( column: ReadPredicateSelector>, direction: 'asc' | 'desc' = 'asc' - ): QueryView { + ): QueryView { const copy = this.copy(); copy.base.orderBy(this.name(this.column(column)), direction); return finishParameterizedQuery(copy) as any; @@ -1053,7 +1140,7 @@ export class SchemaQueryBuilder< orderByRaw( sql: string, bindings: A & WithoutParameters> = [] as any - ): QueryView { + ): QueryView { const captured = captureReadRaw(this.knex, sql, bindings); const copy = this.copy(); copy.base.orderByRaw(captured()); @@ -1062,14 +1149,17 @@ export class SchemaQueryBuilder< /** Group rows before typed aggregate projection. */ groupBy( ...columns: ReadPredicateSelector>[] - ): QueryView> { + ): QueryView< + SchemaQueryBuilder, + Connected + > { const copy = this.copy(); copy.base.groupBy(columns.map(c => this.name(this.column(c)))); copy.grouped = true; return finishParameterizedQuery(copy) as any; } /** Limit parent rows; relation limits apply independently within each parent. */ - limit(count: number): QueryView { + limit(count: number): QueryView { if (!Number.isInteger(count) || count < 0) throw new ReadSchemaError('Limit must be a non-negative integer'); const copy = this.copy(); @@ -1077,7 +1167,7 @@ export class SchemaQueryBuilder< return finishParameterizedQuery(copy) as any; } /** Skip parent rows; use a deterministic order for pagination. */ - offset(count: number): QueryView { + offset(count: number): QueryView { if (!Number.isInteger(count) || count < 0) throw new ReadSchemaError('Offset must be a non-negative integer'); const copy = this.copy(); @@ -1091,11 +1181,14 @@ export class SchemaQueryBuilder< */ include< K extends keyof Relations & string, - Child extends ReadQueryShape = SchemaAwareQuery> + Child extends ReadQueryShape = SchemaAwareQuery< + Related, + Connected + > >( selector: K | ((relations: { [P in keyof Relations]: P }) => K), customize?: ( - query: SchemaAwareQuery> + query: SchemaAwareQuery, Connected> ) => Child & CheckParameterState< AttachParameters< @@ -1115,8 +1208,10 @@ export class SchemaQueryBuilder< >, Relations, false, - AttachParameters, `relation:${K}`> - > + AttachParameters, `relation:${K}`>, + Connected + >, + Connected > { if (Object.values(this.fields).some(f => f.aggregate) || this.grouped) throw new ReadSchemaError( @@ -1157,7 +1252,9 @@ export class SchemaQueryBuilder< this.knex(getTableName(foreign)) ); if (customize) { - const customized = customize(child as any); + const customized = customize( + finishParameterizedQuery(child) as any + ); if (!child.sameSource(customized)) { if (customized instanceof Promise) void customized.catch(() => {}); @@ -1228,8 +1325,8 @@ export class SchemaQueryBuilder< /** @internal Reuse a captured child query across polymorphic parent branches. */ includeFrom( name: string, - prepared: SchemaQueryBuilder - ): QueryView { + prepared: SchemaQueryBuilder + ): QueryView { const loaded = prepared.loaded.find(relation => relation.name === name); if (!loaded) throw new ReadSchemaError(`Unknown prepared relation: ${name}`); @@ -1241,11 +1338,11 @@ export class SchemaQueryBuilder< F extends ReadObject, K extends string, Required extends boolean = true, - Child extends ReadQueryShape = SchemaAwareQuery + Child extends ReadQueryShape = SchemaAwareQuery >( spec: Omit, 'foreignQuery' | 'mappers'>, customize?: ( - query: SchemaAwareQuery + query: SchemaAwareQuery ) => Child & CheckParameterState< AttachParameters< @@ -1266,8 +1363,10 @@ export class SchemaQueryBuilder< >, Relations & Record>, false, - AttachParameters, `relation:${K}`> - > + AttachParameters, `relation:${K}`>, + Connected + >, + Connected > { if ('foreignQuery' in spec || 'mappers' in spec) throw new ReadSchemaError( @@ -1298,14 +1397,14 @@ export class SchemaQueryBuilder< joinMany< F extends ReadObject, K extends string, - Child extends ReadQueryShape = SchemaAwareQuery + Child extends ReadQueryShape = SchemaAwareQuery >( spec: Omit< JoinManySpec, 'orderBy' | 'foreignQuery' | 'mappers' >, customize?: ( - query: SchemaAwareQuery + query: SchemaAwareQuery ) => Child & CheckParameterState< AttachParameters< @@ -1320,8 +1419,10 @@ export class SchemaQueryBuilder< AddField>, Relations & Record>, false, - AttachParameters, `relation:${K}`> - > + AttachParameters, `relation:${K}`>, + Connected + >, + Connected > { if ('foreignQuery' in spec || 'mappers' in spec || 'orderBy' in spec) throw new ReadSchemaError( @@ -1476,7 +1577,7 @@ export class SchemaQueryBuilder< } /** Return an independent mutable Knex snapshot, never the query's owned state. */ toKnexQuery(): Knex.QueryBuilder { - return this.compile(); + return nativeSql(this.compile()); } /** Configure an isolated Knex SELECT once, declaring the complete raw row output. */ @@ -1484,7 +1585,7 @@ export class SchemaQueryBuilder< configure: (query: Knex.QueryBuilder) => Knex.QueryBuilder | undefined, options: QueryOutput ): OpaqueQuery { - const sql = this.compile(); + const sql = this.toKnexQuery(); const result = configure(sql); if (result !== undefined && result !== sql) { if (result instanceof Promise) void result.catch(() => {}); @@ -1662,10 +1763,12 @@ export class SchemaQueryBuilder< return this.execute().then(resolve, reject); } /** Bind an independent query graph to a caller-owned transaction. */ - transacting(trx: Knex.Transaction): QueryView { + transacting(trx: Knex.Transaction): QueryView { const copy = this.copy(); copy.base.transacting(trx); - Object.assign(copy, { knex: trx }); + const connection = transactionConnection(this.knex, trx); + Object.assign(copy, { knex: connection }); + if (connection !== trx) copy.base = bindSql(copy.base, connection); copy.loaded = this.loaded.map(r => ({ ...r, query: unwrapParameterizedQuery(r.query.transacting(trx)) as any @@ -1687,6 +1790,22 @@ export class SchemaQueryBuilder< ...predicateParameters(this.predicates), ...this.loaded.flatMap(r => readerParameters(r.query)) ], + connect: knex => { + const copy = this.copy(); + const bound = bindDescriptionConnection(knex); + Object.assign(copy, { knex: bound }); + copy.base = bindSql(this.base, bound); + copy.defaults = this.defaults + ? bindSql(this.defaults, bound) + : undefined; + copy.loaded = this.loaded.map(relation => ({ + ...relation, + query: relation.query[COMPILED_READER]().connect!( + knex + ) as AnyReadQuery + })); + return copy; + }, compile: () => this.compile(undefined, COMPILE_PARAMETERS), decode: row => { if (checkOrphan && (row as any).__read_cti_present == null) diff --git a/libs/knex-schema/src/aliased-query.ts b/libs/knex-schema/src/aliased-query.ts index 06f8ce78..3c4396f7 100644 --- a/libs/knex-schema/src/aliased-query.ts +++ b/libs/knex-schema/src/aliased-query.ts @@ -13,6 +13,11 @@ import { import { getTableName } from './extension.js'; import { ALLOWED_OPS } from './operations/helpers.js'; import { SchemaQueryBuilder } from './SchemaQueryBuilder.js'; +import { + bindDescriptionConnection, + bindSql, + transactionConnection +} from './sql-description.js'; import { isSqlIdentifier } from './sql-identifiers.js'; type TableSchema = ObjectSchemaBuilder; @@ -165,6 +170,14 @@ export class AliasedQuerySource { return this.knex; } + /** @internal Materialize a captured plan without rerunning selectors or scopes. */ + bindConnection(knex: Knex): AliasedQuerySource { + const copy = this.cloneReadSource(); + copy.knex = bindDescriptionConnection(knex); + copy.sql = bindSql(this.sql, copy.knex); + return copy; + } + /** * Create a read-only query for one aliased schema. * Prefer query(knex, alias(schema, name)) so the table-context type is inferred. @@ -371,8 +384,11 @@ export class AliasedQuerySource { /** * Append raw ordering with Knex bindings; the caller owns aliases and SQL syntax. */ - orderByRaw(sql: string, bindings: readonly Knex.RawBinding[] = []): this { - this.sql.orderByRaw(sql, bindings); + orderByRaw( + sql: string | Knex.Raw, + bindings: readonly Knex.RawBinding[] = [] + ): this { + this.sql.orderByRaw(sql as string, bindings); return this; } @@ -466,8 +482,10 @@ export class AliasedQuerySource { */ transacting(trx: Knex.Transaction): AliasedQuerySource { const copy = this.cloneReadSource(); - Object.assign(copy, { knex: trx }); + const connection = transactionConnection(this.knex, trx); + Object.assign(copy, { knex: connection }); copy.sql = this.sql.clone().transacting(trx); + if (connection !== trx) copy.sql = bindSql(copy.sql, connection); copy.tables = new Map(this.tables); copy.selected = this.selected; copy.decoders = { ...this.decoders }; diff --git a/libs/knex-schema/src/compiled-query.ts b/libs/knex-schema/src/compiled-query.ts index 270b9282..a5ed9e0d 100644 --- a/libs/knex-schema/src/compiled-query.ts +++ b/libs/knex-schema/src/compiled-query.ts @@ -10,6 +10,10 @@ import { } from './parameter.js'; import type { BoundQuerySql, UnderlyingQuery } from './parameter-types.js'; import { ReadSchemaError } from './read-schema.js'; +import { + actualConnection, + isConnectionDescription +} from './sql-description.js'; /** @internal Internal compilation is the only path allowed to emit placeholder slots. */ export const COMPILE_PARAMETERS = Symbol('compile-query-parameters'); @@ -23,6 +27,7 @@ export interface CompiledReader { compile(): Knex.QueryBuilder; decode(row: unknown): unknown; bind(values: ParameterValues): unknown; + connect?(knex: Knex): unknown; } type Reader = { [COMPILED_READER](): CompiledReader }; @@ -38,6 +43,31 @@ const facades = new WeakMap(); const orders = new WeakMap(); const caches = new WeakMap(); const runtimes = new WeakMap(); +const boundReaders = new WeakMap>(); + +function withConnection(reader: Reader, knex: Knex): Reader { + if ( + typeof knex !== 'function' || + isConnectionDescription(knex) || + !knex.client + ) + throw new ReadSchemaError( + 'Supply a Knex connection or transaction as the first argument' + ); + const connection = actualConnection(knex); + let bindings = boundReaders.get(reader); + if (!bindings) { + bindings = new WeakMap(); + boundReaders.set(reader, bindings); + } + let bound = bindings.get(connection); + if (!bound) { + bound = runtimeFor(reader).connect!(connection) as Reader; + copyParameterOrder(reader, bound); + bindings.set(connection, bound); + } + return bound; +} function runtimeFor(reader: Reader): CompiledReader { let runtime = runtimes.get(reader); @@ -252,13 +282,19 @@ const blocked = new Set([ export function finishParameterizedQuery(value: T): T { const reader = unwrapParameterizedQuery(value) as T & Reader; if (!reader || typeof reader[COMPILED_READER] !== 'function') return value; - const uses = reader[COMPILED_READER]().uses; + const runtime = reader[COMPILED_READER](); + const uses = runtime.uses; + const needsConnection = isConnectionDescription(runtime.knex); validateParameterUses(uses); names(reader, uses); - if (!uses.length) return reader; + if (!uses.length && !needsConnection) return reader; const previous = facades.get(reader); if (previous) return previous as T; - const callable = (...args: unknown[]) => execute(reader, args); + const resolve = (args: unknown[]): [Reader, unknown[]] => + needsConnection + ? [withConnection(reader, args[0] as Knex), args.slice(1)] + : [reader, args]; + const callable = (...args: unknown[]) => execute(...resolve(args)); // Keep source checks and instanceof working; actual class methods are bound // to the captured reader, never to the function object. Object.setPrototypeOf(callable, Object.getPrototypeOf(reader)); @@ -266,14 +302,29 @@ export function finishParameterizedQuery(value: T): T { get(_target, prop) { if (prop === 'then') return undefined; if (prop === 'query') - return (...args: unknown[]) => - reader[COMPILED_READER]().bind(valuesFor(reader, args)); + return (...args: unknown[]) => { + const [bound, values] = resolve(args); + return bound[COMPILED_READER]().bind( + valuesFor(bound, values) + ); + }; if (prop === 'toSQL') - return (...args: unknown[]) => inspect(reader, args); - if (blocked.has(prop)) + return (...args: unknown[]) => inspect(...resolve(args)); + if ( + blocked.has(prop) || + (needsConnection && + [ + 'ref', + 'transacting', + 'whereExists', + 'whereNotExists', + 'orWhereExists', + 'orWhereNotExists' + ].includes(String(prop))) + ) return () => { throw new ReadSchemaError( - `Bind query parameters before calling ${String(prop)}()` + `Bind query ${needsConnection ? 'connection and parameters' : 'parameters'} before calling ${String(prop)}()` ); }; const member = Reflect.get(reader, prop, reader); diff --git a/libs/knex-schema/src/index.ts b/libs/knex-schema/src/index.ts index 3aec5c01..5b8ae383 100644 --- a/libs/knex-schema/src/index.ts +++ b/libs/knex-schema/src/index.ts @@ -118,6 +118,7 @@ export type { ParameterState, ParametersOf, QueryArguments, + QueryDefinition, QueryView } from './parameter-types.js'; /** @internal Shared query/ORM fluent typing. */ diff --git a/libs/knex-schema/src/parameter-types.ts b/libs/knex-schema/src/parameter-types.ts index 0b1607fd..96ddc3ee 100644 --- a/libs/knex-schema/src/parameter-types.ts +++ b/libs/knex-schema/src/parameter-types.ts @@ -1,4 +1,5 @@ import type { InferType } from '@cleverbrush/schema'; +import type { Knex } from 'knex'; import type { QueryParameter } from './parameter.js'; /** @internal Query argument state; origins allow branch removal without stale arguments. */ @@ -21,6 +22,7 @@ export type UnderlyingQuery = Q extends { readonly [QUERY_SOURCE]: infer S } /** @internal */ export interface ParameterReader { readonly parameters: unknown; + readonly connection: unknown; readonly result: unknown; } @@ -266,6 +268,39 @@ type UnboundTerminal = | 'onConflict' | 'save'; +type ConnectionTerminal = + | UnboundTerminal + | 'ref' + | 'transacting' + | 'whereExists' + | 'whereNotExists' + | 'orWhereExists' + | 'orWhereNotExists'; + +/** A reusable, non-thenable SELECT definition, independent of any Knex client. */ +export type QueryDefinition< + Q, + P extends ParameterState = ParametersOf +> = Omit & { + readonly [QUERY_SOURCE]: Q; + /** Execute using the caller-owned connection or transaction and positional values. */ + ( + knex: Knex, + ...args: QueryArguments

+ ): Promise[] : never>; + /** Bind a fresh ordinary reader for composition, pagination or supported writes. */ + query( + knex: Knex, + ...args: QueryArguments

+ ): Q extends { + readonly [PARAMETER_READER]: infer F extends ParameterReader; + } + ? (F & { readonly parameters: []; readonly connection: true })['result'] + : never; + /** Inspect SQL; compilation is cached per immutable definition and Knex instance. */ + toSQL(knex: Knex, ...args: QueryArguments

): BoundQuerySql; +}; + /** SQL with positional value placeholders and independently snapshotted bindings. */ export interface BoundQuerySql { readonly sql: string; @@ -292,7 +327,11 @@ export type ParameterizedQuery = Omit< }; /** @internal Ordinary readers keep their existing API until a parameter is added. */ -export type QueryView = - ParametersOf extends readonly [] - ? Q - : ParameterizedQuery>; +export type QueryView = [ + Connected, + ParametersOf +] extends [true, readonly []] + ? Q + : Connected extends false + ? QueryDefinition + : ParameterizedQuery>; diff --git a/libs/knex-schema/src/query-definition.test-d.ts b/libs/knex-schema/src/query-definition.test-d.ts new file mode 100644 index 00000000..e53ecbc5 --- /dev/null +++ b/libs/knex-schema/src/query-definition.test-d.ts @@ -0,0 +1,138 @@ +import Knex from 'knex'; +import { expectTypeOf, test } from 'vitest'; +import { + aggregate, + alias, + date, + defineEntity, + eq, + number, + object, + parameter, + query, + string +} from './index.js'; + +const knex = Knex({ client: 'pg' }); +const User = object({ + id: number().primaryKey(), + name: string(), + balance: number().decimal(24, 6), + birthday: date().optional() +}).hasTableName('users'); + +test('connection state and parameter order survive immutable table composition', async () => { + const all = query(User); + expectTypeOf(all).parameters.toEqualTypeOf<[Knex.Knex]>(); + const byId = all.where(t => t.id, parameter('id')); + expectTypeOf(byId).parameters.toEqualTypeOf<[Knex.Knex, number]>(); + expectTypeOf(byId).not.toBeAny(); + expectTypeOf( + (await byId.query(knex, 1).update({ name: 'Jane' }))[0].name + ).toEqualTypeOf(); + const names = byId + .orderBy('id') + .limit(2) + .select(t => ({ name: t.name })); + expectTypeOf(names).parameters.toEqualTypeOf<[Knex.Knex, number]>(); + expectTypeOf(await names(knex, 1)).toEqualTypeOf<{ name: string }[]>(); + expectTypeOf(await names.query(knex, 1).first()).toEqualTypeOf< + { name: string } | undefined + >(); + expectTypeOf( + names.query(knex, 1).where(t => t.name, parameter('new')) + ).parameters.toEqualTypeOf<[string]>(); + // @ts-expect-error connection is required + byId(1); + // @ts-expect-error wrong parameter type + byId(knex, '1'); + // @ts-expect-error missing value + byId(knex); + // @ts-expect-error no implicit connection for metadata-only definitions + all.first(); + // @ts-expect-error native references require binding + all.ref(t => t.id); + // @ts-expect-error bind explicitly before writes + byId.update({ name: 'Jane' }); + // @ts-expect-error selected readers remain non-writable after binding + names.query(knex, 1).update({ name: 'Jane' }); + const nullable = all + .where(t => t.birthday, parameter('when')) + .where(t => t.balance, parameter('amount')); + expectTypeOf(nullable).parameters.toEqualTypeOf< + [Knex.Knex, Date | null, string] + >(); + expectTypeOf(all.select('id', 'name')).parameters.toEqualTypeOf< + [Knex.Knex] + >(); + expectTypeOf(all.count()).parameters.toEqualTypeOf<[Knex.Knex]>(); + expectTypeOf( + all + .groupBy('name') + .select(t => ({ name: t.name, count: aggregate.count() })) + ).parameters.toEqualTypeOf<[Knex.Knex]>(); + expectTypeOf(all.distinct('name')).parameters.toEqualTypeOf<[Knex.Knex]>(); +}); + +test('nested definitions, groups, aliases and variants retain strong types', async () => { + const Task = defineEntity( + object({ + id: number().primaryKey(), + ownerId: number(), + owner: User.optional() + }).hasTableName('tasks') + ).belongsTo( + t => t.owner, + t => t.ownerId, + t => t.id + ); + const nested = query(Task.schema) + .include('owner', q => + q.where(t => t.name, parameter('name')).select('name') + ) + .where(t => t.id, parameter('id')); + expectTypeOf(nested).parameters.toEqualTypeOf< + [Knex.Knex, string, number] + >(); + expectTypeOf(await nested(knex, 'Jane', 1)).toEqualTypeOf< + { id: number; ownerId: number; owner: { name: string } }[] + >(); + const grouped = query(User).where(p => + p + .where(t => t.id, parameter('id')) + .orWhere(t => t.name, parameter('name')) + ); + expectTypeOf(grouped).parameters.toEqualTypeOf< + [Knex.Knex, number, string] + >(); + const joined = query(alias(User, 'u')) + .leftJoin(alias(Task.schema, 't'), t => eq(t.u.id, t.t.ownerId)) + .where(t => t.u.name, parameter('name')) + .select(t => ({ id: t.u.id, task: t.t.id })); + expectTypeOf(joined).parameters.toEqualTypeOf<[Knex.Knex, string]>(); + expectTypeOf(await joined(knex, 'Jane')).toEqualTypeOf< + { id: number; task: number | null }[] + >(); + const Asset = defineEntity( + object({ id: number().primaryKey(), kind: string() }).hasTableName( + 'assets' + ) + ) + .discriminator('kind') + .stiVariant('note', object({ text: string() })) + .stiVariant('task', Task); + const variants = query(Asset.schema) + .forVariant('note', q => q.where(t => t.text, parameter('text'))) + .where(t => t.id, parameter('id')); + expectTypeOf(variants).parameters.toEqualTypeOf< + [Knex.Knex, string, number] + >(); + expectTypeOf(variants.selectVariants(['task'])).parameters.toEqualTypeOf< + [Knex.Knex, number] + >(); + const relation = query(Asset.schema).includeVariant('task', 'owner', q => + q.where(t => t.name, parameter('name')) + ); + expectTypeOf(relation).parameters.toEqualTypeOf<[Knex.Knex, string]>(); + expectTypeOf(variants.query(knex, 'hi', 1)).not.toBeFunction(); +}); diff --git a/libs/knex-schema/src/query-definition.test.ts b/libs/knex-schema/src/query-definition.test.ts new file mode 100644 index 00000000..dce3db5b --- /dev/null +++ b/libs/knex-schema/src/query-definition.test.ts @@ -0,0 +1,254 @@ +import Knex from 'knex'; +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { + aggregate, + alias, + date, + defineEntity, + eq, + number, + object, + parameter, + query, + string +} from './index.js'; + +const User = object({ + id: number().primaryKey(), + name: string().hasColumnName('display_name'), + age: number(), + birthday: date().optional(), + profile: object({ score: number(), label: string() }).optional() +}).hasTableName('users'); +const Task = defineEntity( + object({ + id: number().primaryKey(), + ownerId: number(), + owner: User.optional() + }).hasTableName('tasks') +).belongsTo( + t => t.owner, + t => t.ownerId, + t => t.id +); +const knex = Knex({ client: 'pg' }); +afterEach(() => vi.restoreAllMocks()); + +describe('connection-independent query definitions', () => { + it('exposes stable schemas and is non-thenable without creating a native builder', async () => { + const builder = vi.spyOn(knex.client, 'queryBuilder'); + const compiler = vi.spyOn(knex.client, 'queryCompiler'); + const root = query(User); + expect(typeof root).toBe('function'); + expect((root as any).then).toBeUndefined(); + const definition = query(User) + .select(t => ({ name: t.name })) + .limit(2); + expect(typeof definition).toBe('function'); + expect((definition as any).then).toBeUndefined(); + expect(await definition).toBe(definition); + expect(definition.rowSchema.parse({ name: 'Jane' })).toEqual({ + name: 'Jane' + }); + expect(builder).not.toHaveBeenCalled(); + expect(compiler).not.toHaveBeenCalled(); + expect(definition.toSQL(knex).sql).toContain('"display_name"'); + expect(definition.query(knex).rowSchema).toBe(definition.rowSchema); + }); + + it('binds independent readers and compiles once per definition and client', () => { + const selector = vi.fn((t: any) => t.id); + const group = vi.fn((p: any) => p.where(selector, parameter('id'))); + const definition = query(User).where(group).select('id'); + const compiler = vi.spyOn(knex.client, 'queryCompiler'); + expect((definition as any).toSQL(knex, 10).bindings).toEqual([10]); + const count = compiler.mock.calls.length; + expect(count).toBeGreaterThan(0); + expect((definition as any).toSQL(knex, 20).bindings).toEqual([20]); + expect(compiler).toHaveBeenCalledTimes(count); + const bound = (definition as any).query(knex, 30); + expect(bound.toQuery()).toContain('30'); + expect(bound.where('age', 21).toQuery()).toContain('21'); + expect(bound.toQuery()).not.toContain('21'); + expect(group).toHaveBeenCalledTimes(1); + expect(selector).toHaveBeenCalledTimes(1); + const other = Knex({ + client: 'pg', + wrapIdentifier: (value, original) => original(`different_${value}`) + }); + expect((definition as any).toSQL(other, 40).sql).toContain( + 'different_users' + ); + expect((definition as any).toSQL(knex, 50).sql).not.toContain( + 'different_' + ); + expect(group).toHaveBeenCalledTimes(1); + }); + + it('captures scopes and raw values once, including default ordering and pagination', () => { + const bindings = ['Jane']; + const defaults = vi.fn((q: any) => + q.where('age', '>=', 18).orderBy('name').limit(9) + ); + const named = vi.fn((q: any) => + q.whereRaw('display_name = ?', bindings) + ); + const definition = query( + User.defaultScope(defaults).scope('named', named) + ) + .scoped('named') + .limit(2); + bindings[0] = 'changed'; + const sql = definition.toSQL(knex); + expect(sql.bindings).toEqual([18, 'Jane', 2]); + expect(sql.sql).toContain('order by'); + expect(definition.unscoped().toSQL(knex).bindings).toEqual(['Jane', 2]); + expect(definition.query(knex).toQuery()).toContain('Jane'); + expect(defaults).toHaveBeenCalledTimes(1); + expect(named).toHaveBeenCalledTimes(1); + }); + + it('captures JSON paths, aggregates, aliases and joins with connection-specific identifiers', () => { + const nested = query(User) + .where(t => t.profile.score, '>', parameter('score')) + .select(t => ({ label: t.profile.label })); + expect(nested.toSQL(knex, 2).sql).toContain('#>>'); + expect(nested.toSQL(knex, 2).bindings).toContain(2); + const join = vi.fn((t: any) => eq(t.u.id, t.task.ownerId)); + const projection = vi.fn((t: any) => ({ + name: t.u.name, + count: aggregate.count(t.task.id) + })); + const joined = query(alias(User, 'u')) + .leftJoin(alias(Task.schema, 'task'), join) + .groupBy(t => t.u.name) + .select(projection) + .orderByRaw('?? desc', ['u.display_name']); + expect(joined.toSQL(knex).sql).toContain('left join'); + expect(joined.query(knex).toQuery()).toContain('count('); + expect(join).toHaveBeenCalledTimes(1); + expect(projection).toHaveBeenCalledTimes(1); + }); + + it('retains relation and polymorphic metadata without replaying customizers', () => { + const customize = vi.fn((q: any) => + q.where('name', parameter('name')).select('name') + ); + const definition = query(Task.schema) + .include('owner', customize) + .where(t => t.id, parameter('id')); + expect((definition as any).toSQL(knex, 'Jane', 1).bindings).toEqual( + query(knex, Task.schema) + .include('owner', q => + q.where('name', parameter('name')).select('name') + ) + .where(t => t.id, parameter('id')) + .toSQL('Jane', 1).bindings + ); + expect((definition as any).query(knex, 'John', 2).toQuery()).toContain( + 'John' + ); + expect(customize).toHaveBeenCalledTimes(1); + const Asset = defineEntity( + object({ id: number().primaryKey(), kind: string() }).hasTableName( + 'assets' + ) + ) + .discriminator('kind') + .stiVariant('note', object({ text: string() })) + .stiVariant('task', Task); + const variant = vi.fn((q: any) => q.where('text', parameter('text'))); + const polymorphic = query(Asset.schema) + .forVariant('note', variant) + .where(t => t.id, parameter('id')); + const rowSchema = polymorphic.rowSchema; + expect((polymorphic as any).toSQL(knex, 'hello', 1).bindings).toContain( + 'hello' + ); + expect((polymorphic as any).query(knex, 'world', 2).rowSchema).toBe( + rowSchema + ); + expect( + (polymorphic.selectVariants(['task']) as any).toSQL(knex, 3) + .bindings + ).toContain(3); + expect(variant).toHaveBeenCalledTimes(1); + }); + + it('rejects missing connections, bad arguments and connection-dependent escapes', () => { + const definition = query(User).where(t => t.id, parameter('id')); + expect(() => (definition as any).toSQL(1)).toThrow(/Knex connection/); + expect(() => (definition as any).toSQL(knex, 'wrong')).toThrow(); + expect(() => (definition as any).toSQL(knex)).toThrow(/Expected 1/); + for (const method of [ + 'ref', + 'apply', + 'selectRaw', + 'first', + 'update', + 'transacting' + ]) + expect(() => (definition as any)[method]()).toThrow(/Bind query/); + expect(() => query(User).whereRaw('?', [knex.raw('1')])).toThrow( + /Bind a connection/ + ); + expect(() => + query(User).whereIn('id', knex('users').select('id')) + ).toThrow(/Bind a connection/); + expect(() => query(User).where('id', knex.raw('1'))).toThrow( + /Bind a connection/ + ); + const json = query(User).whereJsonPath('profile', '$.score', '=', 3); + expect(() => + json + .query(Knex({ client: 'sqlite3', useNullAsDefault: true })) + .toQuery() + ).toThrow(/only supported on PostgreSQL/); + }); + + it('snapshots definition values and inspected bindings independently', () => { + const birthday = new Date('2026-01-01T00:00:00Z'); + const profile = { score: 3, label: 'captured' }; + const definition = query(User) + .where('birthday', birthday) + .where('profile', profile) + .whereRaw('display_name = ?', ['Jane']); + birthday.setUTCFullYear(2040); + profile.label = 'changed'; + const first = definition.toSQL(knex); + expect(first.bindings).toContainEqual(new Date('2026-01-01T00:00:00Z')); + expect(first.bindings).toContainEqual({ score: 3, label: 'captured' }); + ( + first.bindings.find(value => value instanceof Date) as Date + ).setUTCFullYear(2050); + expect(definition.toSQL(knex).bindings).toContainEqual( + new Date('2026-01-01T00:00:00Z') + ); + }); + + it('hands native Knex snapshots to escape hatches without changing event listeners', () => { + const bound = query(User).query(knex); + const rowSchema = bound.rowSchema; + const raw = bound.toKnexQuery(); + const listener = vi.fn(); + raw.on('query', listener); + expect(raw.listeners('query')).toContain(listener); + raw.removeListener('query', listener); + expect(raw.listeners('query')).not.toContain(listener); + const opaque = bound.apply( + sql => { + sql.on('query', listener); + expect(sql.listeners('query')).toContain(listener); + sql.removeListener('query', listener); + return sql.clearSelect().select({ id: 'id' }); + }, + { output: object({ id: number() }) } + ); + const opaqueSql = opaque.toKnexQuery(); + opaqueSql.on('query', listener); + expect(opaqueSql.listeners('query')).toContain(listener); + opaqueSql.removeListener('query', listener); + expect(opaqueSql.listeners('query')).not.toContain(listener); + expect(bound.rowSchema).toBe(rowSchema); + }); +}); diff --git a/libs/knex-schema/src/query.ts b/libs/knex-schema/src/query.ts index 7c5e517e..658c49b6 100644 --- a/libs/knex-schema/src/query.ts +++ b/libs/knex-schema/src/query.ts @@ -6,17 +6,33 @@ import { isTableAlias, type TableAlias } from './aliased-query.js'; +import { finishParameterizedQuery } from './compiled-query.js'; import { getTableName } from './extension.js'; +import type { QueryView } from './parameter-types.js'; import { QuerySource } from './QuerySource.js'; import type { ReadObject } from './read-schema.js'; import { createReadQuery, type SchemaAwareQuery } from './SchemaQueryBuilder.js'; +import { describeConnection } from './sql-description.js'; // Register the private SQL/write planner before creating relation queries. void QuerySource; +/** Define a reusable aliased SELECT. Supply Knex at invocation or query(knex, ...args). */ +export function query( + schema: TableAlias +): QueryView, never, [], false>, false>; +/** + * Define an immutable, non-thenable query without creating a Knex client. + * Selectors and scopes run once during construction. Supply a connection or + * transaction when calling the definition, query(knex, ...args) or toSQL(knex, ...args). + */ +export function query( + schema: S +): SchemaAwareQuery; + /** Create an immutable, lazy query with an automatically inferred row schema. */ export function query( knex: Knex, @@ -27,18 +43,29 @@ export function query( knex: Knex, schema: S ): SchemaAwareQuery; -export function query( - knex: Knex, - schema: S | TableAlias, +export function query( + connectionOrSchema: Knex | ReadObject | TableAlias, + suppliedSchema?: ReadObject | TableAlias, ...unsupported: unknown[] -): SchemaAwareQuery | AliasedQueryBuilder> { +): any { if (unsupported.length) throw new TypeError( 'Raw query sources are not supported; use apply(..., { output })' ); + const knex = + suppliedSchema === undefined + ? describeConnection() + : (connectionOrSchema as Knex); + const schema = (suppliedSchema ?? connectionOrSchema) as + | ReadObject + | TableAlias; if (isTableAlias(schema)) - return new AliasedQueryBuilder(new AliasedQuerySource(knex, schema)); - return createReadQuery(knex, schema, knex(getTableName(schema))); + return finishParameterizedQuery( + new AliasedQueryBuilder(new AliasedQuerySource(knex, schema)) + ); + return finishParameterizedQuery( + createReadQuery(knex, schema, knex(getTableName(schema))) + ); } /** Connection-bound query factory. Query configuration is lazy and immutable. */ diff --git a/libs/knex-schema/src/read-predicates.ts b/libs/knex-schema/src/read-predicates.ts index 0941dd51..4fcbc33a 100644 --- a/libs/knex-schema/src/read-predicates.ts +++ b/libs/knex-schema/src/read-predicates.ts @@ -24,6 +24,10 @@ import { type WithoutParameters } from './parameter-types.js'; import { type ReadSchema, ReadSchemaError } from './read-schema.js'; +import { + assertPortableBindings, + isConnectionDescription +} from './sql-description.js'; const finishGroup = Symbol('finishReadPredicateGroup'); @@ -106,9 +110,15 @@ export function captureReadRaw( bindings: readonly Knex.RawBinding[] = [] ): () => Knex.Raw { assertNoParameters(bindings); + if (isConnectionDescription(knex)) { + assertPortableBindings(bindings); + const captured = bindings.map(copyBinding); + return () => knex.raw(sql, captured.map(copyBinding)); + } return captureSql(knex, knex.raw(sql, [...bindings])); } function captureSubquery(knex: Knex, query: Knex.QueryBuilder): () => Knex.Raw { + if (isConnectionDescription(knex)) assertPortableBindings(query); if ( !query || typeof query.toSQL !== 'function' || @@ -130,6 +140,7 @@ function captureSubquery(knex: Knex, query: Knex.QueryBuilder): () => Knex.Raw { /** @internal Snapshot concrete bindings; placeholders require a typed predicate. */ export function captureValue(knex: Knex, value: any): () => any { assertNoParameters(value); + if (isConnectionDescription(knex)) assertPortableBindings(value); if (value && typeof value.toSQL === 'function') return typeof value.clone === 'function' ? captureSubquery(knex, value) @@ -179,6 +190,8 @@ export abstract class ReadPredicates< /** Quote a mapped column for trusted raw SQL or correlated subqueries. */ ref(selector: ReadPredicateSelector): Knex.Ref | Knex.Raw { const { knex, column } = this.readPredicateContext(); + if (isConnectionDescription(knex)) + throw new ReadSchemaError('Bind a connection before calling ref()'); const resolved = column(selector); return typeof resolved === 'string' ? knex.ref(resolved) : resolved; } @@ -613,6 +626,7 @@ export abstract class ReadPredicates< const context = this.readPredicateContext(); const name = context.column(column); if ( + !isConnectionDescription(context.knex) && !['pg', 'postgres', 'postgresql'].includes( context.knex.client.config.client as string ) @@ -620,21 +634,33 @@ export abstract class ReadPredicates< throw new ReadSchemaError( 'whereJsonPath() is only supported on PostgreSQL' ); - if (operator === '@?' || operator === '@@') - return this.whereRaw(`?? ${operator === '@?' ? '@\\?' : '@@'} ?`, [ - name, - path - ]); - if (!ALLOWED_OPS.has(operator.toLowerCase())) + const predicateOperator = operator === '@?' || operator === '@@'; + if (!predicateOperator && !ALLOWED_OPS.has(operator.toLowerCase())) throw new ReadSchemaError('Unsupported JSON comparison operator'); - return this.whereRaw( - `jsonb_path_query_first(??, ?) ${operator} ?::jsonb`, - [ - name, - path.startsWith('$') ? path : `$.${path}`, - JSON.stringify(value) - ] + const captured = captureReadRaw( + context.knex, + predicateOperator + ? `?? ${operator === '@?' ? '@\\?' : '@@'} ?` + : `jsonb_path_query_first(??, ?) ${operator} ?::jsonb`, + predicateOperator + ? [name, path] + : [ + name, + path.startsWith('$') ? path : `$.${path}`, + JSON.stringify(value) + ] ); + return this.addReadPredicate(query => { + if ( + !['pg', 'postgres', 'postgresql'].includes( + query.client.config.client as string + ) + ) + throw new ReadSchemaError( + 'whereJsonPath() is only supported on PostgreSQL' + ); + query.whereRaw(captured()); + }); } } diff --git a/libs/knex-schema/src/sql-description.ts b/libs/knex-schema/src/sql-description.ts new file mode 100644 index 00000000..1ffda3a3 --- /dev/null +++ b/libs/knex-schema/src/sql-description.ts @@ -0,0 +1,247 @@ +import type { Knex } from 'knex'; +import { copyBinding } from './parameter.js'; +import { ReadSchemaError } from './read-schema.js'; + +// This is an internal SQL operation graph, not a Knex client. It records only +// library-owned planner operations. Public native SQL escape hatches require a +// connection; consumer selectors/scopes/customizers run before recording SQL. +const DESCRIPTION = Symbol('sql-description'); +const CONNECTION = Symbol('description-connection'); +type Operation = readonly [method: string, args: readonly unknown[]]; +type Native = Knex.QueryBuilder | Knex.Raw; + +class SqlDescription { + readonly operations: Operation[] = []; + constructor( + readonly method: string, + readonly args: readonly unknown[] + ) {} +} + +function describe(method: string, args: readonly unknown[]): any { + const description = new SqlDescription(method, args.map(snapshot)); + const proxy = new Proxy(description, { + get(target, key) { + if (key === DESCRIPTION) return target; + if (key === 'then') return undefined; + if (key === 'clone') + return () => { + const copy = describe(target.method, target.args); + copy[DESCRIPTION].operations.push(...target.operations); + return copy; + }; + if (key === 'toSQL' || key === 'toQuery') + return () => { + throw new ReadSchemaError( + 'Bind a connection before compiling SQL' + ); + }; + if (typeof key !== 'string' || key.startsWith('_')) + throw new ReadSchemaError( + `Unsupported SQL description access: ${String(key)}` + ); + return (...values: unknown[]) => { + target.operations.push([key, values.map(snapshot)]); + return proxy; + }; + } + }); + return proxy; +} + +function descriptionOf(value: any): SqlDescription | undefined { + return value && typeof value === 'object' ? value[DESCRIPTION] : undefined; +} + +function snapshot(value: any): any { + if (descriptionOf(value)) return value.clone(); + if (Array.isArray(value)) return value.map(snapshot); + if (value && Object.getPrototypeOf(value) === Object.prototype) + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [key, snapshot(item)]) + ); + return copyBinding(value); +} + +/** @internal Build SQL descriptions without instantiating a client or choosing a dialect. */ +export function describeConnection(): Knex { + return new Proxy((...args: unknown[]) => describe('table', args), { + get(_target, key) { + if (key === CONNECTION) return true; + if (key === 'then') return undefined; + if (typeof key !== 'string' || key === 'client') + throw new ReadSchemaError( + 'A query definition has no connection' + ); + return (...args: unknown[]) => describe(key, args); + } + }) as unknown as Knex; +} + +/** @internal Whether a planner is capturing an unbound SQL description. */ +export function isConnectionDescription(knex: Knex): boolean { + return (knex as any)[CONNECTION] === true; +} + +/** @internal Native SQL objects cannot be imported into connection-independent definitions. */ +export function assertPortableBindings(value: unknown): void { + if (descriptionOf(value)) return; + if (value && typeof (value as any).toSQL === 'function') + throw new ReadSchemaError( + 'Bind a connection before using native SQL expressions or subqueries' + ); + if (Array.isArray(value)) value.forEach(assertPortableBindings); + else if (value && Object.getPrototypeOf(value) === Object.prototype) + Object.values(value).forEach(assertPortableBindings); +} + +const connections = new WeakMap(); +const nativeConnections = new WeakMap(); +const nativeBuilders = new WeakMap(); +const builderConnections = new WeakMap(); + +/** @internal Public escape hatches receive native Knex objects, not planner adapters. */ +export function nativeSql(query: T): T { + return (nativeBuilders.get(query) ?? query) as T; +} + +/** @internal Connection supplied to a deferred internal planner callback. */ +export function connectionForSql(query: Knex.QueryBuilder): Knex { + const connection = builderConnections.get(query); + if (!connection) throw new ReadSchemaError('SQL planner is not bound'); + return connection; +} + +/** @internal Recover the supplied connection, including transaction identity. */ +export function actualConnection(knex: Knex): Knex { + return nativeConnections.get(knex) ?? knex; +} + +/** @internal Preserve fragment resolution when a bound definition enters a transaction. */ +export function transactionConnection( + source: Knex, + trx: Knex.Transaction +): Knex { + return nativeConnections.has(source) ? bindDescriptionConnection(trx) : trx; +} + +/** @internal Materialize library SQL descriptions with the supplied client's configuration. */ +export function materializeSql(value: T, knex: Knex): T { + const connection = actualConnection(knex); + const description = descriptionOf(value); + if (description) { + const args = description.args.map(arg => + materializeSql(arg, connection) + ); + let result = (connection as any)[description.method](...args); + for (const [method, values] of description.operations) + result = result[method]( + ...values.map(arg => materializeSql(arg, connection)) + ); + return result; + } + if (Array.isArray(value)) + return value.map(arg => materializeSql(arg, connection)) as T; + if (value && typeof value === 'object') { + const native = nativeBuilders.get(value); + if (native) return native as T; + if (Object.getPrototypeOf(value) === Object.prototype) + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [ + key, + materializeSql(item, connection) + ]) + ) as T; + } + // Only internal SQL callbacks can enter a description. Bind their nested + // builders too, so captured fragments always use the execution connection. + if (typeof value === 'function' && 'client' in value) + return actualConnection(value as unknown as Knex) as T; + if (typeof value === 'function') + return function (this: Knex.QueryBuilder, ...args: unknown[]) { + return value.apply( + bindSql(this, connection), + args.map(arg => + isBuilder(arg) ? bindSql(arg, connection) : arg + ) + ); + } as T; + return copyBinding(value); +} + +function isBuilder(value: any): value is Knex.QueryBuilder { + return ( + !!value && + typeof value === 'object' && + typeof value.toSQL === 'function' && + typeof value.clone === 'function' + ); +} + +/** @internal Retain ordinary Knex behavior while resolving captured SQL fragments at its boundary. */ +export function bindSql(value: T, knex: Knex): T { + const connection = actualConnection(knex); + const native = materializeSql(value, connection); + const proxy = new Proxy(native, { + get(target, key, receiver) { + if (key === DESCRIPTION) return undefined; + const member = Reflect.get(target, key, receiver); + if (typeof member !== 'function') return member; + if (['then', 'catch', 'finally'].includes(String(key))) + return member.bind(target); + return (...args: unknown[]) => { + const result = member.apply( + target, + args.map(arg => materializeSql(arg, connection)) + ); + if (result === target) return proxy; + return isBuilder(result) ? bindSql(result, connection) : result; + }; + } + }); + nativeBuilders.set(proxy, native); + builderConnections.set(proxy, connection); + return proxy; +} + +/** @internal A bound planner uses the real client, never an ambient/global connection. */ +export function bindDescriptionConnection(knex: Knex): Knex { + const connection = actualConnection(knex); + const cached = connections.get(connection); + if (cached) return cached; + const bound = new Proxy(connection, { + apply(target, _this, args) { + return bindSql( + (target as any)( + ...args.map(arg => materializeSql(arg, connection)) + ), + connection + ); + }, + get(target, key) { + if (key === CONNECTION) return false; + const member = Reflect.get(target, key, target); + if (typeof member !== 'function' || key === 'client') return member; + return (...args: unknown[]) => { + const result = member.apply( + target, + args.map(arg => materializeSql(arg, connection)) + ); + return isBuilder(result) ? bindSql(result, connection) : result; + }; + } + }); + connections.set(connection, bound); + nativeConnections.set(bound, connection); + return bound; +} + +/** @internal Defer private planner inspection until a real Knex builder exists. */ +export function configureSql( + query: Knex.QueryBuilder, + configure: (query: Knex.QueryBuilder) => void +): Knex.QueryBuilder { + if (descriptionOf(query)) return query.modify(configure); + configure(query); + return query; +} diff --git a/libs/orm/README.md b/libs/orm/README.md index 0e011e36..3bc4e3ee 100644 --- a/libs/orm/README.md +++ b/libs/orm/README.md @@ -249,6 +249,28 @@ for scoped search, subquery snapshots, pagination, and raw-SQL boundaries. ## Parameterized compiled reads +For module-level reads that receive an injected connection only at execution, +`query` and `parameter` are also re-exported from this package: + +```ts +// data/user-queries.ts +import { parameter, query } from '@cleverbrush/orm'; +import { UserSchema } from './user-schema.js'; + +export const findUser = query(UserSchema) + .where(user => user.id, parameter('id')); + +// api/handlers/get-user.ts — db is the injected DbContext +const user = await findUser.query(db.knex, userId).first(); +const rows = await findUser(db.knex, userId); +``` + +These connection-independent reads return detached schema-backed rows, not +entities registered in a DbContext's identity map. Use DbSet queries below when +tracking, `find()` helpers or context-managed changes are required. See the +[multi-file query-definition example](../knex-schema/README.md#connection-independent-query-definitions) +for transactions, connection-specific SQL caching and binding before writes. + `parameter` is re-exported by `@cleverbrush/orm`. Adding a named placeholder to a DbSet query creates a callable SELECT with schema-inferred positional arguments: diff --git a/libs/orm/src/dbcontext.ts b/libs/orm/src/dbcontext.ts index 1ac50861..40096b35 100644 --- a/libs/orm/src/dbcontext.ts +++ b/libs/orm/src/dbcontext.ts @@ -138,20 +138,22 @@ function buildContext( ): DbContext { const sets = {} as { [K in keyof TMap]: DbSet }; for (const key of Object.keys(entities) as Array) { - sets[key] = makeDbSet(knex, entities[key]) as DbSet; + sets[key] = makeDbSet(knex, entities[key]); } const ctx = { ...sets, knex, withTransaction(trx: Knex.Transaction): DbContext { - return buildContext(trx as unknown as Knex, entities); + return buildContext(trx as unknown as Knex, entities); }, async transaction( callback: (db: DbContext) => Promise ): Promise { return knex.transaction(async (trx: Knex.Transaction) => { - return callback(buildContext(trx as unknown as Knex, entities)); + return callback( + buildContext(trx as unknown as Knex, entities) + ); }); } } satisfies DbContext; @@ -180,11 +182,7 @@ function buildTrackedContext( return item; }); }; - sets[key as keyof TMap] = makeDbSet( - knex, - entities[key], - onResults - ) as DbSet; + sets[key] = makeDbSet(knex, entities[key], onResults); } const ctx: TrackedDbContext = { @@ -192,7 +190,7 @@ function buildTrackedContext( knex, withTransaction(trx: Knex.Transaction): DbContext { - return buildTrackedContext( + return buildTrackedContext( trx as unknown as Knex, entities, tracker @@ -204,7 +202,7 @@ function buildTrackedContext( ): Promise { return knex.transaction(async (trx: Knex.Transaction) => { return callback( - buildTrackedContext( + buildTrackedContext( trx as unknown as Knex, entities, tracker @@ -325,7 +323,7 @@ export function createDb( }; tracker.registerEntitySet(cfg); } - return buildTrackedContext(knex, entities, tracker); + return buildTrackedContext(knex, entities, tracker); } - return buildContext(knex, entities); + return buildContext(knex, entities); } diff --git a/libs/orm/src/query-definition.test-d.ts b/libs/orm/src/query-definition.test-d.ts new file mode 100644 index 00000000..2552523e --- /dev/null +++ b/libs/orm/src/query-definition.test-d.ts @@ -0,0 +1,21 @@ +import type { Knex } from 'knex'; +import { expectTypeOf, test } from 'vitest'; +import { number, object, parameter, query, string } from './index.js'; + +const User = object({ id: number().primaryKey(), name: string() }).hasTableName( + 'users' +); +const findUser = query(User).where(t => t.id, parameter('id')); + +test('ORM re-exports connection-independent definitions with strongly typed bound readers', () => { + expectTypeOf(findUser).parameters.toEqualTypeOf<[Knex, number]>(); + expectTypeOf(findUser).returns.toEqualTypeOf< + Promise<{ id: number; name: string }[]> + >(); + const knex = null as unknown as Knex; + expectTypeOf(findUser.query(knex, 1).first()).toEqualTypeOf< + Promise<{ id: number; name: string } | undefined> + >(); + // @ts-expect-error standalone definitions do not acquire DbContext lookup/tracking helpers + findUser.query(knex, 1).find(1); +}); diff --git a/libs/orm/src/result-types.ts b/libs/orm/src/result-types.ts index da902ef1..8d07062f 100644 --- a/libs/orm/src/result-types.ts +++ b/libs/orm/src/result-types.ts @@ -8,9 +8,13 @@ import type { Entity, EntityRelations, EntitySchema, + ObjectReadSchema, + PolymorphicRowSchema, PrimaryKeyOf, + ReadRelations, + ReadVariantMetadata, RelationInfo, - SchemaAwareQuery + VariantReadSchemas } from '@cleverbrush/knex-schema'; import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; @@ -31,7 +35,17 @@ import type { InferType, ObjectSchemaBuilder } from '@cleverbrush/schema'; * @public */ export type EntityResult> = InferType< - SchemaAwareQuery>['rowSchema'] + // Derive metadata directly, without recursively instantiating a query's + // complete fluent API while constructing a generic DbContext. + ReadVariantMetadata> extends { + discriminator: string; + variants: Record; + } + ? PolymorphicRowSchema>> + : ObjectReadSchema< + EntitySchema, + keyof ReadRelations> + > >; /** diff --git a/websites/docs/app/knex-schema/page.tsx b/websites/docs/app/knex-schema/page.tsx index c221763d..95719182 100644 --- a/websites/docs/app/knex-schema/page.tsx +++ b/websites/docs/app/knex-schema/page.tsx @@ -413,6 +413,59 @@ const rows = await read.where(t => t.id, taskId);`) +

+

Define now, supply the connection at execution

+

+ Keep reusable queries in their own modules. Their row + schemas and argument types are available without Knex; + handlers supply an injected connection or transaction. +

+
+                         user.id, parameter('id'));
+export const names = query(UserSchema).select('id', 'name');
+
+// api/handlers/get-user.ts
+import type { Knex } from 'knex';
+import { findUser } from '../../data/user-queries.js';
+
+export async function getUser(knex: Knex, id: number) {
+    return findUser.query(knex, id).first();
+}
+
+// Direct SELECT or SQL inspection:
+await findUser(knex, 10);
+await names(knex); // parameterless definition
+findUser.toSQL(knex, 10);`)
+                            }}
+                        />
+                    
+

+ Definitions are immutable and non-thenable. Selectors, + scopes and relation/variant customizers run once during + configuration. Compiled PostgreSQL SELECTs are cached + per definition and actual Knex instance, never across + unrelated clients; each call executes with fresh + bindings. +

+

+ Bind with .query(knex, ...values) before + pagination, native SQL escape hatches or supported + writes. The caller owns transaction lifetime. Existing + connection-first queries and ORM DbSets remain + supported. +

+ + Multi-file examples, transactions and cache boundaries + +
+

Parameterized compiled queries

diff --git a/websites/docs/app/orm/page.tsx b/websites/docs/app/orm/page.tsx index a8eb1fca..b3d41bdb 100644 --- a/websites/docs/app/orm/page.tsx +++ b/websites/docs/app/orm/page.tsx @@ -539,6 +539,19 @@ npx cb-orm db push` {/* ── See Also ─────────────────────────────────────── */}

Reliable query composition

+

+ For module-level queries, import query and{' '} + parameter from{' '} + @cleverbrush/orm, define{' '} + query(UserSchema), and supply{' '} + db.knex when calling it. These reads are + detached; use DbSets when identity tracking and ORM + lookup helpers are needed. See{' '} + + connection-independent query definitions + + . +

Named parameters make DbSet reads callable: use{' '}