Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion PARITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ core, so all three SDKs change at once. **No drift.**
|----------------------------|------------------------------------------------|------------------------------------------------------|---------------------------------------------------------|
| Constructor | `DirSQL(root=None, *, tables=None, ignore=None, no_ignore=False, config=None, persist=False, persist_path=None, extensions=None)` | `DirSQL::builder().root(..).tables(..).ignore(..).config(..).persist(Option<path>).extensions(..).build()` (also `DirSQL::new`/`with_ignore` shortcuts) | `new DirSQL(configPath)` or `new DirSQL({ root?, tables?, ignore?, config?, persist?, persistPath?, extensions?, noIgnore? })` + `await db.ready` |
| Query (read-only; rejects non-SELECT) | `db.query(sql) -> list[dict]` | `db.query(sql) -> Result<Vec<Row>>` | `await db.query(sql) -> Record[]` (runs on libuv threadpool) |
| Query carrying the projection order | — (drift, #1173) | `db.query_ordered(sql) -> Result<QueryResult>` (`columns` + `rows`) | — (drift, #1174) |
| Query carrying the projection order | — (drift, #1173) | `db.query_ordered(sql) -> Result<QueryResult>` (`columns` + `rows`) | `await db.queryOrdered(sql) -> QueryResult` (`columns` + `rows`) |
| Start watcher | `db._start_watcher()` | `db.start_watching()` | `await db.startWatcher()` (runs on libuv threadpool) |
| Poll events | `db._poll_events(ms)` | `db.poll_events(duration)` | `await db.pollEvents(ms)` (runs on libuv threadpool) |
| Watch (channel/stream) | `async for event in db.watch()` (via `_async.py`) | `db.watch() -> WatchStream` (channel) | `for await (const ev of db.watch())` |
Expand Down Expand Up @@ -504,6 +504,7 @@ incl. #313).
| `ddl` batch SQLite rejects: rolled back, error prefixed with the entry (#956) | core | Y (`ddl_batch.rs`, `sdk.rs`) | core |
| Declared `name` resolving to a virtual table errors (#956) | core | Y (`ddl_batch.rs`) | core |
| Query rejects writes | Y | Y | Y |
| Query reports the projection order | N (#1173) | Y (`column_order.rs`) | Y (`query-ordered.test.ts`) |
| Write-rejection edge matrix (leading comments, mixed case, CTE/whitespace allowed) | core | Y (`readonly_query.rs`) | core |
| Internal `_dirsql_*` columns hidden from `SELECT *` | Y | Y | Y |
| `_dirsql_*` filter robustness (comment/string-literal bypass) | core | Y (`code_review_findings.rs`) | core |
Expand Down
19 changes: 19 additions & 0 deletions docs/reference/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,25 @@ than `Number.MAX_SAFE_INTEGER`, 2^53 − 1) cannot be represented as a JS
than return a rounded value. Python's `int` is unbounded, so it always
round-trips.

### `queryOrdered`

::: code-group

```typescript [TypeScript]
await db.queryOrdered(sql: string) -> { columns: string[]; rows: Record<string, unknown>[] }
```

```rust [Rust]
db.query_ordered(sql: &str) -> Result<QueryResult> // QueryResult { columns, rows }
```

:::

Runs `query` and also returns the column names in the order the SELECT list
names them. Use it to render columns in the order the caller asked for: a
row's keys do not carry that order. In TypeScript, object keys that look like
integers always enumerate first.

### `watch`

::: code-group
Expand Down
6 changes: 6 additions & 0 deletions packages/ts/changelog.d/2026-09-24-query-ordered.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
**Added**

- `db.queryOrdered(sql)` resolves to `{ columns, rows }`: the rows `query`
returns, plus the column names in the order the SELECT list names them. A
row object cannot carry that order, since integer-like keys always enumerate
first. The `QueryResult` type is exported from the package root. (#1174)
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"command": "pnpm test:e2e",
"ran_at": 1790250286,
"exit_code": 0,
"commit": "42282d85744861cc8ffbebb66dbb97bcfe31988e",
"branch": "claude/1174-ts-projection-order"
}
26 changes: 26 additions & 0 deletions packages/ts/migrations.d/2026-09-24-query-ordered.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
### `queryOrdered` reports the projection order

**Summary**

New `db.queryOrdered(sql)` resolves to `{ columns, rows }`, where `columns`
lists the result's column names in the order the SELECT list names them.
Additive: `query` is unchanged.

**Required changes**

_None._

**Deprecations removed**

_None._

**Behavior changes without code changes**

_None._

**Verification**

```ts
const { columns } = await db.queryOrdered('SELECT name, 1 AS "2" FROM users');
// columns: ["name", "2"] -- a row's keys enumerate "2" first
```
81 changes: 79 additions & 2 deletions packages/ts/napi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@

use dirsql::extension_resolution::{self, ConfigSource as CoreConfigSource, PlanEntry};
use dirsql::{
DirSQL as CoreDirSQL, Extension, PreparedBuild, RawFileEvent, Row, RowEvent as CoreRowEvent,
Table, Value, flatten_row_event,
DirSQL as CoreDirSQL, Extension, PreparedBuild, QueryResult as CoreQueryResult, RawFileEvent,
Row, RowEvent as CoreRowEvent, Table, Value, flatten_row_event,
};
use napi::Task;
use napi::bindgen_prelude::*;
Expand Down Expand Up @@ -327,6 +327,13 @@ impl DirSQL {
AsyncTask::new(QueryTask { inner, sql })
}

/// [`query`](Self::query), plus the column order the SELECT list names.
#[napi(js_name = "queryOrdered", ts_return_type = "Promise<QueryResult>")]
pub fn query_ordered(&self, sql: String) -> AsyncTask<QueryOrderedTask> {
let inner = self.inner.borrow().as_ref().cloned();
AsyncTask::new(QueryOrderedTask { inner, sql })
}

/// Start the file watcher. Must be called before pollEvents.
///
/// Runs on the libuv threadpool so the JS event loop stays responsive
Expand Down Expand Up @@ -469,6 +476,53 @@ impl Task for QueryTask {
}
}

/// One query's rows plus the column order the SELECT list names. A row is a
/// JS object, which cannot carry that order: integer-like keys enumerate
/// first, in ascending order, whatever order they were inserted in.
///
/// Output-only (`object_from_js = false`): JS never constructs one.
#[napi(object, object_from_js = false)]
pub struct QueryResult {
pub columns: Vec<String>,
#[napi(ts_type = "Record<string, unknown>[]")]
pub rows: Vec<HashMap<String, JsRowValue>>,
}

fn query_result_to_js(result: &CoreQueryResult) -> Result<QueryResult> {
Ok(QueryResult {
columns: result.columns.clone(),
rows: result
.rows
.iter()
.map(value_row_to_js)
.collect::<Result<_>>()?,
})
}

/// [`QueryTask`], for `DirSQL::query_ordered`.
pub struct QueryOrderedTask {
inner: Option<CoreDirSQL>,
sql: String,
}

impl Task for QueryOrderedTask {
type Output = QueryResult;
type JsValue = QueryResult;

fn compute(&mut self) -> Result<Self::Output> {
let inner = self
.inner
.as_ref()
.ok_or_else(|| Error::new(Status::GenericFailure, "DirSQL instance closed"))?;
let result = inner.query_ordered(&self.sql).map_err(to_napi_err)?;
query_result_to_js(&result)
}

fn resolve(&mut self, _env: Env, output: Self::Output) -> Result<Self::JsValue> {
Ok(output)
}
}

/// Runs `DirSQL::start_watching` on the libuv threadpool. Idempotent on the
/// core side, so repeated calls from JS are still safe.
pub struct StartWatcherTask {
Expand Down Expand Up @@ -1027,6 +1081,29 @@ mod tests {
assert!(value_row_to_js(&row).is_err());
}

#[test]
fn query_result_to_js_keeps_the_column_order_and_rows() {
let js = query_result_to_js(&CoreQueryResult {
columns: vec!["z".into(), "k".into()],
rows: vec![one_row()],
})
.unwrap();
assert_eq!(js.columns, vec!["z".to_string(), "k".to_string()]);
assert!(matches!(js.rows[0].get("k"), Some(JsRowValue::Integer(7))));
}

#[test]
fn query_result_to_js_propagates_unsafe_integer() {
let row = HashMap::from([("k".to_string(), Value::Integer(JS_MAX_SAFE_INTEGER + 1))]);
assert!(
query_result_to_js(&CoreQueryResult {
columns: vec!["k".into()],
rows: vec![row],
})
.is_err()
);
}

#[test]
fn row_event_to_js_insert() {
let ev = CoreRowEvent::Insert {
Expand Down
8 changes: 8 additions & 0 deletions packages/ts/src/core.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,14 @@ describe("NativeDirSQL interface", () => {
expect(result).not.toBeInstanceOf(Promise);
});

it("includes queryOrdered() resolving to columns plus rows", async () => {
const result = { columns: ["b", "a"], rows: [{ a: 1, b: 2 }] };
const queryOrdered: NativeDirSQL["queryOrdered"] = vi
.fn()
.mockResolvedValue(result);
expect(await queryOrdered("SELECT b, a")).toEqual(result);
});

it("includes close() method for cleanup (#598)", () => {
const instance = {
query: vi.fn().mockResolvedValue([]),
Expand Down
8 changes: 7 additions & 1 deletion packages/ts/src/core.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
// Lazy access to the napi-rs core module.

import type { ExtensionSpec, RowEvent, ScanFailure } from "./dirsql.js";
import type {
ExtensionSpec,
QueryResult,
RowEvent,
ScanFailure,
} from "./dirsql.js";
import { loadNativeCore as defaultLoadNativeCore } from "./load-native-core.js";
import type { TableDef } from "./table.js";

// Shape of the napi-rs-exposed class. The `DirSQL` wrapper drives this.
export interface NativeDirSQL {
query(sql: string): Promise<Record<string, unknown>[]>;
queryOrdered(sql: string): Promise<QueryResult>;
startWatcher(): Promise<void>;
pollEvents(timeoutMs: number): Promise<RowEvent[]>;
// Synchronous: reads a list the scan already produced, no threadpool hop.
Expand Down
10 changes: 10 additions & 0 deletions packages/ts/src/dirsql.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ vi.mock("./resolve-config-extensions.js", async () => ({

type FakeInner = {
query: ReturnType<typeof vi.fn>;
queryOrdered: ReturnType<typeof vi.fn>;
startWatcher: ReturnType<typeof vi.fn>;
pollEvents: ReturnType<typeof vi.fn>;
scanFailures: ReturnType<typeof vi.fn>;
Expand All @@ -37,6 +38,7 @@ function installFakeCore(inner: FakeInner) {
function makeInner(overrides: Partial<FakeInner> = {}): FakeInner {
return {
query: vi.fn().mockResolvedValue([]),
queryOrdered: vi.fn().mockResolvedValue({ columns: [], rows: [] }),
startWatcher: vi.fn().mockResolvedValue(undefined),
pollEvents: vi.fn().mockResolvedValue([]),
scanFailures: vi.fn(() => []),
Expand Down Expand Up @@ -317,6 +319,14 @@ describe("DirSQL", () => {
expect(inner.query).toHaveBeenCalledWith("SELECT 1");
});

it("queryOrdered awaits ready then forwards to the inner instance", async () => {
const result = { columns: ["b", "a"], rows: [{ a: 1, b: 2 }] };
inner.queryOrdered.mockResolvedValue(result);
const db = new DirSQL({ root: "/d" });
expect(await db.queryOrdered("SELECT b, a")).toEqual(result);
expect(inner.queryOrdered).toHaveBeenCalledWith("SELECT b, a");
});

it("scanFailures awaits ready then forwards to the inner instance", async () => {
const failures = [{ path: "bad.json", message: "boom" }];
const withFailures = makeInner({ scanFailures: vi.fn(() => failures) });
Expand Down
25 changes: 25 additions & 0 deletions packages/ts/src/dirsql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,17 @@ export interface RowEvent {
filePath?: string | null;
}

/** The result of {@link DirSQL.queryOrdered}. */
export interface QueryResult {
/**
* Column names in the order the SELECT list names them. Iterate this, not
* a row's keys: object keys that look like integers enumerate first.
*/
columns: string[];
/** The rows, exactly as {@link DirSQL.query} returns them. */
rows: Record<string, unknown>[];
}

/**
* One file the initial scan could not index.
*
Expand Down Expand Up @@ -223,6 +234,20 @@ export class DirSQL {
return this._inner.query(sql);
}

/**
* {@link query}, plus the column order the SELECT list names. Use it to
* render columns in the order the caller asked for.
*
* ```ts
* const { columns, rows } = await db.queryOrdered("SELECT path, size FROM './'");
* // columns: ["path", "size"]
* ```
*/
async queryOrdered(sql: string): Promise<QueryResult> {
await this.ready;
return this._inner.queryOrdered(sql);
}

/**
* The files the initial scan could not index, each with its root-relative
* `path` and the hook's own `message`.
Expand Down
8 changes: 7 additions & 1 deletion packages/ts/src/index.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import * as api from "./index.js";
import type { ScanFailure } from "./index.js";
import type { QueryResult, ScanFailure } from "./index.js";

describe("public barrel", () => {
it("re-exports the runtime values", () => {
Expand All @@ -17,6 +17,12 @@ describe("public barrel", () => {
expect(failure.message).toBe("boom");
});

it("re-exports the QueryResult type with its documented shape", () => {
const result: QueryResult = { columns: ["a"], rows: [{ a: 1 }] };
expect(result.columns).toEqual(["a"]);
expect(result.rows).toEqual([{ a: 1 }]);
});

it("exposes exactly the public runtime exports", () => {
expect(Object.keys(api).sort()).toEqual(["DirSQL", "Table"].sort());
});
Expand Down
1 change: 1 addition & 0 deletions packages/ts/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ export { DirSQL } from "./dirsql.js";
export type {
DirSQLOptions,
ExtensionSpec,
QueryResult,
RowEvent,
ScanFailure,
} from "./dirsql.js";
Expand Down
59 changes: 59 additions & 0 deletions packages/ts/tests/integration/binding/query-ordered.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import { readFileSync } from "node:fs";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { DirSQL } from "dirsql";
import { afterEach, beforeEach, describe, expect, it } from "vitest";

describe("queryOrdered", () => {
let dir: string;
let db: DirSQL;

beforeEach(async () => {
dir = await mkdtemp(join(tmpdir(), "dirsql-query-ordered-"));
await writeFile(
join(dir, "users.json"),
JSON.stringify([
{ name: "Alice", age: 30 },
{ name: "Bob", age: 25 },
]),
);
db = new DirSQL({
root: dir,
tables: [
{
name: "users",
ddl: "CREATE TABLE users (age INTEGER, name TEXT)",
glob: "users.json",
onFile: (filePath: string) =>
JSON.parse(readFileSync(filePath, "utf8")),
},
],
});
});

afterEach(async () => {
await rm(dir, { recursive: true, force: true });
});

it("reports the columns in the order the SELECT list names them", async () => {
const result = await db.queryOrdered(
"SELECT name, age, 1 AS a FROM users ORDER BY name",
);
expect(result.columns).toEqual(["name", "age", "a"]);
});

it("carries the same rows query returns", async () => {
const sql = "SELECT name, age FROM users ORDER BY name";
const result = await db.queryOrdered(sql);
expect(result.rows).toEqual(await db.query(sql));
expect(result.rows[0]).toEqual({ name: "Alice", age: 30 });
});

it("reports the columns of a query that matches no rows", async () => {
const result = await db.queryOrdered(
"SELECT name, age FROM users WHERE age > 100",
);
expect(result).toEqual({ columns: ["name", "age"], rows: [] });
});
});
Loading
Loading