From 02263bb932c20cf8255455f01d853c4df906e58a Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Fri, 17 Jul 2026 16:05:15 +1200 Subject: [PATCH 1/5] =?UTF-8?q?agents:=20sandbox=20guardrails=20=E2=80=94?= =?UTF-8?q?=20engine=20caps,=20typed=20resource=20errors,=20network=20watc?= =?UTF-8?q?hdog=20(contract=200.3.0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TypeScript mirror of chdb feat/agents-gate-g3 (vendored descriptors.json / cases.jsonl / CONTRACT.md byte-identical, hint strings byte-identical): - Engine-side row bound (max_result_rows=cap+1, break); optional maxMemoryUsage / maxResultBytes; per-call maxRows clamped. - Resource errors (158/159/241/307/396) -> ChDBResourceError with a model-facing hint; ACCESS_DENIED renamed ALLOWLIST_DENIED. - networkTimeout (default 60s): fast-fail engine baseline + Promise.race deadline on url()/s3()/remote()/... queries; expiry throws NETWORK_TIMEOUT with a hint, poisons the tool, parks the session (in-flight native call can't be cancelled). vitest 466 passed / 11 skipped against the locally built addon. --- integrations/agents/CONTRACT.md | 54 ++++++- integrations/agents/conformance/cases.jsonl | 16 +- integrations/agents/descriptors.d.mts | 2 + integrations/agents/descriptors.json | 2 +- integrations/agents/descriptors.mjs | 8 +- integrations/agents/errors.d.mts | 16 +- integrations/agents/errors.mjs | 49 +++++- integrations/agents/index.d.mts | 1 + integrations/agents/index.mjs | 1 + integrations/agents/safety.d.mts | 2 + integrations/agents/safety.mjs | 32 ++++ integrations/agents/tool.d.mts | 17 +++ integrations/agents/tool.mjs | 136 +++++++++++++++-- .../integrations/agents-conformance.test.ts | 11 ++ test/v3/integrations/agents-tool.test.ts | 142 +++++++++++++++++- 15 files changed, 460 insertions(+), 29 deletions(-) diff --git a/integrations/agents/CONTRACT.md b/integrations/agents/CONTRACT.md index 6e47c19..df33995 100644 --- a/integrations/agents/CONTRACT.md +++ b/integrations/agents/CONTRACT.md @@ -11,7 +11,7 @@ > change in a minor release while it stabilizes across bindings; pin a version if > you depend on it. > -> **Contract version: `0.2.0`** — exposed as `CONTRACT_VERSION` and via +> **Contract version: `0.3.0`** — exposed as `CONTRACT_VERSION` and via > `capabilities()` in every binding. Downstream consumers probe capabilities > instead of guessing from a package version (see *Versioning & capabilities*). @@ -125,6 +125,19 @@ to TS, which has no in-process pandas). Bindings may omit it. - Every result is capped by `max_rows` (and a `max_bytes` guard). When the engine produced more than the cap, the result sets `truncated = true`. Truncation is **never silent**. +- The row cap is **also enforced engine-side** (0.3.0): the session sets + `max_result_rows = max_rows + 1`, `result_overflow_mode='break'`, + `max_block_size=8192` — an unbounded SELECT stops at block granularity + instead of materializing everything before the trim, and the `+1` keeps the + `truncated` flag exact. A per-call `max_rows` above the constructor cap is + clamped to it (the engine bound is fixed at construction). +- The engine bound guards mistakes, not adversaries: a `SETTINGS` clause can + override engine limits under readonly=2 (the client-side trim still applies). + Engine quirk on affected builds (fixed upstream, unreleased): a query-level + `SETTINGS` clause persists for the whole session — cases using one run on a + dedicated tool. +- Truncated envelope results carry a `hint` (narrow, don't re-run unchanged); + wording binding-identical. - `max_bytes` counts **UTF-8 bytes of each row's compact JSON encoding** (no spaces, non-ASCII kept raw). This is normative because it is model-visible: counting UTF-16 units or ASCII-escaped characters instead moves the truncation @@ -142,22 +155,51 @@ to TS, which has no in-process pandas). Bindings may omit it. (`UNKNOWN_TOOL` / `INVALID_ARGUMENT`), never as a raised exception. - Direct library methods raise a typed error (`ChDBError` and subclasses). - Error classification is shared: parse `Code: N. DB::Exception: . (TYPE)`; - map by code (`164→READONLY`, `62→SYNTAX`, `46/47/60/81/115→UNKNOWN_*`). + map by code (`164→READONLY`, `62→SYNTAX`, `46/47/60/81/115→UNKNOWN_*`, + `158/159/241/307/396→RESOURCE` — rows / timeout / memory / bytes / result-size limits). +- **The resource family (0.3.0)** gets its own error class + (`ChDBResourceError` or equivalent) and a `hint` field: the SQL was valid, + narrow it — don't abandon, don't retry unchanged. Hint wording + binding-identical. - Binding-side validation failures (no engine round-trip) use `code: 0` with a shared `type`: `INVALID_ARGUMENT` (bad argument value, e.g. a non-numeric - cap or an unknown `tool_specs` dialect), `ACCESS_DENIED` (allowlist), + cap or an unknown `tool_specs` dialect), `ALLOWLIST_DENIED` (allowlist; + **renamed from `ACCESS_DENIED` in 0.3.0** — it collided with engine error + 497 of the same name), `CONFIG_MISMATCH` (a caller-provided session whose readonly state conflicts - with the declared mode), `UNKNOWN_TOOL` (bad `call()` name), `TOOL_ERROR` - (any other non-engine failure surfaced through the envelope). + with the declared mode), `NETWORK_TIMEOUT` (the network watchdog abandoned a + query referencing a remote source — carries a hint; see P5), `UNKNOWN_TOOL` + (bad `call()` name), `TOOL_ERROR` (any other non-engine failure surfaced + through the envelope, including any query attempted after a watchdog + abandonment poisoned the tool). ### P5 — Resource and source controls (normative, optional-per-deployment) - **Query timeout** — an optional `max_execution_time` (seconds) bounds runaway queries at the engine (`TIMEOUT_EXCEEDED`). Off by default; set per deployment. +- **Memory bound (0.3.0)** — optional `max_memory_usage` (bytes); exceeding + raises `MEMORY_LIMIT_EXCEEDED`. Off by default. +- **Network watchdog (0.3.0)** — `network_timeout` (seconds, default 60, + None/0 disables) applies to queries referencing a network-source table + function (the shared `NETWORK_TABLE_FUNCTIONS` set; `file()` and local lake + variants exempt). Two layers: at construction the session gets a fast-fail + baseline (`http_connection_timeout=min(nt,10)`, receive/send = nt, + `http_max_tries=1`, `http_make_head_request=0`); at query time the engine + call runs under a `network_timeout` deadline and expiry raises/envelopes + `NETWORK_TIMEOUT` with a hint. The watchdog is load-bearing, not cosmetic: + a black-holed HTTPS endpoint hangs the engine call unboundedly on affected + builds (blocked TLS handshake ignores every engine-side timeout), so there + is no error to translate unless the binding imposes one. The abandoned call + cannot be cancelled: afterwards the tool is **poisoned** (further queries → + `TOOL_ERROR`, `close()` leaves the session alone, the session leaks for + process lifetime). Bindings document this; they don't hide it. +- **Result-bytes backstop (0.3.0)** — optional `max_result_bytes`; break mode + truncates **without a flag**, so set it well above `max_bytes` and treat it + as an OOM backstop only. Off by default. - **File allowlist** — an optional list of path prefixes. When set, raw SQL is scanned for **every table function the running engine exposes** (live `system.table_functions` snapshot, unioned with a static fallback) that is not in the shared safe-by-construction set: each such call must carry a - **literal** source argument inside the allowlist, else `ACCESS_DENIED`. The + **literal** source argument inside the allowlist, else `ALLOWLIST_DENIED`. The scan runs over masked SQL — string literals and comments blanked position-preserving, backtick/double-quote-wrapped function names matched — so a path-looking string never false-positives and quoting, comments, or a diff --git a/integrations/agents/conformance/cases.jsonl b/integrations/agents/conformance/cases.jsonl index d166956..9b1554b 100644 --- a/integrations/agents/conformance/cases.jsonl +++ b/integrations/agents/conformance/cases.jsonl @@ -1,4 +1,4 @@ -{"fixture": "chdb-agents-conformance", "contract_version": "0.2.0"} +{"fixture": "chdb-agents-conformance", "contract_version": "0.3.0"} {"id": "p1_create_blocked", "pillar": "P1", "desc": "read-only blocks DDL at the engine", "method": "query", "args": {"sql": "CREATE TABLE conformance_t (a Int32) ENGINE = Memory"}, "expect": {"error_type": "READONLY"}} {"id": "p1_escape_blocked", "pillar": "P1", "desc": "read-only cannot be lowered (no escape)", "method": "query", "args": {"sql": "SET readonly = 0"}, "expect": {"error_type": "READONLY"}} {"id": "p1_file_allowed", "pillar": "P1", "desc": "CANARY: readonly=2 still allows file() (readonly=1 would reject)", "method": "query", "args": {"sql": "SELECT toInt32(count()) AS c FROM file('{{fixtures}}/sample.csv')"}, "expect": {"rows": [{"c": 2}]}} @@ -13,7 +13,7 @@ {"id": "introspect_sample_limit", "pillar": "introspection", "desc": "get_sample_data honors limit", "method": "get_sample_data", "args": {"target": "numbers(100)", "limit": 2}, "expect": {"row_count": 2}} {"id": "introspect_list_functions", "pillar": "introspection", "desc": "list_functions returns names", "method": "list_functions", "args": {"limit": 50}, "expect": {"min_len": 1}} {"id": "safety_timeout", "pillar": "safety", "desc": "max_execution_time bounds a runaway query", "tool": {"max_execution_time": 1}, "method": "query", "args": {"sql": "SELECT count() FROM numbers(100000000000)"}, "expect": {"error_type": "TIMEOUT_EXCEEDED"}} -{"id": "safety_allowlist", "pillar": "safety", "desc": "file_allowlist blocks file() paths outside the allowlist", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ACCESS_DENIED"}} +{"id": "safety_allowlist", "pillar": "safety", "desc": "file_allowlist blocks file() paths outside the allowlist", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ALLOWLIST_DENIED"}} {"id": "catalog_attachments", "pillar": "catalog", "desc": "a read-only tool can query files attached at construction", "tool": {"read_only": true, "attachments": {"rep": "{{fixtures}}/sample.csv"}}, "method": "query", "args": {"sql": "SELECT toInt32(count()) AS c FROM rep"}, "expect": {"rows": [{"c": 2}]}} {"id": "p3_maxbytes_utf8", "pillar": "P3", "desc": "max_bytes counts UTF-8 bytes of compact JSON (not UTF-16 units or ASCII-escaped chars)", "tool": {"max_bytes": 60}, "method": "query", "args": {"sql": "SELECT '汉字汉字汉字' AS s FROM numbers(5)"}, "expect": {"truncated": true, "row_count": 2}} {"id": "p3_maxrows_non_numeric", "pillar": "P3", "desc": "a non-numeric per-call max_rows is a typed error, never a silently disabled cap", "method": "query", "args": {"sql": "SELECT 1", "max_rows": "lots"}, "expect": {"error_type": "INVALID_ARGUMENT"}} @@ -21,8 +21,14 @@ {"id": "p2_empty_database_rejected", "pillar": "P2", "desc": "an explicit empty-string database is rejected, not silently treated as unqualified", "method": "call", "args": {"name": "describe_table", "arguments": {"target": "tables", "database": ""}}, "expect": {"envelope_ok": false, "error_type": "TOOL_ERROR"}} {"id": "optional_dataframe_query", "pillar": "optional", "desc": "co-located DataFrame query via Python() (capability-gated: dataframe_query)", "requires": "dataframe_query", "method": "dataframe_query", "args": {"sql": "SELECT toInt32(sum(a)) AS s FROM Python(t)", "dataframes": {"t": {"a": [1, 2, 3]}}}, "expect": {"rows": [{"s": 6}]}} {"id": "p4_malformed_arguments", "pillar": "P4", "desc": "a non-object call() arguments payload returns a typed envelope, not a throw", "method": "call", "args": {"name": "run_select_query", "arguments": "SELECT 1"}, "expect": {"envelope_ok": false, "error_type": "INVALID_ARGUMENT"}} -{"id": "p1_scanner_backtick_bypass", "pillar": "P1", "desc": "a backtick-quoted function name cannot bypass the allowlist gate", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM `file`('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ACCESS_DENIED"}} -{"id": "p1_scanner_comment_between", "pillar": "P1", "desc": "a comment between function name and paren cannot hide the call from the gate", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file/**/('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ACCESS_DENIED"}} -{"id": "p1_scanner_non_literal_arg", "pillar": "P1", "desc": "a computed (non-literal) source argument is denied under an allowlist, not skipped", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file(concat('{{fixtures}}','/sample.csv'))"}, "expect": {"error_type": "ACCESS_DENIED"}} +{"id": "p1_scanner_backtick_bypass", "pillar": "P1", "desc": "a backtick-quoted function name cannot bypass the allowlist gate", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM `file`('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ALLOWLIST_DENIED"}} +{"id": "p1_scanner_comment_between", "pillar": "P1", "desc": "a comment between function name and paren cannot hide the call from the gate", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file/**/('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ALLOWLIST_DENIED"}} +{"id": "p1_scanner_non_literal_arg", "pillar": "P1", "desc": "a computed (non-literal) source argument is denied under an allowlist, not skipped", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file(concat('{{fixtures}}','/sample.csv'))"}, "expect": {"error_type": "ALLOWLIST_DENIED"}} {"id": "p1_scanner_string_literal_inert", "pillar": "P1", "desc": "a path-looking string literal does not false-positive the gate", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT 'file(''/etc/passwd'')' AS s"}, "expect": {"rows": [{"s": "file('/etc/passwd')"}]}} {"id": "p1_scanner_allowlisted_ok", "pillar": "P1", "desc": "an allowlisted literal path still works under the gate", "tool": {"file_allowlist": ["{{fixtures}}"]}, "method": "query", "args": {"sql": "SELECT toInt32(count()) AS c FROM file('{{fixtures}}/sample.csv')"}, "expect": {"rows": [{"c": 2}]}} +{"id": "p3_engine_row_bound", "pillar": "P3", "desc": "an unbounded over-cap SELECT is bounded engine-side (max_result_rows=cap+1, break) and still flagged", "method": "query", "args": {"sql": "SELECT toInt32(number) AS n FROM numbers(100000)"}, "expect": {"truncated": true, "row_count": 1000}} +{"id": "p3_percall_cap_clamped", "pillar": "P3", "desc": "a per-call max_rows above the constructor cap is clamped (the engine bound was fixed at construction; honoring more would truncate silently)", "tool": {"max_rows": 5}, "method": "query", "args": {"sql": "SELECT toInt32(number) AS n FROM numbers(10)", "max_rows": 50}, "expect": {"truncated": true, "row_count": 5}} +{"id": "p5_resource_error_typed", "pillar": "P5", "desc": "an engine resource limit surfaces as its engine type (resource family: 158/159/241/307/396); own tool: a SETTINGS clause persists on a chdb session (upstream quirk)", "tool": {}, "method": "query", "args": {"sql": "SELECT number FROM numbers(100) SETTINGS max_result_rows = 10, result_overflow_mode = 'throw'"}, "expect": {"error_type": "TOO_MANY_ROWS_OR_BYTES"}} +{"id": "p4_truncation_hint", "pillar": "P4", "desc": "a truncated envelope result carries a recovery hint for the model", "method": "call", "args": {"name": "run_select_query", "arguments": {"sql": "SELECT toInt32(number) AS n FROM numbers(2000)"}}, "expect": {"envelope_ok": true, "result_has_hint": true}} +{"id": "p4_resource_hint", "pillar": "P4", "desc": "a resource-limit error envelope carries a recovery hint (narrow, don't abandon or retry unchanged); own tool: a SETTINGS clause persists on a chdb session (upstream quirk)", "tool": {}, "method": "call", "args": {"name": "run_select_query", "arguments": {"sql": "SELECT number FROM numbers(100) SETTINGS max_result_rows = 10, result_overflow_mode = 'throw'"}}, "expect": {"envelope_ok": false, "error_type": "TOO_MANY_ROWS_OR_BYTES", "error_has_hint": true}} +{"id": "p5_network_baseline_settings", "pillar": "P5", "desc": "network_timeout injects the engine-side fast-fail network baseline (tries=1, no HEAD probe, small connect timeout)", "tool": {"network_timeout": 30}, "method": "query", "args": {"sql": "SELECT toInt32(getSetting('http_max_tries')) AS tries, toInt32(getSetting('http_make_head_request')) AS head, toInt32(getSetting('http_connection_timeout')) AS conn"}, "expect": {"rows": [{"tries": 1, "head": 0, "conn": 10}]}} diff --git a/integrations/agents/descriptors.d.mts b/integrations/agents/descriptors.d.mts index 5f0ea6d..b816fc1 100644 --- a/integrations/agents/descriptors.d.mts +++ b/integrations/agents/descriptors.d.mts @@ -29,6 +29,8 @@ export interface Capabilities { attachments: boolean file_allowlist: boolean max_execution_time: boolean + resource_caps: boolean + network_watchdog: boolean async: boolean streaming: boolean [feature: string]: boolean diff --git a/integrations/agents/descriptors.json b/integrations/agents/descriptors.json index 82bccb4..bca2238 100644 --- a/integrations/agents/descriptors.json +++ b/integrations/agents/descriptors.json @@ -1,6 +1,6 @@ { "$comment": "Single source of truth for the model-visible agent-tool surface (names, descriptions, argument schemas). Governed by CONTRACT.md; a copy of this file is vendored byte-identical in every chdb-io binding (chdb-node integrations/agents/descriptors.json). Bindings GENERATE their tool_specs()/framework schemas from this file — hand-editing a generated schema instead of this file is a contract violation. Changing this file is a contract change: bump contract_version here, in the code constant, and in conformance/cases.jsonl's header.", - "contract_version": "0.2.0", + "contract_version": "0.3.0", "tools": [ { "name": "run_select_query", diff --git a/integrations/agents/descriptors.mjs b/integrations/agents/descriptors.mjs index f856cc1..8969735 100644 --- a/integrations/agents/descriptors.mjs +++ b/integrations/agents/descriptors.mjs @@ -20,7 +20,7 @@ import { ChDBError } from './errors.mjs' // The agent-tool contract version (semver). Bumped whenever descriptors.json, // conformance/cases.jsonl, or normative CONTRACT.md text changes. Tests assert // it equals the contract_version field of both data files. -export const CONTRACT_VERSION = '0.2.0' +export const CONTRACT_VERSION = '0.3.0' const DESCRIPTORS_PATH = join(dirname(fileURLToPath(import.meta.url)), 'descriptors.json') let cache = null @@ -123,6 +123,12 @@ export function capabilities() { attachments: true, file_allowlist: true, max_execution_time: true, + // engine-enforced result/memory bounds + typed resource errors with + // a recovery hint (contract 0.3.0) + resource_caps: true, + // binding-side deadline for network-source queries + engine-side + // fast-fail baseline (contract 0.3.0) + network_watchdog: true, async: true, streaming: false, }, diff --git a/integrations/agents/errors.d.mts b/integrations/agents/errors.d.mts index 3e01e6c..d0a6d11 100644 --- a/integrations/agents/errors.d.mts +++ b/integrations/agents/errors.d.mts @@ -2,21 +2,35 @@ export interface ChDBErrorObject { code: number type: string message: string + /** Model-facing recovery instruction (present on resource-limit errors). */ + hint?: string } export class ChDBError extends Error { code: number type: string - constructor(message: string, opts?: { code?: number; type?: string }) + hint: string | null + constructor(message: string, opts?: { code?: number; type?: string; hint?: string | null }) toObject(): ChDBErrorObject } /** A write/DDL was rejected because the tool session is read-only (code 164). */ export class ChDBReadOnlyError extends ChDBError {} +/** + * The query hit an engine resource limit (rows / bytes / time / memory); + * the SQL is valid — carries a `hint` telling the model to narrow the query. + */ +export class ChDBResourceError extends ChDBError {} /** Parse / type / argument error in the submitted SQL. */ export class ChDBSyntaxError extends ChDBError {} /** Unknown table / database / function / column / setting. */ export class ChDBUnknownObjectError extends ChDBError {} +/** The recovery instruction attached to every resource-limit error (binding-identical wording). */ +export declare const RESOURCE_HINT: string + +/** The recovery instruction attached to a NETWORK_TIMEOUT watchdog error (binding-identical wording). */ +export declare const NETWORK_HINT: string + /** Return a typed ChDBError for a raw engine exception or message string. */ export function parseError(excOrMessage: unknown): ChDBError diff --git a/integrations/agents/errors.mjs b/integrations/agents/errors.mjs index df419ca..ba7444a 100644 --- a/integrations/agents/errors.mjs +++ b/integrations/agents/errors.mjs @@ -15,17 +15,24 @@ // message body stays in the message and the real trailing type wins. const ERR_RE = /Code:\s*(\d+)\.\s*DB::Exception:\s*([\s\S]*)\(([A-Z0-9_]+)\)/ +// Base error. `code`/`type`/`message` are always populated; `hint` is an +// optional model-facing recovery instruction (set for resource-limit errors, +// where the model must learn "narrow the query" rather than "give up" or +// "retry unchanged"). export class ChDBError extends Error { - constructor(message, { code = 0, type = 'UNKNOWN' } = {}) { + constructor(message, { code = 0, type = 'UNKNOWN', hint = null } = {}) { super(message) this.name = 'ChDBError' this.code = code this.type = type this.message = message + this.hint = hint } toObject() { - return { code: this.code, type: this.type, message: this.message } + const d = { code: this.code, type: this.type, message: this.message } + if (this.hint) d.hint = this.hint + return d } } @@ -37,6 +44,20 @@ export class ChDBReadOnlyError extends ChDBError { } } +/** + * The query hit an engine resource limit (rows / bytes / time / memory). + * + * Distinct from a logic error: the SQL is valid, the result was just too big + * or too slow. Carries a `hint` telling the model how to shrink the query, so + * an agent distinguishes "narrow and retry" from "abandon". + */ +export class ChDBResourceError extends ChDBError { + constructor(message, opts) { + super(message, opts) + this.name = 'ChDBResourceError' + } +} + /** Parse / type / argument error in the submitted SQL. */ export class ChDBSyntaxError extends ChDBError { constructor(message, opts) { @@ -53,6 +74,13 @@ export class ChDBUnknownObjectError extends ChDBError { } } +// Hint on NETWORK_TIMEOUT: the model must switch strategy, not wait or retry. +export const NETWORK_HINT = + 'The query referenced a remote source (url()/s3()/...) and did not return ' + + 'within the network deadline. Network egress may be disabled or firewalled ' + + 'in this environment. Use file() on data already available locally, or ask ' + + 'the operator to enable egress. Do not retry the same query unchanged.' + // ClickHouse error code -> ChDBError subclass. Kept small and explicit; the // CONTRACT lists exactly these so other languages classify identically. const CODE_TO_CLASS = { @@ -63,8 +91,22 @@ const CODE_TO_CLASS = { 60: ChDBUnknownObjectError, // UNKNOWN_TABLE 81: ChDBUnknownObjectError, // UNKNOWN_DATABASE 115: ChDBUnknownObjectError, // UNKNOWN_SETTING + 158: ChDBResourceError, // TOO_MANY_ROWS + 159: ChDBResourceError, // TIMEOUT_EXCEEDED + 241: ChDBResourceError, // MEMORY_LIMIT_EXCEEDED + 307: ChDBResourceError, // TOO_MANY_BYTES + 396: ChDBResourceError, // TOO_MANY_ROWS_OR_BYTES (max_result_rows/bytes) } +// The recovery instruction attached to every resource-limit error. Wording is +// model-facing: name the fix (filter / project / aggregate / limit) and forbid +// the two failure loops (verbatim retry, silent abandonment). +export const RESOURCE_HINT = + 'The query exceeded a resource limit; the SQL itself is valid. ' + + 'Narrow it and retry: add a WHERE filter, select fewer columns, ' + + 'aggregate before returning, or add/lower LIMIT. ' + + 'Do not retry the same query unchanged.' + const TYPE_TO_CLASS = { READONLY: ChDBReadOnlyError, } @@ -86,5 +128,6 @@ export function parseError(excOrMessage) { // greedy msg keeps the trailing ". " that precedes the (TYPE); trim it const msg = m[2].trim().replace(/\.+$/, '').trim() const Cls = CODE_TO_CLASS[code] || TYPE_TO_CLASS[type] || ChDBError - return new Cls(msg, { code, type }) + const hint = Cls === ChDBResourceError ? RESOURCE_HINT : null + return new Cls(msg, { code, type, hint }) } diff --git a/integrations/agents/index.d.mts b/integrations/agents/index.d.mts index f29c143..86b998e 100644 --- a/integrations/agents/index.d.mts +++ b/integrations/agents/index.d.mts @@ -12,6 +12,7 @@ export type { export { ChDBError, ChDBReadOnlyError, + ChDBResourceError, ChDBSyntaxError, ChDBUnknownObjectError, parseError, diff --git a/integrations/agents/index.mjs b/integrations/agents/index.mjs index 81f19d7..edd591c 100644 --- a/integrations/agents/index.mjs +++ b/integrations/agents/index.mjs @@ -10,6 +10,7 @@ export { CONTRACT_VERSION, capabilities, loadDescriptors, toolSpecs } from './de export { ChDBError, ChDBReadOnlyError, + ChDBResourceError, ChDBSyntaxError, ChDBUnknownObjectError, parseError, diff --git a/integrations/agents/safety.d.mts b/integrations/agents/safety.d.mts index 658aab5..88ba6d5 100644 --- a/integrations/agents/safety.d.mts +++ b/integrations/agents/safety.d.mts @@ -11,6 +11,8 @@ export function pathAllowed(path: string, allowlist: string[] | null | undefined export const SAFE_TABLE_FUNCTIONS: Set /** Static fallback set of external source table functions (lowercase). */ export const FALLBACK_KNOWN_TABLE_FUNCTIONS: Set +/** Table functions whose source is reached over the network (lowercase); drives the network watchdog. */ +export const NETWORK_TABLE_FUNCTIONS: Set /** * Every non-safe table-function call in `sql` (masked scan: string literals and * comments blanked, quoted function names matched) whose lowercase name is in diff --git a/integrations/agents/safety.mjs b/integrations/agents/safety.mjs index cd113ca..6e25a20 100644 --- a/integrations/agents/safety.mjs +++ b/integrations/agents/safety.mjs @@ -168,6 +168,38 @@ export const FALLBACK_KNOWN_TABLE_FUNCTIONS = new Set([ 'prometheusqueryrange', ]) +// Table functions reached over the network; drives the watchdog (CONTRACT P5). +// file()/sqlite and the *local* lake variants are deliberately absent. +export const NETWORK_TABLE_FUNCTIONS = new Set([ + 'url', + 'urlcluster', + 'urlwithheaders', + 's3', + 's3cluster', + 'gcs', + 'azureblobstorage', + 'azureblobstoragecluster', + 'remote', + 'remotesecure', + 'hdfs', + 'hdfscluster', + 'mongodb', + 'postgresql', + 'mysql', + 'redis', + 'odbc', + 'jdbc', + 'icebergs3', + 'icebergs3cluster', + 'icebergazure', + 'icebergazurecluster', + 'iceberghdfs', + 'iceberghdfscluster', + 'deltalake', + 'deltalakeazure', + 'hudi', +]) + // Single pass over the SQL: a token is either a string literal, a line comment, // or a block comment. Left-to-right alternation guarantees that once a construct // opens, its body is consumed up to the matching close before any other rule can diff --git a/integrations/agents/tool.d.mts b/integrations/agents/tool.d.mts index 05d2fad..6c4e25a 100644 --- a/integrations/agents/tool.d.mts +++ b/integrations/agents/tool.d.mts @@ -8,8 +8,13 @@ export interface QueryResultObject { columnNames: string[] elapsedS: number | null bytesRead: number | null + /** Model-facing recovery instruction (present when `truncated` is true). */ + hint?: string } +/** The recovery instruction attached to a truncated envelope result (binding-identical wording). */ +export declare const TRUNCATION_HINT: string + /** Result of a query: decoded rows plus honest truncation / stat metadata. */ export class QueryResult { rows: Array> @@ -44,6 +49,15 @@ export interface ChDBToolOptions { maxBytes?: number /** Optional engine wall-clock bound in seconds; a runaway query raises TIMEOUT_EXCEEDED. */ maxExecutionTime?: number | null + /** + * Deadline in seconds for network-source queries (default 60; null/0 disables). + * On expiry the query fails NETWORK_TIMEOUT and the tool is poisoned (CONTRACT.md P5). + */ + networkTimeout?: number | null + /** Optional engine memory bound in bytes; exceeding it raises MEMORY_LIMIT_EXCEEDED. */ + maxMemoryUsage?: number | null + /** Optional engine-side result-bytes backstop (truncates unflagged; set well above maxBytes). */ + maxResultBytes?: number | null /** Optional allowlist of path prefixes for file()/s3()/url() and attachments. */ fileAllowlist?: string[] | null /** Files to register as views before the read-only lock: { name: path | [path, format] }. */ @@ -64,6 +78,9 @@ export class ChDBTool { readonly maxRows: number readonly maxBytes: number readonly maxExecutionTime: number | null + readonly networkTimeout: number | null + readonly maxMemoryUsage: number | null + readonly maxResultBytes: number | null readonly fileAllowlist: string[] | null query(sql: string, opts?: { params?: Record | null; maxRows?: number | null }): Promise listDatabases(): Promise diff --git a/integrations/agents/tool.mjs b/integrations/agents/tool.mjs index 35918fe..7e948f6 100644 --- a/integrations/agents/tool.mjs +++ b/integrations/agents/tool.mjs @@ -19,15 +19,20 @@ import { Session } from '../../index.mjs' import { toolSpecs } from './descriptors.mjs' -import { ChDBError, ChDBReadOnlyError, parseError } from './errors.mjs' +import { ChDBError, ChDBReadOnlyError, NETWORK_HINT, parseError } from './errors.mjs' import { FALLBACK_KNOWN_TABLE_FUNCTIONS, + NETWORK_TABLE_FUNCTIONS, findSourceCalls, pathAllowed, quoteIdent, quoteString, } from './safety.mjs' +// Sessions abandoned by the network watchdog: the native call is still in +// flight inside them, so freeing them is UB — parked for process lifetime. +const ABANDONED_SESSIONS = [] + // Coerce a numeric argument to an integer, or throw a typed INVALID_ARGUMENT. // A non-numeric cap must fail loudly: Number('lots') is NaN, and every NaN // comparison is false, so before this guard a garbage maxRows silently @@ -43,6 +48,14 @@ function intArg(value, name) { return n } +// Model-facing recovery instruction attached to a truncated result on the +// envelope path. A bare `truncated: true` teaches nothing; the model must learn +// that the full result exists and how to get the part it needs. +export const TRUNCATION_HINT = + 'Result truncated at the row/byte cap; more rows exist. ' + + 'If you need them, aggregate, filter with WHERE, or select fewer columns ' + + 'instead of re-running the same query.' + /** Result of a query: decoded rows plus honest truncation / stat metadata. */ export class QueryResult { constructor(rows, truncated, columnNames, elapsedS = null, bytesRead = null) { @@ -55,7 +68,7 @@ export class QueryResult { } toObject() { - return { + const d = { rows: this.rows, rowCount: this.rowCount, truncated: this.truncated, @@ -63,6 +76,8 @@ export class QueryResult { elapsedS: this.elapsedS, bytesRead: this.bytesRead, } + if (this.truncated) d.hint = TRUNCATION_HINT + return d } } @@ -82,11 +97,14 @@ export class ChDBTool { #session #ownsSession #knownTableFunctions + #poisoned = false /** * @param {{ * path?: string, readOnly?: boolean, maxRows?: number, maxBytes?: number, - * maxExecutionTime?: number|null, fileAllowlist?: string[]|null, + * maxExecutionTime?: number|null, networkTimeout?: number|null, + * maxMemoryUsage?: number|null, + * maxResultBytes?: number|null, fileAllowlist?: string[]|null, * attachments?: Record|null, * session?: import('../../index.mjs').Session|null * }} [opts] @@ -98,6 +116,9 @@ export class ChDBTool { maxRows = 1000, maxBytes = 1_000_000, maxExecutionTime = null, + networkTimeout = 60, + maxMemoryUsage = null, + maxResultBytes = null, fileAllowlist = null, attachments = null, session = null, @@ -108,6 +129,21 @@ export class ChDBTool { this.maxBytes = Math.max(1, intArg(maxBytes, 'maxBytes')) this.maxExecutionTime = maxExecutionTime == null ? null : Math.max(0, intArg(maxExecutionTime, 'maxExecutionTime')) + // Binding-side deadline (seconds) for NETWORK_TABLE_FUNCTIONS queries: a + // firewalled endpoint can hang the engine past every engine-side timeout. + // null/0 disables. + this.networkTimeout = !networkTimeout ? null : Math.max(1, intArg(networkTimeout, 'networkTimeout')) + // Engine-side memory bound for the whole query (bytes). Exceeding it + // raises MEMORY_LIMIT_EXCEEDED (loud, typed) — the primary OOM guard in + // memory-tight deployments (sandboxes). Off by default. + this.maxMemoryUsage = + maxMemoryUsage == null ? null : Math.max(1, intArg(maxMemoryUsage, 'maxMemoryUsage')) + // Engine-side bound on the result set's size (bytes). With the break + // overflow mode this truncates at block granularity WITHOUT a flag, so + // it is a coarse backstop, not the honest cap: set it well above + // maxBytes so the flagged client-side cap fires first. Off by default. + this.maxResultBytes = + maxResultBytes == null ? null : Math.max(1, intArg(maxResultBytes, 'maxResultBytes')) // null = no allowlist (all paths allowed); a list = only these prefixes. this.fileAllowlist = fileAllowlist && fileAllowlist.length ? [...fileAllowlist] : null @@ -142,10 +178,37 @@ export class ChDBTool { } // Exact 64-bit integers survive JSON as strings rather than lossy floats. this.#session.query('SET output_format_json_quote_64bit_integers=1', 'CSV') + // Engine-side result bound. The decode path buffers the engine's whole + // JSON payload before the client-side row/byte trim, so without this an + // unbounded SELECT materializes everything in memory before maxRows + // ever applies — in a 512MB sandbox that is an OOM, not a truncation. + // max_result_rows is the row cap + 1 under result_overflow_mode='break': + // the engine stops producing at block granularity, and the one extra row + // is what keeps the truncated flag exact (data.length > cap). The per-call + // maxRows is clamped to the constructor cap for the same reason. + this.#session.query('SET max_block_size=8192', 'CSV') + this.#session.query("SET result_overflow_mode='break'", 'CSV') + this.#session.query(`SET max_result_rows=${this.maxRows + 1}`, 'CSV') + if (this.maxResultBytes != null) { + this.#session.query(`SET max_result_bytes=${this.maxResultBytes}`, 'CSV') + } + if (this.maxMemoryUsage != null) { + // loud engine OOM guard: exceeding raises MEMORY_LIMIT_EXCEEDED + this.#session.query(`SET max_memory_usage=${this.maxMemoryUsage}`, 'CSV') + } if (this.maxExecutionTime != null) { // engine-side wall-clock bound; a runaway query raises TIMEOUT_EXCEEDED this.#session.query(`SET max_execution_time=${this.maxExecutionTime}`, 'CSV') } + if (this.networkTimeout != null) { + // Engine-side fast-fail baseline: one attempt, no HEAD probe, small + // connect timeout. The watchdog in query() is the backstop. + this.#session.query(`SET http_connection_timeout=${Math.min(this.networkTimeout, 10)}`, 'CSV') + this.#session.query(`SET http_receive_timeout=${this.networkTimeout}`, 'CSV') + this.#session.query(`SET http_send_timeout=${this.networkTimeout}`, 'CSV') + this.#session.query('SET http_max_tries=1', 'CSV') + this.#session.query('SET http_make_head_request=0', 'CSV') + } // Attachments must be materialized BEFORE the read-only lock, because // CREATE VIEW is a write that readonly=2 rejects. This is why read-only // tools declare files at construction rather than via attachFile(). @@ -212,16 +275,35 @@ export class ChDBTool { if (typeof sql !== 'string' || sql.trim() === '') { throw new ChDBError('sql must be a non-empty string') } + if (this.#poisoned) { + throw new ChDBError( + 'a previous network-source query was abandoned after its deadline; ' + + "this tool's engine session may be blocked — create a new ChDBTool", + { type: 'TOOL_ERROR' }, + ) + } this.#enforceAllowlist(sql) - const cap = maxRows == null ? this.maxRows : Math.max(1, intArg(maxRows, 'maxRows')) + // The constructor cap is the ceiling: the engine-side max_result_rows was + // fixed at construction, so honoring a larger per-call cap is impossible — + // clamping keeps the truncated flag honest instead of silently under-filling. + const cap = + maxRows == null + ? this.maxRows + : Math.min(Math.max(1, intArg(maxRows, 'maxRows')), this.maxRows) let obj try { const hasParams = params && Object.keys(params).length > 0 - const res = hasParams - ? await this.#session.queryBindAsync(sql, params, { format: 'JSON' }) - : await this.#session.queryAsync(sql, { format: 'JSON' }) + const engine = hasParams + ? this.#session.queryBindAsync(sql, params, { format: 'JSON' }) + : this.#session.queryAsync(sql, { format: 'JSON' }) + const res = + this.networkTimeout && findSourceCalls(sql, NETWORK_TABLE_FUNCTIONS).length > 0 + ? await this.#raceNetworkDeadline(engine) + : await engine obj = res.json() || {} } catch (e) { + // watchdog/typed errors pass through, never re-wrapped + if (e instanceof ChDBError) throw e // Malformed / non-JSON engine output and engine errors alike become a // typed ChDBError rather than a bare error leaking to the caller. throw parseError(e) @@ -252,6 +334,35 @@ export class ChDBTool { return new QueryResult(rows, truncated, cols, stats.elapsed ?? null, stats.bytes_read ?? null) } + // Network watchdog (CONTRACT P5). The in-flight native call cannot be + // cancelled, so on expiry the tool is poisoned and the session parked. + async #raceNetworkDeadline(engine) { + let timer + const deadline = new Promise((_, reject) => { + timer = setTimeout(() => { + reject( + new ChDBError(`network-source query did not return within ${this.networkTimeout}s`, { + type: 'NETWORK_TIMEOUT', + hint: NETWORK_HINT, + }), + ) + }, this.networkTimeout * 1000) + }) + try { + return await Promise.race([engine, deadline]) + } catch (e) { + if (e instanceof ChDBError && e.type === 'NETWORK_TIMEOUT') { + this.#poisoned = true + ABANDONED_SESSIONS.push(this.#session) + // the abandoned promise must never surface as an unhandled rejection + engine.catch(() => {}) + } + throw e + } finally { + clearTimeout(timer) + } + } + // With a fileAllowlist set, every non-safe table-function call in the SQL // must carry a literal source argument inside the allowlist. The scan runs // over masked SQL (string literals/comments blanked, quoted function names @@ -267,12 +378,12 @@ export class ChDBTool { if (arg === null) { throw new ChDBError( `table function ${JSON.stringify(fn)} without a literal source argument is not allowed when file_allowlist is set`, - { type: 'ACCESS_DENIED' }, + { type: 'ALLOWLIST_DENIED' }, ) } if (!pathAllowed(arg, this.fileAllowlist)) { throw new ChDBError(`source path not in file_allowlist: ${JSON.stringify(arg)}`, { - type: 'ACCESS_DENIED', + type: 'ALLOWLIST_DENIED', }) } } @@ -283,7 +394,7 @@ export class ChDBTool { #createFileView(name, path, format = null) { if (this.fileAllowlist && !pathAllowed(path, this.fileAllowlist)) { throw new ChDBError(`attach path not in file_allowlist: ${JSON.stringify(path)}`, { - type: 'ACCESS_DENIED', + type: 'ALLOWLIST_DENIED', }) } let src = `file(${quoteString(path)}` @@ -457,6 +568,11 @@ export class ChDBTool { } close() { + if (this.#poisoned) { + // session is parked in ABANDONED_SESSIONS; freeing it mid-call is UB + this.#session = null + return + } if (this.#ownsSession && this.#session) { try { this.#session.close() diff --git a/test/v3/integrations/agents-conformance.test.ts b/test/v3/integrations/agents-conformance.test.ts index 776a86d..b2f3920 100644 --- a/test/v3/integrations/agents-conformance.test.ts +++ b/test/v3/integrations/agents-conformance.test.ts @@ -60,6 +60,11 @@ function toolFrom(cfg: any): any { maxRows: c.max_rows, maxBytes: c.max_bytes, maxExecutionTime: c.max_execution_time ?? null, + // absent -> undefined -> the binding's default (60); an explicit JSON + // null/0 must stay disabled, so no ?? fallback here + networkTimeout: c.network_timeout, + maxMemoryUsage: c.max_memory_usage ?? null, + maxResultBytes: c.max_result_bytes ?? null, fileAllowlist: c.file_allowlist ?? null, attachments: c.attachments ?? null, }) @@ -123,6 +128,12 @@ describe('agents conformance (CONTRACT.md / cases.jsonl)', () => { if (exp.envelope_ok !== undefined) { expect(result.ok).toBe(exp.envelope_ok) if (exp.error_type) expect(result.error.type).toBe(exp.error_type) + if (exp.result_has_hint !== undefined) { + expect(Boolean(result.result?.hint)).toBe(exp.result_has_hint) + } + if (exp.error_has_hint !== undefined) { + expect(Boolean(result.error?.hint)).toBe(exp.error_has_hint) + } return } if (exp.rows !== undefined) expect(result.rows).toEqual(exp.rows) diff --git a/test/v3/integrations/agents-tool.test.ts b/test/v3/integrations/agents-tool.test.ts index 67eb5e1..6a18e3b 100644 --- a/test/v3/integrations/agents-tool.test.ts +++ b/test/v3/integrations/agents-tool.test.ts @@ -3,7 +3,9 @@ import { Session } from '../../../index.js' // @ts-ignore - .mjs base resolved at runtime import { ChDBTool } from '../../../integrations/agents/tool.mjs' // @ts-ignore -import { ChDBError } from '../../../integrations/agents/errors.mjs' +import { ChDBError, ChDBResourceError, RESOURCE_HINT } from '../../../integrations/agents/errors.mjs' +// @ts-ignore +import { TRUNCATION_HINT } from '../../../integrations/agents/tool.mjs' // @ts-ignore import { chdbTools } from '../../../integrations/ai-sdk.mjs' // @ts-ignore @@ -17,7 +19,7 @@ import { AGENT_TOOL_DESCRIPTORS } from '../../../integrations/agents/framework.m describe('ChDBTool resource lifetime', () => { it('closes an owned session when constructor setup throws (no leak)', () => { - // A bad attachment under a fileAllowlist throws ACCESS_DENIED during setup; + // A bad attachment under a fileAllowlist throws ALLOWLIST_DENIED during setup; // the Session the tool just created must be closed before the rethrow. expect( () => @@ -191,6 +193,142 @@ describe('argument validation (CONTRACT.md P3)', () => { }) }) +describe('resource caps and hints (CONTRACT.md P3/P5, contract 0.3.0)', () => { + it('classifies an engine resource limit as ChDBResourceError carrying the recovery hint', async () => { + // DEDICATED tool: a query-level SETTINGS clause persists for the session on + // a chdb session (engine quirk documented in CONTRACT.md), so this must + // never run on a shared tool instance. + const t = new ChDBTool({ readOnly: true }) + try { + let err: any + try { + await t.query( + "SELECT number FROM numbers(100) SETTINGS max_result_rows = 10, result_overflow_mode = 'throw'", + ) + } catch (e) { + err = e + } + expect(err).toBeInstanceOf(ChDBResourceError) + expect(err.type).toBe('TOO_MANY_ROWS_OR_BYTES') + expect(err.hint).toBe(RESOURCE_HINT) + expect(err.toObject().hint).toBe(RESOURCE_HINT) + } finally { + t.close() + } + }) + + it('clamps a per-call maxRows above the constructor cap (engine bound fixed at construction)', async () => { + const t = new ChDBTool({ readOnly: true, maxRows: 5 }) + try { + const r = await t.query('SELECT toInt32(number) AS n FROM numbers(10)', { maxRows: 50 }) + expect(r.rowCount).toBe(5) + expect(r.truncated).toBe(true) + } finally { + t.close() + } + }) + + it('adds the truncation hint to a truncated envelope result (and only then)', async () => { + const t = new ChDBTool({ readOnly: true, maxRows: 3 }) + try { + const truncated = (await t.query('SELECT toInt32(number) AS n FROM numbers(10)')).toObject() + expect(truncated.truncated).toBe(true) + expect(truncated.hint).toBe(TRUNCATION_HINT) + const full = (await t.query('SELECT 1 AS x')).toObject() + expect(full.truncated).toBe(false) + expect('hint' in full).toBe(false) + } finally { + t.close() + } + }) + + it('validates maxMemoryUsage / maxResultBytes as typed INVALID_ARGUMENT', () => { + for (const opts of [{ maxMemoryUsage: 'lots' }, { maxResultBytes: 'lots' }]) { + let err: any + try { + new ChDBTool(opts as any) + } catch (e) { + err = e + } + expect(err).toBeInstanceOf(ChDBError) + expect(err.type).toBe('INVALID_ARGUMENT') + } + }) +}) + +describe('network watchdog (CONTRACT.md P5, contract 0.3.0)', () => { + // Stands in for an engine call blocked on a firewalled endpoint (the real + // black hole is not portable); sync query() serves the constructor probe/SETs. + function slowSession() { + return { + query(sql: string) { + return sql.includes("getSetting('readonly')") ? '0' : '' + }, + queryAsync() { + return new Promise((resolve) => + setTimeout(() => resolve({ json: () => ({ data: [] }) }), 2500), + ) + }, + } + } + + it('deadline fires with NETWORK_TIMEOUT + hint, poisons the tool, close() stays safe', async () => { + const tool = new ChDBTool({ session: slowSession() as any, readOnly: false, networkTimeout: 1 }) + const t0 = Date.now() + let err: any + try { + await tool.query("SELECT count() FROM url('https://example.invalid/x.csv', 'CSV')") + } catch (e) { + err = e + } + expect(err).toBeInstanceOf(ChDBError) + expect(err.type).toBe('NETWORK_TIMEOUT') + expect(err.hint).toBeTruthy() + expect(Date.now() - t0).toBeLessThan(2000) + // poisoned: even a local query must fail with TOOL_ERROR + let err2: any + try { + await tool.query('SELECT 1') + } catch (e) { + err2 = e + } + expect(err2).toBeInstanceOf(ChDBError) + expect(err2.type).toBe('TOOL_ERROR') + // close() must drop the reference without freeing the parked session + expect(() => tool.close()).not.toThrow() + }) + + it('envelope path carries NETWORK_TIMEOUT type + hint', async () => { + const tool = new ChDBTool({ session: slowSession() as any, readOnly: false, networkTimeout: 1 }) + const out: any = await tool.call('run_select_query', { + sql: "SELECT 1 FROM s3('https://example.invalid/x.parquet')", + }) + expect(out.ok).toBe(false) + expect(out.error.type).toBe('NETWORK_TIMEOUT') + expect(out.error.hint).toBeTruthy() + tool.close() + }) + + it('local queries bypass the watchdog', async () => { + const tool = new ChDBTool({ networkTimeout: 1 }) + try { + const r = await tool.query('SELECT toInt32(1) AS x') + expect(r.rows).toEqual([{ x: 1 }]) + } finally { + tool.close() + } + }) + + it('networkTimeout: null disables the watchdog', () => { + const tool = new ChDBTool({ networkTimeout: null }) + try { + expect(tool.networkTimeout).toBeNull() + } finally { + tool.close() + } + }) +}) + describe('adapter toolset close()', () => { it('exposes a non-enumerable close() that is not treated as a tool', () => { const s = new Session('') From 3bc56d6f489a4cc84d8abb9bebf517f32c6c399c Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 20 Jul 2026 11:02:24 +1200 Subject: [PATCH 2/5] agents: fix network baseline (handshake honors send/receive); real-hang test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mirror of the Python reference fixes: send/receive timeouts join connection_timeout at min(networkTimeout, 10) — the TLS handshake is bounded by max(send, receive), so the old nt-sized values defeated fast-fail. An engine-side Poco::TimeoutException on a network-source query now carries the same hint as a watchdog expiry. Re-vendored CONTRACT.md + cases.jsonl. Node-specific: a parked watchdog-abandoned session is now closed once its native call settles — chdb-node binds one data directory per process, so leaving it live forever would block every future tool, making the 'create a new ChDBTool' advice unactionable. The real-hang test (local black-holed TLS endpoint) covers both the watchdog and the fresh-tool recovery. Co-Authored-By: Claude Fable 5 --- integrations/agents/CONTRACT.md | 12 ++-- integrations/agents/conformance/cases.jsonl | 2 +- integrations/agents/tool.mjs | 45 +++++++++--- test/v3/integrations/agents-tool.test.ts | 79 +++++++++++++++++++++ 4 files changed, 123 insertions(+), 15 deletions(-) diff --git a/integrations/agents/CONTRACT.md b/integrations/agents/CONTRACT.md index df33995..0344b22 100644 --- a/integrations/agents/CONTRACT.md +++ b/integrations/agents/CONTRACT.md @@ -182,10 +182,14 @@ to TS, which has no in-process pandas). Bindings may omit it. None/0 disables) applies to queries referencing a network-source table function (the shared `NETWORK_TABLE_FUNCTIONS` set; `file()` and local lake variants exempt). Two layers: at construction the session gets a fast-fail - baseline (`http_connection_timeout=min(nt,10)`, receive/send = nt, - `http_max_tries=1`, `http_make_head_request=0`); at query time the engine - call runs under a `network_timeout` deadline and expiry raises/envelopes - `NETWORK_TIMEOUT` with a hint. The watchdog is load-bearing, not cosmetic: + baseline (`http_connection_timeout` / `http_send_timeout` / + `http_receive_timeout` = min(nt, 10), `http_max_tries=1`, + `http_make_head_request=0` — the TLS handshake is bounded by + max(send, receive), not by connection_timeout, and one attempt costs ~4-5x + the setting); at query time the engine call runs under a `network_timeout` + deadline and expiry raises/envelopes `NETWORK_TIMEOUT` with a hint. An + engine-side timeout (`Poco::TimeoutException`) on a network-source query + gets the same hint attached. The watchdog is load-bearing, not cosmetic: a black-holed HTTPS endpoint hangs the engine call unboundedly on affected builds (blocked TLS handshake ignores every engine-side timeout), so there is no error to translate unless the binding imposes one. The abandoned call diff --git a/integrations/agents/conformance/cases.jsonl b/integrations/agents/conformance/cases.jsonl index 9b1554b..98ad766 100644 --- a/integrations/agents/conformance/cases.jsonl +++ b/integrations/agents/conformance/cases.jsonl @@ -31,4 +31,4 @@ {"id": "p5_resource_error_typed", "pillar": "P5", "desc": "an engine resource limit surfaces as its engine type (resource family: 158/159/241/307/396); own tool: a SETTINGS clause persists on a chdb session (upstream quirk)", "tool": {}, "method": "query", "args": {"sql": "SELECT number FROM numbers(100) SETTINGS max_result_rows = 10, result_overflow_mode = 'throw'"}, "expect": {"error_type": "TOO_MANY_ROWS_OR_BYTES"}} {"id": "p4_truncation_hint", "pillar": "P4", "desc": "a truncated envelope result carries a recovery hint for the model", "method": "call", "args": {"name": "run_select_query", "arguments": {"sql": "SELECT toInt32(number) AS n FROM numbers(2000)"}}, "expect": {"envelope_ok": true, "result_has_hint": true}} {"id": "p4_resource_hint", "pillar": "P4", "desc": "a resource-limit error envelope carries a recovery hint (narrow, don't abandon or retry unchanged); own tool: a SETTINGS clause persists on a chdb session (upstream quirk)", "tool": {}, "method": "call", "args": {"name": "run_select_query", "arguments": {"sql": "SELECT number FROM numbers(100) SETTINGS max_result_rows = 10, result_overflow_mode = 'throw'"}}, "expect": {"envelope_ok": false, "error_type": "TOO_MANY_ROWS_OR_BYTES", "error_has_hint": true}} -{"id": "p5_network_baseline_settings", "pillar": "P5", "desc": "network_timeout injects the engine-side fast-fail network baseline (tries=1, no HEAD probe, small connect timeout)", "tool": {"network_timeout": 30}, "method": "query", "args": {"sql": "SELECT toInt32(getSetting('http_max_tries')) AS tries, toInt32(getSetting('http_make_head_request')) AS head, toInt32(getSetting('http_connection_timeout')) AS conn"}, "expect": {"rows": [{"tries": 1, "head": 0, "conn": 10}]}} +{"id": "p5_network_baseline_settings", "pillar": "P5", "desc": "network_timeout injects the engine-side fast-fail network baseline (tries=1, no HEAD probe, small connect/send/receive timeouts — the TLS handshake is bounded by max(send, receive))", "tool": {"network_timeout": 30}, "method": "query", "args": {"sql": "SELECT toInt32(getSetting('http_max_tries')) AS tries, toInt32(getSetting('http_make_head_request')) AS head, toInt32(getSetting('http_connection_timeout')) AS conn, toInt32(getSetting('http_send_timeout')) AS snd, toInt32(getSetting('http_receive_timeout')) AS rcv"}, "expect": {"rows": [{"tries": 1, "head": 0, "conn": 10, "snd": 10, "rcv": 10}]}} diff --git a/integrations/agents/tool.mjs b/integrations/agents/tool.mjs index 7e948f6..78cc6d3 100644 --- a/integrations/agents/tool.mjs +++ b/integrations/agents/tool.mjs @@ -30,7 +30,7 @@ import { } from './safety.mjs' // Sessions abandoned by the network watchdog: the native call is still in -// flight inside them, so freeing them is UB — parked for process lifetime. +// flight inside them, so freeing them is UB — parked until that call settles. const ABANDONED_SESSIONS = [] // Coerce a numeric argument to an integer, or throw a typed INVALID_ARGUMENT. @@ -201,11 +201,13 @@ export class ChDBTool { this.#session.query(`SET max_execution_time=${this.maxExecutionTime}`, 'CSV') } if (this.networkTimeout != null) { - // Engine-side fast-fail baseline: one attempt, no HEAD probe, small - // connect timeout. The watchdog in query() is the backstop. - this.#session.query(`SET http_connection_timeout=${Math.min(this.networkTimeout, 10)}`, 'CSV') - this.#session.query(`SET http_receive_timeout=${this.networkTimeout}`, 'CSV') - this.#session.query(`SET http_send_timeout=${this.networkTimeout}`, 'CSV') + // Fast-fail baseline: one attempt, no HEAD probe. The TLS handshake honors + // max(send, receive) — not connection_timeout — and one attempt costs ~4-5x + // the setting (verified against chdb-core main), so keep all three small. + const capS = Math.min(this.networkTimeout, 10) + this.#session.query(`SET http_connection_timeout=${capS}`, 'CSV') + this.#session.query(`SET http_receive_timeout=${capS}`, 'CSV') + this.#session.query(`SET http_send_timeout=${capS}`, 'CSV') this.#session.query('SET http_max_tries=1', 'CSV') this.#session.query('SET http_make_head_request=0', 'CSV') } @@ -336,6 +338,8 @@ export class ChDBTool { // Network watchdog (CONTRACT P5). The in-flight native call cannot be // cancelled, so on expiry the tool is poisoned and the session parked. + // NB: the abandoned native op keeps the process alive until it settles + // (pending napi work refs the event loop) — exit is delayed, not hung forever. async #raceNetworkDeadline(engine) { let timer const deadline = new Promise((_, reject) => { @@ -353,11 +357,32 @@ export class ChDBTool { } catch (e) { if (e instanceof ChDBError && e.type === 'NETWORK_TIMEOUT') { this.#poisoned = true - ABANDONED_SESSIONS.push(this.#session) - // the abandoned promise must never surface as an unhandled rejection - engine.catch(() => {}) + const session = this.#session + const owned = this.#ownsSession + ABANDONED_SESSIONS.push(session) + // Swallow the abandoned rejection; once the native call settles, close + // an owned parked session — chdb-node binds ONE data directory per + // process, so leaving it live would block every future Session/tool. + engine.then(() => {}, () => {}).then(() => { + const i = ABANDONED_SESSIONS.indexOf(session) + if (i !== -1) ABANDONED_SESSIONS.splice(i, 1) + if (owned) { + try { + session.close() + } catch { + /* best effort */ + } + } + }) + throw e } - throw e + const err = e instanceof ChDBError ? e : parseError(e) + // An engine-side timeout on a network source (Poco::TimeoutException, + // code 1001) deserves the same guidance as a watchdog expiry. + if (err.hint == null && String(err.message).includes('Poco::TimeoutException')) { + err.hint = NETWORK_HINT + } + throw err } finally { clearTimeout(timer) } diff --git a/test/v3/integrations/agents-tool.test.ts b/test/v3/integrations/agents-tool.test.ts index 6a18e3b..ab18082 100644 --- a/test/v3/integrations/agents-tool.test.ts +++ b/test/v3/integrations/agents-tool.test.ts @@ -1,3 +1,4 @@ +import net from 'node:net' import { describe, it, expect } from 'vitest' import { Session } from '../../../index.js' // @ts-ignore - .mjs base resolved at runtime @@ -327,6 +328,84 @@ describe('network watchdog (CONTRACT.md P5, contract 0.3.0)', () => { tool.close() } }) + + it('attaches the network hint to an engine-side Poco timeout (code 1001)', async () => { + // the engine rejects on its own (baseline socket timeouts fired) before the + // watchdog: the parsed error must carry the same recovery hint + const session = { + query(sql: string) { + return sql.includes("getSetting('readonly')") ? '0' : '' + }, + queryAsync() { + return Promise.reject( + new Error('Code: 1001. DB::Exception: Poco::TimeoutException: Timeout. (STD_EXCEPTION)'), + ) + }, + } + const tool = new ChDBTool({ session: session as any, readOnly: false, networkTimeout: 30 }) + try { + let err: any + try { + await tool.query("SELECT 1 FROM url('https://example.invalid/x.csv', 'CSV')") + } catch (e) { + err = e + } + expect(err).toBeInstanceOf(ChDBError) + expect(err.code).toBe(1001) + expect(err.hint).toBeTruthy() + } finally { + tool.close() + } + }) + + it('real black-holed endpoint: watchdog fires, a fresh tool works after the abandoned call settles', async () => { + // Real url() against a local server that accepts and never answers: the TLS + // handshake blocks inside the engine until its socket timeouts fire (~4-7x + // the 2s baseline), so the 2s watchdog must win the race. + const held: net.Socket[] = [] + const srv = net.createServer((sock) => { + held.push(sock) // keep referenced: a GC'd socket sends RST and errors fast + }) + await new Promise((r) => srv.listen(0, '127.0.0.1', r)) + const port = (srv.address() as net.AddressInfo).port + try { + const tool = new ChDBTool({ networkTimeout: 2 }) + const t0 = Date.now() + let err: any + try { + await tool.query(`SELECT count() FROM url('https://127.0.0.1:${port}/x.csv', 'LineAsString')`) + } catch (e) { + err = e + } + expect(err).toBeInstanceOf(ChDBError) + expect(err.type).toBe('NETWORK_TIMEOUT') + expect(err.hint).toBeTruthy() + expect(Date.now() - t0).toBeLessThan(10_000) + tool.close() + + // One data directory per process: a fresh tool becomes constructible only + // once the abandoned native call settles and its parked session is closed. + let fresh: any = null + const deadline = Date.now() + 40_000 + while (fresh == null && Date.now() < deadline) { + try { + fresh = new ChDBTool({ networkTimeout: 2 }) + } catch { + await new Promise((r) => setTimeout(r, 250)) + } + } + expect(fresh, 'fresh tool once the abandoned call settled').toBeTruthy() + try { + const r = await fresh.query('SELECT toInt32(42) AS x') + expect(r.rows).toEqual([{ x: 42 }]) + } finally { + fresh.close() + } + } finally { + srv.close() + for (const c of held) c.destroy() + } + }, 60_000) }) describe('adapter toolset close()', () => { From 4a4a2f13cab3617f17d6a14575bf33465ea624a0 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 20 Jul 2026 11:04:42 +1200 Subject: [PATCH 3/5] agents: re-vendor CONTRACT (parked-session release wording) --- integrations/agents/CONTRACT.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/integrations/agents/CONTRACT.md b/integrations/agents/CONTRACT.md index 0344b22..3e0d7d3 100644 --- a/integrations/agents/CONTRACT.md +++ b/integrations/agents/CONTRACT.md @@ -194,8 +194,11 @@ to TS, which has no in-process pandas). Bindings may omit it. builds (blocked TLS handshake ignores every engine-side timeout), so there is no error to translate unless the binding imposes one. The abandoned call cannot be cancelled: afterwards the tool is **poisoned** (further queries → - `TOOL_ERROR`, `close()` leaves the session alone, the session leaks for - process lifetime). Bindings document this; they don't hide it. + `TOOL_ERROR`, `close()` leaves the session alone). The parked session is + released — closed when tool-owned — if the abandoned call ever settles; it + leaks for process lifetime only if the call never returns. In bindings with + a single-active-session constraint (TS), constructing a replacement tool may + need to wait for that settle. Bindings document this; they don't hide it. - **Result-bytes backstop (0.3.0)** — optional `max_result_bytes`; break mode truncates **without a flag**, so set it well above `max_bytes` and treat it as an OOM backstop only. Off by default. From c54c3cbb49d15c907ae8d78e659390676454c905 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 20 Jul 2026 11:55:52 +1200 Subject: [PATCH 4/5] agents: only null/0 disable networkTimeout; other values validate Mirrors the Python fix: '' silently disabled the watchdog instead of raising INVALID_ARGUMENT. --- integrations/agents/tool.mjs | 7 ++++++- test/v3/integrations/agents-tool.test.ts | 12 ++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/integrations/agents/tool.mjs b/integrations/agents/tool.mjs index 78cc6d3..134ff85 100644 --- a/integrations/agents/tool.mjs +++ b/integrations/agents/tool.mjs @@ -132,7 +132,12 @@ export class ChDBTool { // Binding-side deadline (seconds) for NETWORK_TABLE_FUNCTIONS queries: a // firewalled endpoint can hang the engine past every engine-side timeout. // null/0 disables. - this.networkTimeout = !networkTimeout ? null : Math.max(1, intArg(networkTimeout, 'networkTimeout')) + // Only null/undefined/0 disable the watchdog; anything else (including '') + // must validate instead of silently disabling a guardrail. + this.networkTimeout = + networkTimeout == null || networkTimeout === 0 + ? null + : Math.max(1, intArg(networkTimeout, 'networkTimeout')) // Engine-side memory bound for the whole query (bytes). Exceeding it // raises MEMORY_LIMIT_EXCEEDED (loud, typed) — the primary OOM guard in // memory-tight deployments (sandboxes). Off by default. diff --git a/test/v3/integrations/agents-tool.test.ts b/test/v3/integrations/agents-tool.test.ts index ab18082..3842b2a 100644 --- a/test/v3/integrations/agents-tool.test.ts +++ b/test/v3/integrations/agents-tool.test.ts @@ -420,3 +420,15 @@ describe('adapter toolset close()', () => { s.close() }) }) + +describe('networkTimeout validation (contract 0.3.0)', () => { + it('null and 0 disable; empty string is INVALID_ARGUMENT, not silent-disable', async () => { + const a = new ChDBTool({ networkTimeout: null }) + expect(a.networkTimeout).toBeNull() + a.close() + const b = new ChDBTool({ networkTimeout: 0 }) + expect(b.networkTimeout).toBeNull() + b.close() + expect(() => new ChDBTool({ networkTimeout: '' })).toThrowError(/must be an integer|INVALID_ARGUMENT/) + }) +}) From 97c893b7332da8a85b4f1027c9c8158fd5b67d47 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 20 Jul 2026 12:52:56 +1200 Subject: [PATCH 5/5] test: merge duplicate tool.mjs imports --- test/v3/integrations/agents-tool.test.ts | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/test/v3/integrations/agents-tool.test.ts b/test/v3/integrations/agents-tool.test.ts index 3842b2a..418bf51 100644 --- a/test/v3/integrations/agents-tool.test.ts +++ b/test/v3/integrations/agents-tool.test.ts @@ -2,12 +2,10 @@ import net from 'node:net' import { describe, it, expect } from 'vitest' import { Session } from '../../../index.js' // @ts-ignore - .mjs base resolved at runtime -import { ChDBTool } from '../../../integrations/agents/tool.mjs' +import { ChDBTool, TRUNCATION_HINT } from '../../../integrations/agents/tool.mjs' // @ts-ignore import { ChDBError, ChDBResourceError, RESOURCE_HINT } from '../../../integrations/agents/errors.mjs' // @ts-ignore -import { TRUNCATION_HINT } from '../../../integrations/agents/tool.mjs' -// @ts-ignore import { chdbTools } from '../../../integrations/ai-sdk.mjs' // @ts-ignore import { CONTRACT_VERSION, capabilities, loadDescriptors, toolSpecs } from '../../../integrations/agents/descriptors.mjs'