Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions docs/AGENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2501,12 +2501,6 @@ the role's own grants are the whole boundary (A3).
and never what it said, so an empty completion reaches it too and a model's recorded
`retryEmptyTurn: false` decides nothing. Pinned as it behaves rather than narrowed, because the
narrowing would move behaviour five passing runs were measured under.
- **B76** — the aggregated capture that fixed the wide-catalog refusal now ADMITS the image's own
extension views, so a grounded run on TimescaleDB, Cloudberry or AlloyDB Omni reasons over an
inventory that is mostly internal objects (measured: AlloyDB's least-privilege role sees 67 extension
views beside its 2 user tables). Not a privilege leak — the role genuinely sees them — but grounding
noise; the object set was the same under the flat projection, which refused before any of it reached
a run.

**Settled as limits rather than as work.** The seven below have no entry in `docs/BACKLOG.md`, and

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The B76 removal took the blank line before this paragraph with it, so "Settled as limits rather than as work." now renders as a continuation of the B65 bullet instead of a new paragraph. One blank line above this line fixes it.

that is the point: each is how the product behaves, stated where a reader of this document will meet
Expand Down
30 changes: 1 addition & 29 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ None of it is a GitHub issue.
- [Security Phase 2 deferrals](#security-phase-2-deferrals) — C3–C11 · 7
- [Security Phase 3 deferrals](#security-phase-3-deferrals) — K4
- [Agent M1 deferrals (#328)](#agent-m1-deferrals-328) — A1–A5 · 4
- [Agent M2 deferrals (#329)](#agent-m2-deferrals-329) — B2–B76 · 22
- [Agent M2 deferrals (#329)](#agent-m2-deferrals-329) — B2–B75 · 21

---

Expand Down Expand Up @@ -1835,31 +1835,3 @@ so), and none of them has been measured.

**Done when:** a resume onto a repointed connection does one stated thing, and the run's own record
says which.

### B76. The aggregated grounding capture admits the image's own extension views

The aggregation that fixed the row-cap refusal made a stock image's wide catalog capture SUCCEED instead
of refusing, and the price
is that it succeeds by admitting the image's own objects. Measured live on 2026-09-01 against the
`compat` profile images (`database-compose.yml`), with two user tables seeded:

- TimescaleDB (`timescale/timescaledb:latest-pg17`): 385 column rows aggregate to 46 tables.
- Cloudberry (`woblerr/cloudberry:2.1.0-incubating`): 479 to 67 as gpadmin, 53 as a least-privilege role.
- AlloyDB Omni (`google/alloydbomni:17.9.0`): 542 to 70 as postgres, 69 as a least-privilege role.

AlloyDB is the sharp case: the least-privilege role's 69 tables are 2 user tables plus 67 extension views,
and where they SIT is the part that makes them noise rather than a schema to exclude — 49 of them are
installed into `public` itself (`google_db_advisor_*`, `hypopg_list_indexes`), beside 14 in `google_ml` and
4 in `ai`. Not a privilege leak —
`information_schema.columns` applies its own visibility rules and the role genuinely sees those views —
but grounding noise: the inventory a run reasons over is mostly objects the user did not create, and the
model-facing pack spends `AGENT_CONTEXT_PACK_MAX_CHARS` ranking them against the objective. The old flat
projection had the same object set; it just refused before any of it reached a run.

The candidate fixes are the ones the row-cap fix recorded and set aside: filter by `table_type`, or
exclude the objects the object browser already treats as internal. That fix refuted only the
schema-exclusion variant AS A FIX FOR THE ROW CAP; as a fix for grounding noise the question is open and
unmeasured.

**Done when:** a run grounded on one of these images reasons over the user's objects — or the internal
ones are excluded — with a test per shape (TimescaleDB, Cloudberry, AlloyDB Omni).
20 changes: 14 additions & 6 deletions docs/providers/postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -1083,12 +1083,20 @@ path is reachable from server code and not from a request.
Four things about the PostgreSQL side of that layer are worth knowing here:

- **The catalog read is a composed bounded read**, not a new operation. `inspect_schema` takes a
schema/table selector and the server writes
`SELECT … FROM information_schema.columns WHERE table_schema NOT IN ('pg_catalog', …)`, executed as
`sql.query.read` like any other statement. The model never supplies that SQL. Selectors are quoted
with `quoteLiteral` because `queryReadOnly` binds no parameters, and a selector carrying a
backslash is refused outright rather than quoted — the dialect-less span reader treats it as an
escape, so `'a\'` would read as an unterminated literal.
schema/table selector and the server writes the `columns` statement itself, executed as
`sql.query.read` like any other statement. The model never supplies that SQL. The statement's
`WHERE` excludes the engine's own objects three ways: the full engine-builtin schema list (not
only `pg_catalog` / `information_schema`) and every schema an extension created (`pg_depend` on
`pg_namespace`, `deptype = 'e'`) — both copied from this provider's object browser — plus every
relation an extension created (`pg_depend` on `pg_class`, `deptype = 'e'`), which is new on the
agent path and carried by all four catalog reads, not only the column one. The relation test is
the one that reaches AlloyDB Omni's extension views, 49 of which sit in `public` itself where no
schema filter can reach them; a user's own views are never extension-owned, so they stay in the
inventory. Measured live on 2026-09-07 with two user tables seeded: 46 → 2 object rows on
TimescaleDB, 67 → 2 on Cloudberry and 70 → 2 on AlloyDB Omni.
Selectors are quoted with `quoteLiteral` because `queryReadOnly` binds no parameters, and a selector
carrying a backslash is refused outright rather than quoted — the dialect-less span reader treats it
as an escape, so `'a\'` would read as an unterminated literal.
- **A run reads three catalog inventories at its start (#329 T8), not one.** `inspect_schema` takes
a `kind` — `columns` (the default), `relations` (foreign keys, from `pg_constraint` with
`unnest(conkey, confkey) WITH ORDINALITY` pairing the two sides) and `indexes` (from `pg_index`
Expand Down
155 changes: 151 additions & 4 deletions src/lib/agent/composed-sql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,119 @@ function equalsClause(column: string, value: string | undefined, field: string,
return ` AND ${column} = ${quoteLiteral(assertSelector(value, field), dialect)}`;
}

// ─── PostgreSQL internal-object exclusion ─────────────────────────────────

/**
* The engine-builtin schemas a PostgreSQL grounding read must not treat as user
* data. This is the SAME set the provider's object browser uses
* (`SYSTEM_SCHEMAS` in `src/lib/db/providers/sql/postgres.ts`), copied rather
* than imported because the agent side must not depend on the provider module.
* `tests/unit/lib/agent/composed-sql.test.ts` pins the two lists together so
* they cannot drift. Each group is sorted by engine and checked against that
* engine's own documentation or, where the doc is silent, a live instance.
*/
const POSTGRES_SYSTEM_SCHEMAS = [
// PostgreSQL itself.
"pg_catalog",
"information_schema",
"pg_toast",
// Materialize - materialize.com/docs/sql/system-catalog/
"mz_catalog",
"mz_internal",
"mz_introspection",
// CockroachDB - cockroachlabs.com/docs/stable/system-catalogs enumerates exactly
// four schemas; the two below are the ones stock PostgreSQL does not also have.
"crdb_internal",
"pg_extension",
// TimescaleDB - the extension's own sql/pre_install/schemas.sql creates all seven.
// `_timescaledb_internal` is the one that floods: it holds every hypertable chunk.
"_timescaledb_catalog",
"_timescaledb_config",
"_timescaledb_functions",
"_timescaledb_internal",
"_timescaledb_cache",
"timescaledb_experimental",
"timescaledb_information",
// Apache Cloudberry - cloudberry.apache.org create-and-manage-schemas documents the
// first three. `pg_ext_aux` is not in that page but holds the PAX auxiliary tables
// (pg_pax_tables, pg_pax_fastsequence) on a live 2.1.0 instance, so it is here on
// measurement rather than on the doc's authority.
"gp_toolkit",
"pg_aoseg",
"pg_bitmapindex",
"pg_ext_aux",
] as const;

// Rendered once. Callers interpolate this into a `NOT IN (...)` clause.
const POSTGRES_SYSTEM_SCHEMA_LIST = POSTGRES_SYSTEM_SCHEMAS.map((schema) => `'${schema}'`).join(", ");

/**
* Schemas an EXTENSION created. AlloyDB Omni is deliberately absent from the
* name list above: its `google_ml` and `ai` schemas are extension-created, and
* those are names a user could plausibly choose for schemas of their own, so
* hiding them by name would make the user's tables vanish. Ownership is the
* question the name was standing in for, and `pg_depend` answers it directly.
* Same query the provider composes (`EXTENSION_OWNED_SCHEMAS_SQL`).
*/
const POSTGRES_EXTENSION_OWNED_SCHEMAS_SQL =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provider does not read pg_depend / pg_extension unguarded. It carries a fallback: isMissingExtensionCatalogError -> withoutExtensionOwnershipTest (src/lib/db/providers/sql/postgres.ts:458 and :484), because Materialize raises on those catalogs.

The agent path has no equivalent. composeCatalogRead returns one composed statement and there is no rewrite chain around it, so on a connection typed postgres that is actually Materialize (and postgres is in AGENT_EXECUTION_ENGINES) all four catalog reads fail: the context capture goes silently unavailable and inspect_schema returns a DB failure.

The live verification covered TimescaleDB, Cloudberry and AlloyDB Omni, all of which do have pg_depend, so it could not see this. Either mirror the provider's fallback or state the Materialize exclusion explicitly.

"SELECT n.nspname FROM pg_namespace n " +
"JOIN pg_depend d ON d.objid = n.oid AND d.classid = 'pg_namespace'::regclass AND d.deptype = 'e' " +
"JOIN pg_extension e ON e.oid = d.refobjid";

/**
* RELATIONS (tables, views, materialized views, sequences) an extension created
* — the B76 sharp case. AlloyDB Omni installs 49 of its 67 extension views into
* `public` itself, where no schema filter can reach them, so the OBJECT must be
* excluded by ownership rather than by where it sits. This is the same
* `pg_depend` ownership test with `classid = 'pg_class'::regclass`: it returns
* the extension's own relations and nothing a user created, so a user's own
* views survive.
*/
const POSTGRES_EXTENSION_OWNED_RELATIONS_SQL =
"SELECT n.nspname, c.relname FROM pg_class c " +
"JOIN pg_namespace n ON n.oid = c.relnamespace " +
"JOIN pg_depend d ON d.objid = c.oid AND d.classid = 'pg_class'::regclass AND d.deptype = 'e' " +
"JOIN pg_extension e ON e.oid = d.refobjid";

/**
* The full "this schema is not the engine's own" test for one column: a fixed
* list of engine-builtin schemas, plus anything an extension created. Mirrors
* the provider's `schemaExclusion(column)`.
*/
function postgresSchemaExclusion(column: string): string {
return `${column} NOT IN (${POSTGRES_SYSTEM_SCHEMA_LIST}) AND ${column} NOT IN (${POSTGRES_EXTENSION_OWNED_SCHEMAS_SQL})`;
}

/**
* The relation-level ownership test for one (schema, table) pair. Every
* PostgreSQL catalog read carries it, so `inspect_schema` answers the same
* object set whatever `kind` it is asked for: a relation an extension created
* is absent from columns, indexes, statistics and relations alike, and only a
* user's own objects survive.
*/
function postgresRelationExclusion(schemaColumn: string, tableColumn: string): string {
return `(${schemaColumn}, ${tableColumn}) NOT IN (${POSTGRES_EXTENSION_OWNED_RELATIONS_SQL})`;
}

/**
* The same statement with both ownership tests removed, leaving the fixed
* engine-builtin schema list alone. This is the agent-side mirror of the
* provider's `withoutExtensionOwnershipTest`: the driver serves PostgreSQL-wire
* engines nobody here has run, and one without `pg_depend`/`pg_extension`
* raises on those names, so the read retries with the fixed list alone. Kept as
* a pure string rewrite so the caller decides when to use it.
*/
export function withoutExtensionOwnershipTest(sql: string): string {
const withoutSchemas = sql.replace(
/\s+AND\s+[\w.]+ NOT IN \(SELECT n\.nspname FROM pg_namespace n JOIN pg_depend[^)]*\)/g,
"",
);
return withoutSchemas.replace(
/\s+AND\s+\([\w.]+,\s*[\w.]+\) NOT IN \(SELECT n\.nspname,\s*c\.relname FROM pg_class c JOIN pg_namespace n[^)]*\)/g,
"",
);
}

/**
* The column inventory, one row per TABLE with its columns aggregated (B52).
*
Expand Down Expand Up @@ -189,14 +302,43 @@ function equalsClause(column: string, value: string | undefined, field: string,
* bounds the prompt independently (`MAX_COLUMNS_PER_TABLE`,
* `AGENT_CONTEXT_PACK_MAX_CHARS`), so a wide table does not spend the context
* window on itself.
*
* This change closes B76: the aggregation that fixed the wide-catalog refusal
* then admitted the image's own objects. The WHERE clause below now excludes
* them three ways — the full engine-builtin schema list plus every schema an
* extension created (both copied from the provider's object browser), AND every
* RELATION an extension created (`pg_depend` with `classid = 'pg_class'::regclass`,
* new on the agent path). The relation test is the load-bearing half on AlloyDB
* Omni, whose 67 extension views mostly sit in `public` itself: no schema
* filter can reach them, an ownership test can, and a user's own views are
* never extension-owned so they survive.
*
* Every PostgreSQL catalog read carries the relation test, not only this one:
* `inspect_schema` serves `indexes`, `statistics` and `relations` straight to
* the model without passing them through `buildPostgresTables`, so a read that
* filtered by schema alone would show a PostGIS-owned `spatial_ref_sys` in the
* index and statistics inventories while hiding it from the column inventory.
* The exclusion is applied where each statement identifies its own table, and
* the four reads agree on the object set.
*
* Measured live on 2026-09-07 against the three `compat` images with two user
* tables seeded: the column read answers 46 → 2 object rows on TimescaleDB,
* 67 → 2 on Cloudberry and 70 → 2 on AlloyDB Omni. The sharp case is proven on
* AlloyDB, not assumed: the schema half alone leaves 51 objects (the 2 user
* tables plus the 49 extension views installed into `public`), `pg_depend`
* reports 68 extension-owned relations there, and the relation ownership test
* is what removes them. The committed tests pin the SQL SHAPE (the composed
* text carries each filter); the row counts above are the live BEHAVIOUR, and
* the two are asserted at different layers for that reason.
*/
function composePostgresCatalog(selector: AgentCatalogSelector): string {
return (
"SELECT table_schema, table_name, json_agg(json_build_object(" +
"'name', column_name, 'type', data_type, 'nullable', is_nullable) " +
"ORDER BY ordinal_position) AS columns " +
"FROM information_schema.columns " +
"WHERE table_schema NOT IN ('pg_catalog', 'information_schema')" +
`WHERE ${postgresSchemaExclusion("table_schema")}` +
` AND ${postgresRelationExclusion("table_schema", "table_name")}` +
equalsClause("table_schema", selector.schema, "schema", "postgres") +
equalsClause("table_name", selector.table, "table", "postgres") +
" GROUP BY table_schema, table_name ORDER BY table_schema, table_name"
Expand Down Expand Up @@ -250,7 +392,9 @@ function composePostgresRelations(selector: AgentCatalogSelector): string {
"JOIN LATERAL unnest(c.conkey, c.confkey) WITH ORDINALITY AS k(attnum, fattnum, ord) ON true " +
"JOIN pg_attribute att ON att.attrelid = c.conrelid AND att.attnum = k.attnum " +
"JOIN pg_attribute fatt ON fatt.attrelid = c.confrelid AND fatt.attnum = k.fattnum " +
"WHERE c.contype = 'f' AND rn.nspname NOT IN ('pg_catalog', 'information_schema')" +
"WHERE c.contype = 'f' AND " +
postgresSchemaExclusion("rn.nspname") +
` AND ${postgresRelationExclusion("rn.nspname", "rel.relname")}` +
equalsClause("rn.nspname", selector.schema, "schema", "postgres") +
equalsClause("rel.relname", selector.table, "table", "postgres") +
" ORDER BY rn.nspname, rel.relname, k.ord"
Expand Down Expand Up @@ -302,7 +446,8 @@ function composePostgresIndexes(selector: AgentCatalogSelector): string {
"JOIN pg_namespace n ON n.oid = t.relnamespace " +
"JOIN LATERAL unnest(ix.indkey) WITH ORDINALITY AS k(attnum, ord) ON true " +
"LEFT JOIN pg_attribute att ON att.attrelid = t.oid AND att.attnum = k.attnum " +
"WHERE n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast')" +
`WHERE ${postgresSchemaExclusion("n.nspname")}` +
` AND ${postgresRelationExclusion("n.nspname", "t.relname")}` +
equalsClause("n.nspname", selector.schema, "schema", "postgres") +
equalsClause("t.relname", selector.table, "table", "postgres") +
" ORDER BY n.nspname, t.relname, i.relname, k.ord"
Expand Down Expand Up @@ -355,7 +500,9 @@ function composePostgresStatistics(selector: AgentCatalogSelector): string {
"JOIN pg_namespace n ON n.oid = c.relnamespace " +
"LEFT JOIN pg_stats s ON s.schemaname = n.nspname AND s.tablename = c.relname " +
"WHERE c.relkind IN ('r', 'p') " +
"AND n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast')" +
"AND " +
postgresSchemaExclusion("n.nspname") +
` AND ${postgresRelationExclusion("n.nspname", "c.relname")}` +
equalsClause("n.nspname", selector.schema, "schema", "postgres") +
equalsClause("c.relname", selector.table, "table", "postgres") +
" ORDER BY n.nspname, c.relname, s.attname"
Expand Down
34 changes: 32 additions & 2 deletions src/lib/agent/tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ import {
AgentComposedSqlError,
composeCatalogRead,
composeEstimatingExplain,
withoutExtensionOwnershipTest,
} from "./composed-sql";
import type { AgentDeadlineDenyCode, AgentRunDeadline } from "./deadline";
import {
Expand Down Expand Up @@ -2271,6 +2272,18 @@ export async function readStatementForGrounding(
});
}

/**
* Whether a database error names the two catalogs the extension ownership tests
* join against. A postgres-typed connection can reach an engine the driver
* serves but nobody here has run, and one without those catalogs raises on
* them; the catalog read then retries without the ownership tests — the same
* decision the provider makes in `isMissingExtensionCatalogError`.
*/
function isMissingExtensionCatalogError(message: string): boolean {
const lowered = message.toLowerCase();
return lowered.includes("pg_depend") || lowered.includes("pg_extension");
}

async function readCatalog(
context: AgentToolContext,
selectorSchema: z.ZodType<{ kind?: AgentCatalogKind; schema?: string; table?: string }>,
Expand All @@ -2286,7 +2299,7 @@ async function readCatalog(
} catch (error) {
return composedSqlOutcome(error);
}
const outcome = await executeAgentOperation(context, {
const request: AgentOperationRequest = {
operationId: "sql.query.read",
sql,
grounding,
Expand Down Expand Up @@ -2317,7 +2330,24 @@ async function readCatalog(
// case-sensitively, so declaring a raw `MAIN` would compose fine and then be
// denied against a `["main"]` allowlist.
...(selector.schema === undefined ? {} : { target: { schema: normalizeDeclaredSchema(context, selector.schema) } }),
});
};
let outcome = await executeAgentOperation(context, request);

// A postgres-typed connection can reach an engine the driver serves but nobody
// here has run, one without the two catalogs the ownership tests join against
// (it raises on pg_depend/pg_extension). The provider drops the tests on that
// error and retries with the fixed schema list alone; mirror it here.
if (
context.connection.type === "postgres" &&
outcome.kind === "refused" &&
outcome.refusal.class === "database-error" &&
isMissingExtensionCatalogError(outcome.refusal.message)
) {
const fallback = withoutExtensionOwnershipTest(sql);
if (fallback !== sql) {
outcome = await executeAgentOperation(context, { ...request, sql: fallback });
}
}
/*
A catalog read that matched NO OBJECT is a refusal, not a result.

Expand Down
Loading
Loading