Skip to content

feat: add scoped introspection aligned with Crystal #2 - #1823

Open
Zetazzz wants to merge 2 commits into
mainfrom
feat/crystal2-scoped-introspection
Open

feat: add scoped introspection aligned with Crystal #2#1823
Zetazzz wants to merge 2 commits into
mainfrom
feat/crystal2-scoped-introspection

Conversation

@Zetazzz

@Zetazzz Zetazzz commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Small GraphQL APIs currently pay catalog introspection costs for unrelated schemas in the same PostgreSQL database. Add an opt-in, CNC-owned replacement plugin that follows service schemas and their dependency closure.

Supersedes #1719 and follows the configuration and query behavior in Crystal #2 (441cef73b0a12ed24343b7f38241af28b6454280).

Behavior

  • Copy the static parameterized SQL into graphile-scoped-introspection; no upstream patches or upgrades.
  • Configure per service with gather.pgScopedIntrospection: omitted/false uses stock, true uses scoped defaults (catalogTypes: all), and an options object supports explicit catalog policy and capability extensions.
  • Roots come from the actual PgService schemas. Dependencies cross schemas automatically; query parameters and validation share one normalized plan. Invalid options, unknown service names, and missing root schemas fail explicitly.
  • The server forwards existing graphile.preset / graphile.extends, loads the replacement lazily, preserves introspectionRole, and coalesces handler creation before async preset work.
  • Preserve upstream cache/watch/events and client release semantics. The scoped helper contract is pinned to graphile-build-pg@5.1.3; stock without scoped configuration does not load that contract guard.

CNC configuration:

graphile: {
  preset: {
    gather: { pgScopedIntrospection: { main: true } },
  },
}

Temporary copies and upstream migration

The copied scoped SQL, validation logic, and adapted introspection helper code are a temporary compatibility bridge for CNC's currently pinned Graphile dependencies. The intent is to consume the Crystal implementation upstream, rather than maintain a permanent CNC fork.

Once this functionality is merged into upstream Crystal and available in a compatible published release, upgrade CNC's dependencies and replace the local copies/replacement plugin with the upstream implementation. Retain only CNC-specific configuration wiring that is still needed, preserve the per-service configuration semantics where supported, and rerun the schema/runtime equivalence and default-performance checks before removing the bridge. If upstream changes the configuration contract, handle that migration explicitly.

Validation

  • 163 relevant Jest tests passed: plugin 28 (including real PostgreSQL), server 40 (wiring/single-flight plus current standing/document-gate regressions), env 11, perf harness 84.
  • CJS/ESM builds passed for the eight selected packages, including updated shared errors/context dependencies.
  • Frozen offline install passed; ESLint has zero errors (one existing unused-catch warning).
  • PostgreSQL 18.6 benchmark: 42 measured fresh processes, seven samples per case/scale; all schema hashes and actual table/relation/function checks passed.

Default performance

Stock versus main: true only; target fixed at 9 tables / 8 functions. Values are medians on a shared host.

Unrelated schemas Stock build Scoped build Build change
0 1.204 s 4.199 s +248.8%
10 2.395 s 4.023 s +68.0%
50 6.630 s 4.267 s -35.6%

At 50 unrelated schemas, retained heap drops from 103.2 to 26.5 MiB. The default scoped query is slower in smaller catalogs, so the result is workload-dependent and is not a universal speedup or a production latency claim.

The PR includes methodology and results, a reproducible runner, environment metadata, and individual samples.

@tenki-reviewer

tenki-reviewer Bot commented Sep 10, 2026

Copy link
Copy Markdown

Review complete. 🟡 1 medium

💬 Inline comments (1)

  • 🟡 Align version sentinel with peer rangeplugin.ts:64

This PR introduces graphile/graphile-scoped-introspection, a plugin that restricts catalog introspection to a declared set of schemas (plus their dependency closure) to cut introspection cost on large databases, and wires it into the GraphQL server via new middleware and graphql/types exports, with a perf-harness benchmark comparing stock vs scoped modes.

Files Change
graphile-scoped-introspection/src/* New plugin, query builder, and index implementing scoped catalog introspection with single-flight caching
graphile-scoped-introspection/__tests__/* Test suites for plugin contracts, query validation, cache lifecycle, runtime, and mixed modes
graphql/server/src/middleware/* Lazy-loading scoped preset middleware and updated graphile handler wiring
graphql/types/src/* New exported types for scoped introspection configuration
packages/perf-harness/* Benchmark suite, worker, and sample data for scoped introspection

Note: the high-recall sweep pass did not publish output (hit its step cap), so coverage relies on the lens, domain, and holistic passes; the verifier adjudicated 7 candidates (4 rejected, 2 merged).

Reviewed commit: 66c6bf8

@tenki-reviewer tenki-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adds a schema-scoped PostGraphile introspection plugin with a new graphile-scoped-introspection package, server middleware for lazy preset loading, type exports, and a perf-harness benchmark suite.

Key findings

  • 🟡 Align version sentinel with peer rangeplugin.ts:64

Comment thread graphile/graphile-scoped-introspection/src/plugin.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant