Skip to content

agent, control-plane-api: publications authorize via Snapshot - #3387

Open
bbartman wants to merge 14 commits into
masterfrom
bmb/2781-publications-snapshot-authz
Open

agent, control-plane-api: publications authorize via Snapshot#3387
bbartman wants to merge 14 commits into
masterfrom
bmb/2781-publications-snapshot-authz

Conversation

@bbartman

Copy link
Copy Markdown
Contributor

Part of #2781. Migrates publications' authorization enforcement off internal.user_roles() SQL and onto the in-process authorization Snapshot, following the pattern established for discovers in #3373 and #3386.

This first commit is the red-green test phase:

  • Three characterization tests pin today's SQL-enforced behaviors, which must survive the swap unchanged: drafting a spec requires admin to its name; a data-plane the user cannot read is indistinguishable from a missing one (both when explicitly requested and when reached as a storage mapping's default plane); and an unreadable-but-unused storage-mapping plane is silently tolerated.
  • One test is committed RED: test_publication_no_data_plane_requests_snapshot_refresh asserts that a denied data-plane cancels the Snapshot's revoke token to request an early background refresh, which the SQL path never does. Following commits turn it green.

The full migration plan is in the first comment below. Draft until the remaining commits land.

@bbartman

bbartman commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Migration plan: publications onto the authorization Snapshot

RLS surface reachable from publications

Every internal.user_roles() evaluation in the publications call chain is reached through three touchpoints, all publications-private — no function copies are needed:

  1. resolve_live_specs — user capability per catalog name. fetch_live_specs(..., fetch_user_capabilities=verify_user_authz, ...) evaluates a materialized user_roles($1) CTE; the Rust side then enforces admin-on-drafted-names and read-on-referenced-names. The SQL is already flag-gated, and every other caller passes false.
  2. resolve_live_specs — data-plane name read authorization. The data_plane_names CTE filter (marked for strangling in agent: discovers authorize data-planes via Snapshot #3386). Note it runs unconditionally — controller publications (verify_user_authz: false) also rely on the system user passing this filter, so the Snapshot-based replacement applies to them identically, as PGControlPlane::discover already does.
  3. ExpandDraftget_connected_live_specsfetch_expanded_live_specs. Computes user_capability per expanded row; Rust filters to admin. Single caller, single construction site.

Verified clean: the commit path (quotas, persist_updates, verify_unchanged_revisions), PruneUnboundCollections, connector image/annotation checks, resolve_inferred_schemas, and draft::load_draft have no user-scoped predicates. Out of scope: spec_capabilities (plain role_grants join, spec-to-spec authz, not user RLS), the RLS insert policies on publications/drafts (PostgREST-side), and the other user_roles users (storage-mappings directive, create_data_plane/update_l2_reporting ops/ checks, GraphQL).

Design decisions

  • Snapshot threading follows the Discover precedent: DraftPublication carries a pinned &Snapshot; PublicationsExecutor gains a snapshot_watch and pins one Snapshot per poll, so authorization cannot flip between expand, build, and retry attempts. The other DraftPublication sites (PGControlPlane::publish, create_data_plane, update_l2_reporting, test harness) update mechanically — PGControlPlane and the harness already hold the watch.
  • ExpandDraft holds the same pinned snapshot as a field rather than changing the Initialize trait signature.
  • In-process equivalents: tables::UserGrant::get_user_capability is a drop-in for the SQL max(capability) where starts_with(...); UserGrant::is_authorized(..., Capability::Read) replaces the data-plane name filter. Error strings are preserved exactly; a denied data-plane name stays indistinguishable from a missing one.
  • Accepted behavior divergence (Storage mapping wizard private plane grants #2848): user_roles() only walks the grant graph downward, while Snapshot authorization also walks upward to ancestor subjects. Sub-prefix admins therefore become correctly more permissive against ancestor-subject grants (private data-plane prefixes in practice). This is the intended fix, not a regression; the characterization tests pin mainline behavior only.
  • Fetch/decide split: live_specs fetches are pure data reads; authorization is evaluated in Rust at each decision site against the pinned Snapshot, never synthesized onto rows as capability fields (the field synthesis attempted in Update validation/publication to use snapshots #3155 is what produced its legacy-capability-promotion review finding). Authorize-before-fetch applies only where the query keys are user-chosen names and a denial must be indistinguishable from absence — data-plane names. This converges on the same end-state shape as control-plane, agent: move catalog authorization in-process against a pinned Snapshot (2/7) #3342's get_connected_live_specs, minimizing rebase friction with the stack.
  • Staleness: on any Snapshot-evaluated authorization denial that fails the publication — a drafted name the user can't admin, a referenced name they can't read, or a denied data-plane — cancel the Snapshot's revoke token to request an early background refresh, so a manual retry after a fresh grant succeeds sooner. (Data-plane denial matches discovers; the user-capability cancels go further, since under SQL those checks were never stale.) ExpandDraft's admin filter does not cancel: filtering merely narrows expansion and doesn't fail the publication. The full stale-snapshot deferral (PublicationState.awaiting_snapshot_after, already pre-staged as dead code) stays a follow-up PR.

Commit sequence

  1. ✅ Red-green tests pinning current publication authorization behavior (this PR's first commit).
  2. ✅ Thread the pinned Snapshot: PublicationsExecutor gains snapshot_watch, DraftPublication carries the snapshot; mechanical updates at the other construction sites. No behavior change. (fc23a15d70b)
  3. resolve_live_specs user capabilities via Snapshot: flip fetch_user_capabilities to false, compute with get_user_capability; revoke.cancel() when a denial pushes an authorization error. (d3d2fe3e262)
  4. ✅ Data-plane names via Snapshot: delete the marked SQL block, partition names in-process, revoke.cancel() on denial (turns the RED test green). (4199c8291bb)
  5. ExpandDraft admin filter in-process: fetch_expanded_live_specs becomes a pure data fetch — drop user_id, the internal.user_roles() subquery, and the role_grants subquery (nothing on the expansion path reads spec_capabilities; select a '[]' placeholder while the shared row struct still carries the field). get_connected_live_specs takes the pinned &Snapshot and filters per row via tables::UserGrant::get_user_capability. No revoke.cancel(): filtering narrows expansion, it doesn't fail the publication. (7180bb21602)
  6. ✅ Strangle-complete cleanup: no caller passes fetch_user_capabilities=true anymore, so remove the flag, the user_roles CTE, and the user_id parameter from fetch_live_specs (it fed only the CTE), and delete the now-reader-less user_capability field from LiveSpec. spec_capabilities and its role_grants join stay, resolve-path only: spec-to-spec authz remains out of scope. (1e141e363ea)

After this PR, the remaining internal.user_roles users are the storage-mappings directive, the ops/ admin checks in create_data_plane/update_l2_reporting, and the PostgREST _ext-view estate.

@bbartman

Copy link
Copy Markdown
Contributor Author

Decision: Snapshot grant-walk helpers (review follow-up)

Code review flagged the recurring argument bundle (&snapshot.role_grants, &snapshot.user_grants, user_id, name) at this PR's authorization decision sites as a data clump / message chain.

Decision: hide the walk behind methods on the object that owns the data — Snapshot::user_capability(user_id, name) and Snapshot::is_user_authorized(user_id, name, capability) — and convert only the call sites this PR introduces (the three resolve_live_specs checks and the get_connected_live_specs expansion filter). The pre-existing bare tables::UserGrant::* call sites (server/*.rs, discovers, live_specs::partition_by_authorization) are deliberately left as-is and migrate opportunistically as later #2781 slices touch them — same strangler discipline as the rest of the series. This also pre-lands the end-state shape from the stack-2 reference implementation, which put these helpers on Snapshot.

@bbartman
bbartman marked this pull request as ready for review August 21, 2026 14:07
@bbartman

Copy link
Copy Markdown
Contributor Author

One of the other things that is absent here is that we still need to refactor out spec_capabilities but I'm leaving that for another PR/follow up.

Test-first phase for moving publications' authorization enforcement off
internal.user_roles() SQL and onto the pinned authorization Snapshot
(issue #2781), mirroring the discovers migration (#3373, #3386).

Harness: async_publication takes a caller-chosen data-plane name, with
user_publication_in_plane exposing it; user_publication and
create_user_publication delegate with the default test plane.

Four new integration tests:
- test_publication_drafted_name_requires_admin,
  test_publication_no_data_plane, and
  test_publication_storage_mapping_unreadable_plane are
  characterization tests pinning today's SQL behavior: drafting a spec
  requires admin on its name; a denied data-plane is indistinguishable
  from a missing one, whether requested explicitly or reached as a
  storage mapping's default plane; and an unreadable-but-unused mapping
  plane is silently tolerated. They pass now and must keep passing
  after the swap.
- test_publication_no_data_plane_requests_snapshot_refresh asserts the
  new Snapshot-based behavior and is committed RED: the SQL path never
  cancels the Snapshot's revoke token. Following commits turn it green.
Thread the authorization Snapshot through the publications call chain,
ahead of moving publications' authorization enforcement off
internal.user_roles() SQL and onto the Snapshot (issue #2781),
mirroring the discovers migration (#3373, #3386). No behavior change:
the Snapshot is carried but not yet consulted.

- DraftPublication carries a Snapshot pinned for the entire
  publication, so that once the following commits consult it,
  authorization decisions cannot flip between initialization, build,
  and retry attempts.
- ExpandDraft holds the same pinned Snapshot as a field -- rather than
  changing the Initialize trait signature -- as it's the only
  Initialize which evaluates authorization.
- PublicationsExecutor gains a snapshot_watch and pins one Snapshot
  per poll, as DiscoverExecutor does.
- The remaining construction sites update mechanically:
  PGControlPlane::publish pins from its watch; the create-data-plane
  and update-l2-reporting handlers use the request Envelope's pinned
  Snapshot; and the test harness refreshes the shared watch before
  each publications poll (as it already did for discovers), which
  makes the RED refresh test's explicit refresh redundant.
…apshot

Evaluate resolve_live_specs' two user-capability decisions -- admin on
drafted names and read on referenced names -- in-process against the
pinned Snapshot via tables::UserGrant::get_user_capability, instead of
the internal.user_roles() CTE of fetch_live_specs (issue #2781). The
fetch's user-capability flag is now always false; removing it outright
is left to the final cleanup commit, as publications were its last user.

Either denial now cancels the pinned Snapshot's revoke token, following
the discovers precedent: the needed grant may have been created after
the Snapshot was taken, and requesting an early background refresh
narrows the staleness window for a manual retry.
test_publication_drafted_name_requires_admin gains assertions pinning
this: a denial cancels, and a successful publication does not.

Publisher::build gains the pinned &Snapshot, threaded from
DraftPublication through try_publish and into resolve_live_specs.
Tests which call Publisher::build directly bypass the publications
executor, so they now refresh and pin a Snapshot after tenant setup.

Accepted divergence (#2848): Snapshot evaluation also walks upward to
ancestor grant subjects, so sub-prefix admins become correctly *more*
permissive than the replaced SQL.
Replace the last internal.user_roles() evaluation in the publications
path -- the data_plane_names CTE filter of resolve_live_specs -- with
in-process evaluation against the pinned Snapshot (issue #2781),
completing for publications what discovers established in #3386.

Names from storage mappings and the explicit build parameter are
partitioned via tables::UserGrant::is_authorized at Capability::Read,
and only authorized names reach the SQL fetch, so a denied plane stays
exactly as invisible as a missing one. As with the SQL it replaces,
the check applies unconditionally: controller publications by the
system user included. Planes referenced by id are fetched unchecked,
as those ids come from live specs which were themselves authorized.

A denied name cancels the pinned Snapshot's revoke token, following
the discovers precedent: the grant may have been created after the
Snapshot was taken, and requesting an early background refresh narrows
the staleness window for a manual retry. The cancel fires at partition
time, before it's known whether the denied plane is actually used, so
an unreadable-but-unused storage-mapping plane also triggers it: the
publication still succeeds, at the cost of one early refresh.

Turns the remaining RED test from the first commit green:
test_publication_no_data_plane_requests_snapshot_refresh. The three
characterization tests keep passing unchanged, and the .sqlx entry for
the rewritten query is regenerated.
Drop the per-row `internal.user_roles()` and `role_grants` sub-queries from
`fetch_expanded_live_specs`, making it a pure data fetch. The caller now
evaluates the user's capability over each expanded row in-process against
the pinned authorization Snapshot, consistent with the rest of the
publications authorization path.
…pecs

No caller passes fetch_user_capabilities any more: user capability is
evaluated in-process against the pinned authorization Snapshot at every
decision site. Remove the flag, the internal.user_roles() CTE, and the
user_id bind from fetch_live_specs, and delete the now-reader-less
user_capability field from LiveSpec (with its placeholder in
fetch_expanded_live_specs).

get_live_specs_unfiltered also drops its user_id parameter, which
existed only to feed the removed bind.

spec_capabilities and its role_grants join stay, resolve-path only:
spec-to-spec authorization remains out of scope.
Retense the migration-era test comments (the characterization header and
the formerly-RED refresh test), fix the '[]' spec_capabilities rationale
(the column simply has no reader; expansion output is re-fetched on the
resolve path), qualify the plane-by-id exemptions (injected ops
collections and unverified publications), and correct the
user_publication_in_plane doc. The sqlx cache swap follows the SQL
comment edit.
The awaiting_snapshot_after member was pre-staged for a stale-snapshot
deferral mechanism that is now cancelled for good (see #3389, which
emptied DiscoverState for the same reason). The empty struct remains so
that persisted task state stays decodable across a deploy in either
direction.
Snapshot::user_capability and Snapshot::is_user_authorized wrap the
tables::UserGrant evaluators over the Snapshot's own grant tables, so
publication decision sites no longer thread (&snapshot.role_grants,
&snapshot.user_grants, ...) by hand. Only this PR's call sites are
converted; the pre-existing bare-call estate migrates opportunistically
with later #2781 slices.
Snapshot::request_refresh names what denial sites mean — request an
early background refresh — rather than how it happens (cancelling the
revoke token), and homes the staleness rationale that was previously
duplicated across resolve_live_specs' denial sites.
TestHarness::pinned_snapshot homes the refresh-then-pin ritual (and its
rationale) used by tests that call Publisher::build directly, and
add_private_plane homes the unreadable-plane fixture shared by the
publication authorization tests.
Close the asymmetric coverage of Snapshot refresh requests: a denied
referenced name cancels; an unused-but-denied storage-mapping plane
still cancels eagerly at partition time despite the publication
succeeding; and neither spec-to-spec denials nor expansion filtering
cancel.
Fixes comments flagged by a review of the branch as stale or misplaced,
so each describes present behavior rather than the migration's history:

- user_publications.rs: the storage-mapping pinning rationale sat on
  add_private_plane; move it onto pin_storage_mapping_planes, which it
  actually describes.
- harness.rs: the manually-driven Snapshot watch now serves both the
  discovers and publications executors; the construction-site comment
  and the GraphQL-timeout note still claimed it was discovers-only.
- publications/mod.rs: verify_user_authz: false skips user permission
  checks, but spec-to-spec capability checks and the data-plane name
  read filter still apply; the doc claimed all checks were skipped.
- specs.rs: state the data-plane filter's unconditionality as a present
  invariant instead of narrating the SQL filter it replaced.
test_publication_system_user_data_plane_filter drives
TestControlPlane::publish -- the system user with verify_user_authz:
false, as controllers publish -- against a plane outside the system
user's grants, pinning that the data-plane read filter is evaluated
unconditionally: the denial fails identically to a missing plane and
cancels the pinned Snapshot's revoke token, while the system user's
ops/ grant admits the shared test plane. TestControlPlane gains a
snapshot_refresh accessor because cancellation must be observed on
the very Snapshot the publish pinned: a cancelled revoke makes the
watch's next token() fetch a fresh one.

Review cleanups alongside:
- The publish sites bind the watch token once and deref inline,
  dropping their second snapshot bindings.
- The data-plane partition uses the file's imported bare Capability
  idiom.
- Delete the reader-less ExpandedRow from db_complete.
- Rename the harness's discover_snapshot_* watch plumbing to
  executor_snapshot_*: it serves the publications executor too.
@bbartman
bbartman force-pushed the bmb/2781-publications-snapshot-authz branch from 3bc000f to 65d3b80 Compare August 27, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant