agent: discovers require SpecEdit to the capture

[bbartman]

What

Next slice of the #2781 strangler series (following #3373 and #3386): the discovers executor now proves the user's authorization over the specs a discover touches, rather than only filtering. All decisions evaluate against the Snapshot pinned for the operation.

• SpecEdit on the capture, hard-fail: an unconditional precheck on capture_name runs ahead of data-plane resolution. A user without SpecEdit gets the new terminal JobStatus::NotAuthorized plus a single grant-based draft error, and the Snapshot's revoke token is cancelled to request an early background refresh (the NoDataPlane precedent).
• Live-capture merge-base fetch filters at SpecEdit | CatalogRead: the fetch also discloses the live model into the user's draft, and disclosure is CatalogRead's axis. Every real bundle conveying SpecEdit conveys CatalogRead too, so this is a no-op in practice — it keeps the fetch's read semantics locally correct rather than leaning on that bundling convention.
• Collections keep the silent CatalogRead filter, verbatim: an unauthorized binding-target collection remains indistinguishable from a nonexistent one (control-plane, agent: discovers authorize live specs via Snapshot #3373's doctrine).
• The auto-discover path (filter_user_authz: false, system user) is untouched.

Reviewer callout: behavior change for CatalogRead-only users

Previously a user lacking edit authority on the capture had it silently drafted as brand-new (filtered-as-absent), with the authorization error surfacing only at publish. Now the discover fails up-front with NotAuthorized. This is a deliberate security-posture change: a user who cannot SpecEdit the capture can never publish the result, so silent success was a UX trap, and the up-front check is cheaper than a doomed connector RPC.

This does not create an existence oracle: the check is a pure function of the grant graph and the requested name — its outcome is identical whether or not a spec exists at that name, which test_discover_not_authorized_capture pins by asserting byte-identical outcomes for an existing and a nonexistent unauthorized name. (The data-plane check, by contrast, must conflate unauthorized-with-missing because it is existence-conditional; this check never looks anything up.)

Also note the legacy-capability fallout, pinned by test: legacy read (Viewer) and write (Writer) grants convey CatalogRead but not SpecEdit and are now refused; admin conveys SpecEdit and succeeds. Nothing usable is lost — publications already require admin.

Scope note: secrets in specs

CatalogRead remains the platform's spec-read boundary. A discover merge copies CatalogRead-visible live collection models (including any derive section) into the user's draft — the identical surface flowctl catalog pull-specs already grants, so this PR neither widens nor narrows it. Secret-in-spec exposure (unencrypted configs, plaintext redactSalt) is explicitly out of scope here and is owned by #3366 (first-class secrets) and #2963 (redactSalt encryption).

No retry mechanism, by design

There is deliberately no stale-snapshot deferral or retry machinery, now or planned: an authorization failure is terminal, the revoke cancellation narrows the staleness window to roughly MIN_REFRESH_INTERVAL, and the error text asks the user to retry in a moment. DiscoverState, which was carried as dead state for a planned deferral mechanism (#3279), is accordingly emptied.

UI

The dashboard will render job_status.type == "notAuthorized" as an unknown status until it learns the new variant; the draft error text is the user-facing message in the interim. The wire tag is pinned by a serde test.

Tests

Written first, red-green (each phase is its own commit):

• test_discover_not_authorized_capture: terminal NotAuthorized, grant-based draft error, cancelled revoke token — asserted identically for existing and nonexistent unauthorized names (the no-oracle guard).
• test_discover_capture_requires_spec_edit: read → refused, write → refused, admin → success, via a bare user holding a single direct grant.
• test_discover_not_authorized_wins_over_missing_plane: the SpecEdit check precedes data-plane resolution.
• test_job_status_not_authorized_serde: pins the UI-facing wire tag.
• Removed test_discover_filters_unauthorized_capture, which pinned the prior filter semantics.
• Unchanged and still green: test_discover_merge_filters_unauthorized_collection (the collections filter), test_user_discovers, test_discover_no_data_plane, created_at, and the auto_discovers suite.

🤖 Generated with Claude Code

[bbartman]

Plan of record

The negotiated plan this PR implements, with each decision and its rationale.

Goal. The discover flow proves the user's authorization over the specs it reads, against the pinned Snapshot: SpecEdit on the capture being discovered (the spec the discover validates and edits), and CatalogRead on all binding-target collections.

Decisions

1. Secret-exposure posture: CatalogRead remains the platform's spec-read boundary. A collection spec can embed a connector config (derivations: derive.using.connector.config) and redact_salt, and the merge copies CatalogRead-visible live models whole into the user's draft — but this is the identical surface pull-specs already grants. Out of scope here; owned by First-Class Secrets in Estuary #3366 / Support encryption of redactSalt in capture/derivation specs #2963.
2. Filter or fail: hard-fail on the capture's SpecEdit check (prefix-based, leaks nothing, runs before the connector RPC, and a user who can't SpecEdit the capture can never publish the result). Collections keep the silent CatalogRead filter — binding targets are only fully known after the connector responds, and an existing capture can legitimately carry bindings into prefixes the discovering user can't read; a hard failure there would also be an existence oracle.
3. Failure shape: new JobStatus::NotAuthorized variant plus one grant-based draft error, so the UI shows something actionable before it learns the new status type. Guardrails: error wording speaks only to grants (never "exists but not accessible"), and the check is unconditional — never nested inside an "if a live spec exists" branch. Both are pinned by tests.
4. Precheck mechanics: SpecEdit check runs first, before the data-plane check (most fundamental gate, most actionable signal); NotAuthorized cancels snapshot.revoke like NoDataPlane does; the live-capture fetch filters at SpecEdit | CatalogRead — the hard check gates the operation, the fetch filter carries the disclosure semantics, so a hypothetical SpecEdit-without-CatalogRead grant may discover but has the live model treated as absent.
5. Staleness: no deferral or retry mechanism, now or ever. revoke.cancel() narrows the staleness window; the error text asks the user to retry in a moment. DiscoverState is emptied accordingly (its awaiting_snapshot_after member existed only for the now-cancelled deferral plan).
6. Untouched: the merge-phase collections filter (verbatim), and the auto-discover / controller path (filter_user_authz: false, system user).

Test plan

• test_discover_not_authorized_capture — terminal status, grant-based error, cancelled revoke token; identical outcomes for an existing and a nonexistent unauthorized name (a future refactor that makes the check existence-conditional turns this red).
• test_discover_capture_requires_spec_edit — legacy grant loop: read and write refused (CatalogRead/Writer without SpecEdit), admin succeeds.
• test_discover_not_authorized_wins_over_missing_plane — pins check ordering.
• test_job_status_not_authorized_serde — pins the {"type":"notAuthorized"} wire tag.
• Negative space that must stay green: test_discover_merge_filters_unauthorized_collection, test_user_discovers, test_discover_no_data_plane, created_at, auto_discovers, test_prepare_discover.

Commit shape

Red-green, per the #3386 convention: commit 1 adds the NotAuthorized variant and all tests (verified red — the grant-loop test fails with NoDataPlane where NotAuthorized is expected); commit 2 adds the precheck, fetch change, and DiscoverState cleanup (all 14 discover-related tests green).

Follow-ups (outside this PR)

• estuary/ui: teach the dashboard the notAuthorized job status.
• First-Class Secrets in Estuary #3366 / Support encryption of redactSalt in capture/derivation specs #2963 own the secrets-in-specs exposure this PR deliberately does not change.

[jgraettinger]

Some comments.

I would prefer that fetching live collections is also a hard error if CatalogRead isn't met, instead of silently filtering, but am happy with tackling SpecEdit for the capture alone if that's an easier change to ship.