From 0c35ecb58b5e0181fd61b6b4e98cc36f8f79dd02 Mon Sep 17 00:00:00 2001 From: Pamela Chia Date: Mon, 2 Nov 2026 12:08:47 +0000 Subject: [PATCH 001/101] fix(cli): drop entitlements lookup at enveloped plan gates (#7395) `branches create`, `vanity-subdomains activate`, and `vanity-subdomains check-availability` ran an entitlements lookup (get project, then get organization entitlements) after any 4xx without the `entitlement_required` envelope. The Management API already returns that envelope on all three gates, so the lookup only ran for errors unrelated to the plan: two extra requests, and on an org without the feature, an upgrade hint and a `plan_limit` classification for an unrelated error such as a validation failure. I removed the lookup and its permission declarations at those three commands. The CLI still prints the upgrade hint from the envelope on gated denials and fires `cli_upgrade_suggested` for `branches create` and `vanity-subdomains activate` (`check-availability` still suppresses the event), unchanged since #7271. **Note:** I left `branches update` and the `sso` commands on the lookup for now: the SAML routes return no envelope yet, and I haven't confirmed that the persistent-branch gate on `branches update` returns one. ## Linear - part of GROWTH-1050 --- .../commands/branches/create/SIDE_EFFECTS.md | 20 ++- .../branches/create/create.command.ts | 2 - .../branches/create/create.handler.ts | 26 ++-- .../create/create.integration.test.ts | 64 ++++----- .../activate/SIDE_EFFECTS.md | 6 +- .../activate/activate.command.ts | 2 - .../activate/activate.handler.ts | 1 - .../check-availability/SIDE_EFFECTS.md | 4 +- .../check-availability.command.ts | 2 - .../check-availability.handler.ts | 1 - .../vanity-subdomains.integration.test.ts | 124 ++++++++++-------- 11 files changed, 120 insertions(+), 132 deletions(-) diff --git a/apps/cli/src/commands/branches/create/SIDE_EFFECTS.md b/apps/cli/src/commands/branches/create/SIDE_EFFECTS.md index 001d674f95..a4b49f9ea3 100644 --- a/apps/cli/src/commands/branches/create/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/branches/create/SIDE_EFFECTS.md @@ -22,13 +22,11 @@ Target: hosted, consent default Yes — see [Command target](../../../../docs/co ## API Routes -| Method | Path | Auth | Request body | Response (used fields) | -| ------ | ----------------------------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `POST` | `/v1/projects/{ref}/branches` | Bearer token | `{branch_name, is_default: false, git_branch?, region?, desired_instance_size?, persistent?, with_data?, notify_url?}` | full `BranchResponse` | -| `GET` | `/v1/projects/{ref}` (on 4xx gated) | Bearer token | none | `{organization_slug}` | -| `GET` | `/v1/organizations/{slug}/entitlements` (gated) | Bearer token | none | `[{feature: {key}, hasAccess, ...}]` | -| `GET` | `/v1/projects` | Bearer token | none | `[{id, name, organization_slug, region, …}]` — no project ref from `--remote=`, `SUPABASE_PROJECT_ID` or `.temp/project-ref`, with stdin and stdout TTYs and a non-JSON output format (project picker) | -| `GET` | `/v1/projects/{ref}` | Bearer token | none | `{ref, name, organization_id, organization_slug}` — linked-project cache miss only (best-effort, after the command) | +| Method | Path | Auth | Request body | Response (used fields) | +| ------ | ----------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `POST` | `/v1/projects/{ref}/branches` | Bearer token | `{branch_name, is_default: false, git_branch?, region?, desired_instance_size?, persistent?, with_data?, notify_url?}` | full `BranchResponse` | +| `GET` | `/v1/projects` | Bearer token | none | `[{id, name, organization_slug, region, …}]` — no project ref from `--remote=`, `SUPABASE_PROJECT_ID` or `.temp/project-ref`, with stdin and stdout TTYs and a non-JSON output format (project picker) | +| `GET` | `/v1/projects/{ref}` | Bearer token | none | `{ref, name, organization_id, organization_slug}` — linked-project cache miss only (best-effort, after the command) | ## Environment Variables @@ -54,10 +52,10 @@ Target: hosted, consent default Yes — see [Command target](../../../../docs/co ## Telemetry Events Fired -| Event | When | Notable properties / groups | -| ----------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `cli_command_executed` | post-run, success or failure (via wrapper) | `exit_code`, `duration_ms`, `flags` (`--remote=` → the ref, a bare `--remote` → `true`; `--project-ref` whitelisted when ref-shaped, otherwise redacted) | -| `cli_upgrade_suggested` | on 4xx with `branching_limit` entitlement gated | `{feature_key: "branching_limit", org_slug}` | +| Event | When | Notable properties / groups | +| ----------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `cli_command_executed` | post-run, success or failure (via wrapper) | `exit_code`, `duration_ms`, `flags` (`--remote=` → the ref, a bare `--remote` → `true`; `--project-ref` whitelisted when ref-shaped, otherwise redacted) | +| `cli_upgrade_suggested` | on a 4xx whose body is the `entitlement_required` envelope | `{feature_key: "branching_limit", org_slug}` | ## Output diff --git a/apps/cli/src/commands/branches/create/create.command.ts b/apps/cli/src/commands/branches/create/create.command.ts index e102804ccb..7cc03bdf62 100644 --- a/apps/cli/src/commands/branches/create/create.command.ts +++ b/apps/cli/src/commands/branches/create/create.command.ts @@ -4,7 +4,6 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; import { projectRefPicker } from "../../../config/project-ref.layer.ts"; -import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; @@ -111,7 +110,6 @@ export const branchesCreateCommand = Command.make("create", config).pipe( }, branchesParentTarget, projectRefPicker, - upgradeSuggestion, linkedProjectCache, ), ), diff --git a/apps/cli/src/commands/branches/create/create.handler.ts b/apps/cli/src/commands/branches/create/create.handler.ts index 5443436b87..3294498b62 100644 --- a/apps/cli/src/commands/branches/create/create.handler.ts +++ b/apps/cli/src/commands/branches/create/create.handler.ts @@ -104,22 +104,20 @@ export const branchesCreate = Effect.fn("branches.create")(function* (flags: Bra Effect.tapError(() => creating?.fail() ?? Effect.void), // Runs the plan-gate check before mapping the error, even for a gated 4xx. Effect.catch( - gateMapError( - { projectRef: ref, featureKey: "branching_limit" }, - (cause, upgradeSuggested) => - mapCreateErrorRaw(cause).pipe( - Effect.catchTag("BranchesCreateUnexpectedStatusError", (mapped) => - Effect.fail( - new BranchesCreateUnexpectedStatusError({ - status: mapped.status, - body: mapped.body, - message: mapped.message, - upgradeSuggested, - entitlement: mapped.entitlement, - }), - ), + gateMapError({ projectRef: ref }, (cause, upgradeSuggested) => + mapCreateErrorRaw(cause).pipe( + Effect.catchTag("BranchesCreateUnexpectedStatusError", (mapped) => + Effect.fail( + new BranchesCreateUnexpectedStatusError({ + status: mapped.status, + body: mapped.body, + message: mapped.message, + upgradeSuggested, + entitlement: mapped.entitlement, + }), ), ), + ), ), ), ); diff --git a/apps/cli/src/commands/branches/create/create.integration.test.ts b/apps/cli/src/commands/branches/create/create.integration.test.ts index 363440864b..21bca71f14 100644 --- a/apps/cli/src/commands/branches/create/create.integration.test.ts +++ b/apps/cli/src/commands/branches/create/create.integration.test.ts @@ -66,19 +66,6 @@ function projectResponse() { }; } -function entitlementResponse(opts: { readonly featureKey: string; readonly hasAccess: boolean }) { - return { - entitlements: [ - { - feature: { key: opts.featureKey, type: "boolean" as const }, - hasAccess: opts.hasAccess, - type: "boolean" as const, - config: { enabled: !opts.hasAccess }, - }, - ], - }; -} - const tempRoot = useTempWorkdir("supabase-branches-create-int-"); interface SetupOpts { @@ -87,8 +74,6 @@ interface SetupOpts { readonly response?: unknown; readonly status?: number; readonly network?: "fail"; - readonly gated?: boolean; - readonly featureKey?: string; readonly yes?: boolean; readonly stdinIsTty?: boolean; readonly promptConfirmResponses?: ReadonlyArray; @@ -99,7 +84,6 @@ const hostedCreatePolicy = Layer.succeed(TargetPolicy, { group: "hosted", consen function apiOptions(opts: SetupOpts): MockCommandPlatformApiOpts { const status = opts.status ?? 201; const body = opts.response ?? CREATED; - const featureKey = opts.featureKey ?? "branching_limit"; return { network: opts.network, handler: (request) => @@ -107,16 +91,6 @@ function apiOptions(opts: SetupOpts): MockCommandPlatformApiOpts { if (request.method === "POST" && request.url.includes("/branches")) { return jsonResponse(request, status, body); } - if (request.method === "GET" && request.url.endsWith(`/v1/projects/${VALID_REF}`)) { - return jsonResponse(request, 200, projectResponse()); - } - if (request.method === "GET" && request.url.includes("/entitlements")) { - return jsonResponse( - request, - 200, - entitlementResponse({ featureKey, hasAccess: !(opts.gated ?? false) }), - ); - } return jsonResponse(request, 200, null); }), }; @@ -414,22 +388,24 @@ describe("branches create integration", () => { }).pipe(Effect.provide(layer)); }); - it.effect("fires cli_upgrade_suggested with feature_key=branching_limit on 402 gated", () => { - const { layer, analytics, api } = setup({ status: 402, gated: true }); + it.effect("makes one request and suggests no upgrade on a 4xx without the envelope", () => { + const { layer, out, analytics, api } = setup({ + status: 400, + response: { message: "Invalid branch name" }, + }); return Effect.gen(function* () { - yield* Effect.exit(branchesCreate({ ...baseFlags, name: Option.some("feat-x") })); + const exit = yield* Effect.exit( + branchesCreate({ ...baseFlags, name: Option.some("feat-x") }), + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(classifyCliCauseActionability(exit.cause).error_category).not.toBe("plan_limit"); + } expect(api.requests.map(({ method, url }) => `${method} ${url}`)).toEqual([ `POST https://api.supabase.com/v1/projects/${VALID_REF}/branches`, - `GET https://api.supabase.com/v1/projects/${VALID_REF}`, - `GET https://api.supabase.com/v1/organizations/${ORG_SLUG}/entitlements`, - ]); - assertPermissionDrift({ command: "branches create", requests: api.requests }); - expect(analytics.captured).toEqual([ - { - event: "cli_upgrade_suggested", - properties: { feature_key: "branching_limit", org_slug: ORG_SLUG }, - }, ]); + expect(out.stderrText).not.toContain("Upgrade your plan:"); + expect(analytics.captured).toHaveLength(0); }).pipe(Effect.provide(layer)); }); @@ -507,7 +483,17 @@ describe("branches create integration", () => { }); it.effect("writes linked-project cache + telemetry on the upgrade-suggest failure path", () => { - const { layer, telemetry, cache } = setupTracked({ status: 402, gated: true }); + const { layer, telemetry, cache } = setupTracked({ + status: 402, + response: { + message: "Branching is supported only on the Pro plan or above", + error: { + code: "entitlement_required", + feature: "branching_limit", + upgrade_url: "https://supabase.com/dashboard/org/env-org/billing", + }, + }, + }); return Effect.gen(function* () { yield* Effect.exit(branchesCreate({ ...baseFlags, name: Option.some("feat-x") })); expect(telemetry.flushed).toBe(true); diff --git a/apps/cli/src/commands/vanity-subdomains/activate/SIDE_EFFECTS.md b/apps/cli/src/commands/vanity-subdomains/activate/SIDE_EFFECTS.md index d96a538e52..7666b32bfc 100644 --- a/apps/cli/src/commands/vanity-subdomains/activate/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/vanity-subdomains/activate/SIDE_EFFECTS.md @@ -25,8 +25,6 @@ Target: hosted, consent default No — see [Command target](../../../../docs/com | `GET` | `/v1/projects/{parent}/branches` | Bearer token | none | `[{id, name, project_ref, is_default}]` — only for `--remote=` (or the deprecated `--project-ref=`), and for a bare `--remote` followed by a word that could name a branch; `{parent}` is `SUPABASE_PROJECT_ID`, else the linked project | | `GET` | `/v1/branches/{id}` | Bearer token | none | `{ref, status, …}` — only for `--remote=` when no project is linked | | `POST` | `/v1/projects/{ref}/vanity-subdomain/activate` | Bearer token | `{ vanity_subdomain: string }` | `{ custom_domain: string }` | -| `GET` | `/v1/projects/{ref}` | Bearer token | none | `{organization_slug}` — upgrade-gate side-call on a 4xx with no plan-gate error in the body | -| `GET` | `/v1/organizations/{slug}/entitlements` | Bearer token | none | `{entitlements[].feature.key, .hasAccess}` (key `vanity_subdomain`) — upgrade-gate side-call on a 4xx with no plan-gate error in the body | | `GET` | `/v1/projects/{ref}` | Bearer token | none | `{ref, name, organization_id, organization_slug}` — `linkedProjectCache` fill, best-effort, skipped when `.temp/linked-project.json` exists, `.temp/project-ref` names another ref, or no token is available | ## Environment Variables @@ -57,7 +55,7 @@ Target: hosted, consent default No — see [Command target](../../../../docs/com | Event | When | Notable properties / groups | | ----------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------ | | `cli_command_executed` | post-run, success or failure (via wrapper); not fired when the `--experimental` gate is closed | `exit_code`, `duration_ms`, `flags` | -| `cli_upgrade_suggested` | gated 4xx responses only | `feature_key=vanity_subdomain`, `org_slug` | +| `cli_upgrade_suggested` | 4xx whose body is the `entitlement_required` envelope | `feature_key=vanity_subdomain`, `org_slug` | ## Output @@ -88,7 +86,7 @@ One `result` event with the full response object. open), even when the API call fails, but only on a cache miss with a token when `GET /v1/projects/{ref}` succeeds and `.temp/project-ref` does not name another ref. A closed gate writes nothing. -- On gated 4xx responses this command prints an upgrade suggestion and fires `cli_upgrade_suggested`. +- On a 4xx whose body is the `entitlement_required` envelope this command prints an upgrade suggestion and fires `cli_upgrade_suggested`. - `--desired-subdomain` is required, but only enforced in the handler (after gate → login → ref resolution) rather than at parse time, using the fixed wording `required flag(s) "desired-subdomain" not set`. The gate/login/ref errors win over the missing diff --git a/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts b/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts index 905fba538d..dfe369822f 100644 --- a/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts +++ b/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts @@ -8,7 +8,6 @@ import { branchResolution } from "../../../command-internal/target/branch-resolu import { projectTargetFlags } from "../../../command-internal/target/target-flags.ts"; import { projectRefPicker } from "../../../config/project-ref.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; -import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; import { RESOURCE_OUTPUT_FORMATS } from "../../../command-internal/output-formats.ts"; @@ -54,7 +53,6 @@ export const vanitySubdomainsActivateCommand = Command.make("activate", config). }, ], }, - upgradeSuggestion, projectRefPicker, linkedProjectCache, branchResolution, diff --git a/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts b/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts index 93d9aa98a9..fc3e6dd81a 100644 --- a/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts +++ b/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts @@ -74,7 +74,6 @@ export const vanitySubdomainsActivate = Effect.fn("vanity-subdomains.activate")( Effect.gen(function* () { const upgradeSuggested = yield* suggestUpgrade({ projectRef: ref, - featureKey: "vanity_subdomain", statusCode: mapped.status, response: gateResponse(cause), }); diff --git a/apps/cli/src/commands/vanity-subdomains/check-availability/SIDE_EFFECTS.md b/apps/cli/src/commands/vanity-subdomains/check-availability/SIDE_EFFECTS.md index dc35e60073..06ced207b0 100644 --- a/apps/cli/src/commands/vanity-subdomains/check-availability/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/vanity-subdomains/check-availability/SIDE_EFFECTS.md @@ -25,8 +25,6 @@ Target: hosted, no consent — see [Command target](../../../../docs/command-tar | `GET` | `/v1/projects/{parent}/branches` | Bearer token | none | `[{id, name, project_ref, is_default}]` — only for `--remote=` (or the deprecated `--project-ref=`), and for a bare `--remote` followed by a word that could name a branch; `{parent}` is `SUPABASE_PROJECT_ID`, else the linked project | | `GET` | `/v1/branches/{id}` | Bearer token | none | `{ref, status, …}` — only for `--remote=` when no project is linked | | `POST` | `/v1/projects/{ref}/vanity-subdomain/check-availability` | Bearer token | `{ vanity_subdomain: string }` | `{ available: boolean }` | -| `GET` | `/v1/projects/{ref}` | Bearer token | none | `{organization_slug}` — upgrade-gate side-call on a 4xx with no plan-gate error in the body | -| `GET` | `/v1/organizations/{slug}/entitlements` | Bearer token | none | `{entitlements[].feature.key, .hasAccess}` (key `vanity_subdomain`) — upgrade-gate side-call on a 4xx with no plan-gate error in the body | | `GET` | `/v1/projects/{ref}` | Bearer token | none | `{ref, name, organization_id, organization_slug}` — `linkedProjectCache` fill, best-effort, skipped when `.temp/linked-project.json` exists, `.temp/project-ref` names another ref, or no token is available | ## Environment Variables @@ -55,7 +53,7 @@ Target: hosted, no consent — see [Command target](../../../../docs/command-tar | ---------------------- | ---------------------------------------------------------------------------------------------- | ----------------------------------- | | `cli_command_executed` | post-run, success or failure (via wrapper); not fired when the `--experimental` gate is closed | `exit_code`, `duration_ms`, `flags` | -This command may print an upgrade suggestion for gated 4xx responses, but it does not fire +This command may print an upgrade suggestion for a 4xx whose body is the `entitlement_required` envelope, but it does not fire `cli_upgrade_suggested`. ## Output diff --git a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts index ad5332cc0b..2e21251ac1 100644 --- a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts +++ b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts @@ -8,7 +8,6 @@ import { branchResolution } from "../../../command-internal/target/branch-resolu import { projectTargetFlags } from "../../../command-internal/target/target-flags.ts"; import { projectRefPicker } from "../../../config/project-ref.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; -import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; import { RESOURCE_OUTPUT_FORMATS } from "../../../command-internal/output-formats.ts"; @@ -55,7 +54,6 @@ export const vanitySubdomainsCheckAvailabilityCommand = Command.make( }, ], }, - upgradeSuggestion, projectRefPicker, linkedProjectCache, branchResolution, diff --git a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts index 282be59786..d1a937f196 100644 --- a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts +++ b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts @@ -72,7 +72,6 @@ export const vanitySubdomainsCheckAvailability = Effect.fn("vanity-subdomains.ch // analytics event. const upgradeSuggested = yield* suggestUpgrade({ projectRef: ref, - featureKey: "vanity_subdomain", statusCode: mapped.status, response: gateResponse(cause), trackAnalytics: false, diff --git a/apps/cli/src/commands/vanity-subdomains/vanity-subdomains.integration.test.ts b/apps/cli/src/commands/vanity-subdomains/vanity-subdomains.integration.test.ts index 0e13f8e8c2..1c5717df04 100644 --- a/apps/cli/src/commands/vanity-subdomains/vanity-subdomains.integration.test.ts +++ b/apps/cli/src/commands/vanity-subdomains/vanity-subdomains.integration.test.ts @@ -1,7 +1,6 @@ import type { V1ActivateVanitySubdomainConfigOutput, V1CheckVanitySubdomainAvailabilityOutput, - V1GetProjectOutput, V1GetVanitySubdomainConfigOutput, } from "@supabase/api/effect"; import { describe, expect, it } from "@effect/vitest"; @@ -30,13 +29,13 @@ import { vanitySubdomainsActivate } from "./activate/activate.handler.ts"; import { vanitySubdomainsCheckAvailability } from "./check-availability/check-availability.handler.ts"; import { vanitySubdomainsDelete } from "./delete/delete.handler.ts"; import { vanitySubdomainsGet } from "./get/get.handler.ts"; +import { classifyCliCauseActionability } from "../../shared/telemetry/error-actionability.ts"; import { YesFlag } from "../../command-internal/global-flags.ts"; import { TargetPolicy } from "../../command-internal/target/target-policy.ts"; type VanityConfig = typeof V1GetVanitySubdomainConfigOutput.Type; type AvailabilityResponse = typeof V1CheckVanitySubdomainAvailabilityOutput.Type; type ActivateResponse = typeof V1ActivateVanitySubdomainConfigOutput.Type; -type ProjectResponse = typeof V1GetProjectOutput.Type; const tempRoot = useTempWorkdir("supabase-vanity-int-"); @@ -73,21 +72,12 @@ const SAMPLE_ACTIVATE: ActivateResponse = { custom_domain: "example.com", }; -// Schema-valid project body: the upgrade-suggest fallback decodes it with the typed client. -const SAMPLE_PROJECT: ProjectResponse = { - id: VALID_REF, - ref: VALID_REF, - organization_id: "org-123", - organization_slug: "supabase", - name: "Test Project", - region: "us-east-1", - created_at: "2026-01-01T00:00:00Z", - status: "ACTIVE_HEALTHY", - database: { - host: "db.example.com", - version: "15", - postgres_engine: "15", - release_channel: "ga", +const VANITY_GATE_ENVELOPE = { + message: "This feature requires the Pro, Team, or Enterprise organization plan.", + error: { + code: "entitlement_required", + feature: "vanity_subdomain", + upgrade_url: "https://supabase.com/dashboard/org/env-org/billing", }, }; @@ -112,35 +102,6 @@ function runtimeWith(opts: { }); } -// Mocks a billing-gated (402) write endpoint with entitlements reporting no access to -// `vanity_subdomain`, for the upgrade-suggestion branch tests. -function gatedApi(matchWrite: (url: string) => boolean) { - return mockCommandPlatformApi({ - handler: (request) => - Effect.sync(() => { - if (request.method === "POST" && matchWrite(request.url)) { - return jsonResponse(request, 402, {}); - } - if (request.method === "GET" && request.url.endsWith(`/v1/projects/${VALID_REF}`)) { - return jsonResponse(request, 200, SAMPLE_PROJECT); - } - if (request.method === "GET" && request.url.includes("/entitlements")) { - return jsonResponse(request, 200, { - entitlements: [ - { - feature: { key: "vanity_subdomain", type: "boolean" }, - hasAccess: false, - type: "boolean", - config: { enabled: true }, - }, - ], - }); - } - return jsonResponse(request, 200, null); - }), - }); -} - // No project is configured: the picker lists projects, the command makes its own call, and the // real linked-project cache then looks the project up. function pickerComposition(own: { readonly status: number; readonly body: unknown }) { @@ -534,7 +495,7 @@ describe("vanity-subdomains check-availability", () => { it.effect("suggests upgrade for gated checks without firing analytics", () => { const out = mockOutput({ format: "text" }); - const api = gatedApi((url) => url.includes("/check-availability")); + const api = mockCommandPlatformApi({ response: { status: 400, body: VANITY_GATE_ENVELOPE } }); const analytics = mockAnalytics(); const layer = runtimeWith({ out, api, analytics }); @@ -548,11 +509,10 @@ describe("vanity-subdomains check-availability", () => { ); expect(Exit.isFailure(exit)).toBe(true); expect(out.stderrText).toContain("Upgrade your plan:"); + expect(out.stderrText).toContain("/org/env-org/billing"); expect(analytics.captured).toHaveLength(0); expect(requestLines(api)).toEqual([ `POST ${API_URL}/v1/projects/${VALID_REF}/vanity-subdomain/check-availability`, - `GET ${API_URL}/v1/projects/${VALID_REF}`, - `GET ${API_URL}/v1/organizations/supabase/entitlements`, ]); assertPermissionDrift({ command: "vanity-subdomains check-availability", @@ -562,6 +522,34 @@ describe("vanity-subdomains check-availability", () => { }).pipe(Effect.provide(layer)); }); + it.effect("makes one request and suggests no upgrade on a check 4xx without the envelope", () => { + const out = mockOutput({ format: "text" }); + const api = mockCommandPlatformApi({ + response: { status: 400, body: { message: "Invalid subdomain" } }, + }); + const analytics = mockAnalytics(); + const layer = runtimeWith({ out, api, analytics }); + + return Effect.gen(function* () { + const exit = yield* Effect.exit( + vanitySubdomainsCheckAvailability({ + remote: Option.none(), + projectRef: Option.none(), + desiredSubdomain: Option.some("example.com"), + }), + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(classifyCliCauseActionability(exit.cause).error_category).not.toBe("plan_limit"); + } + expect(requestLines(api)).toEqual([ + `POST ${API_URL}/v1/projects/${VALID_REF}/vanity-subdomain/check-availability`, + ]); + expect(out.stderrText).not.toContain("Upgrade your plan:"); + expect(analytics.captured).toHaveLength(0); + }).pipe(Effect.provide(layer)); + }); + // json mode so the spinner is suppressed — exercises the no-task error path. it.effect("fails with a network error on transport failure", () => { const out = mockOutput({ format: "json" }); @@ -772,7 +760,7 @@ describe("vanity-subdomains activate", () => { it.effect("suggests upgrade and fires analytics for gated activation", () => { const out = mockOutput({ format: "text" }); - const api = gatedApi((url) => url.endsWith("/vanity-subdomain/activate")); + const api = mockCommandPlatformApi({ response: { status: 400, body: VANITY_GATE_ENVELOPE } }); const analytics = mockAnalytics(); const layer = runtimeWith({ out, api, analytics }); @@ -786,16 +774,15 @@ describe("vanity-subdomains activate", () => { ); expect(Exit.isFailure(exit)).toBe(true); expect(out.stderrText).toContain("Upgrade your plan:"); + expect(out.stderrText).toContain("/org/env-org/billing"); expect(analytics.captured).toEqual([ { event: "cli_upgrade_suggested", - properties: { feature_key: "vanity_subdomain", org_slug: "supabase" }, + properties: { feature_key: "vanity_subdomain", org_slug: "env-org" }, }, ]); expect(requestLines(api)).toEqual([ `POST ${API_URL}/v1/projects/${VALID_REF}/vanity-subdomain/activate`, - `GET ${API_URL}/v1/projects/${VALID_REF}`, - `GET ${API_URL}/v1/organizations/supabase/entitlements`, ]); assertPermissionDrift({ command: "vanity-subdomains activate", @@ -805,6 +792,37 @@ describe("vanity-subdomains activate", () => { }).pipe(Effect.provide(layer)); }); + it.effect( + "makes one request and suggests no upgrade on an activate 4xx without the envelope", + () => { + const out = mockOutput({ format: "text" }); + const api = mockCommandPlatformApi({ + response: { status: 400, body: { message: "Invalid subdomain" } }, + }); + const analytics = mockAnalytics(); + const layer = runtimeWith({ out, api, analytics }); + + return Effect.gen(function* () { + const exit = yield* Effect.exit( + vanitySubdomainsActivate({ + remote: Option.none(), + projectRef: Option.none(), + desiredSubdomain: Option.some("example.com"), + }), + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(classifyCliCauseActionability(exit.cause).error_category).not.toBe("plan_limit"); + } + expect(requestLines(api)).toEqual([ + `POST ${API_URL}/v1/projects/${VALID_REF}/vanity-subdomain/activate`, + ]); + expect(out.stderrText).not.toContain("Upgrade your plan:"); + expect(analytics.captured).toHaveLength(0); + }).pipe(Effect.provide(layer)); + }, + ); + // json mode so the spinner is suppressed — exercises the no-task error path. it.effect("fails with a network error on transport failure", () => { const out = mockOutput({ format: "json" }); From 3624bb3f6173e6914ff2bdc8f3970667946c4e2d Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Mon, 2 Nov 2026 12:23:07 +0000 Subject: [PATCH 002/101] fix(cli): skip docker's first health check wait (CLI-2901) (#7430) ## TL;DR fixes `supabase start` sitting about 10s on docker's first health check when the services are already up. ## whats improved/fixed? docker runs a container's first health check one full interval after it starts, and our interval is 10s. so `supabase start` kept waiting on that check even when everything was ready sooner. a restart with the issue's excludes went from about 12.8s to 5.2s. a full stack restart went from about 14s to 8s `db start` and `db reset` get the same. ## how? containers with a docker healthcheck now get `--health-start-interval 1s` and a 10s start period when `docker version` shows API 1.44 or newer for the client binary, the client in use, and the daemon, so the first check runs about 1s in and the interval stays 10s after. kong's check lived only in its image, so its spec now sets the same `kong health` check and gets those flags too. > [!NOTE] > on engines without the flag (docker below API 1.44, podman) containers keep the 10s first check. ## ref: - closes: https://github.com/supabase/cli/issues/7427 --- .../cli/src/command-internal/container-cli.ts | 42 +++++++---- .../container-cli.unit.test.ts | 72 +++++++++++++++++-- .../db-bootstrap/container-lifecycle.ts | 11 ++- .../container-lifecycle.unit.test.ts | 59 +++++++++++++++ .../db-bootstrap/docker-create-args.ts | 31 ++++++++ .../db-bootstrap/recreate-local-database.ts | 7 +- .../db-bootstrap/start-local-database.ts | 6 +- .../cli/src/commands/db/reset/SIDE_EFFECTS.md | 6 +- .../db/reset/reset.integration.test.ts | 14 ++++ .../cli/src/commands/db/start/SIDE_EFFECTS.md | 7 +- .../db/start/start.integration.test.ts | 11 +++ apps/cli/src/commands/start/SIDE_EFFECTS.md | 13 ++++ .../commands/start/services/kong.service.ts | 7 ++ .../start/services/kong.service.unit.test.ts | 7 +- apps/cli/src/commands/start/start.handler.ts | 6 +- .../commands/start/start.integration.test.ts | 59 +++++++++++++++ apps/cli/src/commands/stop/SIDE_EFFECTS.md | 2 +- .../commands/stop/stop.integration.test.ts | 1 + apps/cli/tests/helpers/docker-model.ts | 1 + 19 files changed, 335 insertions(+), 27 deletions(-) diff --git a/apps/cli/src/command-internal/container-cli.ts b/apps/cli/src/command-internal/container-cli.ts index c6f165ea50..5371f4ba02 100644 --- a/apps/cli/src/command-internal/container-cli.ts +++ b/apps/cli/src/command-internal/container-cli.ts @@ -307,18 +307,35 @@ function isDockerApiVersionAtLeast(version: string, minVersion: string): boolean * fall back to Podman, whose `volume prune` has no `--all` flag to gate. */ export const dockerSupportsVolumePruneAllFlag = (spawner: Spawner) => + dockerApiVersionsSatisfy(spawner, (version) => isDockerApiVersionAtLeast(version, "1.42")); + +/** + * `docker create --health-start-interval` needs API >= 1.44 on both ends and is a hard failure + * below that. The Podman CLI rejects the flag, and a `docker` shim for it fails this format or + * prints Podman's own `6.x`, so a major other than 1 is not Docker's API. The real `docker` client + * on a Podman socket reporting 1.44 passes: Podman keeps the start period and drops the start + * interval. + */ +export const dockerSupportsHealthStartInterval = (spawner: Spawner) => + dockerApiVersionsSatisfy( + spawner, + (version) => version.startsWith("1.") && isDockerApiVersionAtLeast(version, "1.44"), + ); + +/* Reads the client's own and effective API versions and the daemon's, `false` on any failure. + `DOCKER_API_VERSION` can lift the effective one past what the client binary supports. */ +const dockerApiVersionsSatisfy = (spawner: Spawner, supported: (version: string) => boolean) => Effect.scoped( Effect.gen(function* () { - const child = yield* spawner.spawn( - ChildProcess.make( - "docker", - ["version", "--format", "{{.Client.APIVersion}} {{.Server.APIVersion}}"], - { - stdin: "ignore", - stdout: "pipe", - stderr: "ignore", - }, - ), + const child = yield* spawnRuntime( + spawner, + "docker", + [ + "version", + "--format", + "{{.Client.DefaultAPIVersion}} {{.Client.APIVersion}} {{.Server.APIVersion}}", + ], + { stdin: "ignore", stdout: "pipe", stderr: "ignore" }, ); const [exitCode, stdout] = yield* Effect.all( [child.exitCode.pipe(Effect.map(Number)), collectText(child.stdout)], @@ -326,9 +343,6 @@ export const dockerSupportsVolumePruneAllFlag = (spawner: Spawner) => ); if (exitCode !== 0) return false; const versions = stdout.trim().split(/\s+/); - return ( - versions.length === 2 && - versions.every((version) => isDockerApiVersionAtLeast(version, "1.42")) - ); + return versions.length === 3 && versions.every(supported); }), ).pipe(Effect.orElseSucceed(() => false)); diff --git a/apps/cli/src/command-internal/container-cli.unit.test.ts b/apps/cli/src/command-internal/container-cli.unit.test.ts index d65317e74c..d686fdbdaf 100644 --- a/apps/cli/src/command-internal/container-cli.unit.test.ts +++ b/apps/cli/src/command-internal/container-cli.unit.test.ts @@ -6,6 +6,7 @@ import { containerCliExitCode, containerRuntimeNotFoundMessage, describeContainerCliFailure, + dockerSupportsHealthStartInterval, dockerSupportsVolumePruneAllFlag, isContainerNotFoundMessage, spawnContainerCli, @@ -17,7 +18,11 @@ function mockSpawner( readonly bothMissing?: boolean; readonly exitCode?: number; readonly stdout?: string; - readonly apiVersions?: { readonly client: string; readonly server: string }; + readonly apiVersions?: { + readonly clientDefault?: string; + readonly client: string; + readonly server: string; + }; } = {}, ) { const spawned: Array<{ readonly command: string; readonly args: ReadonlyArray }> = []; @@ -40,6 +45,10 @@ function mockSpawner( const stdout = opts.apiVersions !== undefined && args[0] === "version" ? (args[args.indexOf("--format") + 1] ?? "") + .replaceAll( + "{{.Client.DefaultAPIVersion}}", + opts.apiVersions.clientDefault ?? opts.apiVersions.client, + ) .replaceAll("{{.Client.APIVersion}}", opts.apiVersions.client) .replaceAll("{{.Server.APIVersion}}", opts.apiVersions.server) : opts.stdout; @@ -229,6 +238,57 @@ describe("container CLI process spans", () => { ); }); +describe("dockerSupportsHealthStartInterval", () => { + it.live.each([ + { name: "both ends at the 1.44 minimum", client: "1.44", server: "1.44" }, + { name: "a new client against a newer daemon", client: "1.47", server: "1.56" }, + { name: "a three-part API version", client: "1.44.0", server: "1.44.0" }, + ])("returns true for $name", ({ client, server }) => { + const mock = mockSpawner({ apiVersions: { client, server } }); + return dockerSupportsHealthStartInterval(mock.spawner).pipe( + Effect.map((supported) => { + expect(supported).toBe(true); + }), + ); + }); + + it.live.each([ + { + name: "a client below API 1.44", + options: { apiVersions: { client: "1.43", server: "1.56" } }, + }, + { + name: "an old client with DOCKER_API_VERSION forced up", + options: { apiVersions: { clientDefault: "1.43", client: "1.44", server: "1.56" } }, + }, + { + name: "a daemon below API 1.44", + options: { apiVersions: { client: "1.56", server: "1.43" } }, + }, + { + name: "a docker shim printing Podman's own version", + options: { apiVersions: { client: "6.1.3", server: "6.1.3" } }, + }, + { + name: "a shim whose major starts with a 1", + options: { apiVersions: { client: "10.1.0", server: "10.1.0" } }, + }, + { name: "fewer than three versions", options: { stdout: "1.56 1.56" } }, + { + name: "the version command exiting non-zero", + options: { exitCode: 1, stdout: "1.56 1.56 1.56" }, + }, + { name: "docker not being installed", options: { dockerMissing: true } }, + ])("returns false for $name", ({ options }) => { + const mock = mockSpawner(options); + return dockerSupportsHealthStartInterval(mock.spawner).pipe( + Effect.map((supported) => { + expect(supported).toBe(false); + }), + ); + }); +}); + describe("dockerSupportsVolumePruneAllFlag", () => { it.live( "returns true when the client and daemon both report an API version at or above 1.42", @@ -240,7 +300,11 @@ describe("dockerSupportsVolumePruneAllFlag", () => { expect(mock.spawned).toEqual([ { command: "docker", - args: ["version", "--format", "{{.Client.APIVersion}} {{.Server.APIVersion}}"], + args: [ + "version", + "--format", + "{{.Client.DefaultAPIVersion}} {{.Client.APIVersion}} {{.Server.APIVersion}}", + ], }, ]); }), @@ -266,8 +330,8 @@ describe("dockerSupportsVolumePruneAllFlag", () => { name: "versions that only compare above 1.42 lexicographically", options: { apiVersions: { client: "1.9", server: "1.9" } }, }, - { name: "a single version instead of a client and daemon pair", options: { stdout: "1.51" } }, - { name: "more than two versions", options: { stdout: "1.51 1.51 1.51" } }, + { name: "a single version", options: { stdout: "1.51" } }, + { name: "more than three versions", options: { stdout: "1.51 1.51 1.51 1.51" } }, { name: "empty output", options: { stdout: "" } }, ])("returns false for $name", ({ options }) => { const mock = mockSpawner(options); diff --git a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts index 85457f9137..1292e7ecca 100644 --- a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts +++ b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts @@ -34,6 +34,7 @@ import { import { buildStartContainerCreateArgs, applyBitbucketStartContainerFilter, + applyHealthStartInterval, isDockerClientEnvKey, type StartContainerSpec, } from "./docker-create-args.ts"; @@ -124,6 +125,11 @@ export interface ContainerOpts { readonly projectId: string; /** Passed in (rather than read via env here) so this module stays a pure effect orchestrator. */ readonly isBitbucketPipeline: boolean; + /** + * Whether the engine takes `--health-start-interval`, probed once per run by each command that + * reads Docker's health status. Shadow databases leave it unset: nothing reads theirs. + */ + readonly healthStartInterval?: boolean; /** * Stamped onto every created container as {@link CLI_WORKDIR_LABEL} so a later `stop` or * {@link rollbackStart} can find this directory from the container's own label alone, without @@ -716,7 +722,10 @@ export function createContainer( } } - const finalSpec = applyBitbucketStartContainerFilter(labeledSpec, opts.isBitbucketPipeline); + const finalSpec = applyHealthStartInterval( + applyBitbucketStartContainerFilter(labeledSpec, opts.isBitbucketPipeline), + opts.healthStartInterval === true, + ); const createArgs = buildStartContainerCreateArgs(finalSpec); // `isDockerClientEnvKey` keys are already emitted inline as `-e KEY=value` above; see diff --git a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts index 648474c306..c75a315ad9 100644 --- a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts @@ -297,6 +297,65 @@ describe("createContainer", () => { ); }); + describe("Docker's first health check", () => { + const steady = { test: ["CMD", "true"], intervalSeconds: 10, timeoutSeconds: 2, retries: 3 }; + const steadyFlags = { + "--health-cmd": "true", + "--health-interval": "10s", + "--health-timeout": "2s", + "--health-retries": "3", + }; + const startFlags = { "--health-start-period": "10s", "--health-start-interval": "1s" }; + + it.live.each([ + { + name: "adds start flags where the engine takes them", + healthcheck: steady, + supported: true, + expected: { ...steadyFlags, ...startFlags }, + }, + { + name: "keeps a service's own start period", + healthcheck: { ...steady, startPeriodSeconds: 20 }, + supported: true, + expected: { ...steadyFlags, ...startFlags, "--health-start-period": "20s" }, + }, + { + name: "adds none where the engine rejects them", + healthcheck: steady, + supported: false, + expected: steadyFlags, + }, + { + name: "adds none for a caller that never asked", + healthcheck: steady, + expected: steadyFlags, + }, + { name: "adds none without a healthcheck", supported: true, expected: {} }, + ])("$name", ({ healthcheck, supported, expected }) => { + const mock = alwaysSucceed(); + return createContainer( + mock.spawner, + { ...baseSpec, binds: [], ...(healthcheck === undefined ? {} : { healthcheck }) }, + { + projectId: "proj", + isBitbucketPipeline: false, + workdir: workdir.current, + extraHosts: [], + ...(supported === undefined ? {} : { healthStartInterval: supported }), + }, + ).pipe( + Effect.map(() => { + const args = mock.spawned.find((spawned) => spawned[0] === "create") ?? []; + const flags = args.flatMap((arg, index) => + arg.startsWith("--health-") ? [[arg, args[index + 1]]] : [], + ); + expect(Object.fromEntries(flags)).toEqual(expected); + }), + ); + }); + }); + it.live("fails with ContainerStartError, unmodified, on a plain start failure", () => { const mock = mockSpawner((args) => { if (args[0] === "create") return { exitCode: 0, stdout: "abc\n" }; diff --git a/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts b/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts index d379bc7a39..1c38d17ae1 100644 --- a/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts +++ b/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts @@ -23,6 +23,8 @@ interface StartHealthcheckSpec { readonly retries?: number; /** Emitted as `--health-start-period s`; omitted when unset. */ readonly startPeriodSeconds?: number; + /** Emitted as `--health-start-interval s`; see {@link applyHealthStartInterval}. */ + readonly startIntervalSeconds?: number; } /** One `-p :[/]` entry. */ @@ -203,6 +205,12 @@ function buildHealthcheckArgs(healthcheck: StartHealthcheckSpec): ReadonlyArray< if (healthcheck.startPeriodSeconds !== undefined) { args.push("--health-start-period", formatDockerDurationSeconds(healthcheck.startPeriodSeconds)); } + if (healthcheck.startIntervalSeconds !== undefined) { + args.push( + "--health-start-interval", + formatDockerDurationSeconds(healthcheck.startIntervalSeconds), + ); + } return args; } @@ -288,3 +296,26 @@ export function applyBitbucketStartContainerFilter( securityOpt: [], }; } + +/* Docker returns to `--health-interval` after the first pass or the end of the start period. */ +const HEALTH_START_INTERVAL_SECONDS = 1; +/* The start interval only runs inside a start period, whose failures do not count. One steady + interval keeps `unhealthy` where it was, or one interval later if a slow probe straddles it. */ +const HEALTH_START_PERIOD_SECONDS = 10; + +/* Runs a container's first health checks right after it starts instead of a full interval later. + `supported` comes from `dockerSupportsHealthStartInterval`; without it the spec is unchanged. */ +export function applyHealthStartInterval( + spec: StartContainerSpec, + supported: boolean, +): StartContainerSpec { + if (!supported || spec.healthcheck === undefined) return spec; + return { + ...spec, + healthcheck: { + ...spec.healthcheck, + startPeriodSeconds: spec.healthcheck.startPeriodSeconds ?? HEALTH_START_PERIOD_SECONDS, + startIntervalSeconds: HEALTH_START_INTERVAL_SECONDS, + }, + }; +} diff --git a/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts b/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts index ce496f87c0..08d02e7f2d 100644 --- a/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts @@ -23,7 +23,7 @@ import { ErrorActionabilityId, } from "../../shared/telemetry/error-actionability.ts"; import { isSqlState } from "../connect-errors.ts"; -import { isContainerNotFoundMessage } from "../container-cli.ts"; +import { dockerSupportsHealthStartInterval, isContainerNotFoundMessage } from "../container-cli.ts"; import { checkDbToml } from "../db-config.toml-read.ts"; import { DbConnection, type DbSession } from "../db-connection.service.ts"; import { DbExecError, type DbConnectError } from "../db-connection.errors.ts"; @@ -292,7 +292,10 @@ const recreateLocalDatabase15 = ( ...input.postgresSpec, image: resolvedPostgresImage, }); - yield* createContainer(spawner, postgresSpec, input.containerOpts); + yield* createContainer(spawner, postgresSpec, { + ...input.containerOpts, + healthStartInterval: yield* dockerSupportsHealthStartInterval(spawner), + }); // Never swallowed — reset has no `--from-backup`-equivalent gate. yield* waitForPostgresReady( diff --git a/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts b/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts index 37ac986ec0..b4323b62b9 100644 --- a/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts @@ -28,6 +28,7 @@ import { parseUrl } from "../storage-url.ts"; import { loadLocalProjectContext } from "../local-project-context.ts"; import { cliProjectFilterValue, localDbContainerId } from "../docker-ids.ts"; import { isBitbucketPipeline } from "../bitbucket-pipeline.ts"; +import { dockerSupportsHealthStartInterval } from "../container-cli.ts"; import { holdProjectLifecycleLock } from "../project-lifecycle-lock.ts"; import { dbNotRunningError } from "../status-errors.ts"; import { buildLocalDbContainerInputs } from "./local-container-inputs.ts"; @@ -267,7 +268,10 @@ const startLocalDatabaseInScope = Effect.fn("DbBootstrap.startLocalDatabase")(fu hostname, dbContainerId, dbPort: values.dbPort, - containerOpts, + containerOpts: { + ...containerOpts, + healthStartInterval: yield* dockerSupportsHealthStartInterval(spawner), + }, // `db reset` has no `fromBackup` concept at all, so `postgresSpecBase` omits it. postgresSpec: { ...postgresSpecBase, fromBackup }, // Only the `db` container's own image, resolved lazily, right where it would be diff --git a/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md index 021aabc7ac..d1b9e63710 100644 --- a/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md @@ -100,6 +100,7 @@ equivalent, PG15) or `InitSchema14`/`ApplyApiPrivileges` (PG14). | `docker container inspect supabase_db_` | local path | `AssertSupabaseDbIsRunning` probe (Podman fallback) — spawned only when the direct Engine-API request over the active local socket/named pipe gives no definitive answer | | `docker stop ` | local path, PG15 | stop the satellites before removing the db container so their `unless-stopped` restarts cannot race the one-shot init jobs (issue #6915), not-found tolerated per service | | `docker container rm -f supabase_db_` / `docker volume rm -f ` | local path, PG15 | remove the existing container/volume before recreating (Podman fallback) | +| `docker version --format ...` | local path, PG15 | once before `docker create`: API >= 1.44 on both ends adds the health start flags (see below) | | `docker network create` / `docker volume create` / `docker create` / `docker start` | local path, PG15 | recreate the Postgres container (same primitives `db start` uses) | | `docker run --rm ` | local path, PG15, per enabled service | the one-shot `initSchema15` migrate jobs (`startSetupLocalDatabase`) | | `docker restart ` | local path, PG14 | `RestartDatabase` — pg_cron must restart after `pg_terminate_backend` | @@ -115,7 +116,10 @@ published Postgres address (2s dial cap). Before the command returns, it waits u Docker's first health verdict on the database container (polling `docker container inspect` every 1s until the status leaves `starting`, the container stops running, or Docker cannot launch its health check), so a following `supabase status` does not see `starting`. That wait never fails the -reset. +reset. When `docker version` reports API >= 1.44 on both ends, `docker create` also gets +`--health-start-period 10s --health-start-interval 1s`, so its first check runs about 1s after start +instead of 10s. Older engines and the Podman CLI get neither flag; the `docker` client on a Podman +socket reporting API >= 1.44 gets both, Podman drops the interval, and there is no speedup. No subprocess delegation on either target — the remote path's `--experimental` schema-files apply is fully native. diff --git a/apps/cli/src/commands/db/reset/reset.integration.test.ts b/apps/cli/src/commands/db/reset/reset.integration.test.ts index 9fa7fd6d70..6db891878a 100644 --- a/apps/cli/src/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/commands/db/reset/reset.integration.test.ts @@ -1208,6 +1208,20 @@ describe("db reset", () => { ); }); + it.effect("emits the health-start flags on a supported engine", () => { + const base = defaultLocalResetRoute(); + const { layer, child } = setup(tmp.current, { + toml: 'project_id = "test"\n', + args: ["db", "reset"], + isLocal: true, + route: (args) => (args[0] === "version" ? { stdout: ["1.44 1.44 1.56"] } : base(args)), + }); + return Effect.gen(function* () { + yield* dbReset(DEFAULT_FLAGS).pipe(Effect.provide(layer)); + expect(createArgs(child.spawned)).toContain("--health-start-interval"); + }); + }); + it.effect( "recreates when Docker cannot exec the database's health check but it accepts connections", () => { diff --git a/apps/cli/src/commands/db/start/SIDE_EFFECTS.md b/apps/cli/src/commands/db/start/SIDE_EFFECTS.md index 62bdc1dc41..69995b0ccc 100644 --- a/apps/cli/src/commands/db/start/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/start/SIDE_EFFECTS.md @@ -81,7 +81,11 @@ composition reuses too — see that command's `SIDE_EFFECTS.md`): 9. Wait up to 15s for Docker's first health verdict, so `supabase db start && supabase status` does not see `starting`: poll `docker container inspect` every 1s until the health status leaves `starting`, the container stops running, or Docker cannot launch its health check. Never - fails the command, whatever the outcome, and is not covered by the rollback below. + fails the command, whatever the outcome, and is not covered by the rollback below. When the + `docker version` probe reports API >= 1.44 on both ends, `docker create` also gets + `--health-start-period 10s --health-start-interval 1s`, so the first check runs about 1s after + start instead of 10s. Older engines and the Podman CLI get neither flag; the `docker` client on a + Podman socket reporting API >= 1.44 gets both, Podman drops the interval, and there is no speedup. Any failure from step 1 onward prints `Stopping containers...` to stderr and rolls back via `rollbackDatabaseStart`, which only touches what this command creates: the db container @@ -135,6 +139,7 @@ only spawns on fallback. | `docker volume inspect supabase_db_` | when not already running — the pre-create fresh-volume probe | | `docker image inspect` / `docker pull` (registry-fallback resolve) | when not already running — resolves the Postgres image | | `docker volume create --label ...` | when not already running, unless a `--from-backup` restore onto an existing volume (fails first) | +| `docker version --format ...` | when not already running, once: API >= 1.44 on both ends adds the health start flags (see step 9) | | `docker create` + optional `docker cp - :/` + `docker start` | when not already running — `cp` streams one in-memory tar archive containing the pgsodium root key at its exact path when the selected entrypoint needs it | | `docker cp :/var/lib/postgresql/data/PG_VERSION -` | when the volume already existed, between `docker create` and `docker start` — the data-directory probe; its stdout is discarded | | `docker container inspect` (repeated) | readiness polling every 500ms up to `db.health_timeout`, then health-settle polling every 1s for up to 15s | diff --git a/apps/cli/src/commands/db/start/start.integration.test.ts b/apps/cli/src/commands/db/start/start.integration.test.ts index f60124ae61..5889b11a77 100644 --- a/apps/cli/src/commands/db/start/start.integration.test.ts +++ b/apps/cli/src/commands/db/start/start.integration.test.ts @@ -1039,6 +1039,17 @@ describe("db start", () => { }); }); + it.effect("emits the health-start flags on a supported engine", () => { + const base = freshVolumeRoute(defaultRoute()); + const { layer, child } = setup({ + route: (args) => (args[0] === "version" ? { stdout: ["1.44 1.44 1.56"] } : base(args)), + }); + return Effect.gen(function* () { + yield* dbStart(DEFAULT_FLAGS).pipe(Effect.provide(layer)); + expect(createArgs(child.spawned)).toContain("--health-start-interval"); + }); + }); + it.effect("starts when Docker's health check cannot exec but the host can connect", () => { const s = setup({ route: execFailedHealthRoute(freshVolumeRoute(defaultRoute())) }); return Effect.gen(function* () { diff --git a/apps/cli/src/commands/start/SIDE_EFFECTS.md b/apps/cli/src/commands/start/SIDE_EFFECTS.md index 5d2a4d32d3..2c4c72505a 100644 --- a/apps/cli/src/commands/start/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/start/SIDE_EFFECTS.md @@ -110,6 +110,19 @@ the Postgres container (polling `docker container inspect` every 1s until the st `starting`, the container stops running, or Docker cannot launch its health check), so a following `supabase status` or `supabase start` does not see `starting`. It never fails the run. +Before creating any container, `start` runs `docker version --format +'{{.Client.DefaultAPIVersion}} {{.Client.APIVersion}} {{.Server.APIVersion}}'` once. When all three +are Docker API versions >= 1.44, every container whose spec sets a Docker healthcheck also gets +`--health-start-interval 1s` and, unless its service sets its own, `--health-start-period 10s`. The +first check then runs about 1s after the container starts instead of one 10s interval later, and the +interval returns to 10s after the first pass. Failures inside the period do not count, so a failing +service with 3 retries reaches `unhealthy` about 30s after it starts, or one interval later if a slow +probe straddles the end of the period; the wait's budget and error type do not change. Below API +1.44, when `docker version` fails, or with the Podman CLI or a `docker` shim for it, +`--health-start-interval` is not passed and only Mailpit and Logflare keep their own start period. +The real `docker` client on a Podman socket reporting 1.44 gets them: Podman keeps the start period +and drops the start interval, so there is no speedup there. + When a container is running but Docker could not launch its health check, it is judged from the host instead of failing. This applies only when the container's last recorded probe has exit code `-1` with an `OCI runtime exec failed ... unable to start container process` output, whatever diff --git a/apps/cli/src/commands/start/services/kong.service.ts b/apps/cli/src/commands/start/services/kong.service.ts index 199f36ca75..269059de71 100644 --- a/apps/cli/src/commands/start/services/kong.service.ts +++ b/apps/cli/src/commands/start/services/kong.service.ts @@ -267,6 +267,13 @@ export function buildKongContainerSpec( { containerPort: "8443" }, { containerPort: String(KONG_NGINX_TEMPLATE_SERVER_PORT) }, ], + /* The image's own check, spelled out so the health start flags can apply to it. */ + healthcheck: { + test: ["CMD-SHELL", "kong health"], + intervalSeconds: 10, + timeoutSeconds: 10, + retries: 10, + }, restartPolicy: "unless-stopped", networkId: input.networkId, networkAliases: KONG_NETWORK_ALIASES, diff --git a/apps/cli/src/commands/start/services/kong.service.unit.test.ts b/apps/cli/src/commands/start/services/kong.service.unit.test.ts index 8fa7a2d9c7..8483cb0e40 100644 --- a/apps/cli/src/commands/start/services/kong.service.unit.test.ts +++ b/apps/cli/src/commands/start/services/kong.service.unit.test.ts @@ -151,7 +151,12 @@ describe("buildKongContainerSpec", () => { expect(spec.networkId).toBe("supabase_network_proj"); expect(spec.networkAliases).toEqual(["kong", "api.supabase.internal"]); expect(spec.labels).toEqual({}); - expect(spec.healthcheck).toBeUndefined(); + expect(spec.healthcheck).toEqual({ + test: ["CMD-SHELL", "kong health"], + intervalSeconds: 10, + timeoutSeconds: 10, + retries: 10, + }); }).pipe(Effect.provide(BunPath.layer)); }); diff --git a/apps/cli/src/commands/start/start.handler.ts b/apps/cli/src/commands/start/start.handler.ts index 94c11f5c73..77786e00bb 100644 --- a/apps/cli/src/commands/start/start.handler.ts +++ b/apps/cli/src/commands/start/start.handler.ts @@ -38,7 +38,10 @@ import { resolveApiTlsPath, resolveEmailTemplateContentPath, } from "../../command-internal/config-validate.ts"; -import { isContainerNotFoundMessage } from "../../command-internal/container-cli.ts"; +import { + dockerSupportsHealthStartInterval, + isContainerNotFoundMessage, +} from "../../command-internal/container-cli.ts"; import { checkDbToml } from "../../command-internal/db-config.toml-read.ts"; import { resolveEdgeRuntimeImage } from "../../command-internal/edge-runtime-image.ts"; import { @@ -796,6 +799,7 @@ export const start = Effect.fn("start")(function* (flags: StartFlags) { const startOpts: ContainerOpts = { projectId, isBitbucketPipeline: inBitbucketPipeline, + healthStartInterval: yield* dockerSupportsHealthStartInterval(spawner), workdir: cliSettings.workdir, extraHosts, }; diff --git a/apps/cli/src/commands/start/start.integration.test.ts b/apps/cli/src/commands/start/start.integration.test.ts index c7aef1a51f..c428ea6474 100644 --- a/apps/cli/src/commands/start/start.integration.test.ts +++ b/apps/cli/src/commands/start/start.integration.test.ts @@ -1741,6 +1741,65 @@ describe("start integration", () => { }).pipe(Effect.provide(BunServices.layer)), ); + describe("Docker's first health check", () => { + const withVersionAnswer = (answer: RouteResult) => { + const base = defaultRoute(); + return (args: ReadonlyArray) => (args[0] === "version" ? answer : base(args)); + }; + const healthFlagsFor = (spawned: ReadonlyArray, service: string) => { + const args = + spawned.find( + (s) => + s.args[0] === "create" && + containerNameFromCreateArgs(s.args).includes(`_${service}_`), + )?.args ?? []; + return Object.fromEntries( + args.flatMap((arg, index) => + arg.startsWith("--health-") ? [[arg, args[index + 1]]] : [], + ), + ); + }; + + it.effect("emits the health-start flags on a supported engine, asking Docker once", () => + Effect.gen(function* () { + const { layer, child } = yield* setup({ + route: withVersionAnswer({ stdout: ["1.44 1.44 1.44"] }), + }); + + yield* start(flags()).pipe(Effect.provide(layer)); + + expect(child.spawned.filter((s) => s.args[0] === "version")).toHaveLength(1); + for (const service of ["db", "auth", "realtime", "storage", "studio", "kong"]) { + expect(healthFlagsFor(child.spawned, service)).toMatchObject({ + "--health-interval": "10s", + "--health-start-period": "10s", + "--health-start-interval": "1s", + }); + } + expect(healthFlagsFor(child.spawned, "rest")).toEqual({}); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect.each([ + { name: "an engine below API 1.44", answer: { stdout: ["1.43 1.43 1.43"] } }, + { name: "a docker shim for Podman", answer: { stdout: ["6.1.3 6.1.3 6.1.3"] } }, + { name: "a failing docker version", answer: { exitCode: 1 } }, + ])("passes none of the start flags on $name", ({ answer }) => + Effect.gen(function* () { + const { layer, child } = yield* setup({ route: withVersionAnswer(answer) }); + + yield* start(flags()).pipe(Effect.provide(layer)); + + const creates = child.spawned.filter((s) => s.args[0] === "create"); + expect(creates.length).toBeGreaterThan(0); + for (const create of creates) { + expect(create.args).not.toContain("--health-start-interval"); + } + expect(healthFlagsFor(child.spawned, "auth")).not.toHaveProperty("--health-start-period"); + }).pipe(Effect.provide(BunServices.layer)), + ); + }); + it.effect( "starts Studio in a fresh project on Podman, which rejects missing bind sources", () => diff --git a/apps/cli/src/commands/stop/SIDE_EFFECTS.md b/apps/cli/src/commands/stop/SIDE_EFFECTS.md index 84fa021d7e..333bbea781 100644 --- a/apps/cli/src/commands/stop/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/stop/SIDE_EFFECTS.md @@ -163,7 +163,7 @@ flags.noBackup`. `--backup=false` alone does **not** delete volumes; only `--no-backup` does. - Volume prune gates `--all` on the Docker client and daemon API versions (`container-cli.ts`'s `dockerSupportsVolumePruneAllFlag`, checked via `docker version --format -'{{.Client.APIVersion}} {{.Server.APIVersion}}'`) — Docker requires both to be >= 1.42. This +'{{.Client.DefaultAPIVersion}} {{.Client.APIVersion}} {{.Server.APIVersion}}'`) — the CLI requires all three to be >= 1.42. This isn't cosmetic: a pre-23.0 Docker CLI doesn't know `--all` on `volume prune`, and a newer CLI annotates it `version: "1.42"` and rejects it when the daemon is below 1.42, before pruning runs, so sending it unconditionally hard-fails the whole call instead of just diff --git a/apps/cli/src/commands/stop/stop.integration.test.ts b/apps/cli/src/commands/stop/stop.integration.test.ts index f892cc4423..7fe01bcf0c 100644 --- a/apps/cli/src/commands/stop/stop.integration.test.ts +++ b/apps/cli/src/commands/stop/stop.integration.test.ts @@ -188,6 +188,7 @@ function defaultRoute( return { stdout: [ format + .replaceAll("{{.Client.DefaultAPIVersion}}", dockerClientApiVersion) .replaceAll("{{.Client.APIVersion}}", dockerClientApiVersion) .replaceAll("{{.Server.APIVersion}}", dockerServerApiVersion), ], diff --git a/apps/cli/tests/helpers/docker-model.ts b/apps/cli/tests/helpers/docker-model.ts index 9c17b88394..774a44df70 100644 --- a/apps/cli/tests/helpers/docker-model.ts +++ b/apps/cli/tests/helpers/docker-model.ts @@ -191,6 +191,7 @@ export function dockerModel(hooks: DockerModelHooks = {}) { exitCode: 0, stdout: [ format + .replaceAll("{{.Client.DefaultAPIVersion}}", "1.45") .replaceAll("{{.Client.APIVersion}}", "1.45") .replaceAll("{{.Server.APIVersion}}", "1.45"), ], From c3f3ef2ef3496355c8c003290311fb4436ce0d94 Mon Sep 17 00:00:00 2001 From: Prashansa Kulshrestha Date: Mon, 2 Nov 2026 14:57:57 +0000 Subject: [PATCH 003/101] ci(cli): judge block-scoped benchmark timings by per-block spread (#7433) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to #7412. Block-scoped workloads started one stack per block of five launches, so a ten-run workload gave each build only two stack sessions. Launches in one block share a stack session, so session-to-session spread could push the branch median outside the base range, and an A/A run (same CLI code as base and branch) was flagged as slower or faster. - Blocks are now two launches, so each build gets five stack sessions at ten runs, and each measured launch records its block index in the report. - With at least three paired blocks, the block is the unit of evidence. The median of the per-block deltas must clear the existing 5 percent and 5 ms floor, a strict majority of blocks must move the same way past that floor, and the median delta must exceed 1.5 times the larger interquartile range of the two builds' block medians. One odd block cannot create or hide a verdict, and mixed-sign blocks read as within noise. - Run-scoped workloads keep the existing rule (the branch median outside the base min-max range) and also need the median delta to exceed 1.5 times the larger interquartile range of the two builds' launches, so a wide spread reads as noise. - `estimatedSeconds` for block-scoped workloads is rescaled for the extra stack sessions, and `tracing-monitoring.md` describes the new rule. False positives may not be entirely eliminated. A follow-up could allow re-running one or more chosen workloads, so a reported regression can be confirmed without re-running the whole benchmark. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- apps/cli/bench/workloads.full.json | 26 ++-- apps/cli/docs/tracing-monitoring.md | 13 +- .../cli/scripts/bench-cli.integration.test.ts | 16 ++- apps/cli/scripts/bench-cli.ts | 17 ++- apps/cli/scripts/bench-cli.unit.test.ts | 15 +++ apps/cli/scripts/bench-report.ts | 84 +++++++++++-- apps/cli/scripts/bench-report.unit.test.ts | 114 +++++++++++++++--- 7 files changed, 237 insertions(+), 48 deletions(-) diff --git a/apps/cli/bench/workloads.full.json b/apps/cli/bench/workloads.full.json index 90fef53a28..0b17661da1 100644 --- a/apps/cli/bench/workloads.full.json +++ b/apps/cli/bench/workloads.full.json @@ -48,7 +48,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "db-query", @@ -61,7 +61,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "db-reset", @@ -90,7 +90,7 @@ "warmup": 1, "traceRuns": 1, "timeoutSeconds": 120, - "estimatedSeconds": 60 + "estimatedSeconds": 180 }, { "id": "gen-types-small", @@ -104,7 +104,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "gen-types-medium", @@ -118,7 +118,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "gen-types-large", @@ -132,7 +132,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 100 + "estimatedSeconds": 250 }, { "id": "db-diff-cold", @@ -183,7 +183,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 100 + "estimatedSeconds": 250 }, { "id": "db-lint", @@ -197,7 +197,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "migration-up", @@ -239,7 +239,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "stack-start-docker", @@ -266,7 +266,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-stop-docker", @@ -294,7 +294,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-db-reset-docker", @@ -336,7 +336,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-stop-native", @@ -364,7 +364,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-db-reset-native", diff --git a/apps/cli/docs/tracing-monitoring.md b/apps/cli/docs/tracing-monitoring.md index cdc4b92766..c42cee4b1e 100644 --- a/apps/cli/docs/tracing-monitoring.md +++ b/apps/cli/docs/tracing-monitoring.md @@ -178,9 +178,14 @@ the timing ranges, API calls per command, and the slowest phases. It reports: responses (the build did not follow the recorded scenario), non-zero or differing exit codes, missing traces, and too few runs for a timing verdict. Other count changes, such as 1 to 2 calls, appear in the API calls table with the changed endpoints but are not flagged. -- Wall time: base ("before") and branch ("after") medians and ranges. A verdict of slower or faster needs the branch - median outside the base min-max range, a change of at least 5 percent and 5 ms, and at least 5 - measured runs; otherwise it reads "within noise", or "not comparable" when a build failed. +- Wall time: base ("before") and branch ("after") medians and ranges. A verdict of slower or faster needs a + change of at least 5 percent and 5 ms that is also larger than 1.5 times the wider build's + interquartile range, a branch median beyond the base min-max range for run-scoped workloads, and + at least 5 measured runs; otherwise it reads "within noise", or "not + comparable" when a build failed. Block-scoped workloads record a block index per launch and, from + three paired blocks, judge block medians instead: the median per-block change must pass the same + tests and more than half of the blocks must move the same way past the floor, so one outlier + stack session can neither flag nor hide a change. Thresholds are named constants in `bench-report.ts`. - API calls per workload, base and branch. - Phase deltas: the three largest span deltas for each workload with a slower or faster verdict. @@ -297,7 +302,7 @@ described above: | Field | Meaning | | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | | `setup`, `teardown` | Lists of CLI arguments run untimed with the build under test. `teardown` always runs, even after a failure or timeout. | -| `setupScope` | `run` (default) repeats setup and teardown for every launch; `block` starts one stack per block of five launches. | +| `setupScope` | `run` (default) repeats setup and teardown for every launch; `block` starts one stack per block of two launches. | | `stack` | Sets `SUPABASE_EXPERIMENTAL_STACK=1` on every launch of the workload, the only variable a workload can set. | | `runtime` | `docker` or `native`. A stack workload must pass the same `--runtime` to every `stack start`. | | `fixture` | A directory under `apps/cli/bench/fixtures/` copied into the project's `supabase/` directory after `init`. | diff --git a/apps/cli/scripts/bench-cli.integration.test.ts b/apps/cli/scripts/bench-cli.integration.test.ts index 6744dbe420..0254e056bd 100644 --- a/apps/cli/scripts/bench-cli.integration.test.ts +++ b/apps/cli/scripts/bench-cli.integration.test.ts @@ -204,13 +204,21 @@ describe("runWorkloadBench", () => { expect(report.base).toHaveLength(6); expect(report.branch).toHaveLength(6); const boots = calls.filter((call) => call.command === "boot"); - expect(boots.map((call) => call.build)).toEqual(["base", "branch", "branch", "base"]); - expect(calls.filter((call) => call.command === "halt")).toHaveLength(4); - // One warmup plus five measured launches share the first block's stack. + expect(boots.map((call) => call.build)).toEqual([ + "base", + "branch", + "branch", + "base", + "base", + "branch", + ]); + expect(calls.filter((call) => call.command === "halt")).toHaveLength(6); + expect(report.base.map((sample) => sample.block)).toEqual([0, 0, 1, 1, 2, 2]); + // One warmup plus two measured launches share the first block's stack. const firstBlock = calls.filter((call) => call.home === boots[0]?.home); expect(firstBlock.map((call) => call.command)).toEqual([ "boot", - ...Array.from({ length: 6 }, () => "work"), + ...Array.from({ length: 3 }, () => "work"), "halt", ]); }), diff --git a/apps/cli/scripts/bench-cli.ts b/apps/cli/scripts/bench-cli.ts index a048619fa6..5f8cc7c8ca 100644 --- a/apps/cli/scripts/bench-cli.ts +++ b/apps/cli/scripts/bench-cli.ts @@ -463,11 +463,13 @@ interface LaunchOutcome { /** Planned units of work for one build: a run-scoped unit holds one launch, a block-scoped one many. */ export interface PlannedUnit { readonly build: Build; + /** Block index for block-scoped units. */ + readonly block?: number; readonly launches: ReadonlyArray<"warmup" | "timed">; } /** Measured launches per block for `block`-scoped workloads. */ -export const BLOCK_SIZE = 5; +export const BLOCK_SIZE = 2; /** * Units in execution order. `run` scope reuses {@link buildRunPlan}, one launch per unit. @@ -495,7 +497,7 @@ export function buildUnitPlan( ...Array.from({ length: measured }, () => "timed" as const), ]; const order: ReadonlyArray = block % 2 === 0 ? ["base", "branch"] : ["branch", "base"]; - for (const build of order) units.push({ build, launches }); + for (const build of order) units.push({ build, block, launches }); } return units; } @@ -644,7 +646,7 @@ export const runWorkloadBench = ( const plan = buildUnitPlan(runs, warmup, scope); const started = yield* Clock.currentTimeNanos; - const timed: Record> = { + const timed: Record> = { base: [], branch: [], }; @@ -668,10 +670,14 @@ export const runWorkloadBench = ( (options.cpuProf && build === "branch" && !alreadyProfiled), ); - const record = (build: Build, outcomes: ReadonlyArray) => { + const record = (build: Build, outcomes: ReadonlyArray, block?: number) => { for (const outcome of outcomes) { if (outcome.kind === "timed") { - timed[build].push({ wallMs: outcome.wallMs, exitCode: outcome.exitCode }); + timed[build].push({ + wallMs: outcome.wallMs, + exitCode: outcome.exitCode, + ...(block === undefined ? {} : { block }), + }); } else if (outcome.kind === "traced") { traced[build].push({ exitCode: outcome.exitCode, trace: outcome.trace }); } @@ -728,6 +734,7 @@ export const runWorkloadBench = ( teardownFailures, onProfile, }), + planned.block, ); } diff --git a/apps/cli/scripts/bench-cli.unit.test.ts b/apps/cli/scripts/bench-cli.unit.test.ts index 15f8b80855..f577d8aa74 100644 --- a/apps/cli/scripts/bench-cli.unit.test.ts +++ b/apps/cli/scripts/bench-cli.unit.test.ts @@ -137,6 +137,21 @@ describe("buildUnitPlan", () => { }); }); +describe("buildUnitPlan block sizing", () => { + it("gives ten runs five blocks per build by default, each tagged with its block index", () => { + const plan = buildUnitPlan(10, 1, "block"); + + for (const build of ["base", "branch"] as const) { + expect(plan.filter((unit) => unit.build === build).map((unit) => unit.block)).toEqual([ + 0, 1, 2, 3, 4, + ]); + } + expect( + plan.every((unit) => unit.launches.filter((kind) => kind === "timed").length === 2), + ).toBe(true); + }); +}); + describe("parseBenchArgs for stack runs", () => { it("lets explicit --runs and --warmup override the manifest, and defaults otherwise", () => { expect(parseBenchArgs(required)).toMatchObject({ diff --git a/apps/cli/scripts/bench-report.ts b/apps/cli/scripts/bench-report.ts index a866d304d2..dede4dc6a0 100644 --- a/apps/cli/scripts/bench-report.ts +++ b/apps/cli/scripts/bench-report.ts @@ -29,8 +29,13 @@ export interface TimingSummary { readonly min: number; readonly max: number; readonly p90: number; + readonly iqr: number; } +/** Interquartile range by nearest rank, so one extreme value at either end cannot move it. */ +const iqr = (values: ReadonlyArray): number => + percentile(values, 75) - percentile(values, 25); + export function summarizeTimings(values: ReadonlyArray): TimingSummary { const sorted = [...values].sort((a, b) => a - b); return { @@ -38,6 +43,7 @@ export function summarizeTimings(values: ReadonlyArray): TimingSummary { min: round(sorted[0] ?? 0), max: round(sorted[sorted.length - 1] ?? 0), p90: round(percentile(sorted, 90)), + iqr: round(iqr(sorted)), }; } @@ -289,6 +295,8 @@ const TraceSample = Schema.Struct({ const RunSample = Schema.Struct({ wallMs: Schema.Finite, + /** Block index of a launch from a block-scoped workload; absent for run-scoped launches. */ + block: Schema.optionalKey(Schema.Finite), exitCode: Schema.Finite, /** Exit code of the traced launch, recorded even when no spans could be decoded. */ tracedExitCode: Schema.optionalKey(Schema.Finite), @@ -345,27 +353,88 @@ export const encodeBenchReport = Schema.encodeEffect(BenchReportJson); export const VERDICT_MIN_PCT = 5; export const VERDICT_MIN_MS = 5; export const VERDICT_MIN_SAMPLES = 5; +/** Paired blocks needed before the block, not the launch, is the unit of evidence for a verdict. */ +export const VERDICT_MIN_BLOCKS = 3; +export const VERDICT_IQR_FACTOR = 1.5; export const PHASE_ROWS = 3; export type TimingVerdict = "slower" | "faster" | "within noise" | "not comparable"; +/** Median wall time of one block for each build, paired by block index. */ +export interface BlockPair { + readonly base: number; + readonly branch: number; +} + +const exceedsFloor = (delta: Delta): boolean => + Math.abs(delta.pct) >= VERDICT_MIN_PCT && Math.abs(delta.ms) >= VERDICT_MIN_MS; + /** - * "slower" or "faster" only when the branch median is outside the base min-max range and the - * change is at least {@link VERDICT_MIN_PCT} percent and {@link VERDICT_MIN_MS} ms. + * "slower" or "faster" only when the median change passes the {@link VERDICT_MIN_PCT} percent and + * {@link VERDICT_MIN_MS} ms floor and is larger than {@link VERDICT_IQR_FACTOR} times the wider + * build's interquartile range, so one extreme sample cannot fake a change. Run-scoped timings + * also need the branch median beyond the base range; with at least {@link VERDICT_MIN_BLOCKS} + * paired blocks the spread test runs on block medians instead, and a strict majority of blocks + * must pass the floor in the same direction. */ export function timingVerdict( base: TimingSummary, branch: TimingSummary, - context: { readonly comparable: boolean; readonly samples: number }, + context: { + readonly comparable: boolean; + readonly samples: number; + readonly blocks?: ReadonlyArray; + }, ): TimingVerdict { if (!context.comparable || context.samples < VERDICT_MIN_SAMPLES) return "not comparable"; + const blocks = context.blocks; + if (blocks !== undefined && blocks.length >= VERDICT_MIN_BLOCKS) { + const baseMedians = blocks.map((block) => block.base); + const branchMedians = blocks.map((block) => block.branch); + const reference = median(baseMedians); + const ms = median(blocks.map((block) => block.branch - block.base)); + const pct = reference === 0 ? 0 : (ms / reference) * 100; + if (!exceedsFloor({ ms: round(ms), pct: round(pct) })) return "within noise"; + const agreeing = blocks.filter((block) => { + const blockDelta = deltaOf(block.base, block.branch); + return exceedsFloor(blockDelta) && blockDelta.ms > 0 === ms > 0; + }).length; + if (agreeing * 2 <= blocks.length) return "within noise"; + if (Math.abs(ms) <= VERDICT_IQR_FACTOR * Math.max(iqr(baseMedians), iqr(branchMedians))) { + return "within noise"; + } + return ms > 0 ? "slower" : "faster"; + } const delta = deltaOf(base.median, branch.median); - if (Math.abs(delta.pct) < VERDICT_MIN_PCT || Math.abs(delta.ms) < VERDICT_MIN_MS) { + if (!exceedsFloor(delta)) return "within noise"; + const outsideRange = delta.ms > 0 ? branch.median > base.max : branch.median < base.min; + if (!outsideRange) return "within noise"; + if (Math.abs(delta.ms) <= VERDICT_IQR_FACTOR * Math.max(base.iqr, branch.iqr)) { return "within noise"; } - if (branch.median > base.max) return "slower"; - if (branch.median < base.min) return "faster"; - return "within noise"; + return delta.ms > 0 ? "slower" : "faster"; +} + +/** Pairs the per-block medians of both builds; empty unless every sample carries a block index. */ +export function pairBlocks( + base: ReadonlyArray, + branch: ReadonlyArray, +): ReadonlyArray { + const group = (samples: ReadonlyArray): Map> | undefined => { + const blocks = new Map>(); + for (const sample of samples) { + if (sample.block === undefined) return undefined; + blocks.set(sample.block, [...(blocks.get(sample.block) ?? []), sample.wallMs]); + } + return blocks; + }; + const baseBlocks = group(base); + const branchBlocks = group(branch); + if (baseBlocks === undefined || branchBlocks === undefined) return []; + return [...baseBlocks.entries()].flatMap(([block, values]) => { + const other = branchBlocks.get(block); + return other === undefined ? [] : [{ base: median(values), branch: median(other) }]; + }); } export interface ApiSummary { @@ -559,6 +628,7 @@ export function analyzeWorkload(workload: WorkloadReport, traced: boolean): Work const verdict = timingVerdict(base.timing, branch.timing, { comparable: !differ && runsCleanly(base) && runsCleanly(branch), samples: Math.min(base.samples, branch.samples), + blocks: pairBlocks(workload.base, workload.branch), }); const spans = base.spans === undefined || branch.spans === undefined diff --git a/apps/cli/scripts/bench-report.unit.test.ts b/apps/cli/scripts/bench-report.unit.test.ts index 9f41a41d98..00743cebae 100644 --- a/apps/cli/scripts/bench-report.unit.test.ts +++ b/apps/cli/scripts/bench-report.unit.test.ts @@ -13,6 +13,7 @@ import { parseWorkloadManifest, summarizeSpanRun, summarizeTimings, + pairBlocks, timingVerdict, validateWorkloads, type BenchReport, @@ -51,6 +52,7 @@ describe("summarizeTimings", () => { min: 10, max: 50, p90: 50, + iqr: 20, }); }); }); @@ -180,34 +182,116 @@ describe("classifyHttpSpans", () => { }); }); +const timings = (values: ReadonlyArray) => summarizeTimings(values); +const repeat = (count: number, value: number) => Array.from({ length: count }, () => value); + describe("timingVerdict", () => { - const base = { median: 100, min: 95, max: 108, p90: 107 }; const context = { comparable: true, samples: 10 }; it.each([ - { branch: { median: 120, min: 115, max: 125, p90: 124 }, expected: "slower" }, - { branch: { median: 80, min: 78, max: 90, p90: 88 }, expected: "faster" }, - { branch: { median: 103, min: 99, max: 110, p90: 109 }, expected: "within noise" }, - { branch: { median: 107, min: 99, max: 118, p90: 117 }, expected: "within noise" }, - ])("calls a branch median of $branch.median $expected", ({ branch, expected }) => { - expect(timingVerdict(base, branch, context)).toBe(expected); + { branch: 120, expected: "slower" }, + { branch: 80, expected: "faster" }, + { branch: 103, expected: "within noise" }, + ])("calls a steady branch of $branch ms $expected", ({ branch, expected }) => { + expect(timingVerdict(timings(repeat(10, 100)), timings(repeat(10, branch)), context)).toBe( + expected, + ); }); it("ignores a large percentage change that is under the absolute floor", () => { - expect( - timingVerdict( - { median: 10, min: 9, max: 11, p90: 11 }, - { median: 14, min: 13, max: 15, p90: 15 }, - context, - ), - ).toBe("within noise"); + expect(timingVerdict(timings(repeat(10, 10)), timings(repeat(10, 14)), context)).toBe( + "within noise", + ); }); it("declines to compare when runs are not comparable or too few", () => { - const slow = { median: 200, min: 190, max: 210, p90: 209 }; + const base = timings(repeat(10, 100)); + const slow = timings(repeat(10, 200)); expect(timingVerdict(base, slow, { comparable: false, samples: 10 })).toBe("not comparable"); expect(timingVerdict(base, slow, { comparable: true, samples: 3 })).toBe("not comparable"); }); + + it("flags slower although one branch launch was fast", () => { + const branch = timings([...repeat(9, 125), 90]); + expect(timingVerdict(timings(repeat(10, 100)), branch, context)).toBe("slower"); + }); + + it("reads a branch median inside a noisy base range as noise on real five-run data", () => { + const base = timings([328, 234, 228, 229, 237]); + const branch = timings([303, 317, 273, 268, 313]); + expect(timingVerdict(base, branch, { comparable: true, samples: 5 })).toBe("within noise"); + }); + + it("reads a change past the floor but within 1.5 times the wider spread as noise", () => { + const base = timings([88, 92, 95, 100, 100, 100, 103, 107, 110, 112]); + const branch = timings([105, 110, 112, 115, 115, 115, 118, 120, 122, 125]); + expect(base.iqr).toBe(12); + expect(branch.iqr).toBe(8); + expect(branch.median).toBeGreaterThan(base.max); + expect(timingVerdict(base, branch, context)).toBe("within noise"); + }); +}); + +describe("timingVerdict with blocks", () => { + const context = { comparable: true, samples: 10 }; + const unused = timings(repeat(10, 100)); + const pairs = (...branchMedians: ReadonlyArray) => + branchMedians.map((branch) => ({ base: 100, branch })); + + it("flags slower when four of five blocks are clearly slower and one is fast", () => { + expect( + timingVerdict(unused, unused, { ...context, blocks: pairs(125, 126, 124, 125, 80) }), + ).toBe("slower"); + }); + + it("flags faster when most blocks are clearly faster", () => { + expect(timingVerdict(unused, unused, { ...context, blocks: pairs(75, 74, 76, 75, 120) })).toBe( + "faster", + ); + }); + + it("reads mixed-sign blocks as within noise", () => { + expect(timingVerdict(unused, unused, { ...context, blocks: pairs(180, 70, 130, 75, 85) })).toBe( + "within noise", + ); + }); + + it("reads block medians with wide spread as noise", () => { + const blocks = [ + [222, 276], + [224, 252], + [227, 225], + [233, 299], + [222, 291], + ].map(([base, branch]) => ({ base: base!, branch: branch! })); + expect(timingVerdict(unused, unused, { ...context, blocks })).toBe("within noise"); + }); + + it("falls back to the run path below three blocks", () => { + const branch = timings(repeat(10, 130)); + expect(timingVerdict(unused, branch, { ...context, blocks: pairs(60, 60) })).toBe("slower"); + }); +}); + +describe("pairBlocks", () => { + const at = (block: number, wallMs: number) => ({ + wallMs, + exitCode: 0, + block, + }); + + it("pairs per-block medians of both builds by block index", () => { + expect( + pairBlocks([at(0, 10), at(0, 20), at(1, 30)], [at(0, 40), at(0, 60), at(1, 50)]), + ).toEqual([ + { base: 15, branch: 50 }, + { base: 30, branch: 50 }, + ]); + }); + + it("returns no pairs when a sample has no block index", () => { + expect(pairBlocks([{ wallMs: 10, exitCode: 0 }], [at(0, 10)])).toEqual([]); + }); }); const sample = ( From ba3329c0d2e105bc21c150f6d1e6c4539dccbbd7 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 15:09:33 +0000 Subject: [PATCH 004/101] test(cli): mask default postgres image in container input goldens (#7442) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The local container inputs precedence goldens recorded the default Postgres image tag, so every Postgres bump PR (e.g. supabase/cli#7408) failed integration tests until the goldens were regenerated by hand. The test now redacts the current default database image as ``, keeping the goldens focused on config precedence. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- ...local-container-inputs.precedence.integration.test.ts | 9 ++++++++- .../matched-remote-with-project-id-env.json | 6 +++--- .../matched-remote-with-shell-env.json | 6 +++--- .../local-container-inputs/matched-remote.json | 6 +++--- .../local-container-inputs/no-overrides.json | 6 +++--- .../local-container-inputs/project-dotenv.json | 6 +++--- .../local-container-inputs/shell-env.json | 6 +++--- 7 files changed, 26 insertions(+), 19 deletions(-) diff --git a/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts b/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts index 3cb27f822b..028f197b7a 100644 --- a/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts @@ -4,6 +4,7 @@ import { ConfigProvider, Effect, FileSystem, Layer, Option, Path } from "effect" import { ChildProcessSpawner } from "effect/process"; import * as HttpClient from "effect/http/HttpClient"; import * as HttpClientResponse from "effect/http/HttpClientResponse"; +import { catalogPins } from "@supabase/stack/internal/artifacts"; import { cliConfigValuesTestLayer } from "../../../tests/helpers/config-values-layer.ts"; import { mockRuntimeInfo } from "../../../tests/helpers/mocks.ts"; @@ -16,6 +17,9 @@ import { buildLocalDbContainerInputs } from "./local-container-inputs.ts"; const TARGET_REF = "abcdefghijklmnopqrst"; const GOLDEN_DIR = "./testdata/config-precedence/local-container-inputs"; +const DEFAULT_POSTGRES_IMAGE = catalogPins().find( + (entry) => entry.service === "database" && entry.isDefault, +)!.pin.upstreamImage; const BASE_CONFIG = `project_id = "golden-shadow" @@ -192,7 +196,10 @@ describe("local container inputs precedence goldens", () => { projectEnvValues: configEnvOnly(setup.projectEnvValues ?? {}), }, }; - const goldenText = yield* goldenJson(golden, { [workdir]: "" }); + const goldenText = (yield* goldenJson(golden, { [workdir]: "" })).replaceAll( + `"${DEFAULT_POSTGRES_IMAGE}"`, + `""`, + ); yield* Effect.promise(() => ctx.expect(goldenText).toMatchFileSnapshot(`${GOLDEN_DIR}/${fixture.golden}.json`), ); diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json index dce71ff5c6..c7b02557a0 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": true, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_shell-project-id", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json index 4272d72177..0e84b1fc8b 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": false, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_abcdefghijklmnopqrst", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json index 980df2596f..1fb9218605 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": true, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_abcdefghijklmnopqrst", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json index 2f86ba9c79..b5814743f7 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": true, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_golden-shadow", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json index 3d85d0db5a..0443435132 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": false, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_golden-shadow", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json index a49d200a86..e271fcb6d9 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": false, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_golden-shadow", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, From 02398c63c123d34a0d14bdc12cb0100923a44625 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 15:09:48 +0000 Subject: [PATCH 005/101] fix(api): keep annotated binary request bodies typed as BinaryInput (#7443) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Management API spec now describes the ESZIP update-function body through a shared component that carries a `description` and `contentEncoding`. The generator only rewrote a bare `{ format: "binary" }` annotation to `BinaryInput`, so this body regenerated as `Schema.String` and rejected `Uint8Array`/`ArrayBuffer`/`Blob` uploads (breaking `functions deploy`). The sanitizer now reduces binary string schemas to their type and format, so they keep generating as `BinaryInput`. Generated output on `develop` is unchanged; this unblocks the API sync in https://github.com/supabase/cli/pull/7418, which still needs its v2 PostgREST config fixture updated for the new `db_pool_max_lifetime`/`db_pool_max_idletime` fields. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- packages/api/scripts/generate.ts | 10 ++++++++++ packages/api/scripts/generate.unit.test.ts | 5 +++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/packages/api/scripts/generate.ts b/packages/api/scripts/generate.ts index 5bfd891fe6..da60674cc3 100644 --- a/packages/api/scripts/generate.ts +++ b/packages/api/scripts/generate.ts @@ -307,6 +307,16 @@ export function sanitizeOpenApiSchema( sanitized[key] = rawValue; } + // Binary bodies are rewritten to `BinaryInput` by matching the bare format annotation, so + // extra annotations such as `description` would leave them typed as plain strings. + if (sanitized.type === "string" && sanitized.format === "binary") { + return { + type: "string", + format: "binary", + ...(sanitized.nullable === true ? { nullable: true } : {}), + }; + } + if ( sanitized.type === "string" && sanitized.format === "uuid" && diff --git a/packages/api/scripts/generate.unit.test.ts b/packages/api/scripts/generate.unit.test.ts index a9fed3bd9b..b0d798e946 100644 --- a/packages/api/scripts/generate.unit.test.ts +++ b/packages/api/scripts/generate.unit.test.ts @@ -145,18 +145,19 @@ describe("generate", () => { }); }); - test("drops content annotations from binary strings without touching a property named contentEncoding", () => { + test("reduces binary strings to their format, keeping nullability, without touching a property named contentEncoding", () => { expect( renderOpenApiSchema({ type: "object", additionalProperties: false, properties: { eszip: { type: "string", format: "binary", contentEncoding: "binary" }, + described: { type: "string", format: "binary", description: "Raw ESZIP", nullable: true }, contentEncoding: { type: "string", contentMediaType: "text/plain" }, }, }), ).toBe( - 'Schema.Struct({ "eszip": Schema.optionalKey(Schema.String.annotate({ "format": "binary" })), "contentEncoding": Schema.optionalKey(Schema.String) })', + 'Schema.Struct({ "eszip": Schema.optionalKey(Schema.String.annotate({ "format": "binary" })), "described": Schema.optionalKey(Schema.Union([Schema.String.annotate({ "format": "binary" }), Schema.Null])), "contentEncoding": Schema.optionalKey(Schema.String) })', ); }); From deeaf911774a3adc0641f3878e81277f122edd08 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 15:22:59 +0000 Subject: [PATCH 006/101] chore(stack): bump postgres to 17.11.0.016-r0 (#7408) This bumps postgres from 17.11.0.011-r0 to 17.11.0.016-r0. https://github.com/supabase/slim-services/releases/tag/postgres-17.11.0.016-r0 Planned after supabase/slim-services published 17.11.0.015-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Valleteau --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 0134b90fc6..69f13f15de 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -5,7 +5,7 @@ # a catalog update. Everything else in this file (this comment, kong, pg14, the one-shot job # images) is hand- or Dependabot-managed. Keep a line between image lines: git conflicts on edits to # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. -FROM supabase/postgres:17.11.0.011 AS pg +FROM supabase/postgres:17.11.0.016 AS pg FROM supabase/postgres:15.19.0.016 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 0d0a81ebfa..1bf4bd9c2f 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -127,23 +127,23 @@ const definitions: Readonly> = { database: definition( "postgres", { - upstreamVersion: "17.11.0.011", + upstreamVersion: "17.11.0.016", revision: 0, image: - "ghcr.io/supabase/cli/postgres:17.11.0.011-r0@sha256:9a0eae54eb663be4b8a021382e9d290eb1960398d3b9825856b81d6944c48834", - upstreamImage: "supabase/postgres:17.11.0.011", + "ghcr.io/supabase/cli/postgres:17.11.0.016-r0@sha256:4a8d3f297b52ea72836cf95fcdeb56aff88d2d2efa19b6557bce3d8e98bae02c", + upstreamImage: "supabase/postgres:17.11.0.016", natives: { "darwin-arm64": { - archive: "de01b042eb3da72a3bce4b00b8f07e4b22545d9a2aad83d5b312918eadb39f44", - manifest: "8298f20dac793bb997a3d0110bca2f7296ed038d18ded274a43317360955cbaf", + archive: "5033846ff2dc56a992e9bf929d62d132bd2819656e040d4a359632121a9722e7", + manifest: "0afd146d48203665fb2c8fdfad1e79caba053b9b6d722070bb07ee980f7326fe", }, "linux-amd64": { - archive: "986389e7cdfc81e30c5b2f8e3e15bca98377374c1124b80d1740d0035e657128", - manifest: "10ba0c16e8c5160be1dc7cf4e3698e661c8bd3bf7353b6f78e7022d9cf6a5591", + archive: "29b3bdcbabeb187055018da72dda6875947d113287c365aea5f7cf93fbfac7b0", + manifest: "2cb7d1526df26aaedc7158f7582a7d3c5e2ba75fd32864332f5c8fe21ff8a1ae", }, "linux-arm64": { - archive: "0baab2d07d1437c640aeed7cf60a851c1f81cb85426c8fe5ebfb7ac6ee06dbd0", - manifest: "ae8de4c02229ac55f5db9edd54cb01a674a3d06a858187245af97353707f4556", + archive: "ca9f3927b0d14519b720b674e4a706d318981ceea4b8fffadf6c4c46b689a2ec", + manifest: "dfba4a861d9b95299dc40d69e48ae64e66d8894b50a962b64baebaabeb8e0f55", }, }, }, From feb39e878f00b956f50ffa3e0fa4f4e94462a4ca Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 15:42:00 +0000 Subject: [PATCH 007/101] chore(api): sync Management API OpenAPI spec (#7418) This PR was automatically created to sync the generated `@supabase/api` package with the latest Management API OpenAPI document. Changes were detected in the upstream OpenAPI documents exposed by `https://api.supabase.com/api/v1-json` and `https://api.supabase.com/api/v2-json`. --------- Co-authored-by: jgoux <1443499+jgoux@users.noreply.github.com> Co-authored-by: avallete Co-authored-by: Claude Opus 5.5 Co-authored-by: Andrew Valleteau --- .../src/commands/config/push/push.e2e.test.ts | 2 + .../config/push/push.integration.test.ts | 2 + .../project-config-api-drift.unit.test.ts | 2 + packages/api/src/generated/contracts.ts | 328 ++++- packages/api/src/generated/effect-client.ts | 48 + packages/api/src/generated/openapi.json | 1278 ++++++++++++----- packages/api/src/generated/permissions.ts | 16 +- packages/api/src/internal/client.unit.test.ts | 2 + .../src/project-config/api-attributes.ts | 2 + .../config/src/project-config/registry.ts | 3 +- 10 files changed, 1323 insertions(+), 360 deletions(-) diff --git a/apps/cli/src/commands/config/push/push.e2e.test.ts b/apps/cli/src/commands/config/push/push.e2e.test.ts index 17215fc1a1..de09a2c99e 100644 --- a/apps/cli/src/commands/config/push/push.e2e.test.ts +++ b/apps/cli/src/commands/config/push/push.e2e.test.ts @@ -86,6 +86,8 @@ describe("supabase config push", () => { max_rows: 500, db_pool: null, db_pool_acquisition_timeout: null, + db_pool_max_lifetime: null, + db_pool_max_idletime: null, }); }); } diff --git a/apps/cli/src/commands/config/push/push.integration.test.ts b/apps/cli/src/commands/config/push/push.integration.test.ts index c7d249597b..4dc4a94b07 100644 --- a/apps/cli/src/commands/config/push/push.integration.test.ts +++ b/apps/cli/src/commands/config/push/push.integration.test.ts @@ -139,6 +139,8 @@ const POSTGREST_WRITE_RESPONSE = { max_rows: 0, db_pool: null, db_pool_acquisition_timeout: null, + db_pool_max_lifetime: null, + db_pool_max_idletime: null, }; /** diff --git a/apps/cli/src/shared/config/project-config-api-drift.unit.test.ts b/apps/cli/src/shared/config/project-config-api-drift.unit.test.ts index 2b4c6f7e39..daa6fc9075 100644 --- a/apps/cli/src/shared/config/project-config-api-drift.unit.test.ts +++ b/apps/cli/src/shared/config/project-config-api-drift.unit.test.ts @@ -172,6 +172,8 @@ describe("project-config API type drift guard", () => { max_rows: 500, db_pool_acquisition_timeout: 10, db_pool: null, + db_pool_max_lifetime: null, + db_pool_max_idletime: null, }, database: { major_version: 17, diff --git a/packages/api/src/generated/contracts.ts b/packages/api/src/generated/contracts.ts index 6a546ee8b9..62cd187ef5 100644 --- a/packages/api/src/generated/contracts.ts +++ b/packages/api/src/generated/contracts.ts @@ -4358,6 +4358,9 @@ export const V1GetPerformanceAdvisorsOutput = Schema.Struct({ "log_auth_error_rate_high", "log_storage_error_rate_high", "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit", "project_not_active", "advisor_check_unavailable", ]), @@ -4906,6 +4909,38 @@ export const V1GetPostgrestServiceConfigOutput = Schema.Struct({ ), Schema.Null, ]), + db_pool_max_lifetime: Schema.Union([ + Schema.Number.annotate({ + description: "If `null`, the value is automatically configured to 1800.", + }) + .check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(3600).annotate({ + expected: "a value less than or equal to 3600", + }), + ), + Schema.Null, + ]), + db_pool_max_idletime: Schema.Union([ + Schema.Number.annotate({ + description: "If `null`, the value is automatically configured to 30.", + }) + .check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(60).annotate({ expected: "a value less than or equal to 60" }), + ), + Schema.Null, + ]), jwt_secret: Schema.optionalKey(Schema.String), }); export const V1GetProfileInput = Schema.Record(Schema.String, Schema.Never); @@ -5823,6 +5858,9 @@ export const V1GetSecurityAdvisorsOutput = Schema.Struct({ "log_auth_error_rate_high", "log_storage_error_rate_high", "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit", "project_not_active", "advisor_check_unavailable", ]), @@ -10147,6 +10185,30 @@ export const V1UpdatePostgrestServiceConfigInput = Schema.Struct({ Schema.isLessThanOrEqualTo(60).annotate({ expected: "a value less than or equal to 60" }), ), ), + db_pool_max_lifetime: Schema.optionalKey( + Schema.Number.check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(3600).annotate({ + expected: "a value less than or equal to 3600", + }), + ), + ), + db_pool_max_idletime: Schema.optionalKey( + Schema.Number.check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(60).annotate({ expected: "a value less than or equal to 60" }), + ), + ), }); export const V1UpdatePostgrestServiceConfigOutput = Schema.Struct({ db_schema: Schema.String, @@ -10196,6 +10258,38 @@ export const V1UpdatePostgrestServiceConfigOutput = Schema.Struct({ ), Schema.Null, ]), + db_pool_max_lifetime: Schema.Union([ + Schema.Number.annotate({ + description: "If `null`, the value is automatically configured to 1800.", + }) + .check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(3600).annotate({ + expected: "a value less than or equal to 3600", + }), + ), + Schema.Null, + ]), + db_pool_max_idletime: Schema.Union([ + Schema.Number.annotate({ + description: "If `null`, the value is automatically configured to 30.", + }) + .check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(60).annotate({ expected: "a value less than or equal to 60" }), + ), + Schema.Null, + ]), }); export const V1UpdateProjectApiKeyInput = Schema.Struct({ ref: Schema.String.check( @@ -10713,12 +10807,12 @@ export const V2AssignOrganizationMemberRoleInput = Schema.Struct({ user_id: Schema.String.annotate({ format: "uuid" }).check( Schema.isPattern( new RegExp( - "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$", + "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$", "u", ), ).annotate({ expected: - "a string matching the RegExp ^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$", + "a string matching the RegExp ^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$", }), ), data: Schema.Struct({ @@ -11596,6 +11690,7 @@ export const V2CreateOrganizationInvitationsOutput = Schema.Struct({ "unauthorized", "validation_failed", "agentic_credential_not_found", + "backup_unsupported", "branch_admin_required", "compute_build_context_too_large", "compute_build_in_progress", @@ -12115,6 +12210,30 @@ export const V2DeployAComputeInstanceOutput = Schema.Struct({ }), }), }); +export const V2DownloadProjectBackupInput = Schema.Struct({ + ref: Schema.String.check( + Schema.isMinCodePoints(20).annotate({ expected: "a string with at least 20 code points" }), + ) + .check( + Schema.isMaxCodePoints(20).annotate({ expected: "a string with at most 20 code points" }), + ) + .check( + Schema.isPattern(new RegExp("^[a-z]+$", "u")).annotate({ + expected: "a string matching the RegExp ^[a-z]+$", + }), + ), + data: Schema.Struct({ + type: Schema.Literal("project_backup_file_url").annotate({ description: "Resource type." }), + id: Schema.Number.check(Schema.isFinite().annotate({ expected: "a finite number" })), + }), +}); +export const V2DownloadProjectBackupOutput = Schema.Struct({ + data: Schema.Struct({ + type: Schema.Literal("project_backup_file_url").annotate({ description: "Resource type." }), + id: Schema.Number.check(Schema.isFinite().annotate({ expected: "a finite number" })), + attributes: Schema.Struct({ file_url: Schema.String }), + }), +}); export const V2GetAComputeInstanceInput = Schema.Struct({ ref: Schema.String.check( Schema.isMinCodePoints(20).annotate({ expected: "a string with at least 20 code points" }), @@ -12802,6 +12921,32 @@ export const V2GetProjectConfigOutput = Schema.Struct({ expected: "a value less than or equal to 9007199254740991", }), ), + db_pool_max_lifetime: Schema.Number.check( + Schema.isInt().annotate({ expected: "an integer" }), + ) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(3600).annotate({ + expected: "a value less than or equal to 3600", + }), + ), + db_pool_max_idletime: Schema.Number.check( + Schema.isInt().annotate({ expected: "an integer" }), + ) + .check( + Schema.isGreaterThanOrEqualTo(0).annotate({ + expected: "a value greater than or equal to 0", + }), + ) + .check( + Schema.isLessThanOrEqualTo(60).annotate({ + expected: "a value less than or equal to 60", + }), + ), db_pool: Schema.Union([ Schema.Number.annotate({ description: @@ -13519,6 +13664,68 @@ export const V2ListOrganizationGithubConnectionsOutput = Schema.Struct({ ), }), }); +export const V2ListOrganizationMaintenanceWindowsInput = Schema.Struct({ + slug: Schema.String.check( + Schema.isPattern(new RegExp("^[\\w-]+$", "u")).annotate({ + expected: "a string matching the RegExp ^[\\w-]+$", + }), + ), +}); +export const V2ListOrganizationMaintenanceWindowsOutput = Schema.Struct({ + data: Schema.Array( + Schema.Struct({ + type: Schema.Literal("maintenance_window").annotate({ description: "Resource type." }), + id: Schema.String.annotate({ description: "AWS region." }), + attributes: Schema.Struct({ + window: Schema.Struct({ + day: Schema.Union([ + Schema.Literals(["mon", "tue", "wed", "thu", "fri", "sat", "sun"]).annotate({ + description: "UTC day the window starts.", + }), + OpenString, + ]), + start_utc: Schema.String.annotate({ description: "UTC start time." }), + duration_minutes: Schema.Number.check(Schema.isInt().annotate({ expected: "an integer" })) + .check( + Schema.isGreaterThanOrEqualTo(-9007199254740991).annotate({ + expected: "a value greater than or equal to -9007199254740991", + }), + ) + .check( + Schema.isLessThanOrEqualTo(9007199254740991).annotate({ + expected: "a value less than or equal to 9007199254740991", + }), + ), + }), + current_window: Schema.Union([ + Schema.Struct({ + start_at: Schema.String.annotate({ + description: "Start of the occurrence, inclusive.", + format: "date-time", + }), + end_at: Schema.String.annotate({ + description: "End of the occurrence, exclusive.", + format: "date-time", + }), + }).annotate({ description: "The open occurrence, or null when the window is closed." }), + Schema.Null, + ]), + next_windows: Schema.Array( + Schema.Struct({ + start_at: Schema.String.annotate({ + description: "Start of the occurrence, inclusive.", + format: "date-time", + }), + end_at: Schema.String.annotate({ + description: "End of the occurrence, exclusive.", + format: "date-time", + }), + }), + ).annotate({ description: "The next five occurrences that start after now, in order." }), + }), + }), + ), +}); export const V2ListOrganizationMembersInput = Schema.Struct({ slug: Schema.String.check( Schema.isPattern(new RegExp("^[\\w-]+$", "u")).annotate({ @@ -14077,6 +14284,7 @@ export const V2OrganizationsSlugWebhooksDeliveriesIdGetOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14086,6 +14294,7 @@ export const V2OrganizationsSlugWebhooksDeliveriesIdGetOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", ]).annotate({ description: "Webhook event type." }), OpenString, ]), @@ -14219,6 +14428,7 @@ export const V2OrganizationsSlugWebhooksEndpointsDeleteOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14228,6 +14438,7 @@ export const V2OrganizationsSlugWebhooksEndpointsDeleteOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -14354,6 +14565,7 @@ export const V2OrganizationsSlugWebhooksEndpointsGetOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14363,6 +14575,7 @@ export const V2OrganizationsSlugWebhooksEndpointsGetOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -14503,6 +14716,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdDeleteOutput = Schema.Struct( "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14512,6 +14726,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdDeleteOutput = Schema.Struct( "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -14702,6 +14917,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdDeliveriesGetOutput = Schema. "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14711,6 +14927,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdDeliveriesGetOutput = Schema. "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", ]).annotate({ description: "Webhook event type." }), OpenString, ]), @@ -14849,6 +15066,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdGetOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14858,6 +15076,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdGetOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -14953,6 +15172,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdPatchInput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14962,6 +15182,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdPatchInput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), }), @@ -15062,6 +15283,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdPatchOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15071,6 +15293,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdPatchOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -15141,6 +15364,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdTestPostInput = Schema.Struct "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15150,6 +15374,7 @@ export const V2OrganizationsSlugWebhooksEndpointsIdTestPostInput = Schema.Struct "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", ]).annotate({ description: "Webhook event type." }), }), }), @@ -15205,6 +15430,7 @@ export const V2OrganizationsSlugWebhooksEndpointsPostInput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15214,6 +15440,7 @@ export const V2OrganizationsSlugWebhooksEndpointsPostInput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), }), @@ -15305,6 +15532,7 @@ export const V2OrganizationsSlugWebhooksEndpointsPostOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15314,6 +15542,7 @@ export const V2OrganizationsSlugWebhooksEndpointsPostOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -15504,6 +15733,7 @@ export const V2ProjectsRefWebhooksDeliveriesIdGetOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15513,6 +15743,7 @@ export const V2ProjectsRefWebhooksDeliveriesIdGetOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", ]).annotate({ description: "Webhook event type." }), OpenString, ]), @@ -15658,6 +15889,7 @@ export const V2ProjectsRefWebhooksEndpointsDeleteOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15667,6 +15899,7 @@ export const V2ProjectsRefWebhooksEndpointsDeleteOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -15799,6 +16032,7 @@ export const V2ProjectsRefWebhooksEndpointsGetOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15808,6 +16042,7 @@ export const V2ProjectsRefWebhooksEndpointsGetOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -15954,6 +16189,7 @@ export const V2ProjectsRefWebhooksEndpointsIdDeleteOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15963,6 +16199,7 @@ export const V2ProjectsRefWebhooksEndpointsIdDeleteOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -16159,6 +16396,7 @@ export const V2ProjectsRefWebhooksEndpointsIdDeliveriesGetOutput = Schema.Struct "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16168,6 +16406,7 @@ export const V2ProjectsRefWebhooksEndpointsIdDeliveriesGetOutput = Schema.Struct "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", ]).annotate({ description: "Webhook event type." }), OpenString, ]), @@ -16312,6 +16551,7 @@ export const V2ProjectsRefWebhooksEndpointsIdGetOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16321,6 +16561,7 @@ export const V2ProjectsRefWebhooksEndpointsIdGetOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -16422,6 +16663,7 @@ export const V2ProjectsRefWebhooksEndpointsIdPatchInput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16431,6 +16673,7 @@ export const V2ProjectsRefWebhooksEndpointsIdPatchInput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), }), @@ -16531,6 +16774,7 @@ export const V2ProjectsRefWebhooksEndpointsIdPatchOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16540,6 +16784,7 @@ export const V2ProjectsRefWebhooksEndpointsIdPatchOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -16616,6 +16861,7 @@ export const V2ProjectsRefWebhooksEndpointsIdTestPostInput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16625,6 +16871,7 @@ export const V2ProjectsRefWebhooksEndpointsIdTestPostInput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", ]).annotate({ description: "Webhook event type." }), }), }), @@ -16686,6 +16933,7 @@ export const V2ProjectsRefWebhooksEndpointsPostInput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16695,6 +16943,7 @@ export const V2ProjectsRefWebhooksEndpointsPostInput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), }), @@ -16786,6 +17035,7 @@ export const V2ProjectsRefWebhooksEndpointsPostOutput = Schema.Struct({ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16795,6 +17045,7 @@ export const V2ProjectsRefWebhooksEndpointsPostOutput = Schema.Struct({ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*", ]).annotate({ description: "Webhook event type." }), OpenString, @@ -16832,6 +17083,24 @@ export const V2ProjectsRefWebhooksEndpointsPostOutput = Schema.Struct({ }), }), }); +export const V2RemoveOrganizationMembersInput = Schema.Struct({ + slug: Schema.String.check( + Schema.isPattern(new RegExp("^[\\w-]+$", "u")).annotate({ + expected: "a string matching the RegExp ^[\\w-]+$", + }), + ), + user_id: Schema.String.annotate({ format: "uuid" }).check( + Schema.isPattern( + new RegExp( + "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$", + "u", + ), + ).annotate({ + expected: + "a string matching the RegExp ^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$", + }), + ), +}); export const V2RunProjectAdvisorsInput = Schema.Struct({ ref: Schema.String.check( Schema.isMinCodePoints(20).annotate({ expected: "a string with at least 20 code points" }), @@ -16891,6 +17160,9 @@ export const V2RunProjectAdvisorsInput = Schema.Struct({ "log_auth_error_rate_high", "log_storage_error_rate_high", "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit", ]), }), ) @@ -16898,7 +17170,7 @@ export const V2RunProjectAdvisorsInput = Schema.Struct({ Schema.isMinLength(1).annotate({ expected: "a value with a length of at least 1" }), ) .check( - Schema.isMaxLength(10).annotate({ expected: "a value with a length of at most 10" }), + Schema.isMaxLength(15).annotate({ expected: "a value with a length of at most 15" }), ), }), }), @@ -16953,6 +17225,9 @@ export const V2RunProjectAdvisorsOutput = Schema.Struct({ "log_auth_error_rate_high", "log_storage_error_rate_high", "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit", "project_not_active", "advisor_check_unavailable", ]), @@ -17682,6 +17957,7 @@ export const V2DeleteLogDrainOutput = Schema.Void; export const V2DeleteNotebookOutput = Schema.Void; export const V2DeletePrivateLinkAssociationOutput = Schema.Void; export const V2DeletePrivateLinkAssociationForDatabaseOutput = Schema.Void; +export const V2RemoveOrganizationMembersOutput = Schema.Void; export const V2TransferAProjectOutput = Schema.Void; export const openApiOperationIdMap = { @@ -17869,6 +18145,7 @@ export const openApiOperationIdMap = { "v2-delete-private-link-association": "v2DeletePrivateLinkAssociation", "v2-delete-private-link-association-for-database": "v2DeletePrivateLinkAssociationForDatabase", "v2-deploy-a-compute-instance": "v2DeployAComputeInstance", + "v2-download-project-backup": "v2DownloadProjectBackup", "v2-get-a-compute-instance": "v2GetAComputeInstance", "v2-get-notebook": "v2GetNotebook", "v2-get-project-config": "v2GetProjectConfig", @@ -17876,6 +18153,7 @@ export const openApiOperationIdMap = { "v2-list-log-drains": "v2ListLogDrains", "v2-list-notebooks": "v2ListNotebooks", "v2-list-organization-github-connections": "v2ListOrganizationGithubConnections", + "v2-list-organization-maintenance-windows": "v2ListOrganizationMaintenanceWindows", "v2-list-organization-members": "v2ListOrganizationMembers", "v2-list-organization-projects": "v2ListOrganizationProjects", "v2-list-organization-roles": "v2ListOrganizationRoles", @@ -17907,6 +18185,7 @@ export const openApiOperationIdMap = { "v2-projects-ref-webhooks-endpoints-id-patch": "v2ProjectsRefWebhooksEndpointsIdPatch", "v2-projects-ref-webhooks-endpoints-id-test-post": "v2ProjectsRefWebhooksEndpointsIdTestPost", "v2-projects-ref-webhooks-endpoints-post": "v2ProjectsRefWebhooksEndpointsPost", + "v2-remove-organization-members": "v2RemoveOrganizationMembers", "v2-run-project-advisors": "v2RunProjectAdvisors", "v2-transfer-a-project": "v2TransferAProject", "v2-update-log-drain": "v2UpdateLogDrain", @@ -20500,6 +20779,8 @@ export const operationDefinitions = { "max_rows", "db_pool", "db_pool_acquisition_timeout", + "db_pool_max_lifetime", + "db_pool_max_idletime", ], }, response: { kind: "json" }, @@ -20852,6 +21133,19 @@ export const operationDefinitions = { inputSchema: V2DeployAComputeInstanceInput, outputSchema: V2DeployAComputeInstanceOutput, }, + v2DownloadProjectBackup: { + id: "v2DownloadProjectBackup", + description: "Download project backup", + method: "POST", + path: "/v2/projects/{ref}/backups/download", + pathParams: ["ref"], + queryParams: [], + headerParams: [], + requestBody: { kind: "json", contentType: "application/json", fields: ["data"] }, + response: { kind: "json" }, + inputSchema: V2DownloadProjectBackupInput, + outputSchema: V2DownloadProjectBackupOutput, + }, v2GetAComputeInstance: { id: "v2GetAComputeInstance", description: @@ -20948,6 +21242,20 @@ export const operationDefinitions = { inputSchema: V2ListOrganizationGithubConnectionsInput, outputSchema: V2ListOrganizationGithubConnectionsOutput, }, + v2ListOrganizationMaintenanceWindows: { + id: "v2ListOrganizationMaintenanceWindows", + description: + "Returns the weekly maintenance window for each region, with the open occurrence and the next five.", + method: "GET", + path: "/v2/organizations/{slug}/maintenance/windows", + pathParams: ["slug"], + queryParams: [], + headerParams: [], + requestBody: { kind: "none" }, + response: { kind: "json" }, + inputSchema: V2ListOrganizationMaintenanceWindowsInput, + outputSchema: V2ListOrganizationMaintenanceWindowsOutput, + }, v2ListOrganizationMembers: { id: "v2ListOrganizationMembers", description: @@ -21286,6 +21594,20 @@ export const operationDefinitions = { inputSchema: V2ProjectsRefWebhooksEndpointsPostInput, outputSchema: V2ProjectsRefWebhooksEndpointsPostOutput, }, + v2RemoveOrganizationMembers: { + id: "v2RemoveOrganizationMembers", + description: + "Removes one or more organization members including their roles and project-scoped permissions.", + method: "DELETE", + path: "/v2/organizations/{slug}/members/{user_id}", + pathParams: ["slug", "user_id"], + queryParams: [], + headerParams: [], + requestBody: { kind: "none" }, + response: { kind: "void" }, + inputSchema: V2RemoveOrganizationMembersInput, + outputSchema: V2RemoveOrganizationMembersOutput, + }, v2RunProjectAdvisors: { id: "v2RunProjectAdvisors", description: "Runs the project advisors with the given names", diff --git a/packages/api/src/generated/effect-client.ts b/packages/api/src/generated/effect-client.ts index d1f8bd2150..59c9ea29aa 100644 --- a/packages/api/src/generated/effect-client.ts +++ b/packages/api/src/generated/effect-client.ts @@ -2948,6 +2948,22 @@ export const versionedEffectOperations = { options, ); }), + downloadProjectBackup: ( + input: typeof operationDefinitions.v2DownloadProjectBackup.inputSchema.Type, + options?: SupabaseApiRequestOptions, + ): Effect.Effect< + typeof operationDefinitions.v2DownloadProjectBackup.outputSchema.Type, + SupabaseApiError, + SupabaseApiClient + > => + Effect.gen(function* () { + const client = yield* SupabaseApiClient; + return yield* client.execute<"v2DownloadProjectBackup">( + operationDefinitions.v2DownloadProjectBackup, + input, + options, + ); + }), getAComputeInstance: ( input: typeof operationDefinitions.v2GetAComputeInstance.inputSchema.Type, options?: SupabaseApiRequestOptions, @@ -3060,6 +3076,22 @@ export const versionedEffectOperations = { options, ); }), + listOrganizationMaintenanceWindows: ( + input: typeof operationDefinitions.v2ListOrganizationMaintenanceWindows.inputSchema.Type, + options?: SupabaseApiRequestOptions, + ): Effect.Effect< + typeof operationDefinitions.v2ListOrganizationMaintenanceWindows.outputSchema.Type, + SupabaseApiError, + SupabaseApiClient + > => + Effect.gen(function* () { + const client = yield* SupabaseApiClient; + return yield* client.execute<"v2ListOrganizationMaintenanceWindows">( + operationDefinitions.v2ListOrganizationMaintenanceWindows, + input, + options, + ); + }), listOrganizationMembers: ( input: typeof operationDefinitions.v2ListOrganizationMembers.inputSchema.Type, options?: SupabaseApiRequestOptions, @@ -3460,6 +3492,22 @@ export const versionedEffectOperations = { options, ); }), + removeOrganizationMembers: ( + input: typeof operationDefinitions.v2RemoveOrganizationMembers.inputSchema.Type, + options?: SupabaseApiRequestOptions, + ): Effect.Effect< + typeof operationDefinitions.v2RemoveOrganizationMembers.outputSchema.Type, + SupabaseApiError, + SupabaseApiClient + > => + Effect.gen(function* () { + const client = yield* SupabaseApiClient; + return yield* client.execute<"v2RemoveOrganizationMembers">( + operationDefinitions.v2RemoveOrganizationMembers, + input, + options, + ); + }), runProjectAdvisors: ( input: typeof operationDefinitions.v2RunProjectAdvisors.inputSchema.Type, options?: SupabaseApiRequestOptions, diff --git a/packages/api/src/generated/openapi.json b/packages/api/src/generated/openapi.json index 9f80f6f044..42344c6444 100644 --- a/packages/api/src/generated/openapi.json +++ b/packages/api/src/generated/openapi.json @@ -2164,6 +2164,9 @@ "403": { "description": "Forbidden action" }, + "404": { + "description": "API key not found for project" + }, "429": { "description": "Rate limit exceeded" } @@ -2240,6 +2243,9 @@ "403": { "description": "Forbidden action" }, + "404": { + "description": "API key not found for project" + }, "429": { "description": "Rate limit exceeded" } @@ -2338,6 +2344,9 @@ "403": { "description": "Forbidden action" }, + "404": { + "description": "API key not found for project" + }, "429": { "description": "Rate limit exceeded" } @@ -9248,13 +9257,12 @@ "content": { "application/vnd.denoland.eszip": { "schema": { - "type": "string", - "format": "binary" + "$ref": "#/components/schemas/V1UpdateFunctionBodyEszip" } }, "application/json": { "schema": { - "$ref": "#/components/schemas/V1UpdateFunctionBody" + "$ref": "#/components/schemas/V1UpdateFunctionBodyJson" } } } @@ -12045,6 +12053,114 @@ "x-fga-permissions": [["advisors_read"]] } }, + "/v2/projects/{ref}/backups/download": { + "post": { + "operationId": "v2-download-project-backup", + "parameters": [ + { + "name": "ref", + "required": true, + "in": "path", + "description": "Project ref", + "schema": { + "minLength": 20, + "maxLength": 20, + "pattern": "^[a-z]+$", + "example": "abcdefghijklmnopqrst", + "type": "string" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DownloadBackupBody" + } + } + } + }, + "responses": { + "201": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DownloadBackupResponse_Output" + } + } + } + }, + "401": { + "description": "Unauthorized", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "403": { + "description": "Forbidden action", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "404": { + "description": "Project not found", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "429": { + "description": "Rate limit exceeded", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "500": { + "description": "Failed to download project backup", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + } + }, + "security": [ + { + "bearer": [] + } + ], + "summary": "Download project backup", + "tags": ["Projects"], + "x-badges": [ + { + "name": "OAuth scope: database:write", + "position": "after" + } + ], + "x-endpoint-owners": ["infra", "control-plane"], + "x-fga-permissions": [["backups_write"]], + "x-oauth-scope": "database:write" + } + }, "/v2/projects/{ref}/branches": { "post": { "description": "Creates a database branch from the specified project. Compute and disk size can be set here so the branch is provisioned at the requested size, instead of being resized after creation.", @@ -13816,10 +13932,10 @@ "x-scalar-stability": "experimental" } }, - "/v2/organizations/{slug}/members": { + "/v2/organizations/{slug}/roles": { "get": { - "description": "Returns a cursor-paginated list of organization members including their roles and project-scoped permissions.", - "operationId": "v2-list-organization-members", + "description": "Returns a list of org-level roles for the organization.", + "operationId": "v2-list-organization-roles", "parameters": [ { "name": "slug", @@ -13831,51 +13947,6 @@ "example": "tsrqponmlkjihgfedcba", "type": "string" } - }, - { - "name": "page", - "required": false, - "in": "query", - "schema": { - "properties": { - "size": { - "type": "integer", - "minimum": 1, - "maximum": 100 - }, - "after": { - "type": "string", - "format": "uuid", - "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$" - }, - "before": { - "type": "string", - "format": "uuid", - "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$" - } - }, - "type": "object" - }, - "style": "deepObject" - }, - { - "name": "filter", - "required": false, - "in": "query", - "schema": { - "properties": { - "username": { - "type": "string" - }, - "primary_email": { - "type": "string", - "format": "email", - "pattern": "^(?!\\.)(?!.*\\.\\.)([A-Za-z0-9_'+\\-\\.]*)[A-Za-z0-9_+-]@([A-Za-z0-9][A-Za-z0-9\\-]*\\.)+[A-Za-z]{2,}$" - } - }, - "type": "object" - }, - "style": "deepObject" } ], "responses": { @@ -13884,7 +13955,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2ListMembersResponse_Output" + "$ref": "#/components/schemas/V2ListRolesResponse_Output" } } } @@ -13925,7 +13996,7 @@ "bearer": [] } ], - "summary": "List members of an organization", + "summary": "List roles of an organization", "tags": ["Organizations"], "x-badges": [ { @@ -13938,10 +14009,10 @@ "x-oauth-scope": "organizations:read" } }, - "/v2/organizations/{slug}/members/{user_id}/roles": { - "patch": { - "description": "Assigns an org-wide role when projects is omitted, or creates a project-scoped assignment when projects is provided. Uses an org-level role template id from GET /v2/organizations/{slug}/roles. Stale role assignments are automatically cleaned up: if a role no longer has any projects, it is deleted; overlapping project assignments in other roles are automatically removed to avoid duplication.", - "operationId": "v2-assign-organization-member-role", + "/v2/organizations/{slug}/members/invitations": { + "post": { + "description": "Creates member invitations for an organization. Each invitation can have different role and project scope settings.", + "operationId": "v2-create-organization-invitations", "parameters": [ { "name": "slug", @@ -13953,14 +14024,104 @@ "example": "tsrqponmlkjihgfedcba", "type": "string" } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2CreateInvitationsRequest" + } + } + } + }, + "responses": { + "201": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2CreateInvitationsResponse_Output" + } + } + } + }, + "401": { + "description": "Unauthorized", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } }, + "402": { + "description": "This feature requires the Team, or Enterprise organization plan.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "403": { + "description": "Forbidden action", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "429": { + "description": "Rate limit exceeded", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + } + }, + "security": [ { - "name": "user_id", + "bearer": [] + } + ], + "summary": "Creates organization invitations", + "tags": ["Organizations Members Invitations"], + "x-allowed-plans": ["Team", "Enterprise"], + "x-badges": [ + { + "name": "OAuth scope: organizations:write", + "position": "after" + }, + { + "name": "Only available on Team, Enterprise", + "position": "before" + } + ], + "x-endpoint-owners": ["control-plane"], + "x-fga-permissions": [["members_write"]], + "x-oauth-scope": "organizations:write" + }, + "delete": { + "description": "Bulk delete member invitations for an organization by email address.", + "operationId": "v2-delete-organization-invitations", + "parameters": [ + { + "name": "slug", "required": true, "in": "path", + "description": "Organization slug", "schema": { - "format": "uuid", - "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$", + "pattern": "^[\\w-]+$", + "example": "tsrqponmlkjihgfedcba", "type": "string" } } @@ -13970,7 +14131,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2AssignOrganizationMemberRoleRequest" + "$ref": "#/components/schemas/V2DeleteInvitationsRequest" } } } @@ -13981,7 +14142,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/OrganizationMemberRoleResponse_Output" + "$ref": "#/components/schemas/V2DeleteInvitationsResponse_Output" } } } @@ -14025,16 +14186,6 @@ } } } - }, - "500": { - "description": "Failed to assign organization member role", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/ErrorResponseBody" - } - } - } } }, "security": [ @@ -14042,23 +14193,28 @@ "bearer": [] } ], - "summary": "Assign or change an organization member role", - "tags": ["Organizations"], + "summary": "Deletes organization invitations by email", + "tags": ["Organizations Members Invitations"], "x-allowed-plans": ["Team", "Enterprise"], "x-badges": [ + { + "name": "OAuth scope: organizations:write", + "position": "after" + }, { "name": "Only available on Team, Enterprise", "position": "before" } ], "x-endpoint-owners": ["control-plane"], - "x-fga-permissions": [["organization_admin_write"]] + "x-fga-permissions": [["members_write"]], + "x-oauth-scope": "organizations:write" } }, - "/v2/organizations/{slug}/roles": { + "/v2/organizations/{slug}/members": { "get": { - "description": "Returns a list of org-level roles for the organization.", - "operationId": "v2-list-organization-roles", + "description": "Returns a cursor-paginated list of organization members including their roles and project-scoped permissions.", + "operationId": "v2-list-organization-members", "parameters": [ { "name": "slug", @@ -14070,6 +14226,51 @@ "example": "tsrqponmlkjihgfedcba", "type": "string" } + }, + { + "name": "page", + "required": false, + "in": "query", + "schema": { + "properties": { + "size": { + "type": "integer", + "minimum": 1, + "maximum": 100 + }, + "after": { + "type": "string", + "format": "uuid", + "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$" + }, + "before": { + "type": "string", + "format": "uuid", + "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$" + } + }, + "type": "object" + }, + "style": "deepObject" + }, + { + "name": "filter", + "required": false, + "in": "query", + "schema": { + "properties": { + "username": { + "type": "string" + }, + "primary_email": { + "type": "string", + "format": "email", + "pattern": "^(?!\\.)(?!.*\\.\\.)([A-Za-z0-9_'+\\-\\.]*)[A-Za-z0-9_+-]@([A-Za-z0-9][A-Za-z0-9\\-]*\\.)+[A-Za-z]{2,}$" + } + }, + "type": "object" + }, + "style": "deepObject" } ], "responses": { @@ -14078,7 +14279,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2ListRolesResponse_Output" + "$ref": "#/components/schemas/V2ListMembersResponse_Output" } } } @@ -14119,7 +14320,7 @@ "bearer": [] } ], - "summary": "List roles of an organization", + "summary": "List members of an organization", "tags": ["Organizations"], "x-badges": [ { @@ -14132,10 +14333,10 @@ "x-oauth-scope": "organizations:read" } }, - "/v2/organizations/{slug}/members/invitations": { - "post": { - "description": "Creates member invitations for an organization. Each invitation can have different role and project scope settings.", - "operationId": "v2-create-organization-invitations", + "/v2/organizations/{slug}/members/{user_id}/roles": { + "patch": { + "description": "Assigns an org-wide role when projects is omitted, or creates a project-scoped assignment when projects is provided. Uses an org-level role template id from GET /v2/organizations/{slug}/roles. Stale role assignments are automatically cleaned up: if a role no longer has any projects, it is deleted; overlapping project assignments in other roles are automatically removed to avoid duplication.", + "operationId": "v2-assign-organization-member-role", "parameters": [ { "name": "slug", @@ -14147,6 +14348,16 @@ "example": "tsrqponmlkjihgfedcba", "type": "string" } + }, + { + "name": "user_id", + "required": true, + "in": "path", + "schema": { + "format": "uuid", + "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$", + "type": "string" + } } ], "requestBody": { @@ -14154,18 +14365,18 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2CreateInvitationsRequest" + "$ref": "#/components/schemas/V2AssignOrganizationMemberRoleRequest" } } } }, "responses": { - "201": { + "200": { "description": "", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2CreateInvitationsResponse_Output" + "$ref": "#/components/schemas/OrganizationMemberRoleResponse_Output" } } } @@ -14209,6 +14420,16 @@ } } } + }, + "500": { + "description": "Failed to assign organization member role", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } } }, "security": [ @@ -14216,8 +14437,8 @@ "bearer": [] } ], - "summary": "Creates organization invitations", - "tags": ["Organizations Members Invitations"], + "summary": "Assign or change an organization member role", + "tags": ["Organizations"], "x-allowed-plans": ["Team", "Enterprise"], "x-badges": [ { @@ -14230,12 +14451,14 @@ } ], "x-endpoint-owners": ["control-plane"], - "x-fga-permissions": [["members_write"]], + "x-fga-permissions": [["organization_admin_write"]], "x-oauth-scope": "organizations:write" - }, + } + }, + "/v2/organizations/{slug}/members/{user_id}": { "delete": { - "description": "Bulk delete member invitations for an organization by email address.", - "operationId": "v2-delete-organization-invitations", + "description": "Removes one or more organization members including their roles and project-scoped permissions.", + "operationId": "v2-remove-organization-members", "parameters": [ { "name": "slug", @@ -14247,31 +14470,34 @@ "example": "tsrqponmlkjihgfedcba", "type": "string" } - } - ], - "requestBody": { - "required": true, - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2DeleteInvitationsRequest" - } + }, + { + "name": "user_id", + "required": true, + "in": "path", + "schema": { + "format": "uuid", + "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$", + "type": "string" } } - }, + ], "responses": { - "200": { - "description": "", + "204": { + "description": "" + }, + "401": { + "description": "Unauthorized", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2DeleteInvitationsResponse_Output" + "$ref": "#/components/schemas/ErrorResponseBody" } } } }, - "401": { - "description": "Unauthorized", + "403": { + "description": "Forbidden action", "content": { "application/json": { "schema": { @@ -14280,8 +14506,8 @@ } } }, - "402": { - "description": "This feature requires the Team, or Enterprise organization plan.", + "404": { + "description": "Organization member not found", "content": { "application/json": { "schema": { @@ -14290,8 +14516,8 @@ } } }, - "403": { - "description": "Forbidden action", + "429": { + "description": "Rate limit exceeded", "content": { "application/json": { "schema": { @@ -14300,8 +14526,8 @@ } } }, - "429": { - "description": "Rate limit exceeded", + "500": { + "description": "Failed to remove organization member", "content": { "application/json": { "schema": { @@ -14316,21 +14542,16 @@ "bearer": [] } ], - "summary": "Deletes organization invitations by email", - "tags": ["Organizations Members Invitations"], - "x-allowed-plans": ["Team", "Enterprise"], + "summary": "Removes organization members", + "tags": ["Organizations"], "x-badges": [ { "name": "OAuth scope: organizations:write", "position": "after" - }, - { - "name": "Only available on Team, Enterprise", - "position": "before" } ], "x-endpoint-owners": ["control-plane"], - "x-fga-permissions": [["members_write"]], + "x-fga-permissions": [["organization_admin_write"]], "x-oauth-scope": "organizations:write" } }, @@ -14584,6 +14805,94 @@ "x-oauth-scope": "projects:read" } }, + "/v2/organizations/{slug}/maintenance/windows": { + "get": { + "description": "Returns the weekly maintenance window for each region, with the open occurrence and the next five.", + "operationId": "v2-list-organization-maintenance-windows", + "parameters": [ + { + "name": "slug", + "required": true, + "in": "path", + "description": "Organization slug", + "schema": { + "pattern": "^[\\w-]+$", + "example": "tsrqponmlkjihgfedcba", + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2ListMaintenanceWindowsResponse_Output" + } + } + } + }, + "401": { + "description": "Unauthorized", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "403": { + "description": "Forbidden action", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "404": { + "description": "Organization not found", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + }, + "429": { + "description": "Rate limit exceeded", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponseBody" + } + } + } + } + }, + "security": [ + { + "bearer": [] + } + ], + "summary": "List maintenance windows of an organization", + "tags": ["Organizations"], + "x-badges": [ + { + "name": "OAuth scope: organizations:read", + "position": "after" + } + ], + "x-endpoint-owners": ["infra"], + "x-fga-permissions": [["organization_admin_read"]], + "x-internal": true, + "x-oauth-scope": "organizations:read" + } + }, "/v2/projects/{ref}/webhooks/endpoints": { "get": { "operationId": "v2-projects-ref-webhooks-endpoints-get", @@ -14698,6 +15007,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -14707,6 +15017,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -15473,6 +15784,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -15482,6 +15794,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -16162,6 +16475,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16171,6 +16485,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -16311,6 +16626,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -16320,6 +16636,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -17045,6 +17362,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -17054,6 +17372,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -17871,6 +18190,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -17880,6 +18200,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -18654,6 +18975,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -18663,6 +18985,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -18811,6 +19134,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -18820,6 +19144,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -19724,6 +20049,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -19732,7 +20058,8 @@ "v1.organization.member.role.removed", "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", - "v1.organization.billing.plan.downgraded" + "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked" ], "description": "Webhook event type." }, @@ -21429,6 +21756,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -21437,7 +21765,8 @@ "v1.organization.member.role.removed", "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", - "v1.organization.billing.plan.downgraded" + "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked" ], "description": "Webhook event type." } @@ -21610,6 +21939,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -21618,7 +21948,8 @@ "v1.organization.member.role.removed", "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", - "v1.organization.billing.plan.downgraded" + "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked" ], "description": "Webhook event type." }, @@ -23164,6 +23495,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -23173,6 +23505,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -23937,6 +24270,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -23946,6 +24280,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -24626,6 +24961,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -24635,6 +24971,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -24773,6 +25110,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -24782,6 +25120,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -25505,6 +25844,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -25514,6 +25854,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -26329,6 +26670,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -26338,6 +26680,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -27112,6 +27455,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -27121,6 +27465,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -27267,6 +27612,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -27276,6 +27622,7 @@ "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked", "*" ], "description": "Webhook event type." @@ -28178,6 +28525,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -28186,7 +28534,8 @@ "v1.organization.member.role.removed", "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", - "v1.organization.billing.plan.downgraded" + "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked" ], "description": "Webhook event type." }, @@ -29881,6 +30230,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -29889,7 +30239,8 @@ "v1.organization.member.role.removed", "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", - "v1.organization.billing.plan.downgraded" + "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked" ], "description": "Webhook event type." } @@ -30060,6 +30411,7 @@ "v1.project.branch.created", "v1.project.branch.updated", "v1.project.branch.removed", + "v1.project.secrets.leaked", "v1.organization.member.invitation.created", "v1.organization.member.invitation.canceled", "v1.organization.member.added", @@ -30068,7 +30420,8 @@ "v1.organization.member.role.removed", "v1.organization.member.role.updated", "v1.organization.billing.plan.upgraded", - "v1.organization.billing.plan.downgraded" + "v1.organization.billing.plan.downgraded", + "v1.organization.secrets.leaked" ], "description": "Webhook event type." }, @@ -33472,6 +33825,20 @@ "description": "If `null`, the value is automatically configured to 10.", "nullable": true }, + "db_pool_max_lifetime": { + "type": "integer", + "minimum": 0, + "maximum": 3600, + "description": "If `null`, the value is automatically configured to 1800.", + "nullable": true + }, + "db_pool_max_idletime": { + "type": "integer", + "minimum": 0, + "maximum": 60, + "description": "If `null`, the value is automatically configured to 30.", + "nullable": true + }, "jwt_secret": { "type": "string" } @@ -33481,7 +33848,9 @@ "max_rows", "db_extra_search_path", "db_pool", - "db_pool_acquisition_timeout" + "db_pool_acquisition_timeout", + "db_pool_max_lifetime", + "db_pool_max_idletime" ] }, "V1UpdatePostgrestConfigBody": { @@ -33507,6 +33876,16 @@ "type": "integer", "minimum": 0, "maximum": 60 + }, + "db_pool_max_lifetime": { + "type": "integer", + "minimum": 0, + "maximum": 3600 + }, + "db_pool_max_idletime": { + "type": "integer", + "minimum": 0, + "maximum": 60 } }, "example": { @@ -33542,6 +33921,20 @@ "maximum": 9007199254740991, "description": "If `null`, the value is automatically configured to 10.", "nullable": true + }, + "db_pool_max_lifetime": { + "type": "integer", + "minimum": 0, + "maximum": 3600, + "description": "If `null`, the value is automatically configured to 1800.", + "nullable": true + }, + "db_pool_max_idletime": { + "type": "integer", + "minimum": 0, + "maximum": 60, + "description": "If `null`, the value is automatically configured to 30.", + "nullable": true } }, "required": [ @@ -33549,7 +33942,9 @@ "max_rows", "db_extra_search_path", "db_pool", - "db_pool_acquisition_timeout" + "db_pool_acquisition_timeout", + "db_pool_max_lifetime", + "db_pool_max_idletime" ] }, "V1ProjectRefResponse_Output": { @@ -37322,6 +37717,9 @@ "log_auth_error_rate_high", "log_storage_error_rate_high", "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit", "project_not_active", "advisor_check_unavailable" ] @@ -38756,7 +39154,7 @@ "type": "object", "properties": {} }, - "V1UpdateFunctionBody": { + "V1UpdateFunctionBodyJson": { "type": "object", "properties": { "name": { @@ -38775,6 +39173,13 @@ "verify_jwt": true } }, + "V1UpdateFunctionBodyEszip": { + "type": "string", + "format": "binary", + "contentEncoding": "binary", + "description": "Raw Brotli-compressed ESZIP binary file.\n\n**Validation requirements** - The file must start with the four magic bytes `EZBR` (hex: `45 5A 42 52`).\n- The `ezbr_sha256` query parameter must contain the SHA-256 hash of the uploaded ESZIP payload.\n\n**Example**\n```javascript\nimport { readFile } from 'node:fs/promises';\nimport { brotliCompress } from 'node:zlib';\nimport { promisify } from 'node:util';\nimport { createHash } from 'node:crypto';\n\nconst compress = promisify(brotliCompress);\nconst marker = Buffer.from('EZBR', 'ascii');\n\nconst file = await readFile('./function.eszip');\nconst compressed = await compress(file);\n\nconst payload = Buffer.concat([marker, compressed]);\n\nconst ezbr_sha256 = createHash('sha256')\n .update(payload)\n .digest('hex');\n\nconsole.log(ezbr_sha256);\n```\n\n", + "examples": ["Binary Brotli-compressed ESZIP file"] + }, "V1StorageBucketResponse_Output": { "type": "object", "properties": { @@ -41443,6 +41848,7 @@ "unauthorized", "validation_failed", "agentic_credential_not_found", + "backup_unsupported", "branch_admin_required", "compute_build_context_too_large", "compute_build_in_progress", @@ -42295,7 +42701,7 @@ "properties": { "lints": { "minItems": 1, - "maxItems": 10, + "maxItems": 15, "type": "array", "items": { "type": "object", @@ -42342,7 +42748,10 @@ "log_data_api_error_rate_high", "log_auth_error_rate_high", "log_storage_error_rate_high", - "log_edge_function_error_rate_high" + "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit" ] } }, @@ -42423,6 +42832,9 @@ "log_auth_error_rate_high", "log_storage_error_rate_high", "log_edge_function_error_rate_high", + "realtime_authorization_timeouts", + "realtime_postgres_changes_spike", + "realtime_concurrent_users_near_limit", "project_not_active", "advisor_check_unavailable" ] @@ -42524,6 +42936,55 @@ }, "required": ["data"] }, + "DownloadBackupBody": { + "type": "object", + "properties": { + "data": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Resource type.", + "enum": ["project_backup_file_url"] + }, + "id": { + "type": "number" + } + }, + "required": ["type", "id"] + } + }, + "required": ["data"] + }, + "DownloadBackupResponse_Output": { + "type": "object", + "properties": { + "data": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Resource type.", + "enum": ["project_backup_file_url"] + }, + "id": { + "type": "number" + }, + "attributes": { + "type": "object", + "properties": { + "file_url": { + "type": "string" + } + }, + "required": ["file_url"] + } + }, + "required": ["type", "id", "attributes"] + } + }, + "required": ["data"] + }, "V2CreateBranchRequest": { "type": "object", "properties": { @@ -43013,6 +43474,16 @@ "minimum": -9007199254740991, "maximum": 9007199254740991 }, + "db_pool_max_lifetime": { + "type": "integer", + "minimum": 0, + "maximum": 3600 + }, + "db_pool_max_idletime": { + "type": "integer", + "minimum": 0, + "maximum": 60 + }, "db_pool": { "type": "integer", "minimum": -9007199254740991, @@ -43026,6 +43497,8 @@ "db_extra_search_path", "max_rows", "db_pool_acquisition_timeout", + "db_pool_max_lifetime", + "db_pool_max_idletime", "db_pool" ] }, @@ -44786,227 +45259,6 @@ }, "required": ["data"] }, - "V2ListMembersResponse_Output": { - "type": "object", - "properties": { - "data": { - "type": "array", - "items": { - "type": "object", - "properties": { - "type": { - "type": "string", - "description": "Resource type.", - "enum": ["organization_member"] - }, - "id": { - "type": "string", - "format": "uuid", - "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$" - }, - "attributes": { - "type": "object", - "properties": { - "username": { - "type": "string", - "description": "Member's username", - "nullable": true - }, - "primary_email": { - "type": "string", - "description": "Member's primary email", - "nullable": true - }, - "mfa_enabled": { - "type": "boolean", - "description": "Whether Multi-Factor Authentication is enabled for this member" - }, - "is_sso_user": { - "type": "boolean", - "description": "Whether this member is a Single Sign-On user" - }, - "avatar_url": { - "type": "string", - "description": "Member's avatar URL", - "nullable": true - }, - "roles": { - "type": "array", - "items": { - "type": "object", - "properties": { - "name": { - "type": "string", - "description": "Role name. For project-scoped roles this is the base role name.", - "example": "developer" - }, - "scope": { - "type": "string", - "enum": ["organization", "project"], - "description": "Whether this role applies org-wide or is scoped to specific projects for the user." - }, - "projects": { - "type": "array", - "items": { - "type": "object", - "properties": { - "ref": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": ["ref", "name"] - }, - "description": "Project refs this role is scoped to. Empty array for org-level roles." - } - }, - "required": ["name", "scope", "projects"] - }, - "description": "Roles assigned to this member. Includes both org-level and project-scoped roles." - } - }, - "required": [ - "username", - "primary_email", - "mfa_enabled", - "is_sso_user", - "avatar_url", - "roles" - ] - } - }, - "required": ["type", "id", "attributes"] - } - }, - "links": { - "type": "object", - "properties": { - "first": { - "type": "string", - "description": "URL path to the first page if available.", - "example": "/v2/organizations/my-org/members?page[size]=10", - "nullable": true - }, - "prev": { - "type": "string", - "description": "URL path to the previous page.", - "example": "/v2/organizations/my-org/members?page[size]=10&page[before]=019adf7d-4513-74c5-bb9a-f1bc0f7a95d7", - "nullable": true - }, - "next": { - "type": "string", - "description": "URL path to the next page.", - "example": "/v2/organizations/my-org/members?page[size]=10&page[after]=019adf7d-4513-7062-b292-78b86cc470a4", - "nullable": true - }, - "last": { - "type": "string", - "description": "URL path to the last page if available.", - "example": "/v2/organizations/my-org/members?page[size]=10&page[after]=019adf7d-4513-71ba-b264-21900edb4295", - "nullable": true - } - }, - "required": ["prev", "next"] - } - }, - "required": ["data", "links"] - }, - "V2AssignOrganizationMemberRoleRequest": { - "type": "object", - "properties": { - "data": { - "type": "object", - "properties": { - "type": { - "type": "string", - "description": "Resource type.", - "enum": ["organization_member_role"] - }, - "attributes": { - "type": "object", - "properties": { - "role": { - "type": "string", - "enum": ["owner", "administrator", "developer", "read-only", "no-access"], - "description": "Role name to assign. Must be one of: owner, administrator, developer, read-only. Must be on a Team or Enterprise plan to use the read-only role.", - "example": "developer" - }, - "projects": { - "description": "The projects to assign a project-scoped role for. If omitted, assigns an org-wide role.", - "minItems": 1, - "type": "array", - "items": { - "type": "object", - "properties": { - "ref": { - "type": "string", - "description": "Project ref", - "example": "abcjuqabhgwjjutfvtpa" - } - }, - "required": ["ref"] - } - } - }, - "required": ["role"] - } - }, - "required": ["type", "attributes"] - } - }, - "required": ["data"] - }, - "OrganizationMemberRoleResponse_Output": { - "type": "object", - "properties": { - "data": { - "type": "object", - "properties": { - "type": { - "type": "string", - "description": "Resource type.", - "enum": ["organization_member_role"] - }, - "attributes": { - "type": "object", - "properties": { - "name": { - "type": "string", - "description": "Role name. For project-scoped assignments this is the base role name.", - "example": "developer" - }, - "scope": { - "type": "string", - "enum": ["organization", "project"], - "description": "Whether this role applies org-wide or is scoped to specific projects for the user." - }, - "projects": { - "type": "array", - "items": { - "type": "object", - "properties": { - "ref": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "required": ["ref", "name"] - }, - "description": "Project refs this role is scoped to. Empty array for org-level roles." - } - }, - "required": ["name", "scope", "projects"] - } - }, - "required": ["type", "attributes"] - } - }, - "required": ["data"] - }, "V2ListRolesResponse_Output": { "type": "object", "properties": { @@ -45126,6 +45378,7 @@ "unauthorized", "validation_failed", "agentic_credential_not_found", + "backup_unsupported", "branch_admin_required", "compute_build_context_too_large", "compute_build_in_progress", @@ -45358,6 +45611,227 @@ }, "required": ["data"] }, + "V2ListMembersResponse_Output": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Resource type.", + "enum": ["organization_member"] + }, + "id": { + "type": "string", + "format": "uuid", + "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$" + }, + "attributes": { + "type": "object", + "properties": { + "username": { + "type": "string", + "description": "Member's username", + "nullable": true + }, + "primary_email": { + "type": "string", + "description": "Member's primary email", + "nullable": true + }, + "mfa_enabled": { + "type": "boolean", + "description": "Whether Multi-Factor Authentication is enabled for this member" + }, + "is_sso_user": { + "type": "boolean", + "description": "Whether this member is a Single Sign-On user" + }, + "avatar_url": { + "type": "string", + "description": "Member's avatar URL", + "nullable": true + }, + "roles": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Role name. For project-scoped roles this is the base role name.", + "example": "developer" + }, + "scope": { + "type": "string", + "enum": ["organization", "project"], + "description": "Whether this role applies org-wide or is scoped to specific projects for the user." + }, + "projects": { + "type": "array", + "items": { + "type": "object", + "properties": { + "ref": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": ["ref", "name"] + }, + "description": "Project refs this role is scoped to. Empty array for org-level roles." + } + }, + "required": ["name", "scope", "projects"] + }, + "description": "Roles assigned to this member. Includes both org-level and project-scoped roles." + } + }, + "required": [ + "username", + "primary_email", + "mfa_enabled", + "is_sso_user", + "avatar_url", + "roles" + ] + } + }, + "required": ["type", "id", "attributes"] + } + }, + "links": { + "type": "object", + "properties": { + "first": { + "type": "string", + "description": "URL path to the first page if available.", + "example": "/v2/organizations/my-org/members?page[size]=10", + "nullable": true + }, + "prev": { + "type": "string", + "description": "URL path to the previous page.", + "example": "/v2/organizations/my-org/members?page[size]=10&page[before]=019adf7d-4513-74c5-bb9a-f1bc0f7a95d7", + "nullable": true + }, + "next": { + "type": "string", + "description": "URL path to the next page.", + "example": "/v2/organizations/my-org/members?page[size]=10&page[after]=019adf7d-4513-7062-b292-78b86cc470a4", + "nullable": true + }, + "last": { + "type": "string", + "description": "URL path to the last page if available.", + "example": "/v2/organizations/my-org/members?page[size]=10&page[after]=019adf7d-4513-71ba-b264-21900edb4295", + "nullable": true + } + }, + "required": ["prev", "next"] + } + }, + "required": ["data", "links"] + }, + "V2AssignOrganizationMemberRoleRequest": { + "type": "object", + "properties": { + "data": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Resource type.", + "enum": ["organization_member_role"] + }, + "attributes": { + "type": "object", + "properties": { + "role": { + "type": "string", + "enum": ["owner", "administrator", "developer", "read-only", "no-access"], + "description": "Role name to assign. Must be one of: owner, administrator, developer, read-only. Must be on a Team or Enterprise plan to use the read-only role.", + "example": "developer" + }, + "projects": { + "description": "The projects to assign a project-scoped role for. If omitted, assigns an org-wide role.", + "minItems": 1, + "type": "array", + "items": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "Project ref", + "example": "abcjuqabhgwjjutfvtpa" + } + }, + "required": ["ref"] + } + } + }, + "required": ["role"] + } + }, + "required": ["type", "attributes"] + } + }, + "required": ["data"] + }, + "OrganizationMemberRoleResponse_Output": { + "type": "object", + "properties": { + "data": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Resource type.", + "enum": ["organization_member_role"] + }, + "attributes": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Role name. For project-scoped assignments this is the base role name.", + "example": "developer" + }, + "scope": { + "type": "string", + "enum": ["organization", "project"], + "description": "Whether this role applies org-wide or is scoped to specific projects for the user." + }, + "projects": { + "type": "array", + "items": { + "type": "object", + "properties": { + "ref": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": ["ref", "name"] + }, + "description": "Project refs this role is scoped to. Empty array for org-level roles." + } + }, + "required": ["name", "scope", "projects"] + } + }, + "required": ["type", "attributes"] + } + }, + "required": ["data"] + }, "V2ListProjectsResponse_Output": { "type": "object", "properties": { @@ -45697,6 +46171,101 @@ }, "required": ["data", "links"] }, + "V2ListMaintenanceWindowsResponse_Output": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Resource type.", + "enum": ["maintenance_window"] + }, + "id": { + "type": "string", + "description": "AWS region.", + "example": "us-east-1" + }, + "attributes": { + "type": "object", + "properties": { + "window": { + "type": "object", + "properties": { + "day": { + "type": "string", + "enum": ["mon", "tue", "wed", "thu", "fri", "sat", "sun"], + "description": "UTC day the window starts." + }, + "start_utc": { + "type": "string", + "description": "UTC start time.", + "example": "06:00" + }, + "duration_minutes": { + "type": "integer", + "minimum": -9007199254740991, + "maximum": 9007199254740991, + "example": 120 + } + }, + "required": ["day", "start_utc", "duration_minutes"] + }, + "current_window": { + "type": "object", + "properties": { + "start_at": { + "type": "string", + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Start of the occurrence, inclusive." + }, + "end_at": { + "type": "string", + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "End of the occurrence, exclusive." + } + }, + "required": ["start_at", "end_at"], + "description": "The open occurrence, or null when the window is closed.", + "nullable": true + }, + "next_windows": { + "type": "array", + "items": { + "type": "object", + "properties": { + "start_at": { + "type": "string", + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Start of the occurrence, inclusive." + }, + "end_at": { + "type": "string", + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "End of the occurrence, exclusive." + } + }, + "required": ["start_at", "end_at"] + }, + "description": "The next five occurrences that start after now, in order." + } + }, + "required": ["window", "current_window", "next_windows"] + } + }, + "required": ["type", "id", "attributes"] + } + } + }, + "required": ["data"] + }, "APIErrorObject": { "type": "object", "properties": { @@ -45717,6 +46286,7 @@ "unauthorized", "validation_failed", "agentic_credential_not_found", + "backup_unsupported", "branch_admin_required", "compute_build_context_too_large", "compute_build_in_progress", diff --git a/packages/api/src/generated/permissions.ts b/packages/api/src/generated/permissions.ts index 42196b9d25..51180dad11 100644 --- a/packages/api/src/generated/permissions.ts +++ b/packages/api/src/generated/permissions.ts @@ -344,6 +344,7 @@ export const operationPermissions: Readonly oauthScope: "analytics_config:write", }, "v2-run-project-advisors": { fga: [["advisors_read"]] }, + "v2-download-project-backup": { fga: [["backups_write"]], oauthScope: "database:write" }, "v2-create-a-branch": { fga: [["branching_development_create"], ["branching_production_create"]], oauthScope: "environment:write", @@ -381,8 +382,6 @@ export const operationPermissions: Readonly oauthScope: "edge_functions:write", }, "v2-deploy-a-compute-instance": { fga: [["workers_write"]], oauthScope: "edge_functions:write" }, - "v2-list-organization-members": { fga: [["members_read"]], oauthScope: "organizations:read" }, - "v2-assign-organization-member-role": { fga: [["organization_admin_write"]] }, "v2-list-organization-roles": { fga: [["members_read"]], oauthScope: "organizations:read" }, "v2-create-organization-invitations": { fga: [["members_write"]], @@ -392,6 +391,15 @@ export const operationPermissions: Readonly fga: [["members_write"]], oauthScope: "organizations:write", }, + "v2-list-organization-members": { fga: [["members_read"]], oauthScope: "organizations:read" }, + "v2-assign-organization-member-role": { + fga: [["organization_admin_write"]], + oauthScope: "organizations:write", + }, + "v2-remove-organization-members": { + fga: [["organization_admin_write"]], + oauthScope: "organizations:write", + }, "v2-list-organization-projects": { fga: [["organization_projects_read"]], oauthScope: "projects:read", @@ -400,6 +408,10 @@ export const operationPermissions: Readonly fga: [["organization_projects_read"]], oauthScope: "projects:read", }, + "v2-list-organization-maintenance-windows": { + fga: [["organization_admin_read"]], + oauthScope: "organizations:read", + }, "v2-projects-ref-webhooks-endpoints-get": { fga: [] }, "v2-projects-ref-webhooks-endpoints-post": { fga: [] }, "v2-projects-ref-webhooks-endpoints-delete": { fga: [] }, diff --git a/packages/api/src/internal/client.unit.test.ts b/packages/api/src/internal/client.unit.test.ts index 5ce9c56fe1..267d297bcc 100644 --- a/packages/api/src/internal/client.unit.test.ts +++ b/packages/api/src/internal/client.unit.test.ts @@ -1349,6 +1349,8 @@ describe("makeSupabaseApiClient", () => { db_extra_search_path: "", max_rows: 1000, db_pool_acquisition_timeout: 0, + db_pool_max_lifetime: 0, + db_pool_max_idletime: 0, db_pool: null, }, realtime: { diff --git a/packages/config/src/project-config/api-attributes.ts b/packages/config/src/project-config/api-attributes.ts index 2967b7408c..e8a38d7749 100644 --- a/packages/config/src/project-config/api-attributes.ts +++ b/packages/config/src/project-config/api-attributes.ts @@ -101,6 +101,8 @@ const apiAttributes = Schema.Struct({ max_rows: Schema.optionalKey(ApiNumber), db_pool_acquisition_timeout: Schema.optionalKey(Schema.Unknown), db_pool: Schema.optionalKey(Schema.Unknown), + db_pool_max_lifetime: Schema.optionalKey(Schema.Unknown), + db_pool_max_idletime: Schema.optionalKey(Schema.Unknown), }); // No `realtime.*` field is mapped; keys stay declared as `Schema.Unknown` (rather than dropping diff --git a/packages/config/src/project-config/registry.ts b/packages/config/src/project-config/registry.ts index 4843d1e9e2..2451ed1819 100644 --- a/packages/config/src/project-config/registry.ts +++ b/packages/config/src/project-config/registry.ts @@ -107,7 +107,8 @@ const apiSectionRows: ReadonlyArray = [ }, normalizeDocument: normalizeDocumentMaxRows, }, - // Unmapped (no config counterpart): api.db_pool, api.db_pool_acquisition_timeout. + // Unmapped (no config counterpart): api.db_pool, api.db_pool_acquisition_timeout, + // api.db_pool_max_lifetime, api.db_pool_max_idletime. ]; /** Settings fields whose remote value is a signed int, clamped to uint. */ From 25cb580a6762989e640552bf278fd9bbb40914ec Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 15:46:33 +0000 Subject: [PATCH 008/101] chore(stack): bump postgres to 17.11.0.017-r0 (#7445) This bumps postgres from 17.11.0.016-r0 to 17.11.0.017-r0. https://github.com/supabase/slim-services/releases/tag/postgres-17.11.0.017-r0 Planned after supabase/slim-services published 17.11.0.017-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 69f13f15de..009f35a7dc 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -5,7 +5,7 @@ # a catalog update. Everything else in this file (this comment, kong, pg14, the one-shot job # images) is hand- or Dependabot-managed. Keep a line between image lines: git conflicts on edits to # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. -FROM supabase/postgres:17.11.0.016 AS pg +FROM supabase/postgres:17.11.0.017 AS pg FROM supabase/postgres:15.19.0.016 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 1bf4bd9c2f..c49ed0a264 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -127,23 +127,23 @@ const definitions: Readonly> = { database: definition( "postgres", { - upstreamVersion: "17.11.0.016", + upstreamVersion: "17.11.0.017", revision: 0, image: - "ghcr.io/supabase/cli/postgres:17.11.0.016-r0@sha256:4a8d3f297b52ea72836cf95fcdeb56aff88d2d2efa19b6557bce3d8e98bae02c", - upstreamImage: "supabase/postgres:17.11.0.016", + "ghcr.io/supabase/cli/postgres:17.11.0.017-r0@sha256:2ae0e78c6e87fb9cf0cd8ba6795a597198e45bd2ac6e6f63a4b0ece024a008ef", + upstreamImage: "supabase/postgres:17.11.0.017", natives: { "darwin-arm64": { - archive: "5033846ff2dc56a992e9bf929d62d132bd2819656e040d4a359632121a9722e7", - manifest: "0afd146d48203665fb2c8fdfad1e79caba053b9b6d722070bb07ee980f7326fe", + archive: "543c9b10cee5efda3aa4819970f8760ea0cc52ad7a326e24ed52add97d665886", + manifest: "e058218a46629e2087c68baf5600722e51f694e1acff1062050110c94547a1ff", }, "linux-amd64": { - archive: "29b3bdcbabeb187055018da72dda6875947d113287c365aea5f7cf93fbfac7b0", - manifest: "2cb7d1526df26aaedc7158f7582a7d3c5e2ba75fd32864332f5c8fe21ff8a1ae", + archive: "ae6aa7604938c8b94d87769cff801a81ddd4dbf7b92dca1cfe31baa2de6af46c", + manifest: "8cc9731ff5011e630c142843c863e012dc726024dcb87d7e8e1d34931c2418fa", }, "linux-arm64": { - archive: "ca9f3927b0d14519b720b674e4a706d318981ceea4b8fffadf6c4c46b689a2ec", - manifest: "dfba4a861d9b95299dc40d69e48ae64e66d8894b50a962b64baebaabeb8e0f55", + archive: "72e80f471e39d214cffeb5d3b695f09542af3fdd4b1b0b4b01e7135c0857a37a", + manifest: "27930c8c06808195d7520d179c62d06cf0c57f4788c5419298694288d13c9ece", }, }, }, From 9ae94dd3dae6d1cfd59a75dcd62d99f6507cd7d5 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 15:50:35 +0000 Subject: [PATCH 009/101] chore(stack): bump postgres to 15.19.0.017-r0 (#7444) This bumps postgres from 15.19.0.016-r0 to 15.19.0.017-r0. https://github.com/supabase/slim-services/releases/tag/postgres-15.19.0.017-r0 Planned after supabase/slim-services published 17.11.0.017-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 009f35a7dc..22309a8bd0 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -7,7 +7,7 @@ # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. FROM supabase/postgres:17.11.0.017 AS pg -FROM supabase/postgres:15.19.0.016 AS pg15 +FROM supabase/postgres:15.19.0.017 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). FROM supabase/postgres:14.1.0.89 AS pg14 # New always-on service alias: extend SERVICE_IMAGE_ALIASES in diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index c49ed0a264..b4ccdd5325 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -157,24 +157,24 @@ const definitions: Readonly> = { "bin/psql", ], { - "15.19.0.016": { - upstreamVersion: "15.19.0.016", + "15.19.0.017": { + upstreamVersion: "15.19.0.017", revision: 0, image: - "ghcr.io/supabase/cli/postgres:15.19.0.016-r0@sha256:722ac5537e2b5870253befe3e7edde371305038c03aa779c4cbee894f6d46cdf", - upstreamImage: "supabase/postgres:15.19.0.016", + "ghcr.io/supabase/cli/postgres:15.19.0.017-r0@sha256:9faad1022a7b8a8c17108a4a38f41a124dadcb6a74248abde933bf9e3a6860c3", + upstreamImage: "supabase/postgres:15.19.0.017", natives: { "darwin-arm64": { - archive: "39e86642da1e683d4cb318f6eee667ba3dc880aabe7555e3bec800105269fe98", - manifest: "7d97d341d01e2d2d871cd33235a1d391e9d226c8bc46ba212761f9438b98f162", + archive: "1b521f140a7890dfc657c677f1abe38f59897f7f239b4b13cb10b74cd35b6baf", + manifest: "55b693210d762465f0d1e2693cc832914e5dfbda2b8275a6067fb21ba90fe26e", }, "linux-amd64": { - archive: "cc10dbbb79bcdc4cd34edb9343113a7849a17cabde39d4cbe7c2dbe977063f51", - manifest: "c965c997f85feba8273b30c56e351925c8eb3756e3beecfce351ad1d5446f91d", + archive: "09f2a1fbbe2ade0a286c96ee9f7b217951bc28820c5fc706cbf2a9fdf04476aa", + manifest: "2e2133d5b18d0291fe10349e3ac98e666c46b2adeb23e6cbb287d079b91fd9dc", }, "linux-arm64": { - archive: "35deb5146dfa49e6d4b5fc7eb329aa0cbbcbc2987cb331de81c6f2725f7b4836", - manifest: "e5f831e7e3a5caa255e5f2935115bd186785564483ad66fe6f9b55b08cfaa0c4", + archive: "7c434d619bc5621944afc65827390a30e993d949730d1fe33d967c66beb98fea", + manifest: "0f96b47e43910fd913d6354728904ea25944ac79f03a56234c336b37e51ed886", }, }, }, From b14477c8dd0d4cfb8b38dd6a7d4c0551c9738094 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 16:13:13 +0000 Subject: [PATCH 010/101] test(cli): fix 1 flaky test in live branch create (#7326) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR The live branch helper now re-creates a branch once when the platform drops TLS while setting the branch password, the same way it handles a 57P01 database restart. - `apps/cli/src/commands/branches/get/get.live.test.ts › gets a preview branch by name`: `branches create` failed with `400: Failed to update password: Client network socket disconnected before secure TLS connection was established`. This is a transient platform rejection during create, like the 57P01 restart from #7256, but the helper only re-issued the create for 57P01. The helper's transient-rejection matcher now also covers this documented TLS drop, so it runs the same single clear-and-recreate. A second rejection still fails. `create.live` uses the same helper and gets the fix too. ## ref spotted in: - https://github.com/supabase/cli/actions/runs/40170147136/job/121099157809 (develop) - https://github.com/supabase/cli/actions/runs/40170152974/job/121099176450 (main) --- _Generated by [Claude Code](https://claude.ai/code/session_014y76TaNM2eKGdpmiMQXWqu)_ --------- Co-authored-by: Claude --- .../helpers/branches-live.integration.test.ts | 29 +++++++++++++++++++ apps/cli/tests/helpers/branches-live.ts | 14 +++++---- 2 files changed, 37 insertions(+), 6 deletions(-) diff --git a/apps/cli/tests/helpers/branches-live.integration.test.ts b/apps/cli/tests/helpers/branches-live.integration.test.ts index 3a1f84e957..51b1632609 100644 --- a/apps/cli/tests/helpers/branches-live.integration.test.ts +++ b/apps/cli/tests/helpers/branches-live.integration.test.ts @@ -275,6 +275,35 @@ describe("live branch lifecycle helpers", () => { }), ); + it.effect("re-issues create once after the platform drops TLS while setting the password", () => + Effect.gen(function* () { + let creates = 0; + const cli: BranchCli = (args) => + Effect.gen(function* () { + if (args[1] === "create") { + creates += 1; + return creates === 1 + ? result( + yield* json({ + _tag: "Error", + error: { + message: + 'unexpected create branch status 400: {"message":"Failed to update password: Client network socket disconnected before secure TLS connection was established"}', + }, + }), + "", + 1, + ) + : result(yield* json(branch)); + } + return result(yield* json([defaultBranch])); + }); + const ref = yield* createLiveBranchEffect(cli, project, branch.name); + expect(ref).toBe(branchRef); + expect(creates).toBe(2); + }), + ); + it.effect("fails a create rejected by a second platform database restart", () => Effect.gen(function* () { let creates = 0; diff --git a/apps/cli/tests/helpers/branches-live.ts b/apps/cli/tests/helpers/branches-live.ts index 13e84fea23..0836a11bb6 100644 --- a/apps/cli/tests/helpers/branches-live.ts +++ b/apps/cli/tests/helpers/branches-live.ts @@ -139,17 +139,19 @@ function branchRefFromCreate( ); } -// The platform rejects `branches create` with a 400 when the database it configures restarts mid-create. -const PLATFORM_DATABASE_RESTART = /\b57P01\b|the database system is shutting down/u; +// The platform rejects `branches create` with a 400 when the database it configures restarts, or its +// TLS connection drops while setting the database password, mid-create. +const PLATFORM_TRANSIENT_CREATE_REJECTION = + /\b57P01\b|the database system is shutting down|Failed to update password: Client network socket disconnected before secure TLS connection was established/u; -function isPlatformDatabaseRestart(error: unknown): boolean { +function isPlatformTransientCreateRejection(error: unknown): boolean { return ( error instanceof BranchCommandFailed && - PLATFORM_DATABASE_RESTART.test(`${error.stdout ?? ""}\n${error.stderr}`) + PLATFORM_TRANSIENT_CREATE_REJECTION.test(`${error.stdout ?? ""}\n${error.stderr}`) ); } -/** Runs `branches create`, re-issuing it once when the platform restarted the database mid-create. */ +/** Runs `branches create`, re-issuing it once when the platform rejected it with a transient mid-create failure. */ export function runLiveBranchCreateEffect( cli: BranchCli, project: LiveProject, @@ -170,7 +172,7 @@ export function runLiveBranchCreateEffect( ); return create.pipe( Effect.catch((rejected) => - isPlatformDatabaseRestart(rejected) + isPlatformTransientCreateRejection(rejected) ? clearRejected.pipe( Effect.mapError( (cleanup) => From 0f6748ad8c6df2d0003b23177eb98aa31978d1ab Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 16:17:02 +0000 Subject: [PATCH 011/101] test(stack): fix 1 flaky test in the Realtime catalog check (#7359) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR The Realtime, pg-meta and Studio catalog test hung on Podman without saying where. It now fails before the test timeout, naming the service and phase that stalled and that service's output. - `Realtime.integration.test.ts › serves Realtime, pg-meta, and Studio against the owned database`: on Podman, the test hit the 120s test timeout with only the generator start frame as a clue. The test runs four services in sequence (database, Realtime, pg-meta, Studio), each with a recipe, start, readiness and HTTP step, so a stall anywhere looked the same. The cause is still unknown, so this change is a diagnostic. The scoped body now runs under a 110s guard. Each step records its phase, and each recipe's launch output is collected. A stall fails with `CatalogStallError` naming the service and phase in progress (including stop and root cleanup) and the last part of that service's output. ## ref spotted in: - https://github.com/supabase/cli/actions/runs/40530528881/job/122281757547 --- _Generated by [Claude Code](https://claude.ai/code/session_01MBEk1PCdaXDCUPR16M9Ria)_ --------- Co-authored-by: Claude --- .../src/services/Realtime.integration.test.ts | 294 +++++++++++------- 1 file changed, 189 insertions(+), 105 deletions(-) diff --git a/packages/stack/src/services/Realtime.integration.test.ts b/packages/stack/src/services/Realtime.integration.test.ts index 03c0b72c80..b23901c30b 100644 --- a/packages/stack/src/services/Realtime.integration.test.ts +++ b/packages/stack/src/services/Realtime.integration.test.ts @@ -1,9 +1,10 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, Layer, Redacted } from "effect"; +import { Data, Effect, Fiber, Layer, Redacted, Ref, Stream } from "effect"; import { HttpClient, HttpClientRequest } from "effect/http"; import { makeStandaloneService } from "../../tests/standalone-service.ts"; import { makeServiceRecipe } from "./Catalog.ts"; +import type { CatalogLogs } from "./Recipe.ts"; import { makeDockerHttpRelay, makeDockerTcpRelay } from "../../tests/docker-relay.ts"; import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; @@ -25,120 +26,203 @@ const dockerOptions = (root: string) => ({ engineTarget, }); +class CatalogStallError extends Data.TaggedError("CatalogStallError")<{ + readonly message: string; +}> {} + +/** Tracks the current launch phase and each service's output so a stall names both. */ +const makeLaunchTrace = Effect.gen(function* () { + const phase = yield* Ref.make({ service: "database", step: "setup" }); + const outputs = yield* Ref.make>(new Map()); + const capture = Effect.fnUntraced(function* (service: string, logs: CatalogLogs) { + const decoders = { stdout: new TextDecoder(), stderr: new TextDecoder() }; + yield* Stream.fromSubscription(yield* logs).pipe( + Stream.runForEach(({ stream, bytes }) => + Ref.update(outputs, (current) => + new Map(current).set( + service, + ((current.get(service) ?? "") + decoders[stream].decode(bytes, { stream: true })).slice( + -4_000, + ), + ), + ), + ), + Effect.forkScoped({ startImmediately: true }), + ); + }); + const enter = (service: string, step: string) => Ref.set(phase, { service, step }); + /** Reports a stall without waiting for the stalled scope's finalizers, which may hang too. */ + const guard = + (duration: `${number} seconds`) => + (effect: Effect.Effect) => + Effect.flatMap(Effect.forkDetach(effect), (fiber) => + Fiber.join(fiber).pipe( + Effect.onInterrupt(() => Effect.forkDetach(Fiber.interrupt(fiber))), + Effect.timeoutOrElse({ + duration, + orElse: () => + Effect.all([Ref.get(phase), Ref.get(outputs)]).pipe( + Effect.flatMap(([{ service, step }, output]) => + Effect.fail( + new CatalogStallError({ + message: [ + `${service} ${step} did not finish within ${duration}`, + ...(service === "database" ? [service] : [service, "database"]).map( + (name) => `${name} output:\n${output.get(name) || "(none)"}`, + ), + ].join("\n"), + }), + ), + ), + ), + }), + ), + ); + return { capture, enter, guard }; +}); + describe("service catalog", () => { it.live( "serves Realtime, pg-meta, and Studio against the owned database", () => - Effect.scoped( - Effect.gen(function* () { - const client = yield* HttpClient.HttpClient; - const root = yield* makeDockerDatabaseRoot("catalog-optional-api-"); - const secret = "catalog-optional-api-secret-with-at-least-32-chars"; - const databaseRecipe = yield* makeServiceRecipe( - { - service: "database", - pin: selection.database, - config: { - databasePassword: Redacted.make("postgres"), - jwtSecret: Redacted.make(secret), - jwtExpiry: 3600, + Effect.gen(function* () { + const trace = yield* makeLaunchTrace; + yield* Effect.scoped( + Effect.gen(function* () { + const client = yield* HttpClient.HttpClient; + const root = yield* makeDockerDatabaseRoot("catalog-optional-api-"); + const secret = "catalog-optional-api-secret-with-at-least-32-chars"; + const databaseRecipe = yield* makeServiceRecipe( + { + service: "database", + pin: selection.database, + config: { + databasePassword: Redacted.make("postgres"), + jwtSecret: Redacted.make(secret), + jwtExpiry: 3600, + }, }, - }, - dockerOptions(root), - ); - const database = yield* makeStandaloneService(databaseRecipe.definition, { - id: "database", - config: databaseRecipe.creation, - }); - yield* database.start; - yield* database.ready; - const databaseRelay = yield* makeDockerTcpRelay(databaseRecipe.endpoint("sql")); - const databaseUrl = `postgresql://supabase_admin:postgres@${databaseRelay.host}:${databaseRelay.port}/postgres`; + dockerOptions(root), + ); + yield* trace.capture("database", databaseRecipe.logs); + const database = yield* makeStandaloneService(databaseRecipe.definition, { + id: "database", + config: databaseRecipe.creation, + }); + yield* trace.enter("database", "start"); + yield* database.start; + yield* trace.enter("database", "readiness"); + yield* database.ready; + yield* trace.enter("database", "relay"); + const databaseRelay = yield* makeDockerTcpRelay(databaseRecipe.endpoint("sql")); + const databaseUrl = `postgresql://supabase_admin:postgres@${databaseRelay.host}:${databaseRelay.port}/postgres`; - const realtimeRecipe = yield* makeServiceRecipe( - { - service: "realtime", - pin: selection.realtime, - config: { databaseUrl, jwtSecret: secret }, - }, - dockerOptions(root), - ); - const realtime = yield* makeStandaloneService(realtimeRecipe.definition, { - id: "realtime", - config: realtimeRecipe.creation, - }); - yield* realtime.start; - yield* realtime.ready; - const realtimeEndpoint = yield* realtimeRecipe.endpoint("http"); - const realtimeResponse = yield* client.execute( - HttpClientRequest.get( - `http://${httpHost(realtimeEndpoint)}:${realtimeEndpoint.port}/healthcheck`, - ), - ); - expect(realtimeResponse.status).toBe(200); + yield* trace.enter("realtime", "recipe"); + const realtimeRecipe = yield* makeServiceRecipe( + { + service: "realtime", + pin: selection.realtime, + config: { databaseUrl, jwtSecret: secret }, + }, + dockerOptions(root), + ); + yield* trace.capture("realtime", realtimeRecipe.logs); + const realtime = yield* makeStandaloneService(realtimeRecipe.definition, { + id: "realtime", + config: realtimeRecipe.creation, + }); + yield* trace.enter("realtime", "start"); + yield* realtime.start; + yield* trace.enter("realtime", "readiness"); + yield* realtime.ready; + yield* trace.enter("realtime", "/healthcheck request"); + const realtimeEndpoint = yield* realtimeRecipe.endpoint("http"); + const realtimeResponse = yield* client.execute( + HttpClientRequest.get( + `http://${httpHost(realtimeEndpoint)}:${realtimeEndpoint.port}/healthcheck`, + ), + ); + expect(realtimeResponse.status).toBe(200); + yield* trace.enter("pgmeta", "recipe"); - const pgmetaRecipe = yield* makeServiceRecipe( - { service: "pgmeta", pin: selection.pgmeta, config: { databaseUrl } }, - dockerOptions(root), - ); - const pgmeta = yield* makeStandaloneService(pgmetaRecipe.definition, { - id: "pgmeta", - config: pgmetaRecipe.creation, - }); - yield* pgmeta.start; - yield* pgmeta.ready; - const pgmetaEndpoint = yield* pgmetaRecipe.endpoint("http"); - const pgmetaRelay = yield* makeDockerHttpRelay(pgmetaRecipe.endpoint("http")); - const schemasResponse = yield* client.execute( - HttpClientRequest.get( - `http://${httpHost(pgmetaEndpoint)}:${pgmetaEndpoint.port}/schemas`, - ), - ); - expect(schemasResponse.status).toBe(200); - expect(yield* schemasResponse.text).toContain("public"); + const pgmetaRecipe = yield* makeServiceRecipe( + { service: "pgmeta", pin: selection.pgmeta, config: { databaseUrl } }, + dockerOptions(root), + ); + yield* trace.capture("pgmeta", pgmetaRecipe.logs); + const pgmeta = yield* makeStandaloneService(pgmetaRecipe.definition, { + id: "pgmeta", + config: pgmetaRecipe.creation, + }); + yield* trace.enter("pgmeta", "start"); + yield* pgmeta.start; + yield* trace.enter("pgmeta", "readiness"); + yield* pgmeta.ready; + yield* trace.enter("pgmeta", "relay"); + const pgmetaEndpoint = yield* pgmetaRecipe.endpoint("http"); + const pgmetaRelay = yield* makeDockerHttpRelay(pgmetaRecipe.endpoint("http")); + yield* trace.enter("pgmeta", "/schemas request"); + const schemasResponse = yield* client.execute( + HttpClientRequest.get( + `http://${httpHost(pgmetaEndpoint)}:${pgmetaEndpoint.port}/schemas`, + ), + ); + expect(schemasResponse.status).toBe(200); + expect(yield* schemasResponse.text).toContain("public"); + yield* trace.enter("studio", "recipe"); - const studioRecipe = yield* makeServiceRecipe( - { - service: "studio", - pin: selection.studio, - config: { - databaseUrl, - pgmetaUrl: `http://${pgmetaRelay.host}:${pgmetaRelay.port}`, - analyticsApiKey: "catalog-analytics-key", - apiUrl: "http://localhost:8000", - publicApiUrl: "http://localhost:8000", - jwtSecret: secret, + const studioRecipe = yield* makeServiceRecipe( + { + service: "studio", + pin: selection.studio, + config: { + databaseUrl, + pgmetaUrl: `http://${pgmetaRelay.host}:${pgmetaRelay.port}`, + analyticsApiKey: "catalog-analytics-key", + apiUrl: "http://localhost:8000", + publicApiUrl: "http://localhost:8000", + jwtSecret: secret, + }, }, - }, - dockerOptions(root), - ); - const studio = yield* makeStandaloneService(studioRecipe.definition, { - id: "studio", - config: studioRecipe.creation, - }); - yield* studio.start; - yield* studio.ready; - const studioEndpoint = yield* studioRecipe.endpoint("http"); - const profileResponse = yield* client.execute( - HttpClientRequest.get( - `http://${httpHost(studioEndpoint)}:${studioEndpoint.port}/api/platform/profile`, - ), - ); - expect(profileResponse.status).toBe(200); - const queryResponse = yield* client.execute( - HttpClientRequest.post( - `http://${httpHost(studioEndpoint)}:${studioEndpoint.port}/api/platform/pg-meta/default/query`, - ).pipe(HttpClientRequest.bodyJsonUnsafe({ query: "select current_user" })), - ); - expect(queryResponse.status).toBe(200); - expect(yield* queryResponse.text).toContain('"current_user":"postgres"'); + dockerOptions(root), + ); + yield* trace.capture("studio", studioRecipe.logs); + const studio = yield* makeStandaloneService(studioRecipe.definition, { + id: "studio", + config: studioRecipe.creation, + }); + yield* trace.enter("studio", "start"); + yield* studio.start; + yield* trace.enter("studio", "readiness"); + yield* studio.ready; + yield* trace.enter("studio", "API requests"); + const studioEndpoint = yield* studioRecipe.endpoint("http"); + const profileResponse = yield* client.execute( + HttpClientRequest.get( + `http://${httpHost(studioEndpoint)}:${studioEndpoint.port}/api/platform/profile`, + ), + ); + expect(profileResponse.status).toBe(200); + const queryResponse = yield* client.execute( + HttpClientRequest.post( + `http://${httpHost(studioEndpoint)}:${studioEndpoint.port}/api/platform/pg-meta/default/query`, + ).pipe(HttpClientRequest.bodyJsonUnsafe({ query: "select current_user" })), + ); + expect(queryResponse.status).toBe(200); + expect(yield* queryResponse.text).toContain('"current_user":"postgres"'); - yield* studio.stop; - yield* pgmeta.stop; - yield* realtime.stop; - yield* database.stop; - }), - ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), + yield* trace.enter("studio", "stop"); + yield* studio.stop; + yield* trace.enter("pgmeta", "stop"); + yield* pgmeta.stop; + yield* trace.enter("realtime", "stop"); + yield* realtime.stop; + yield* trace.enter("database", "stop"); + yield* database.stop; + yield* trace.enter("database", "scope teardown"); + }), + ).pipe(trace.guard("110 seconds")); + }).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), { timeout: 120_000 }, ); }); From 06149d787140318684ff2037e48bc2319ed13fe4 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 16:18:52 +0000 Subject: [PATCH 012/101] test(stack): fix 1 flaky test in the native owner-logs check (#7434) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR The native owner-logs test no longer pays a private cold artifact fetch inside its timeout, and a stall now names the step that hung. - `Owner.logs.integration.test.ts › persists native output, serves history and follow, and deletes it on destroy`: the test hit the 30s test timeout while starting native Mailpit. The vitest frame pointed at the `openOwner` helper, not the stalled step. This file kept native artifacts in an OS temp cache (`$TMPDIR/supabase-stack-artifacts`) that no other suite shares, so on a runner without a warm temp dir its first native test fetched Mailpit inside the test timeout (the next native test in the file then took ~12s instead of under 1s). The file now uses the shared test artifact cache (`testArtifactCacheRoot`) like the other stack integration suites. The create, start (artifact preparation), readiness, first-follow and history steps also run under a 25s guard that fails with `NativeMailStallError`, naming the step, the instance id and Mailpit's persisted output, so a remaining stall points at its phase. ## ref spotted in: https://github.com/supabase/cli/actions/runs/41383389524/job/125078843228 (green on the same SHA: https://github.com/supabase/cli/actions/runs/41383048926/job/125077793411) --- _Generated by [Claude Code](https://claude.ai/code/session_01MBEk1LVZyNoX4Zg9Nd5ZMM)_ --------- Co-authored-by: Claude --- .../stack/src/Owner.logs.integration.test.ts | 131 +++++++++++++----- 1 file changed, 96 insertions(+), 35 deletions(-) diff --git a/packages/stack/src/Owner.logs.integration.test.ts b/packages/stack/src/Owner.logs.integration.test.ts index 31de225ccf..b829b84f4f 100644 --- a/packages/stack/src/Owner.logs.integration.test.ts +++ b/packages/stack/src/Owner.logs.integration.test.ts @@ -3,6 +3,7 @@ import { describe, expect, it } from "@effect/vitest"; import { Context, Crypto, + Data, Effect, Exit, Fiber, @@ -11,11 +12,12 @@ import { Path, Queue, Redacted, + Ref, Scope, Stream, } from "effect"; import { HttpClient } from "effect/http"; -import { tmpdir } from "node:os"; +import { testArtifactCacheRoot as cacheRoot } from "../tests/artifact-cache.ts"; import { engineTarget, testEngine } from "../tests/engine-target.ts"; import { ownerFor } from "../tests/owner-rpc.ts"; import type { LogRecord } from "./host/LogRecord.ts"; @@ -25,7 +27,6 @@ import * as StackNamespace from "./StackNamespace.ts"; import type { SavedStack } from "./StackNamespace.ts"; import { selection } from "../tests/pins.ts"; -const cacheRoot = `${tmpdir()}/supabase-stack-artifacts`; const services = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); const stackFor = (id: string, runtime: SavedStack["runtime"]): SavedStack => ({ @@ -96,6 +97,51 @@ const isOutput = (record: LogRecord) => record.kind === "stdout" || record.kind const firstOutput = (records: Stream.Stream) => records.pipe(Stream.filter(isOutput), Stream.take(1), Stream.runCollect); +class NativeMailStallError extends Data.TaggedError("NativeMailStallError")<{ + readonly message: string; +}> {} + +type NativeMailPhase = { + readonly step: string; + readonly id?: string; + readonly history?: (id: string) => Stream.Stream; +}; + +/** Fails a stalled native Mailpit step within the test timeout, naming the step and its persisted output. */ +const guardNativeMail = + (phase: Ref.Ref) => + (effect: Effect.Effect) => + Effect.flatMap(Effect.forkDetach(effect), (fiber) => + Fiber.join(fiber).pipe( + Effect.onInterrupt(() => Effect.forkDetach(Fiber.interrupt(fiber))), + Effect.timeoutOrElse({ + duration: "25 seconds", + orElse: () => + Effect.gen(function* () { + const { step, id, history } = yield* Ref.get(phase); + const output = + id === undefined || history === undefined + ? [] + : yield* history(id).pipe( + Stream.filter(isOutput), + Stream.runCollect, + Effect.timeoutOrElse({ + duration: "2 seconds", + orElse: () => Effect.succeed([]), + }), + Effect.orElseSucceed(() => []), + ); + const text = Array.from(output, (record) => record.text ?? "") + .join("") + .slice(-2_000); + return yield* new NativeMailStallError({ + message: `native Mailpit ${step}${id === undefined ? "" : ` (${id})`} did not finish within 25 seconds; output:\n${text || "(none)"}`, + }); + }), + }), + ), + ); + /** Follows from the oldest record until `count` launch markers are persisted. */ const awaitLaunches = (records: Stream.Stream, count: number) => records.pipe( @@ -107,41 +153,56 @@ const awaitLaunches = (records: Stream.Stream, count: num describe("owner persisted logs", () => { it.live("persists native output, serves history and follow, and deletes it on destroy", () => - Effect.scoped( - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const owner = yield* openOwner("owner-logs-native-", "native"); - const mail = yield* owner.rpc.createService({ - service: "mail", - pin: selection.mail, - config: {}, - endpoints: { http: { port: "auto" } }, - }); - yield* owner.rpc.startService({ id: mail.id }); - yield* owner.rpc.readyService({ id: mail.id }); - - const [followed] = yield* firstOutput(owner.rpc.readLogs({ id: mail.id, follow: true })); - const history = Array.from( - yield* owner.rpc.readLogs({ id: mail.id, follow: false }).pipe(Stream.runCollect), - ); + Effect.gen(function* () { + const phase = yield* Ref.make({ step: "owner start" }); + const enter = (step: string, id?: string) => + Ref.update(phase, (current) => ({ ...current, step, id })); + yield* Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const owner = yield* openOwner("owner-logs-native-", "native"); + yield* Effect.addFinalizer(() => enter("owner teardown")); + yield* Ref.set(phase, { + step: "create", + history: (id) => owner.rpc.readLogs({ id, follow: false }), + }); + const mail = yield* owner.rpc.createService({ + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }); + yield* enter("artifact preparation and start", mail.id); + yield* owner.rpc.startService({ id: mail.id }); + yield* enter("readiness", mail.id); + yield* owner.rpc.readyService({ id: mail.id }); + yield* enter("first followed output", mail.id); + const [followed] = yield* firstOutput(owner.rpc.readLogs({ id: mail.id, follow: true })); + yield* enter("history read", mail.id); + const history = Array.from( + yield* owner.rpc.readLogs({ id: mail.id, follow: false }).pipe(Stream.runCollect), + ); - expect(history[0]).toMatchObject({ kind: "launch", launchId: 1 }); - expect(history).toContainEqual(followed); - const directory = path.join(owner.logsRoot, "mail", mail.id); - expect(yield* fs.exists(directory)).toBe(true); - const offline = yield* LogStore.streamStackLogs({ root: owner.logsRoot }).pipe( - Stream.filter(({ instanceId }) => instanceId === mail.id), - Stream.runCollect, - ); - expect(offline.slice(0, history.length).map(({ position }) => position)).toEqual( - history.map(({ position }) => position), - ); + expect(history[0]).toMatchObject({ kind: "launch", launchId: 1 }); + expect(history).toContainEqual(followed); + const directory = path.join(owner.logsRoot, "mail", mail.id); + expect(yield* fs.exists(directory)).toBe(true); + yield* enter("offline history read", mail.id); + const offline = yield* LogStore.streamStackLogs({ root: owner.logsRoot }).pipe( + Stream.filter(({ instanceId }) => instanceId === mail.id), + Stream.runCollect, + ); + expect(offline.slice(0, history.length).map(({ position }) => position)).toEqual( + history.map(({ position }) => position), + ); - yield* owner.rpc.destroyService({ id: mail.id }); - expect(yield* fs.exists(directory)).toBe(false); - }), - ).pipe(Effect.provide(services)), + yield* enter("destroy", mail.id); + yield* owner.rpc.destroyService({ id: mail.id }); + expect(yield* fs.exists(directory)).toBe(false); + }), + ).pipe(guardNativeMail(phase)); + }).pipe(Effect.provide(services)), ); it.live("streams history through the public API after the owner stops", () => From cf1dc72d4171fc149ba2f4cc687e1cdf26863b02 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 16:36:31 +0000 Subject: [PATCH 013/101] chore(stack): bump storage to v1.89.1-r0 (#7447) This bumps storage from v1.88.0-r0 to v1.89.1-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.89.1-r0 Planned after supabase/slim-services published v1.89.1-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 22309a8bd0..c689e56ba5 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.151.3 AS realtime -FROM supabase/storage-api:v1.88.0 AS storage +FROM supabase/storage-api:v1.89.1 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index b4ccdd5325..cb86b4501c 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -448,23 +448,23 @@ const definitions: Readonly> = { storage: definition( "storage", { - upstreamVersion: "v1.88.0", + upstreamVersion: "v1.89.1", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.88.0-r0@sha256:59c0ac53a25df17578072e8121f8cb20ae2e72c762dcbf0120601aa7d4dbc138", - upstreamImage: "supabase/storage-api:v1.88.0", + "ghcr.io/supabase/cli/storage:v1.89.1-r0@sha256:0f0643cf6d01a80008b9b5c7db2bba942943c44b0dbc359173dc8465141059d8", + upstreamImage: "supabase/storage-api:v1.89.1", natives: { "darwin-arm64": { - archive: "5e820ff84e2a04c104e1ff188eb4afea5cec8ae0c4139410a58f37c38ff06308", - manifest: "8bdd6d989a489104d11151d5a7d108c1ca8a520d6704a86cf9d916b4947b24f9", + archive: "8f07db086600659c3ec6dbd1b8c6c5bd13bd60a0d47377ea3e0c54bf93862167", + manifest: "85513e335ea5517f8380aa4097d5aa22925976f7b5bfd3a6ed4e8dcb1fe7e7b0", }, "linux-amd64": { - archive: "6e1b616343a8623cfa5176a851dc3bff89f11efd3bf3f533d36524597c6ef160", - manifest: "5e0a3be00a0d3d82a5b0566adb6a87a04dfc90feaabb6e4ce90472faa8a07ec2", + archive: "4c2d1542aed89c83f418da9f31489f29ec91e9bd6b8a569cb6b7c8d0b1d2db14", + manifest: "60f29d8177d96e8efdb1967294f0664e1025682228d62546846aa684c3952318", }, "linux-arm64": { - archive: "2d4c3726e6f89fdef9f3c87cea0228a99a63cba2e41d80874309b2cefaffb957", - manifest: "8059cfd35449c7f22b906cf46c1d3f655923c748605188cce830b4994a807387", + archive: "ad8456bbd54c12076093102c74d102c9a288fe1fb7dd66f95575f3ff856a5138", + manifest: "1131244038c9cffc9a7a511fac49e19f079544f7dd3da32dd2652cfddef8f8fd", }, }, }, From a893e9231eb9c5ddcf6b2ca83c3b6db155e8d719 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 16:52:13 +0000 Subject: [PATCH 014/101] test(cli): fix 1 flaky test in live db reset (#7428) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR re-issues the live `db reset` cleanup once when the platform restarts the database, so one dropped connection can't leave a migration behind for the next file - `db reset` live failed in cleanup with `FATAL: terminating connection due to administrator command (SQLSTATE 57P01)` from the pooler on connect. the cleanup reset never ran, so the test's migration stayed in the shared project's history and the following `db push` live file failed with `Remote migration versions not found`. the `db reset` and `db push` cleanups now go through `resetLiveDatabase`, which re-issues the reset once on a platform database restart (the same rule `branches create` already uses) and names the re-issue when it fails again ## ref - spotted in: [failing job](https://github.com/supabase/cli/actions/runs/41176803525/job/124465943942) [green same SHA](https://github.com/supabase/cli/actions/runs/41015125746) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01MBEk1LFBnaQgPd2VvJLo4X Co-authored-by: Claude --- .../src/commands/db/push/push.live.test.ts | 7 +- .../src/commands/db/reset/reset.live.test.ts | 7 +- apps/cli/tests/helpers/live.ts | 26 ++++++ apps/cli/tests/helpers/live.unit.test.ts | 79 ++++++++++++++++++- 4 files changed, 107 insertions(+), 12 deletions(-) diff --git a/apps/cli/src/commands/db/push/push.live.test.ts b/apps/cli/src/commands/db/push/push.live.test.ts index 503a9abbd0..6f9fc3a9cf 100644 --- a/apps/cli/src/commands/db/push/push.live.test.ts +++ b/apps/cli/src/commands/db/push/push.live.test.ts @@ -2,7 +2,7 @@ import { BunServices } from "@effect/platform-bun"; import { Cause, Clock, Effect, Exit, FileSystem, Path, Random } from "effect"; import { expect } from "vitest"; -import { requireLiveSuccess, test, throwWithCleanup } from "../../../../tests/helpers/live.ts"; +import { resetLiveDatabase, test, throwWithCleanup } from "../../../../tests/helpers/live.ts"; test("pushes a local migration to the remote database", ({ cliEffect, @@ -31,10 +31,7 @@ test("pushes a local migration to the remote database", ({ expect(result.stdout).toContain("Finished supabase db push"); }); - const reset = Effect.gen(function* () { - const result = yield* cliEffect(["db", "reset", "--yes", `--remote=${project.dbUrl}`]); - requireLiveSuccess(result, "db reset cleanup after db push"); - }); + const reset = resetLiveDatabase(cliEffect, project, "db reset cleanup after db push"); return yield* Effect.uninterruptibleMask((restore) => Effect.gen(function* () { diff --git a/apps/cli/src/commands/db/reset/reset.live.test.ts b/apps/cli/src/commands/db/reset/reset.live.test.ts index d4ff4a3a43..160d57df40 100644 --- a/apps/cli/src/commands/db/reset/reset.live.test.ts +++ b/apps/cli/src/commands/db/reset/reset.live.test.ts @@ -2,7 +2,7 @@ import { BunServices } from "@effect/platform-bun"; import { Cause, Clock, Effect, Exit, FileSystem, Path, Random } from "effect"; import { expect } from "vitest"; -import { requireLiveSuccess, test, throwWithCleanup } from "../../../../tests/helpers/live.ts"; +import { resetLiveDatabase, test, throwWithCleanup } from "../../../../tests/helpers/live.ts"; test("resets the remote database with local migrations", ({ cliEffect, @@ -31,10 +31,7 @@ test("resets the remote database with local migrations", ({ expect(`${result.stdout}${result.stderr}`).toContain("Resetting remote database"); }); - const resetCleanup = Effect.gen(function* () { - const reset = yield* cliEffect(["db", "reset", "--yes", `--remote=${project.dbUrl}`]); - requireLiveSuccess(reset, "db reset cleanup"); - }); + const resetCleanup = resetLiveDatabase(cliEffect, project, "db reset cleanup"); return yield* Effect.uninterruptibleMask((restore) => Effect.gen(function* () { diff --git a/apps/cli/tests/helpers/live.ts b/apps/cli/tests/helpers/live.ts index 3bfcd04e58..a9c4e7ec99 100644 --- a/apps/cli/tests/helpers/live.ts +++ b/apps/cli/tests/helpers/live.ts @@ -326,6 +326,32 @@ export function removeLiveMigration( ); } +const PLATFORM_DATABASE_RESTART = /\b57P01\b|the database system is shutting down/u; + +function isPlatformDatabaseRestart(output: string): boolean { + return PLATFORM_DATABASE_RESTART.test(output); +} + +/** + * Cleanup `db reset` of the shared project, re-issued once when the platform restarted the + * database, so a leftover migration can't fail the next migration live file. + */ +export const resetLiveDatabase = Effect.fn("Live.resetDatabase")(function* ( + cliEffect: LiveFixtures["cliEffect"], + project: LiveProject, + label: string, +) { + const reset = Effect.suspend(() => + cliEffect(["db", "reset", "--yes", `--remote=${project.dbUrl}`]), + ); + const first = yield* reset; + const restarted = first.exitCode !== 0 && isPlatformDatabaseRestart(first.stderr); + yield* Effect.annotateCurrentSpan({ attempts: restarted ? 2 : 1 }); + if (!restarted) return requireCliSuccess(first, label); + yield* Effect.logWarning(`${label} hit a platform database restart, re-issuing\n${first.stderr}`); + return requireCliSuccess(yield* reset, `${label} re-issued after a platform database restart`); +}); + /** * Proves a postgres-config write through `get`. The platform can serve a stale * read right after the PUT, so after one fail-fast read the value is polled diff --git a/apps/cli/tests/helpers/live.unit.test.ts b/apps/cli/tests/helpers/live.unit.test.ts index 51f8a01d70..6528d851fc 100644 --- a/apps/cli/tests/helpers/live.unit.test.ts +++ b/apps/cli/tests/helpers/live.unit.test.ts @@ -1,7 +1,41 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it } from "@effect/vitest"; +import { Cause, Effect, Exit } from "effect"; import { requireCliSuccess } from "./cli.ts"; -import { throwWithCleanup } from "./live.ts"; +import { + type LiveFixtures, + type LiveProject, + resetLiveDatabase, + throwWithCleanup, +} from "./live.ts"; + +const project: LiveProject = { + ref: "abcdefghijklmnopqrst", + dbUrl: "postgresql://postgres:password@example.com:5432/postgres", + dbPassword: "password", + anonKey: "anon", + serviceRoleKey: "service-role", + functionsUrl: "https://example.com/functions", + storageBucket: "bucket", +}; +const restarted = { + exitCode: 1, + stdout: "", + stderr: + "failed to connect to postgres: server error (FATAL: terminating connection due to administrator command (SQLSTATE 57P01))", +}; +const refused = { exitCode: 1, stdout: "", stderr: "permission denied for schema public" }; +const reset = { exitCode: 0, stdout: "", stderr: "Resetting remote database..." }; + +function scriptedCli(results: ReadonlyArray) { + const calls: string[] = []; + const cliEffect: LiveFixtures["cliEffect"] = (args) => + Effect.sync(() => { + calls.push(args.join(" ")); + return results[calls.length - 1] ?? reset; + }); + return { calls, cliEffect }; +} describe("requireCliSuccess", () => { it("passes a zero exit through", () => { @@ -50,3 +84,44 @@ describe("throwWithCleanup", () => { expect(thrown.errors).toEqual([primary, ...cleanup]); }); }); + +describe("resetLiveDatabase", () => { + const failureOf = (cliEffect: LiveFixtures["cliEffect"]) => + Effect.exit(resetLiveDatabase(cliEffect, project, "db reset cleanup")).pipe( + Effect.map((exit) => (Exit.isFailure(exit) ? String(Cause.squash(exit.cause)) : undefined)), + ); + + it.effect("re-issues the reset once when the platform restarted the database", () => + Effect.gen(function* () { + const { calls, cliEffect } = scriptedCli([restarted, reset]); + + expect(yield* failureOf(cliEffect)).toBeUndefined(); + expect(calls).toEqual([ + `db reset --yes --remote=${project.dbUrl}`, + `db reset --yes --remote=${project.dbUrl}`, + ]); + }), + ); + + it.effect("fails without re-issuing a reset that failed for another reason", () => + Effect.gen(function* () { + const { calls, cliEffect } = scriptedCli([refused, reset]); + + expect(yield* failureOf(cliEffect)).toMatch( + /db reset cleanup failed \(exit 1\)[\s\S]*permission denied/u, + ); + expect(calls).toHaveLength(1); + }), + ); + + it.effect("names the re-issue when the platform restarts the database twice", () => + Effect.gen(function* () { + const { calls, cliEffect } = scriptedCli([restarted, restarted]); + + expect(yield* failureOf(cliEffect)).toMatch( + /db reset cleanup re-issued after a platform database restart failed/u, + ); + expect(calls).toHaveLength(2); + }), + ); +}); From cf193a1920ed57c031b81dfe4ea07fb709568967 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 16:52:47 +0000 Subject: [PATCH 015/101] test(stack): fix 2 flaky tests in network pinning and docker storage (#7288) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR Two stack integration tests failed because of runner state, not because of the code under test. - `Network.integration.test.ts › a pinned shared listener keeps its reservation until its last route closes`: on Windows, the bind failed as a foreign port conflict on 1801. The random pinned port landed on a port that a Windows runner service holds, and the static held-port list cannot anticipate every such port. `pinnedPort` now draws again until no listener answers the drawn loopback port and the bind's own vacancy guard (`Ports.probeVacant`) accepts it. This replaces the static list. - `DockerDatabaseStorage.integration.test.ts › round-trips PostgreSQL 15 data with its catalog image`: the pull failed with `toomanyrequests` after all pull retries. GHCR rate-limited the runner for longer than the pull backoff, and the tests had no image mirror, unlike the stack's production wiring. Three changes give the tests the same ECR fallback the stack uses: - The tests' container runtimes and the spawned helper-owner fixture now use `slimImageMirrors`. - The direct engine runs use the reference the runtime resolved. - The test that checks a custom mirror keeps its explicit mirror. ## ref spotted in: - https://github.com/supabase/cli/actions/runs/39757673300/job/119753742485 - https://github.com/supabase/cli/actions/runs/39812209412/job/119941209064 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01XrrxbNdkjLwsEeJJiJvGs8 --------- Co-authored-by: Claude --- .../stack/src/Network.integration.test.ts | 23 ++++--- .../DockerDatabaseStorage.integration.test.ts | 67 +++++++++++-------- .../docker-storage-publish-loop-fixture.ts | 2 + packages/stack/tests/helper-owner-fixture.ts | 3 +- 4 files changed, 57 insertions(+), 38 deletions(-) diff --git a/packages/stack/src/Network.integration.test.ts b/packages/stack/src/Network.integration.test.ts index e4a5a93ee4..d56bb3971d 100644 --- a/packages/stack/src/Network.integration.test.ts +++ b/packages/stack/src/Network.integration.test.ts @@ -7,6 +7,7 @@ import { createServer, type RequestListener } from "node:http"; // oxlint-disabl import { HttpBody, HttpClient } from "effect/http"; import * as Network from "./Network.ts"; import * as PortReservations from "./namespace/PortReservations.ts"; +import * as Ports from "./Ports.ts"; import { DOCKER_HOST_ALIAS } from "./runtime/Container.ts"; import { freshWrites, joinRoutes, sharedRoutes } from "./host/Endpoints.ts"; import * as StackNamespace from "./StackNamespace.ts"; @@ -742,18 +743,20 @@ it.live("addresses docker runtime endpoints through the stack host alias", () => ).pipe(Effect.provide(NodeServices.layer)), ); -/** Ports Windows runners listen on: RDP, WinRM, Delivery Optimization and Windows services. */ -const windowsHeldPorts = new Set([3389, 5040, 5357, 5985, 5986, 7680]); - /** * A pinned port below the stack's native and auto port ranges and every OS ephemeral range, so * neither another test's auto allocation nor an outbound socket can already hold it. It stays - * under 9000, where other files pin fixed ports. + * under 9000, where other files pin fixed ports, and is redrawn until the bind's own vacancy guard + * would accept it. */ -const pinnedPort = (): number => { +const pinnedPort: Effect.Effect = Effect.gen(function* () { const port = 1024 + (Math.abs(Hash.string(randomUUID())) % (9_000 - 1024)); - return windowsHeldPorts.has(port) ? pinnedPort() : port; -}; + const vacant = Ports.probeVacant(process.platform)("pinned", "127.0.0.1", port).pipe( + Effect.as(true), + Effect.catchTag("PortError", () => Effect.succeed(false)), + ); + return !(yield* Ports.accepts("127.0.0.1", port)) && (yield* vacant) ? port : yield* pinnedPort; +}); it.live("addresses a pinned endpoint from its configured port without a registry row", () => Effect.scoped( @@ -763,7 +766,7 @@ it.live("addresses a pinned endpoint from its configured port without a registry const state = yield* makeTestState(root); yield* state.save(stack("stack")); const target = yield* backend; - const port = pinnedPort(); + const port = yield* pinnedPort; const network = yield* makeTestNetwork({ stackId: "stack", runtime: "native", state }); const namespace = yield* network.register({ id: "db", @@ -786,7 +789,7 @@ it.live("a pinned port is reserved only while its listener is open", () => yield* state.save(stack("first")); yield* state.save(stack("second")); const target = yield* backend; - const port = pinnedPort(); + const port = yield* pinnedPort; const firstEnabled = yield* Ref.make(true); const firstNetwork = yield* makeTestNetwork({ stackId: "first", runtime: "native", state }); const first = yield* firstNetwork.register({ @@ -820,7 +823,7 @@ it.live("a pinned shared listener keeps its reservation until its last route clo const state = yield* makeTestState(root); yield* state.save(stack("stack")); const target = yield* backend; - const port = pinnedPort(); + const port = yield* pinnedPort; const firstEnabled = yield* Ref.make(true); const secondEnabled = yield* Ref.make(true); const network = yield* makeTestNetwork({ stackId: "stack", runtime: "native", state }); diff --git a/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts b/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts index 436f181c09..3e7f050276 100644 --- a/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts +++ b/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts @@ -22,7 +22,7 @@ import { } from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; import * as TestClock from "effect/testing/TestClock"; -import { resolveArtifact, type ArtifactPin } from "../Artifacts.ts"; +import { resolveArtifact, slimImageMirrors, type ArtifactPin } from "../Artifacts.ts"; import { makeContainerRuntime, removeStackContainers, @@ -42,6 +42,19 @@ const dockerTarget: EngineTarget = { engine: "docker", argv: [], daemonId: "test const containerTarget: EngineTarget = { ...engineTarget, argv: [] }; const postgresImage = (pin: ArtifactPin) => resolveArtifact({ service: "database", pin }).pipe(Effect.map(({ image }) => image)); +/** Falls back to the registry mirror the stack wires in production when the primary rate-limits. */ +const containerRuntime = (options: Parameters[0]) => + makeContainerRuntime({ imageMirrors: slimImageMirrors, ...options }); +/** Pulls a database image for direct engine runs, returning the mirror reference when it fell back. */ +const preparedPostgresImage = Effect.fnUntraced(function* ( + pin: ArtifactPin, + target: EngineTarget = dockerTarget, +) { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-image-" }); + const container = yield* containerRuntime({ target, root }); + return yield* container.prepareImage(yield* postgresImage(pin)); +}); const Marker = Schema.Struct({ backend: Schema.Literals(["docker", "host"]), volume: Schema.optionalKey(Schema.String), @@ -203,7 +216,7 @@ const ownerDeathScenario = (mode: "instance" | "shared" | "stall-before-start") const stackId = `storage-owner-death-${yield* crypto.randomUUIDv4}`; const filter = `label=com.supabase.stack=${stackId}`; // Pull before spawning so a cold image cache does not consume the owner's readiness timeout. - const container = yield* makeContainerRuntime({ target: dockerTarget, root }); + const container = yield* containerRuntime({ target: dockerTarget, root }); yield* container.prepareImage(yield* postgresImage(selection.database)); const child = yield* spawner.spawn( ChildProcess.make( @@ -648,7 +661,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(database15); + const helperImage = yield* preparedPostgresImage(database15); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-pg15-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -657,7 +670,7 @@ describe.runIf(testEngine === "docker")( yield* fs.makeDirectory(sourceRoot, { recursive: true }); yield* fs.makeDirectory(targetRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -754,7 +767,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-" }); const stateRoot = path.join(root, "state"); const storageRoot = path.join(stateRoot, "stack", "data"); @@ -764,7 +777,7 @@ describe.runIf(testEngine === "docker")( yield* fs.makeDirectory(sourceRoot, { recursive: true }); yield* fs.makeDirectory(targetRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -961,7 +974,7 @@ describe.runIf(testEngine === "docker")( const cacheRoot = path.join(root, "cache"); const instanceRoot = path.join(storageRoot, "database"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1057,7 +1070,7 @@ describe.runIf(testEngine === "docker")( const cacheRoot = path.join(root, "cache"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); const markerPath = path.join(instanceRoot, ".supabase-database-storage.json"); - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const storage = yield* makeDockerDatabaseStorage({ runtime: "docker", target: dockerTarget, @@ -1069,7 +1082,7 @@ describe.runIf(testEngine === "docker")( fs, path, crypto, - container: yield* makeContainerRuntime({ + container: yield* containerRuntime({ target: dockerTarget, root: instanceRoot, }), @@ -1198,7 +1211,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-unmarked-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -1224,7 +1237,7 @@ describe.runIf(testEngine === "docker")( "-c", "chown -R 100:101 /instance/data /instance/.supabase-snapshots; chmod -R 700 /instance/data /instance/.supabase-snapshots", ]); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1281,7 +1294,7 @@ describe.runIf(testEngine === "docker")( fs, path, crypto, - container: yield* makeContainerRuntime({ + container: yield* containerRuntime({ target: dockerTarget, root, }), @@ -1310,7 +1323,7 @@ describe.runIf(testEngine === "docker")( const instanceRoot = path.join(storageRoot, "recovery"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1377,7 +1390,7 @@ describe.runIf(testEngine === "docker")( const instanceRoot = path.join(storageRoot, "database"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1466,14 +1479,14 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-missing-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); const instanceRoot = path.join(storageRoot, "missing"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1554,14 +1567,14 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-no-marker-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); const instanceRoot = path.join(storageRoot, "sleeping"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1636,7 +1649,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); // `root` is deleted entirely below; nest it under the scope's own temp dir so the test // framework's own cleanup of that outer, still-present directory does not fail. const tempDir = yield* fs.makeTempDirectoryScoped({ @@ -1648,7 +1661,7 @@ describe.runIf(testEngine === "docker")( const instanceRoot = path.join(storageRoot, "sleeping"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1721,7 +1734,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-reopen-" }); const stateRoot = path.join(root, "state"); const storageRoot = path.join(stateRoot, "stack", "data"); @@ -1731,7 +1744,7 @@ describe.runIf(testEngine === "docker")( yield* fs.makeDirectory(sourceRoot, { recursive: true }); yield* fs.makeDirectory(targetRoot, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: dockerTarget, root, }); @@ -1851,7 +1864,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-line-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -1870,7 +1883,7 @@ describe.runIf(testEngine === "docker")( fs, path, crypto, - container: yield* makeContainerRuntime({ target: dockerTarget, root }), + container: yield* containerRuntime({ target: dockerTarget, root }), spawner, }); let volume: string | undefined; @@ -1959,7 +1972,7 @@ describe("Host database storage", { timeout: 120_000 }, () => { fs, path, crypto, - container: yield* makeContainerRuntime({ target: containerTarget, root }), + container: yield* containerRuntime({ target: containerTarget, root }), spawner, }); yield* Effect.addFinalizer(() => @@ -1998,7 +2011,7 @@ describe("Host database storage", { timeout: 120_000 }, () => { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage(selection.database); + const helperImage = yield* preparedPostgresImage(selection.database, containerTarget); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-host-" }); const stateRoot = path.join(root, "state"); const storageRoot = path.join(stateRoot, "stack", "data"); @@ -2032,7 +2045,7 @@ describe("Host database storage", { timeout: 120_000 }, () => { path.join(instanceRoot, ".supabase-database-ready.json"), `{"runtime":"${testEngine}","profile":"supabase"}`, ); - const container = yield* makeContainerRuntime({ + const container = yield* containerRuntime({ target: containerTarget, root, }); diff --git a/packages/stack/tests/docker-storage-publish-loop-fixture.ts b/packages/stack/tests/docker-storage-publish-loop-fixture.ts index 4e25945a35..0664b86ffe 100644 --- a/packages/stack/tests/docker-storage-publish-loop-fixture.ts +++ b/packages/stack/tests/docker-storage-publish-loop-fixture.ts @@ -1,6 +1,7 @@ import { NodeServices } from "@effect/platform-node"; import { Console, Crypto, Effect, FileSystem, Layer, Path } from "effect"; import { ChildProcessSpawner } from "effect/process"; +import { slimImageMirrors } from "../src/Artifacts.ts"; import { makeContainerRuntime } from "../src/runtime/Container.ts"; import { makeDockerDatabaseStorage } from "../src/storage/DockerDatabaseStorage.ts"; import { selection } from "./pins.ts"; @@ -40,6 +41,7 @@ const program = Effect.scoped( const container = yield* makeContainerRuntime({ target, root: instanceRoot, + imageMirrors: slimImageMirrors, }); const storage = yield* makeDockerDatabaseStorage({ runtime: "docker", diff --git a/packages/stack/tests/helper-owner-fixture.ts b/packages/stack/tests/helper-owner-fixture.ts index a1f0e51497..b5420aa134 100644 --- a/packages/stack/tests/helper-owner-fixture.ts +++ b/packages/stack/tests/helper-owner-fixture.ts @@ -1,6 +1,7 @@ import { NodeServices } from "@effect/platform-node"; import { Crypto, Effect, FileSystem, Path, Schema } from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; +import { slimImageMirrors } from "../src/Artifacts.ts"; import { makeContainerRuntime } from "../src/runtime/Container.ts"; import { makeDockerDatabaseStorage } from "../src/storage/DockerDatabaseStorage.ts"; import { makeDockerHelperRegistry } from "../src/storage/DockerHelperRegistry.ts"; @@ -51,7 +52,7 @@ const run = Effect.gen(function* () { { mode: 0o600 }, ); const target = { engine: "docker" as const, argv: [], daemonId: "test-daemon-id" }; - const container = yield* makeContainerRuntime({ target, root }); + const container = yield* makeContainerRuntime({ target, root, imageMirrors: slimImageMirrors }); const helpers = shared ? yield* makeDockerHelperRegistry(yield* crypto.randomUUIDv4) : undefined; const storage = yield* makeDockerDatabaseStorage({ runtime: "docker", From 62cf83683edd96d4e87e51a38d44fc736b1bbb64 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Mon, 2 Nov 2026 16:53:45 +0000 Subject: [PATCH 016/101] test(stack): fix 1 flaky Mailpit test and share the stall guard (#7318) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR The Mailpit launch test hung on Podman without saying where. It now fails within its timeout, naming the phase that stalled and Mailpit's output. The detached stall guard it uses is now one shared test helper, `failOnStall`, which the Realtime and native owner-logs tests also use. - `Mail.integration.test.ts › launches the real Mailpit recipe and serves its HTTP endpoint`: on Podman, the test hit the 30s test timeout with no diagnostic. The error frame pointed at `sendTestEmail`, but this test never sends mail, so that frame is misleading. The test waits on start, readiness and a `/readyz` request. Container readiness may wait up to 60s, longer than the test timeout, and the `/readyz` request has no bound. So a stall in any of the three looked the same. The whole test scope now runs under one 25s guard inside the 30s test timeout, and a `Ref` tracks the current phase. The test also collects the recipe's log stream, so a stall fails with `MailStallError` naming the phase and showing Mailpit's output so far. - `packages/stack/tests/stall-guard.ts` adds `failOnStall(duration, onStall)`. It joins the guarded effect from a detached fiber, so the stall error surfaces at the deadline instead of waiting for teardown, and it interrupts the stalled fiber in the background so scoped cleanup still runs. `Mail`, `Realtime` (#7359) and `Owner.logs` (#7434) integration tests now share it instead of carrying three copies; each keeps its own stall message. ## ref spotted in: - https://github.com/supabase/cli/actions/runs/39920135558/job/120303098411 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude --- .../stack/src/Owner.logs.integration.test.ts | 59 ++++++-------- .../src/services/Mail.integration.test.ts | 81 +++++++++++++------ .../src/services/Realtime.integration.test.ts | 44 +++++----- packages/stack/tests/stall-guard.ts | 15 ++++ 4 files changed, 116 insertions(+), 83 deletions(-) create mode 100644 packages/stack/tests/stall-guard.ts diff --git a/packages/stack/src/Owner.logs.integration.test.ts b/packages/stack/src/Owner.logs.integration.test.ts index b829b84f4f..717d1ab69d 100644 --- a/packages/stack/src/Owner.logs.integration.test.ts +++ b/packages/stack/src/Owner.logs.integration.test.ts @@ -26,6 +26,7 @@ import * as LogStore from "./host/LogStore.ts"; import * as StackNamespace from "./StackNamespace.ts"; import type { SavedStack } from "./StackNamespace.ts"; import { selection } from "../tests/pins.ts"; +import { failOnStall } from "../tests/stall-guard.ts"; const services = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); @@ -108,39 +109,31 @@ type NativeMailPhase = { }; /** Fails a stalled native Mailpit step within the test timeout, naming the step and its persisted output. */ -const guardNativeMail = - (phase: Ref.Ref) => - (effect: Effect.Effect) => - Effect.flatMap(Effect.forkDetach(effect), (fiber) => - Fiber.join(fiber).pipe( - Effect.onInterrupt(() => Effect.forkDetach(Fiber.interrupt(fiber))), - Effect.timeoutOrElse({ - duration: "25 seconds", - orElse: () => - Effect.gen(function* () { - const { step, id, history } = yield* Ref.get(phase); - const output = - id === undefined || history === undefined - ? [] - : yield* history(id).pipe( - Stream.filter(isOutput), - Stream.runCollect, - Effect.timeoutOrElse({ - duration: "2 seconds", - orElse: () => Effect.succeed([]), - }), - Effect.orElseSucceed(() => []), - ); - const text = Array.from(output, (record) => record.text ?? "") - .join("") - .slice(-2_000); - return yield* new NativeMailStallError({ - message: `native Mailpit ${step}${id === undefined ? "" : ` (${id})`} did not finish within 25 seconds; output:\n${text || "(none)"}`, - }); - }), - }), - ), - ); +const guardNativeMail = (phase: Ref.Ref) => + failOnStall( + "25 seconds", + Effect.gen(function* () { + const { step, id, history } = yield* Ref.get(phase); + const output = + id === undefined || history === undefined + ? [] + : yield* history(id).pipe( + Stream.filter(isOutput), + Stream.runCollect, + Effect.timeoutOrElse({ + duration: "2 seconds", + orElse: () => Effect.succeed([]), + }), + Effect.orElseSucceed(() => []), + ); + const text = Array.from(output, (record) => record.text ?? "") + .join("") + .slice(-2_000); + return yield* new NativeMailStallError({ + message: `native Mailpit ${step}${id === undefined ? "" : ` (${id})`} did not finish within 25 seconds; output:\n${text || "(none)"}`, + }); + }), + ); /** Follows from the oldest record until `count` launch markers are persisted. */ const awaitLaunches = (records: Stream.Stream, count: number) => diff --git a/packages/stack/src/services/Mail.integration.test.ts b/packages/stack/src/services/Mail.integration.test.ts index 84a0ed4bde..1439288d4d 100644 --- a/packages/stack/src/services/Mail.integration.test.ts +++ b/packages/stack/src/services/Mail.integration.test.ts @@ -1,12 +1,13 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, Exit, FileSystem, Layer, Path } from "effect"; +import { Data, Effect, Exit, FileSystem, Layer, Path, Ref, Stream } from "effect"; import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/http"; import { makeStandaloneService } from "../../tests/standalone-service.ts"; import { makeServiceRecipe } from "./Catalog.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; import { selection } from "../../tests/pins.ts"; +import { failOnStall } from "../../tests/stall-guard.ts"; const options = (root: string) => ({ stackId: "catalog-mail", @@ -59,31 +60,63 @@ const sendTestEmail = ( yield* HttpClientResponse.filterStatusOk(response); }); +class MailStallError extends Data.TaggedError("MailStallError")<{ readonly message: string }> {} + +const mailStall = (duration: `${number} seconds`, phase: Ref.Ref, logs: Ref.Ref) => + failOnStall( + duration, + Effect.all([Ref.get(phase), Ref.get(logs)]).pipe( + Effect.flatMap(([stalled, output]) => + Effect.fail( + new MailStallError({ + message: `Mailpit ${stalled} did not finish within ${duration}; output:\n${output || "(none)"}`, + }), + ), + ), + ), + ); + describe("service catalog", () => { it.live("launches the real Mailpit recipe and serves its HTTP endpoint", () => - Effect.scoped( - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const client = yield* HttpClient.HttpClient; - const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-" }); - const recipe = yield* makeServiceRecipe( - { service: "mail", pin: selection.mail, config: {} }, - dockerOptions(root), - ); - const instance = yield* makeStandaloneService(recipe.definition, { - id: "mail", - config: recipe.creation, - }); - yield* instance.start; - yield* instance.ready; - const endpoint = yield* recipe.endpoint("http"); - const response = yield* client.execute( - HttpClientRequest.get(`http://${httpHost(endpoint)}:${endpoint.port}/readyz`), - ); - expect(response.status).toBe(200); - yield* instance.stop; - }), - ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), + Effect.gen(function* () { + const phase = yield* Ref.make("setup"); + const logs = yield* Ref.make(""); + yield* Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const client = yield* HttpClient.HttpClient; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-" }); + const recipe = yield* makeServiceRecipe( + { service: "mail", pin: selection.mail, config: {} }, + dockerOptions(root), + ); + const decoders = { stdout: new TextDecoder(), stderr: new TextDecoder() }; + yield* Stream.fromSubscription(yield* recipe.logs).pipe( + Stream.runForEach(({ stream, bytes }) => + Ref.update(logs, (text) => text + decoders[stream].decode(bytes, { stream: true })), + ), + Effect.forkScoped({ startImmediately: true }), + ); + const instance = yield* makeStandaloneService(recipe.definition, { + id: "mail", + config: recipe.creation, + }); + yield* Ref.set(phase, "start"); + yield* instance.start; + yield* Ref.set(phase, "readiness"); + yield* instance.ready; + yield* Ref.set(phase, "/readyz request"); + const endpoint = yield* recipe.endpoint("http"); + const response = yield* client.execute( + HttpClientRequest.get(`http://${httpHost(endpoint)}:${endpoint.port}/readyz`), + ); + expect(response.status).toBe(200); + yield* Ref.set(phase, "stop"); + yield* instance.stop; + yield* Ref.set(phase, "scope teardown"); + }), + ).pipe(mailStall("25 seconds", phase, logs)); + }).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); it.live( diff --git a/packages/stack/src/services/Realtime.integration.test.ts b/packages/stack/src/services/Realtime.integration.test.ts index b23901c30b..2db41c8594 100644 --- a/packages/stack/src/services/Realtime.integration.test.ts +++ b/packages/stack/src/services/Realtime.integration.test.ts @@ -1,6 +1,6 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; -import { Data, Effect, Fiber, Layer, Redacted, Ref, Stream } from "effect"; +import { Data, Effect, Layer, Redacted, Ref, Stream } from "effect"; import { HttpClient, HttpClientRequest } from "effect/http"; import { makeStandaloneService } from "../../tests/standalone-service.ts"; import { makeServiceRecipe } from "./Catalog.ts"; @@ -10,6 +10,7 @@ import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; import { selection } from "../../tests/pins.ts"; +import { failOnStall } from "../../tests/stall-guard.ts"; const options = (root: string) => ({ stackId: "catalog-realtime", @@ -51,33 +52,24 @@ const makeLaunchTrace = Effect.gen(function* () { ); }); const enter = (service: string, step: string) => Ref.set(phase, { service, step }); - /** Reports a stall without waiting for the stalled scope's finalizers, which may hang too. */ - const guard = - (duration: `${number} seconds`) => - (effect: Effect.Effect) => - Effect.flatMap(Effect.forkDetach(effect), (fiber) => - Fiber.join(fiber).pipe( - Effect.onInterrupt(() => Effect.forkDetach(Fiber.interrupt(fiber))), - Effect.timeoutOrElse({ - duration, - orElse: () => - Effect.all([Ref.get(phase), Ref.get(outputs)]).pipe( - Effect.flatMap(([{ service, step }, output]) => - Effect.fail( - new CatalogStallError({ - message: [ - `${service} ${step} did not finish within ${duration}`, - ...(service === "database" ? [service] : [service, "database"]).map( - (name) => `${name} output:\n${output.get(name) || "(none)"}`, - ), - ].join("\n"), - }), - ), + const guard = (duration: `${number} seconds`) => + failOnStall( + duration, + Effect.all([Ref.get(phase), Ref.get(outputs)]).pipe( + Effect.flatMap(([{ service, step }, output]) => + Effect.fail( + new CatalogStallError({ + message: [ + `${service} ${step} did not finish within ${duration}`, + ...(service === "database" ? [service] : [service, "database"]).map( + (name) => `${name} output:\n${output.get(name) || "(none)"}`, ), - ), - }), + ].join("\n"), + }), + ), ), - ); + ), + ); return { capture, enter, guard }; }); diff --git a/packages/stack/tests/stall-guard.ts b/packages/stack/tests/stall-guard.ts new file mode 100644 index 0000000000..e718450742 --- /dev/null +++ b/packages/stack/tests/stall-guard.ts @@ -0,0 +1,15 @@ +import { type Duration, Effect, Fiber } from "effect"; + +/** + * Fails with `onStall` once `duration` passes, without waiting for the stalled effect's finalizers, + * which may hang too; the stalled fiber is interrupted in the background so its cleanup still runs. + */ +export const failOnStall = + (duration: Duration.Input, onStall: Effect.Effect) => + (effect: Effect.Effect): Effect.Effect => + Effect.flatMap(Effect.forkDetach(effect), (fiber) => + Fiber.join(fiber).pipe( + Effect.onInterrupt(() => Effect.forkDetach(Fiber.interrupt(fiber))), + Effect.timeoutOrElse({ duration, orElse: () => onStall }), + ), + ); From f57025de83cc6157782afebda4074f86c49a3843 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 17:23:45 +0000 Subject: [PATCH 017/101] chore(stack): add postgres 17.11.0.017-orioledb-r0 (#7446) This adds postgres 17.11.0.017-orioledb-r0 as a new release line. https://github.com/supabase/slim-services/releases/tag/postgres-17.11.0.017-orioledb-r0 Planned after supabase/slim-services published 17.11.0.017-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- packages/stack/src/Artifacts.ts | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index cb86b4501c..a52c50ad74 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -367,6 +367,27 @@ const definitions: Readonly> = { }, }, }, + "17.11.0.017-orioledb": { + upstreamVersion: "17.11.0.017-orioledb", + revision: 0, + image: + "ghcr.io/supabase/cli/postgres:17.11.0.017-orioledb-r0@sha256:21e4be21bff828cf4164de76b526d695a5f58c7ba7bfeb152fc7a2065e82c9d1", + upstreamImage: "supabase/postgres:17.11.0.017-orioledb", + natives: { + "darwin-arm64": { + archive: "99055a53bd70fd72f5b42edf507f2f97fd9e1fcad85839f34cfb7958e5757752", + manifest: "b0dee24eb192969ec1dd2fbeca6f1f68f03e0a3f5760bd4b862796d4b7500bd1", + }, + "linux-amd64": { + archive: "055f40a1bd648fac00d520f13e27a0b399f0495a327e89f3c3a02fdc462f4330", + manifest: "e284e1fc51a301cfb34866c31cd51224ca8d2adbcbbe96b3c7453530ff4d1b9f", + }, + "linux-arm64": { + archive: "14baf5cfc84330a64e04f7ce28762a60dcde8fcff41bef85ce4c6491b3c7daf4", + manifest: "63f5b996fd842560f33b499bc36b0f8674cac2ab5b8941db22066f5b35ee6420", + }, + }, + }, }, ), rest: definition( From dd0e161875f4f76df6f4ba4b4c2fd0487679c7f3 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 17:35:34 +0000 Subject: [PATCH 018/101] chore(stack): bump storage to v1.89.2-r0 (#7448) This bumps storage from v1.89.1-r0 to v1.89.2-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.89.2-r0 Planned after supabase/slim-services published v1.89.2-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index c689e56ba5..85563294a3 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.151.3 AS realtime -FROM supabase/storage-api:v1.89.1 AS storage +FROM supabase/storage-api:v1.89.2 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index a52c50ad74..5356e4ee5f 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -469,23 +469,23 @@ const definitions: Readonly> = { storage: definition( "storage", { - upstreamVersion: "v1.89.1", + upstreamVersion: "v1.89.2", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.89.1-r0@sha256:0f0643cf6d01a80008b9b5c7db2bba942943c44b0dbc359173dc8465141059d8", - upstreamImage: "supabase/storage-api:v1.89.1", + "ghcr.io/supabase/cli/storage:v1.89.2-r0@sha256:faba36d1b165a52af782ffa5f6e2ad7ca0160a0b52125cf622d7d8991b0898a4", + upstreamImage: "supabase/storage-api:v1.89.2", natives: { "darwin-arm64": { - archive: "8f07db086600659c3ec6dbd1b8c6c5bd13bd60a0d47377ea3e0c54bf93862167", - manifest: "85513e335ea5517f8380aa4097d5aa22925976f7b5bfd3a6ed4e8dcb1fe7e7b0", + archive: "ce36b600d0a9fa77aeb7d8a6e67cfa147f2b55687c86547d0f2d19551add6899", + manifest: "bdd31e46912193868f9c66ac5c21582c44679279f65c4804721a9fe9625bb21e", }, "linux-amd64": { - archive: "4c2d1542aed89c83f418da9f31489f29ec91e9bd6b8a569cb6b7c8d0b1d2db14", - manifest: "60f29d8177d96e8efdb1967294f0664e1025682228d62546846aa684c3952318", + archive: "5312dba58da9adede90b880546a2d7e629217f3e8f89ce3f2907965b4548fd5b", + manifest: "4f83527b182bc00256cbdcd5b3326abe85652e570568b948493bbb899b03b5b5", }, "linux-arm64": { - archive: "ad8456bbd54c12076093102c74d102c9a288fe1fb7dd66f95575f3ff856a5138", - manifest: "1131244038c9cffc9a7a511fac49e19f079544f7dd3da32dd2652cfddef8f8fd", + archive: "00ec8132ed5e39c4cae5ffbf767e4553e7e8e7fdb1dca4eb8539f4e9161b4ed2", + manifest: "75d7bf69670a58ec1dffa64b7b6cbf8a386364db796845ffbfb7d9eb9eb24534", }, }, }, From 8ef9f65b2602a78f12fa7ba5bfefda898069bbc1 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 17:42:24 +0000 Subject: [PATCH 019/101] chore(stack): bump realtime to v2.151.4-r0 (#7449) This bumps realtime from v2.151.3-r0 to v2.151.4-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.151.4-r0 Planned after supabase/slim-services published v2.151.4-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 85563294a3..abc1c70e87 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.151.3 AS realtime +FROM supabase/realtime:v2.151.4 AS realtime FROM supabase/storage-api:v1.89.2 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 5356e4ee5f..ea400911e4 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -443,23 +443,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.151.3", + upstreamVersion: "v2.151.4", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.151.3-r0@sha256:7dfbc7b200b4c426bc069c5b736581a71d2606383ff84bc0d9f29c1b0fd8d413", - upstreamImage: "supabase/realtime:v2.151.3", + "ghcr.io/supabase/cli/realtime:v2.151.4-r0@sha256:3625d9c0db8b313df88198935d0ef4a75ddc596d7cd99a068c942a131a9850f0", + upstreamImage: "supabase/realtime:v2.151.4", natives: { "darwin-arm64": { - archive: "6616f3c45dbf25200e59b12d989ab2a2e97513e87561f7a5c45eea56aa3cd9e6", - manifest: "44f950a123047828103d1955eb9e389e862f1159d429359958c10af04495ce6c", + archive: "853c1f7b4a79f5b79f2c4e23a758ea1cb793430ef9fa9e1c2e6e9112fcf71978", + manifest: "74313b61677a9bd4eb2be32e924ecdd9c734fb87c569d7966d98981e664e36ba", }, "linux-amd64": { - archive: "bf8cc9c184b2702273153ff1897ae890f75301727a586eaf13baedb856ad8198", - manifest: "3a5dded73d9f80ee30c378717e532f397c100debe65218925fb76c6d954811fc", + archive: "6974df885b63b2036cd488dd55a3a939d0090779dcf546a813cc04838aca6020", + manifest: "83a84baff1806a316bf827bc399a2693542ae7e45ee55caff3c1ee2756e204ad", }, "linux-arm64": { - archive: "4f752db9845d4621126974025b2942ac6440c7af0515212b6285cc6f4734e3fc", - manifest: "8bb1e37ffc611d38cc95eb21adbd1e712efd941e930bba5bad1bfe1b5cb7124a", + archive: "57c69a13a73b30d49919e89c2fc98ddef93f37dd38dac9fa88f61fa5138a2ff5", + manifest: "13cceacab3812f848c54f163d2a677fc4a3de11b61b4a30a7a0c47acbc9fe4db", }, }, }, From ce48e09037f8a745d50695f9ccd62ec7ad96e10d Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Mon, 2 Nov 2026 19:43:26 +0000 Subject: [PATCH 020/101] chore(repo): fold the effect lint into the repo lint (CLI-2899) (#7422) ## TL;DR folds the effect lint into `pnpm lint:check`, so one `.oxlintrc.json` drives editors, agents and CI ## whats introduced? - `.oxlintrc.json` extends the `@effect/tsgo` recommended preset, and `.oxlintrc.effect.json` and the `lint:effect:*` scripts go away - `.github/scripts`, the npm launcher shim, compute stack templates and the generated API contracts move to `ignorePatterns`, so they leave the repo lint entirely - `lint:fix` applies suggestions, and `fix:all` runs knip, lint and format in order so fixes never race ## ref - closes: CLI-2899 & CLI-2407 --- .oxlintrc.effect.json | 25 ------------ .oxlintrc.json | 39 +++++++++++-------- AGENTS.md | 12 +++--- apps/cli-e2e/AGENTS.md | 2 +- .../db-bootstrap/local-db-running.ts | 18 ++++----- package.json | 8 ++-- turbo.json | 6 --- 7 files changed, 41 insertions(+), 69 deletions(-) delete mode 100644 .oxlintrc.effect.json diff --git a/.oxlintrc.effect.json b/.oxlintrc.effect.json deleted file mode 100644 index d3209edd72..0000000000 --- a/.oxlintrc.effect.json +++ /dev/null @@ -1,25 +0,0 @@ -{ - "$schema": "./node_modules/@effect/tsgo/oxlint-schema.json", - "extends": ["./.oxlintrc.json", "./node_modules/@effect/tsgo/oxlint-presets/recommended.json"], - "options": { - "denyWarnings": true - }, - "ignorePatterns": [ - "**", - "!packages/api/**", - "!packages/config/**", - "!packages/stack/**", - "!apps/cli-e2e/**", - "!apps/cli/**", - "!apps/docs/**", - "!packages/cli-test-helpers/**", - "!tools/**", - // Last match wins across the whole list: keep bare re-exclusions after every - // `!` entry that would otherwise re-include them. - "apps/cli/src/shared/cli/bin.ts", - "apps/cli/src/shared/compute/stacks/**", - "apps/cli/src/shared/feedback/database.types.ts", - "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts", - "packages/api/src/generated/contracts.ts" - ] -} diff --git a/.oxlintrc.json b/.oxlintrc.json index 51bff7da6a..f48e4f2a84 100644 --- a/.oxlintrc.json +++ b/.oxlintrc.json @@ -1,15 +1,28 @@ { - "$schema": "./node_modules/oxlint/configuration_schema.json", + "$schema": "./node_modules/@effect/tsgo/oxlint-schema.json", + "extends": ["./node_modules/@effect/tsgo/oxlint-presets/recommended.json"], "options": { "typeAware": true, "denyWarnings": true }, "ignorePatterns": [ + /* Reference submodules, populated by `repos:install`. */ ".repos", + /* Generated from the OpenAPI spec by `packages/api/scripts/generate.ts`. */ + "packages/api/src/generated/contracts.ts", // Generated by `supabase gen types` (see `gen:feedback-types` in apps/cli). "apps/cli/src/shared/feedback/database.types.ts", + /* Standalone Node shim built with `bun build --target node` that does not use Effect. */ + "apps/cli/src/shared/cli/bin.ts", + // Templates that run inside the user's deployed container, not in the CLI process. + "apps/cli/src/shared/compute/stacks", + /* Standalone CI automation, not Effect code. `github-scripts-ci.yml` type-checks and tests it. */ + ".github/scripts", + /* Static JSON fixtures served by the e2e replay server. */ "apps/cli-e2e/fixtures", + /* Test fixtures and golden snapshots. */ "**/testdata", + /* Build, coverage and docs output. */ "**/dist", "**/coverage", "**/.next", @@ -51,11 +64,7 @@ "excludeFiles": [ "apps/cli/src/**/*.test.ts", "apps/cli/src/shared/config/cli-config-provider.layer.ts", - "apps/cli/src/shared/config/cli-config-env.ts", - // Standalone node shim built with `bun build --target node`; it cannot import effect. - "apps/cli/src/shared/cli/bin.ts", - // Templates that run inside the user's deployed container, not in the CLI process. - "apps/cli/src/shared/compute/stacks/**" + "apps/cli/src/shared/config/cli-config-env.ts" ], "rules": { "no-restricted-properties": [ @@ -78,15 +87,6 @@ ] } }, - { - // `.github/scripts` is the only `bun:test` consumer; `@types/bun`'s shared `Matchers` - // types `.rejects.toThrow(...)` as returning `void`, even though it must be awaited at - // runtime. Removing the `await` would make the assertion racy. - "files": [".github/scripts/**"], - "rules": { - "typescript/await-thenable": "off" - } - }, { // The namespace owns paths, leases, artifact pins, and port reservations; everywhere // in the repo, outside `src/namespace/**` itself, reaches that state through @@ -118,8 +118,6 @@ "apps/cli/src/**/*.test.ts", "apps/cli/src/shared/config/cli-config-provider.layer.ts", "apps/cli/src/shared/config/cli-config-env.ts", - "apps/cli/src/shared/cli/bin.ts", - "apps/cli/src/shared/compute/stacks/**", "apps/cli/src/cli/complete.ts", "apps/cli/src/command-internal/colors.ts", "apps/cli/src/command-internal/db-config.parse.ts", @@ -574,6 +572,13 @@ } ] } + }, + { + /* The Docker Engine probe needs `node:http`'s `socketPath` for unix and npipe sockets. */ + "files": ["apps/cli/src/command-internal/db-bootstrap/local-db-running.ts"], + "rules": { + "effecttsgo/node-builtin-import": "off" + } } ] } diff --git a/AGENTS.md b/AGENTS.md index 3afcece062..ad5787f493 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ Bun monorepo with workspaces under `apps/` and `packages/`. `pnpm` is the packag Use an existing TypeScript/Bun workspace, especially `packages/api`, as the package-structure reference. Published `apps/cli` and `packages/config` are not private; `apps/docs` and -`packages/cli-*` have their own shapes. Generic lint, format, and unused-code tooling is +`packages/cli-*` have their own shapes. Lint, format, and unused-code tooling is root-owned. ### Config Naming Vocabulary @@ -35,8 +35,10 @@ all TypeScript runtime code in Effect. Promise-returning APIs are allowed only a for consumers that do not use Effect. The skill is authoritative for Effect coding practices when repository instructions conflict. -Effect linting uses oxlint via `.oxlintrc.effect.json`; run `pnpm lint:effect:check` or -`pnpm lint:effect:fix` from the repository root. +Effect linting is part of the repo lint (`.oxlintrc.json` extends the Effect oxlint preset); run +`pnpm lint:check` or `pnpm lint:fix` from the repository root; `lint:fix` also applies suggestion +fixes, so review its diff. `ignorePatterns` keeps `.github/scripts`, the npm launcher shim, compute +stack templates and the generated API contracts and feedback types out of the lint. Instrument by default: when you add or change code that does I/O (HTTP, SQL, filesystem beyond a small read, child processes, containers, waits) or millisecond-scale work, give it a named span @@ -67,9 +69,9 @@ Keep the local feedback loop fast; full CI runs for ready PRs targeting `develop Use `pnpm test` only when its scope fits the change; the CLI's aggregate script includes full E2E, so use `pnpm run test:unit` and `pnpm run test:integration` with affected test files locally. When repo-wide validation is warranted, use root `pnpm check:all` or `pnpm fix:all`. These are the -repo-wide quality entrypoints: `check:all` runs generic and Effect lint, format, knip, type, and +repo-wide quality entrypoints: `check:all` runs lint, format, knip, type, and GitHub Actions workflow checks (jactionlint and shellcheck, installed by mise); `fix:all` applies -generic and Effect lint, format, and knip fixes. Do not use production `as` casts to silence type +lint, format, and knip fixes. Do not use production `as` casts to silence type errors. Use root Turbo entrypoints for live and auxiliary workflows: diff --git a/apps/cli-e2e/AGENTS.md b/apps/cli-e2e/AGENTS.md index 0b75112979..0ee5456bfc 100644 --- a/apps/cli-e2e/AGENTS.md +++ b/apps/cli-e2e/AGENTS.md @@ -80,7 +80,7 @@ The `workspace` fixture is what triggers scenario setup — if you destructure ` Always return `runE2e(...)` from the body: Vitest does not run a returned Effect, so the harness fails a test that requests `run` or `runNoProjectId` and passes without executing it. A custom runner added with `testBehaviour.extend` gets the same check by wrapping its CLI Effect in `runGuard.track(...)`, as `stackRun` does. Pass the test's `signal` so a Vitest timeout interrupts the body. -This workspace is under the Effect lint (`.oxlintrc.effect.json`, warnings are errors), which rejects `async` test bodies, global `fetch`, `process.env` and `JSON.parse` in Effect code. Decode JSON stdout with `decodeJsonOutput(schema)(result.stdout)`, which reports a stdout excerpt on failure. +This workspace is under the Effect lint (`.oxlintrc.json`, warnings are errors), which rejects `async` test bodies, global `fetch`, `process.env` and `JSON.parse` in Effect code. Decode JSON stdout with `decodeJsonOutput(schema)(result.stdout)`, which reports a stdout excerpt on failure. Available fixtures: diff --git a/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts b/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts index df89a0d462..312776a23e 100644 --- a/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts +++ b/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts @@ -390,16 +390,14 @@ export function localDbState( if (!isContainerNotFoundMessage(stderr)) { // Surface the Docker install hint instead of raw stderr when the daemon is unreachable. const daemonDown = isDockerDaemonUnreachable(stderr); - return yield* Effect.fail( - new LocalDbRunningError({ - message: - stderr.length > 0 - ? `failed to inspect service: ${stderr}` - : "failed to inspect service", - daemonDown, - ...(daemonDown ? { suggestion: SUGGEST_DOCKER_INSTALL } : {}), - }), - ); + return yield* new LocalDbRunningError({ + message: + stderr.length > 0 + ? `failed to inspect service: ${stderr}` + : "failed to inspect service", + daemonDown, + ...(daemonDown ? { suggestion: SUGGEST_DOCKER_INSTALL } : {}), + }); } return { exists: false } satisfies LocalDbState; }), diff --git a/package.json b/package.json index e0b88a787c..71200204c3 100644 --- a/package.json +++ b/package.json @@ -15,12 +15,10 @@ "test:integration": "pnpm exec turbo run test:integration:run --continue=always --", "test:e2e": "pnpm exec turbo run supabase#build && pnpm exec turbo run test:e2e:run --only --concurrency=1 --continue=always --", "test:vitest": "bun --bun vitest run --project '!*e2e*' --project '!*live*'", - "check:all": "pnpm exec turbo run types:check lint:check fmt:check knip:check lint:effect:check lint:actions:check", - "fix:all": "pnpm exec turbo run lint:fix fmt:fix knip:fix && pnpm run lint:effect:fix", + "check:all": "pnpm exec turbo run types:check lint:check fmt:check knip:check lint:actions:check", + "fix:all": "pnpm exec turbo run knip:fix && pnpm exec turbo run lint:fix && pnpm exec turbo run fmt:fix", "lint:check": "oxlint --config .oxlintrc.json", - "lint:fix": "oxlint --config .oxlintrc.json --fix", - "lint:effect:check": "oxlint --config .oxlintrc.effect.json", - "lint:effect:fix": "oxlint --config .oxlintrc.effect.json --fix --fix-suggestions", + "lint:fix": "oxlint --config .oxlintrc.json --fix --fix-suggestions", "lint:actions:check": "jactionlint", "fmt:check": "oxfmt --config .oxfmtrc.json --check", "fmt:fix": "oxfmt --config .oxfmtrc.json", diff --git a/turbo.json b/turbo.json index 2cdbbc3bee..2d10e7b252 100644 --- a/turbo.json +++ b/turbo.json @@ -17,12 +17,6 @@ "lint:fix": { "cache": false }, - "lint:effect:check": { - "cache": false - }, - "lint:effect:fix": { - "cache": false - }, "lint:actions:check": { "cache": false }, From 03392adf120076480cf012b9e4d6f27a3ae3cfee Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Mon, 2 Nov 2026 20:10:04 +0000 Subject: [PATCH 021/101] feat(cli): add `SUPABASE_API_MAX_RETRIES` to cap api retries (CLI-2607) (#7394) ## TL;DR adds `SUPABASE_API_MAX_RETRIES` so automation can cap management api retries, where `0` sends each request once. ## whats introduced? the new env var sets the retry count on every management api client the cli builds login and `services` included. `functions deploy` caps its own retry loops at that value and keeps honouring rate limit headers, and a duplicated create still becomes an update even at `0`. an invalid value stops the command before any request, and leaving it unset keeps today's behavior... ```mermaid flowchart TD env["SUPABASE_API_MAX_RETRIES"] --> parse{"CommandSettings.load"} parse -- "unset or empty" --> none["none: today's behaviour"] parse -- "plain digits, e.g. 0 or 2" --> some["some(n)"] parse -- "anything else" --> err["ApiMaxRetriesError, exit 1, 0 requests"] none --> defaults["client retries 5 times, deploy keeps its own loops"] some --> clients["Management API clients: retry.maxRetries = n"] clients --> platform["platform api: projects, secrets, link, ..."] clients --> login["login profile lookup"] clients --> services["services version lookup"] some --> deploy["functions deploy: apiMaxRetries"] deploy --> single["command loops capped, client retries off"] single --> dup["duplicated create still becomes one update"] ``` ## why? the management api client retries retryable failures up to 5 times with no way to change that, so eg: a failing `projects list` sends 6 requests, automation that has to stop on the first failure had no supported way to send each request once.... ## ref - closes: https://github.com/supabase/cli/issues/6958 --- apps/cli/docs/supabase/login.md | 2 + apps/cli/src/SIDE_EFFECTS_TEMPLATE.md | 9 +- .../src/auth/command-platform-api.layer.ts | 1 + .../command-platform-api.layer.unit.test.ts | 39 ++- .../src/command-internal/db-config.service.ts | 6 +- ...e-runtime-script.layer.integration.test.ts | 1 + .../login-api.layer.integration.test.ts | 30 +- .../src/command-internal/login-api.layer.ts | 9 +- .../resolve-token.unit.test.ts | 1 + .../start-export-pointer.integration.test.ts | 1 + .../commands/functions/deploy/SIDE_EFFECTS.md | 1 + .../functions/deploy/deploy.handler.ts | 1 + .../deploy/deploy.integration.test.ts | 271 +++++++++++++++++- apps/cli/src/commands/login/SIDE_EFFECTS.md | 1 + .../cli/src/commands/services/SIDE_EFFECTS.md | 1 + .../src/commands/services/services.handler.ts | 2 + .../services/services.integration.test.ts | 1 + .../src/config/cli-config-key-annotations.ts | 1 + apps/cli/src/config/command-settings.layer.ts | 47 ++- .../command-settings.layer.unit.test.ts | 36 ++- .../src/config/command-settings.service.ts | 2 + .../src/config/project-ref.layer.unit.test.ts | 1 + apps/cli/src/shared/functions/deploy.ts | 187 ++++++++---- .../src/shared/services/services.shared.ts | 16 +- .../telemetry/__fixtures__/error-tags.txt | 1 + .../cli/src/shared/telemetry/event-catalog.ts | 1 + apps/cli/tests/helpers/command-mocks.ts | 5 +- apps/cli/tests/helpers/compute.ts | 1 + 28 files changed, 586 insertions(+), 89 deletions(-) diff --git a/apps/cli/docs/supabase/login.md b/apps/cli/docs/supabase/login.md index 5dbc7df448..66aef424d2 100644 --- a/apps/cli/docs/supabase/login.md +++ b/apps/cli/docs/supabase/login.md @@ -7,3 +7,5 @@ Your access token is stored securely in native credentials storage (the operatin > If this behavior is not desired, such as in a CI environment, you may skip login by specifying the `SUPABASE_ACCESS_TOKEN` environment variable in other commands. The Supabase CLI uses the stored token to access Management APIs for projects, functions, secrets, etc. + +> The Management API client retries retryable failures up to 5 times by default. Set `SUPABASE_API_MAX_RETRIES` to change that. It counts retries after the first attempt, so `0` sends each request once. diff --git a/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md b/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md index 1dec99b0f8..11b03f843e 100644 --- a/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md +++ b/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md @@ -58,10 +58,11 @@ -| Variable | Purpose | Required? | -| ----------------------- | ---------------------------------------------------- | ------------------------------------------------------- | -| `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | -| `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| Variable | Purpose | Required? | +| -------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------- | +| `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | +| `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| `SUPABASE_API_MAX_RETRIES` | Management API client retries after the first attempt (`0` = send once), not command loops | no (falls back to 5) | ## Exit Codes diff --git a/apps/cli/src/auth/command-platform-api.layer.ts b/apps/cli/src/auth/command-platform-api.layer.ts index 112f224e19..fce96774e1 100644 --- a/apps/cli/src/auth/command-platform-api.layer.ts +++ b/apps/cli/src/auth/command-platform-api.layer.ts @@ -67,6 +67,7 @@ export const makeCommandPlatformApi = Effect.gen(function* () { }, { transformClient, + retry: { maxRetries: Option.getOrUndefined(cliSettings.apiMaxRetries) }, }, ); }).pipe(Effect.withSpan("CommandPlatformApi.build")); diff --git a/apps/cli/src/auth/command-platform-api.layer.unit.test.ts b/apps/cli/src/auth/command-platform-api.layer.unit.test.ts index ac98a9add2..944256b937 100644 --- a/apps/cli/src/auth/command-platform-api.layer.unit.test.ts +++ b/apps/cli/src/auth/command-platform-api.layer.unit.test.ts @@ -38,6 +38,7 @@ function mockCliSettings(opts: { userAgent?: string; profile?: string; projectHost?: string; + apiMaxRetries?: number; }) { return Layer.succeed(CommandSettings, { githubToken: Option.none(), @@ -55,6 +56,7 @@ function mockCliSettings(opts: { workdir: "/tmp", explicitWorkdir: false, userAgent: opts.userAgent ?? "SupabaseCLI/0.0.0-dev", + apiMaxRetries: Option.fromUndefinedOr(opts.apiMaxRetries), }); } @@ -184,7 +186,7 @@ function withBaseDeps( ); } -function captureRequests(responseHeaders: Record = {}) { +function captureRequests(responseHeaders: Record = {}, status = 200) { const requests: Array<{ url: string; headers: Readonly>; @@ -195,7 +197,7 @@ function captureRequests(responseHeaders: Record = {}) { HttpClientResponse.fromWeb( request, new Response(JSON.stringify([]), { - status: 200, + status, headers: { "content-type": "application/json", ...responseHeaders }, }), ), @@ -306,6 +308,39 @@ describe("commandPlatformApiLayer", () => { }).pipe(Effect.provide(layer)); }); + it.effect("sends a failing request once when SUPABASE_API_MAX_RETRIES is 0", () => { + // `Retry-After: 0` keeps any retry from waiting on the test clock. + const http = captureRequests({ "Retry-After": "0" }, 500); + const layer = commandPlatformApiLayer.pipe( + Layer.provide(mockCliSettings({ accessToken: VALID_TOKEN, apiMaxRetries: 0 })), + Layer.provide(mockCredentials(Option.none())), + Layer.provide(http.layer), + withBaseDeps(), + ); + return Effect.gen(function* () { + const api = yield* CommandPlatformApi; + const exit = yield* Effect.exit(api.v1.listAllProjects()); + expect(Exit.isFailure(exit)).toBe(true); + expect(http.requests).toHaveLength(1); + }).pipe(Effect.provide(layer)); + }); + + it.effect("keeps the client's default retries when SUPABASE_API_MAX_RETRIES is unset", () => { + const http = captureRequests({ "Retry-After": "0" }, 500); + const layer = commandPlatformApiLayer.pipe( + Layer.provide(mockCliSettings({ accessToken: VALID_TOKEN })), + Layer.provide(mockCredentials(Option.none())), + Layer.provide(http.layer), + withBaseDeps(), + ); + return Effect.gen(function* () { + const api = yield* CommandPlatformApi; + const exit = yield* Effect.exit(api.v1.listAllProjects()); + expect(Exit.isFailure(exit)).toBe(true); + expect(http.requests).toHaveLength(6); + }).pipe(Effect.provide(layer)); + }); + it.effect("targets the configured apiUrl rather than SUPABASE_API_URL env", () => { const http = captureRequests(); const layer = commandPlatformApiLayer.pipe( diff --git a/apps/cli/src/command-internal/db-config.service.ts b/apps/cli/src/command-internal/db-config.service.ts index d2d91992fc..079142b166 100644 --- a/apps/cli/src/command-internal/db-config.service.ts +++ b/apps/cli/src/command-internal/db-config.service.ts @@ -8,6 +8,7 @@ import type { ProjectRefNotLinkedError, } from "../config/project-ref.errors.ts"; import type { ProfileLoadError } from "./profile-load.ts"; +import type { ApiMaxRetriesError } from "../config/command-settings.layer.ts"; import type { LinkedProjectRefReadError } from "../shared/config/project-link.ts"; import type { DbConnectError } from "./db-connection.errors.ts"; import type { LocalDbRunningError } from "./db-bootstrap/local-db-running.ts"; @@ -68,8 +69,9 @@ export type DbConfigError = // the pooler-config fallback. | CommandPlatformApiFactoryError // The lazy linked runtime rebuilds `commandSettingsLayer`, whose strict - // profile resolution can fail inside the resolver effect the same way. - | ProfileLoadError; + // profile and `SUPABASE_API_MAX_RETRIES` checks can fail inside the resolver effect the same way. + | ProfileLoadError + | ApiMaxRetriesError; // The `--linked` path builds a lazy Management API runtime (so `--local`/`--db-url` never need an // access token) from the resolver's captured context, so `resolve`'s R stays `never` and the diff --git a/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts b/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts index 535e628a85..58b58b9a85 100644 --- a/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts +++ b/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts @@ -58,6 +58,7 @@ function makeCliSettings(workdir = "/nonexistent-workdir") { workdir, explicitWorkdir: false, userAgent: "test", + apiMaxRetries: Option.none(), }); } diff --git a/apps/cli/src/command-internal/login-api.layer.integration.test.ts b/apps/cli/src/command-internal/login-api.layer.integration.test.ts index 6e7da75165..4e3175b4da 100644 --- a/apps/cli/src/command-internal/login-api.layer.integration.test.ts +++ b/apps/cli/src/command-internal/login-api.layer.integration.test.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from "@effect/vitest"; import { Effect, Layer, Option } from "effect"; +import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { LoginApi } from "../commands/login/login-api.service.ts"; import { @@ -13,11 +14,14 @@ import { loginApiLayer, loginProfile } from "./login-api.layer.ts"; const PROFILE = { gotrue_id: "gotrue-user-123", primary_email: "a@b.co", username: "ab" }; -function setup(opts: Parameters[0]) { +function setup( + opts: Parameters[0], + apiMaxRetries = Option.none(), +) { const api = mockCommandPlatformApi(opts); const layer = loginApiLayer.pipe( Layer.provide(api.httpClientLayer), - Layer.provide(mockCommandSettings({ workdir: "/unused-workdir" })), + Layer.provide(mockCommandSettings({ workdir: "/unused-workdir", apiMaxRetries })), ); return { api, layer }; } @@ -59,4 +63,26 @@ describe("loginApiLayer", () => { expect(api.requests).toHaveLength(1); }).pipe(Effect.provide(layer)); }); + + it.effect("looks the profile up once when SUPABASE_API_MAX_RETRIES is 0", () => { + const { api, layer } = setup( + { + // `Retry-After: 0` keeps any retry from waiting on the test clock. + handler: (request) => + Effect.succeed( + HttpClientResponse.fromWeb( + request, + new Response("{}", { status: 500, headers: { "Retry-After": "0" } }), + ), + ), + }, + Option.some(0), + ); + return Effect.gen(function* () { + const loginApi = yield* LoginApi; + const gotrueId = yield* loginApi.fetchGotrueId(DEFAULT_API_URL, VALID_TOKEN); + expect(gotrueId).toEqual(Option.none()); + expect(api.requests).toHaveLength(1); + }).pipe(Effect.provide(layer)); + }); }); diff --git a/apps/cli/src/command-internal/login-api.layer.ts b/apps/cli/src/command-internal/login-api.layer.ts index 14c78dbe13..f94ae30aee 100644 --- a/apps/cli/src/command-internal/login-api.layer.ts +++ b/apps/cli/src/command-internal/login-api.layer.ts @@ -88,11 +88,10 @@ export const loginApiLayer = Layer.effect( // host and token. fetchGotrueId: (apiHost: string, token: string) => Effect.gen(function* () { - const api = yield* makeApiClient({ - baseUrl: apiHost, - accessToken: token, - userAgent: cliSettings.userAgent, - }); + const api = yield* makeApiClient( + { baseUrl: apiHost, accessToken: token, userAgent: cliSettings.userAgent }, + { retry: { maxRetries: Option.getOrUndefined(cliSettings.apiMaxRetries) } }, + ); const profile = yield* api.v1.getProfile(); return profile.gotrue_id.length > 0 ? Option.some(profile.gotrue_id) diff --git a/apps/cli/src/command-internal/resolve-token.unit.test.ts b/apps/cli/src/command-internal/resolve-token.unit.test.ts index e52ce3b99c..32830931bd 100644 --- a/apps/cli/src/command-internal/resolve-token.unit.test.ts +++ b/apps/cli/src/command-internal/resolve-token.unit.test.ts @@ -22,6 +22,7 @@ const settings = (accessToken: Option.Option>) => workdirEnvValue: Option.none(), githubToken: Option.none(), userAgent: "SupabaseCLI/test", + apiMaxRetries: Option.none(), }); const credentials = ( diff --git a/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts b/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts index 2e4261f03f..3b7b6c2e7f 100644 --- a/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts @@ -208,6 +208,7 @@ const runStatusEnv = (input: { explicitWorkdir: Option.isSome(workdirFlag), workdirEnvValue: Option.none(), userAgent: "SupabaseCLI/pointer-test", + apiMaxRetries: Option.none(), }); }), ); diff --git a/apps/cli/src/commands/functions/deploy/SIDE_EFFECTS.md b/apps/cli/src/commands/functions/deploy/SIDE_EFFECTS.md index 4ee0ca965b..4074209f75 100644 --- a/apps/cli/src/commands/functions/deploy/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/functions/deploy/SIDE_EFFECTS.md @@ -66,6 +66,7 @@ Docker bundling may pull or run the configured edge-runtime image and uses the | `BITBUCKET_CLONE_DIR` | when defined (including an empty value), skips creating the named Deno-cache volume and omits its bind mount from the bundler `docker run` (Bitbucket's restricted Docker environment rejects both); project values are passed explicitly to Docker setup | no | | `SUPABASE_EDGE_RUNTIME_DENO_VERSION` | overrides `edge_runtime.deno_version` (which bundler image tag to use) when set, from the ambient shell or project dotenv — takes effect even with no `config.toml` on disk | no | | `NPM_CONFIG_REGISTRY` | forwarded into Docker bundling when set (the only npm variable forwarded; `NPM_AUTH_TOKEN` is not) | no | +| `SUPABASE_API_MAX_RETRIES` | Management API retries after the first attempt (`0` = send once; a duplicated create still becomes one update); also lowers each of this command's own Management API retry loops to at most that value with client retries off inside them, and since those loops nest, a request can be sent more times above `0` | no (unset keeps this command's own retries) | | `DEBUG` | enables verbose Docker bundle output when `true` | no | ## Exit Codes diff --git a/apps/cli/src/commands/functions/deploy/deploy.handler.ts b/apps/cli/src/commands/functions/deploy/deploy.handler.ts index f6948248cb..4178684703 100644 --- a/apps/cli/src/commands/functions/deploy/deploy.handler.ts +++ b/apps/cli/src/commands/functions/deploy/deploy.handler.ts @@ -28,6 +28,7 @@ export const functionsDeploy = Effect.fn("functions.deploy")(function* ( yield* deployFunctions(flags, { api, + apiMaxRetries: cliSettings.apiMaxRetries, cwd: cliSettings.workdir, flagCwd: runtimeInfo.cwd, projectRoot: cliSettings.workdir, diff --git a/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts b/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts index 8bcbff1ee3..d10cc61d50 100644 --- a/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts +++ b/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts @@ -127,6 +127,15 @@ const removeTempRoot = Effect.gen(function* () { yield* fs.remove(tempRoot.current, { recursive: true, force: true }); }).pipe(Effect.provide(BunServices.layer), Effect.orDie); +// Each sleep advances the test clock itself, so retry backoff costs no real time. +const withAdvancingClock = (effect: Effect.Effect) => + Effect.flatMap(TestClock.testClockWith(Effect.succeed), (clock) => + Effect.provideService(effect, Clock.Clock, { + ...clock, + sleep: (duration) => clock.adjust(duration), + }), + ); + const writeCliConfig = Effect.fnUntraced(function* ( cwd: string, content = 'project_id = "test-project"\n', @@ -763,9 +772,13 @@ describe("functions deploy", () => { handler: ApiHandler, args: ReadonlyArray = ["functions", "deploy", "hello-world", "--use-api"], rateLimitNotices?: Queue.Enqueue, + apiMaxRetries = Option.none(), ) { const out = mockOutput({ format: "text" }); - const api = mockCommandPlatformApi({ handler }); + const api = mockCommandPlatformApi({ + handler, + clientMaxRetries: Option.getOrElse(apiMaxRetries, () => 0), + }); const layer = Layer.mergeAll( cliConfigValuesTestLayer, buildTestRuntime({ @@ -795,7 +808,7 @@ describe("functions deploy", () => { ).pipe(Layer.provide(out.layer)), }, api, - cliSettings: mockCommandSettings({ workdir: tempRoot.current }), + cliSettings: mockCommandSettings({ workdir: tempRoot.current, apiMaxRetries }), runtimeInfo: mockRuntimeInfo({ cwd: tempRoot.current }), }), approvedHostedMutation, @@ -1222,6 +1235,112 @@ describe("functions deploy", () => { }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); }); + it.effect("caps a failed function list at SUPABASE_API_MAX_RETRIES", () => { + const { layer, api } = apiDeployHarness( + (request) => Effect.succeed(jsonResponse(request, 500, { message: "boom" })), + undefined, + undefined, + Option.some(2), + ); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + + const error = yield* functionsDeploy(baseFlags).pipe(withAdvancingClock, Effect.flip); + + expect(error).toMatchObject({ status: 500 }); + expect( + api.requests.filter((request) => request.url.endsWith(`/${VALID_REF}/functions`)), + ).toHaveLength(3); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + + it.effect("caps the prune function list at SUPABASE_API_MAX_RETRIES", () => { + const { layer, api } = apiDeployHarness( + (request) => Effect.succeed(jsonResponse(request, 500, { message: "boom" })), + ["functions", "deploy", "hello-world", "--use-api", "--no-verify-jwt", "--prune"], + undefined, + Option.some(2), + ); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + + const error = yield* functionsDeploy({ ...baseFlags, noVerifyJwt: true, prune: true }).pipe( + withAdvancingClock, + Effect.flip, + ); + + expect(error).toMatchObject({ status: 500 }); + expect( + api.requests.filter((request) => request.url.endsWith(`/${VALID_REF}/functions`)), + ).toHaveLength(3); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + + it.effect( + "waits out a long rate limit on the function list when SUPABASE_API_MAX_RETRIES is set", + () => { + const { layer, api, out } = apiDeployHarness( + (request) => + Effect.succeed( + HttpClientResponse.fromWeb( + request, + new Response("{}", { status: 429, headers: { "retry-after": "30" } }), + ), + ), + undefined, + undefined, + Option.some(1), + ); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + + const error = yield* functionsDeploy(baseFlags).pipe(withAdvancingClock, Effect.flip); + + expect(error).toMatchObject({ status: 429 }); + expect( + api.requests.filter((request) => request.url.endsWith(`/${VALID_REF}/functions`)), + ).toHaveLength(2); + expect(out.stderrText).toContain( + "Rate limit exceeded while listing functions. Retrying in 30s.", + ); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }, + ); + + it.effect("caps a rate-limited deploy at SUPABASE_API_MAX_RETRIES", () => { + const { layer, api, out } = apiDeployHarness( + (request) => + Effect.succeed( + request.method === "GET" + ? jsonResponse(request, 200, []) + : HttpClientResponse.fromWeb( + request, + new Response("{}", { status: 429, headers: { "retry-after": "0" } }), + ), + ), + undefined, + undefined, + Option.some(2), + ); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + + const error = yield* functionsDeploy(baseFlags).pipe(Effect.flip); + + expect(error).toMatchObject({ status: 429 }); + expect(api.requests.filter((request) => request.method === "POST")).toHaveLength(3); + expect(out.stderrText).toContain("Rate limit exceeded while deploying function hello-world."); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + it.effect("gives up on a persistent rate limit after the command's backoff schedule", () => Effect.gen(function* () { const notices = yield* Queue.unbounded(); @@ -1883,6 +2002,7 @@ describe("functions deploy", () => { readonly deployStatuses: ReadonlyArray; readonly bulkStatuses?: ReadonlyArray; readonly functionStatuses?: Readonly>; + readonly apiMaxRetries?: Option.Option; }) { const out = mockOutput({ format: "text" }); let deployCalls = 0; @@ -1934,7 +2054,10 @@ describe("functions deploy", () => { buildTestRuntime({ out, api, - cliSettings: mockCommandSettings({ workdir: tempRoot.current }), + cliSettings: mockCommandSettings({ + workdir: tempRoot.current, + apiMaxRetries: opts.apiMaxRetries, + }), runtimeInfo: mockRuntimeInfo({ cwd: tempRoot.current }), }), approvedHostedMutation, @@ -1979,6 +2102,30 @@ describe("functions deploy", () => { }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); }); + it.effect("caps a failed bulk update at SUPABASE_API_MAX_RETRIES", () => { + const { api, layer } = setupBulkDeploy({ + deployStatuses: [201, 201], + bulkStatuses: [500, 500, 200], + apiMaxRetries: Option.some(1), + }); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + yield* writeLocalFunction(tempRoot.current, "bye-world"); + + const error = yield* functionsDeploy({ + ...baseFlags, + functionNames: ["hello-world", "bye-world"], + }).pipe(withAdvancingClock, Effect.flip); + + expect(error).toMatchObject({ + message: 'unexpected bulk update status 500: {"message":"bulk update rejected"}', + }); + expect(api.requests.filter((request) => request.method === "PUT")).toHaveLength(2); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + it.effect("skips the bulk update entirely when every upload fails", () => { const { out, api, layer } = setupBulkDeploy({ deployStatuses: [409, 400] }); @@ -2374,6 +2521,7 @@ describe("functions deploy", () => { { ...baseFlags, functionNames: [] }, { api: platformApi, + apiMaxRetries: Option.none(), cwd: tempRoot.current, flagCwd: tempRoot.current, projectRoot: tempRoot.current, @@ -2580,6 +2728,122 @@ describe("functions deploy", () => { }); } + function setupDockerRetries(handler: ApiHandler, apiMaxRetries: number) { + const out = mockOutput({ format: "text" }); + const api = mockCommandPlatformApi({ handler, clientMaxRetries: apiMaxRetries }); + const layer = Layer.mergeAll( + cliConfigValuesTestLayer, + buildTestRuntime({ + out, + api, + cliSettings: mockCommandSettings({ + workdir: tempRoot.current, + apiMaxRetries: Option.some(apiMaxRetries), + }), + runtimeInfo: mockRuntimeInfo({ cwd: tempRoot.current }), + }), + approvedHostedMutation, + mockDockerBundleSpawner().layer, + Stdio.layerTest({ + args: Effect.succeed(["functions", "deploy", "hello-world", "--use-api=false"]), + }), + ); + const writes = () => + api.requests.filter((request) => request.method !== "GET").map((request) => request.method); + const gets = () => + api.requests.filter((request) => request.url.endsWith(`/${VALID_REF}/functions`)); + return { layer, writes, gets, out }; + } + + const deployThroughDocker = Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + return yield* functionsDeploy({ ...baseFlags, useApi: false, useDocker: true }).pipe( + withAdvancingClock, + Effect.flip, + ); + }); + + it.effect("updates a duplicated function once when SUPABASE_API_MAX_RETRIES is 0", () => { + const { layer, writes } = setupDockerRetries( + (request) => + Effect.succeed( + request.method === "GET" + ? jsonResponse(request, 200, []) + : request.method === "POST" + ? jsonResponse(request, 400, { message: "Duplicated function slug" }) + : jsonResponse(request, 500, { message: "boom" }), + ), + 0, + ); + + return Effect.gen(function* () { + expect(yield* deployThroughDocker).toMatchObject({ status: 500 }); + expect(writes()).toEqual(["POST", "PATCH"]); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + + it.effect("caps a rate-limited create at SUPABASE_API_MAX_RETRIES", () => { + const { layer, writes, out } = setupDockerRetries( + (request) => + Effect.succeed( + request.method === "GET" + ? jsonResponse(request, 200, []) + : HttpClientResponse.fromWeb( + request, + new Response("{}", { status: 429, headers: { "x-ratelimit-reset": "30" } }), + ), + ), + 2, + ); + + return Effect.gen(function* () { + expect(yield* deployThroughDocker).toMatchObject({ status: 429 }); + expect(writes()).toEqual(["POST", "POST", "POST"]); + expect(out.stderrText).toContain( + "Rate limit exceeded while creating function hello-world. Retrying in 30s.", + ); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + + it.effect("caps the docker function list at SUPABASE_API_MAX_RETRIES", () => { + const { layer, gets } = setupDockerRetries( + (request) => Effect.succeed(jsonResponse(request, 500, { message: "boom" })), + 2, + ); + + return Effect.gen(function* () { + expect(yield* deployThroughDocker).toMatchObject({ status: 500 }); + expect(gets()).toHaveLength(3); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + + for (const [apiMaxRetries, posts] of [ + [0, 1], + [1, 2], + [5, 4], + ] as const) { + it.effect( + `sends a failing create ${posts} times when SUPABASE_API_MAX_RETRIES is ${apiMaxRetries}`, + () => { + const { layer, writes } = setupDockerRetries( + (request) => + Effect.succeed( + request.method === "GET" + ? jsonResponse(request, 200, []) + : jsonResponse(request, 502, { message: "bad gateway" }), + ), + apiMaxRetries, + ); + + return Effect.gen(function* () { + expect(yield* deployThroughDocker).toMatchObject({ status: 502 }); + expect(writes()).toEqual(Array.from({ length: posts }, () => "POST")); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }, + ); + } + it.live( "resolves the deno v1 edge-runtime image tag when SUPABASE_EDGE_RUNTIME_DENO_VERSION=1 overrides an unset config value", () => { @@ -3106,6 +3370,7 @@ describe("functions deploy", () => { { ...baseFlags, useApi: false, useDocker: true }, { api: platformApi, + apiMaxRetries: Option.none(), cwd: tempRoot.current, flagCwd: tempRoot.current, projectRoot: tempRoot.current, diff --git a/apps/cli/src/commands/login/SIDE_EFFECTS.md b/apps/cli/src/commands/login/SIDE_EFFECTS.md index 4cdbd876ca..e3887419f3 100644 --- a/apps/cli/src/commands/login/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/login/SIDE_EFFECTS.md @@ -35,6 +35,7 @@ captures `cli_login_completed`. | ---------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------- | | `SUPABASE_ACCESS_TOKEN` | non-interactive token source | no (falls back to `--token` → piped stdin → browser flow) | | `SUPABASE_NO_KEYRING` | disables the OS keyring, forcing the file fallback | no | +| `SUPABASE_API_MAX_RETRIES` | Management API profile lookup retries (`0` = none) | no (falls back to 5) | | `CLAUDECODE` / `CLAUDE_CODE` | enables the Claude Code plugin hint (TTY stdout only) | no | | `DO_NOT_TRACK` / `SUPABASE_TELEMETRY_DISABLED` | suppress analytics delivery (state file still written) | no | diff --git a/apps/cli/src/commands/services/SIDE_EFFECTS.md b/apps/cli/src/commands/services/SIDE_EFFECTS.md index f562766237..9eb8041645 100644 --- a/apps/cli/src/commands/services/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/services/SIDE_EFFECTS.md @@ -44,6 +44,7 @@ Tenant calls send `apikey: ` and additionally | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | | `SUPABASE_ACCESS_TOKEN` | auth token for Management API linked-version checks | no (falls back to keyring, then `~/.supabase/access-token`) | | `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| `SUPABASE_API_MAX_RETRIES` | Management API client retries for linked-version checks (`0` = none) | no (falls back to 5) | | `SUPABASE_USE_SLIM_IMAGES` | Ambient `process.env` only (`true`/`1` enable). Rewrites legacy current-pin `SERVICE IMAGE`/`name` fields to `ghcr.io/supabase/cli/`; Kong stays on docker.io. Majors 13/15 list the slim `15.14.1.167` pin when enabled; flag-off keeps `15.8.1.085`. Ignored by the stack backend. | no | | `SUPABASE_EXPERIMENTAL_STACK` | Selects the stack backend (`1`/`true`) or legacy backend (`0`/`false`); unset or empty uses `experimental.stack`. Non-boolean values fail routing. | no | | `SUPABASE_DB_MAJOR_VERSION` | Stack backend PostgreSQL major override; supported values are 15 and 17. | no | diff --git a/apps/cli/src/commands/services/services.handler.ts b/apps/cli/src/commands/services/services.handler.ts index d062da032a..6343e202e6 100644 --- a/apps/cli/src/commands/services/services.handler.ts +++ b/apps/cli/src/commands/services/services.handler.ts @@ -141,6 +141,7 @@ export const services = Effect.fn("services")(function* (_flags: ServicesFlags) projectRef: validLinkedRef.value, accessToken: accessToken.value, userAgent: cliSettings.userAgent, + maxRetries: Option.getOrUndefined(cliSettings.apiMaxRetries), }) : {}; const result = yield* stackServiceVersions(cliSettings.workdir, remote); @@ -210,6 +211,7 @@ export const services = Effect.fn("services")(function* (_flags: ServicesFlags) projectRef: validLinkedRef.value, accessToken: accessToken.value, userAgent: cliSettings.userAgent, + maxRetries: Option.getOrUndefined(cliSettings.apiMaxRetries), }); rows = mergeRemoteServiceVersions(remote, localImageOptions); } diff --git a/apps/cli/src/commands/services/services.integration.test.ts b/apps/cli/src/commands/services/services.integration.test.ts index 872be19f58..5d0e932a50 100644 --- a/apps/cli/src/commands/services/services.integration.test.ts +++ b/apps/cli/src/commands/services/services.integration.test.ts @@ -131,6 +131,7 @@ function setup( workdir: opts.workdir ?? defaultWorkdir.current, explicitWorkdir: false, userAgent: "SupabaseCLI/test", + apiMaxRetries: Option.none(), }), ), Layer.succeed( diff --git a/apps/cli/src/config/cli-config-key-annotations.ts b/apps/cli/src/config/cli-config-key-annotations.ts index 51e3235361..c370e92936 100644 --- a/apps/cli/src/config/cli-config-key-annotations.ts +++ b/apps/cli/src/config/cli-config-key-annotations.ts @@ -296,6 +296,7 @@ export const CLI_NON_CONFIG_ENV_NAMES: Readonly> = { SUPABASE_ACCESS_TOKEN: "platform auth", SUPABASE_ACTIVE_HELP: "shell completion", SUPABASE_ANON_KEY: "functions runtime env", + SUPABASE_API_MAX_RETRIES: "platform api retries", SUPABASE_API_URL: "platform profile", SUPABASE_BASELINE: "pgdata snapshot marker file name", SUPABASE_CA_SKIP_VERIFY: "telemetry signal", diff --git a/apps/cli/src/config/command-settings.layer.ts b/apps/cli/src/config/command-settings.layer.ts index b5ec300e3b..06de31fe16 100644 --- a/apps/cli/src/config/command-settings.layer.ts +++ b/apps/cli/src/config/command-settings.layer.ts @@ -1,5 +1,15 @@ import { findCliProjectRoot } from "@supabase/config/effect"; -import { Config, ConfigProvider, Effect, FileSystem, Layer, Option, Path, Redacted } from "effect"; +import { + Config, + ConfigProvider, + Data, + Effect, + FileSystem, + Layer, + Option, + Path, + Redacted, +} from "effect"; import { CliArgs } from "../shared/cli/cli-args.service.ts"; import { lastExplicitLongFlagValue } from "../shared/cli/flag-groups.ts"; import { CLI_VERSION } from "../shared/cli/version.ts"; @@ -14,6 +24,20 @@ import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { CliEnvNames } from "./cli-config-keys.ts"; import { CommandSettings } from "./command-settings.service.ts"; import { resolveSupabaseHomeValue } from "../shared/config/supabase-home.ts"; +import { + actionability, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../shared/telemetry/error-actionability.ts"; + +/** A `SUPABASE_API_MAX_RETRIES` that is not a non-negative integer, rejected before any API request. */ +export class ApiMaxRetriesError extends Data.TaggedError("ApiMaxRetriesError")<{ + readonly message: string; +}> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidConfig; + } +} function unknownMessage(error: unknown): string { return error instanceof Error ? error.message : String(error); @@ -162,6 +186,26 @@ export const commandSettingsLayer = Layer.unwrap( path, ); + // Empty means unset, like the other SUPABASE_* settings; anything else must be plain + // digits, rejected here before a command reaches the network. + const rawApiMaxRetries = yield* read( + Config.option(Config.String("SUPABASE_API_MAX_RETRIES")), + ); + const apiMaxRetries = yield* Option.match( + Option.filter(rawApiMaxRetries, (value) => value.length > 0), + { + onNone: () => Effect.succeed(Option.none()), + onSome: (value) => + /^\d+$/.test(value) && Number.isSafeInteger(Number(value)) + ? Effect.succeed(Option.some(Number(value))) + : Effect.fail( + new ApiMaxRetriesError({ + message: `invalid SUPABASE_API_MAX_RETRIES "${value}": expected a non-negative integer`, + }), + ), + }, + ); + const userAgent = `SupabaseCLI/${CLI_VERSION}`; yield* Effect.annotateCurrentSpan({ @@ -184,6 +228,7 @@ export const commandSettingsLayer = Layer.unwrap( explicitWorkdir, workdirEnvValue, userAgent, + apiMaxRetries, }); }).pipe(Effect.withSpan("CommandSettings.load")), ); diff --git a/apps/cli/src/config/command-settings.layer.unit.test.ts b/apps/cli/src/config/command-settings.layer.unit.test.ts index a53247d677..e8bcf7a531 100644 --- a/apps/cli/src/config/command-settings.layer.unit.test.ts +++ b/apps/cli/src/config/command-settings.layer.unit.test.ts @@ -19,7 +19,7 @@ import { useTempWorkdir } from "../../tests/helpers/command-mocks.ts"; import { mockRuntimeInfo } from "../../tests/helpers/mocks.ts"; import { debugLoggerLayer } from "../command-internal/debug-logger.layer.ts"; import { ProfileLoadError } from "../command-internal/profile-load.ts"; -import { commandSettingsLayer } from "./command-settings.layer.ts"; +import { ApiMaxRetriesError, commandSettingsLayer } from "./command-settings.layer.ts"; import { CommandSettings } from "./command-settings.service.ts"; const path = Effect.runSync(Effect.provide(Path.Path, BunPath.layer)); @@ -536,6 +536,40 @@ describe("commandSettingsLayer", () => { ), ); + it.effect("captures SUPABASE_API_MAX_RETRIES, treating an empty value as unset", () => + Effect.gen(function* () { + const read = (value: string | undefined) => + Effect.service(CommandSettings).pipe( + Effect.provide( + makeLayer({ env: { SUPABASE_API_MAX_RETRIES: value }, cwd: tempRoot.current }), + ), + Effect.map((config) => config.apiMaxRetries), + ); + expect(yield* read("0")).toEqual(Option.some(0)); + expect(yield* read("3")).toEqual(Option.some(3)); + expect(yield* read("")).toEqual(Option.none()); + expect(yield* read(undefined)).toEqual(Option.none()); + }), + ); + + it.effect("rejects a SUPABASE_API_MAX_RETRIES that is not a non-negative integer", () => + Effect.gen(function* () { + for (const value of ["-1", "1.5", "1e2", " 3", "abc"]) { + const exit = yield* configExit({ + env: { SUPABASE_API_MAX_RETRIES: value }, + cwd: tempRoot.current, + }); + const error = Exit.isFailure(exit) + ? exit.cause.reasons.find(Cause.isFailReason)?.error + : undefined; + expect(error).toBeInstanceOf(ApiMaxRetriesError); + expect(error).toMatchObject({ + message: `invalid SUPABASE_API_MAX_RETRIES "${value}": expected a non-negative integer`, + }); + } + }), + ); + it.effect("captures SUPABASE_PROJECT_ID env", () => Effect.gen(function* () { const config = yield* CommandSettings; diff --git a/apps/cli/src/config/command-settings.service.ts b/apps/cli/src/config/command-settings.service.ts index 58b702e824..b87294cb5b 100644 --- a/apps/cli/src/config/command-settings.service.ts +++ b/apps/cli/src/config/command-settings.service.ts @@ -44,6 +44,8 @@ interface CommandSettingsShape { /** Raw `SUPABASE_WORKDIR` value; `Some("")` differs from an absent variable and is used verbatim. */ readonly workdirEnvValue: Option.Option; readonly userAgent: string; + /** `SUPABASE_API_MAX_RETRIES`: Management API retries after the first attempt; none keeps the client default. */ + readonly apiMaxRetries: Option.Option; } export class CommandSettings extends Context.Service()( diff --git a/apps/cli/src/config/project-ref.layer.unit.test.ts b/apps/cli/src/config/project-ref.layer.unit.test.ts index 433b83d21e..5446b0e3b0 100644 --- a/apps/cli/src/config/project-ref.layer.unit.test.ts +++ b/apps/cli/src/config/project-ref.layer.unit.test.ts @@ -33,6 +33,7 @@ function mockCliSettings(opts: { workdir: string; projectId?: string }) { workdir: opts.workdir, explicitWorkdir: false, userAgent: "SupabaseCLI/0.0.0-dev", + apiMaxRetries: Option.none(), }); } diff --git a/apps/cli/src/shared/functions/deploy.ts b/apps/cli/src/shared/functions/deploy.ts index c198a7ce34..5e4b797056 100644 --- a/apps/cli/src/shared/functions/deploy.ts +++ b/apps/cli/src/shared/functions/deploy.ts @@ -96,6 +96,8 @@ interface FunctionsDeployFlags { interface DeployFunctionsDependencies { readonly api: ApiClient; + /** `SUPABASE_API_MAX_RETRIES`; none keeps this command's own retries. */ + readonly apiMaxRetries: Option.Option; readonly cwd: string; readonly flagCwd: string; readonly projectRoot: string; @@ -1685,11 +1687,14 @@ const bundleFunctionWithDocker = Effect.fn("functions.deploy.bundleWithDocker")( const listRemoteFunctions = Effect.fn("functions.deploy.listRemoteFunctions")(function* ( api: ApiClient, projectRef: string, + apiMaxRetries: Option.Option, ) { return yield* Effect.gen(function* () { - const response = yield* api - .executeRaw(operationDefinitions.v1ListAllFunctions, { ref: projectRef }) - .pipe(Effect.mapError((error) => mapTransportError("failed to list functions", error))); + const response = yield* cappedRequest("listing functions", apiMaxRetries, (options) => + api + .executeRaw(operationDefinitions.v1ListAllFunctions, { ref: projectRef }, options) + .pipe(Effect.mapError((error) => mapTransportError("failed to list functions", error))), + ); const body = yield* response.text.pipe(Effect.orElseSucceed(() => "")); if (response.status === 200) { // A 200 whose body is not the expected JSON is an API-response problem, @@ -1713,9 +1718,11 @@ const listRemoteFunctions = Effect.fn("functions.deploy.listRemoteFunctions")(fu }).pipe( Effect.retry({ schedule: Schedule.exponential(Duration.seconds(1)), - times: 3, + times: cappedRetries(apiMaxRetries, 3), while: (error) => - !(error instanceof FunctionsApiStatusError) || error.status >= 500 || error.status === 429, + !(error instanceof FunctionsApiStatusError) || + error.status >= 500 || + (error.status === 429 && Option.isNone(apiMaxRetries)), }), ); }); @@ -1758,24 +1765,29 @@ function rateLimitDelayText(milliseconds: number) { // The command's own 429 loop is the only retry layer. const NO_CLIENT_RETRY = { retry: { maxRetries: 0 } } as const; -const rateLimitedRequest = Effect.fnUntraced(function* ( +// `SUPABASE_API_MAX_RETRIES` caps a command loop without ever raising it. +const cappedRetries = (apiMaxRetries: Option.Option, commandRetries: number) => + Math.min( + Option.getOrElse(apiMaxRetries, () => commandRetries), + commandRetries, + ); + +interface RateLimitedResponse { + readonly status: number; + readonly headers: Readonly>; +} + +const rateLimitedRequest = Effect.fnUntraced(function* ( action: string, - request: () => Effect.Effect< - { - readonly status: number; - readonly headers: Readonly>; - readonly body: Effect.Effect; - }, - Error - >, + apiMaxRetries: Option.Option, + request: () => Effect.Effect, ) { const output = yield* Output; + const maxRetries = cappedRetries(apiMaxRetries, DEPLOY_RATE_LIMIT_MAX_RETRIES); return yield* Effect.repeat(Effect.suspend(request), ($) => $(Schedule.forever).pipe( Schedule.passthrough, - Schedule.while( - ({ input, attempt }) => input.status === 429 && attempt <= DEPLOY_RATE_LIMIT_MAX_RETRIES, - ), + Schedule.while(({ input, attempt }) => input.status === 429 && attempt <= maxRetries), Schedule.modifyDelay(({ input, attempt, now }) => Effect.succeed(Duration.millis(rateLimitDelayMillis(input.headers, attempt - 1, now))), ), @@ -1789,6 +1801,17 @@ const rateLimitedRequest = Effect.fnUntraced(function* ( ); }); +// Once `SUPABASE_API_MAX_RETRIES` is set, 429s go through the capped loop that reads the rate-limit +// headers, and the client stays out so attempts don't multiply. +const cappedRequest = ( + action: string, + apiMaxRetries: Option.Option, + request: (options?: typeof NO_CLIENT_RETRY) => Effect.Effect, +) => + Option.isSome(apiMaxRetries) + ? rateLimitedRequest(action, apiMaxRetries, () => request(NO_CLIENT_RETRY)) + : request(); + const uploadFunctionSource = Effect.fn("functions.deploy.uploadFunctionSource")(function* ( api: ApiClient, projectRef: string, @@ -1797,6 +1820,7 @@ const uploadFunctionSource = Effect.fn("functions.deploy.uploadFunctionSource")( config: ResolvedDeployFunctionConfig, metadata: SourceDeployMetadata, bundleOnly: boolean, + apiMaxRetries: Option.Option, ) { const output = yield* Output; const files = yield* joinDetached( @@ -1810,33 +1834,36 @@ const uploadFunctionSource = Effect.fn("functions.deploy.uploadFunctionSource")( ), ), ); - const response = yield* rateLimitedRequest(`deploying function ${config.slug}`, () => - api - .executeRaw( - operationDefinitions.v1DeployAFunction, - { - ref: projectRef, - slug: config.slug, - ...(bundleOnly ? { bundleOnly: true } : {}), - body: { - metadata, - file: files, + const response = yield* rateLimitedRequest( + `deploying function ${config.slug}`, + apiMaxRetries, + () => + api + .executeRaw( + operationDefinitions.v1DeployAFunction, + { + ref: projectRef, + slug: config.slug, + ...(bundleOnly ? { bundleOnly: true } : {}), + body: { + metadata, + file: files, + }, }, - }, - NO_CLIENT_RETRY, - ) - .pipe( - // Read the body as text (never failing) so the status check below wins: - // a non-201 with a non-JSON body, or a 201 with malformed JSON, must - // classify as a status/response problem — not fall through - // `mapTransportError` as a network failure. - Effect.map((raw) => ({ - status: raw.status, - headers: raw.headers, - body: raw.text.pipe(Effect.orElseSucceed(() => "")), - })), - Effect.mapError((error) => mapTransportError("failed to deploy function", error)), - ), + NO_CLIENT_RETRY, + ) + .pipe( + // Read the body as text (never failing) so the status check below wins: + // a non-201 with a non-JSON body, or a 201 with malformed JSON, must + // classify as a status/response problem — not fall through + // `mapTransportError` as a network failure. + Effect.map((raw) => ({ + status: raw.status, + headers: raw.headers, + body: raw.text.pipe(Effect.orElseSucceed(() => "")), + })), + Effect.mapError((error) => mapTransportError("failed to deploy function", error)), + ), ); const body = yield* response.body; if (response.status !== 201) { @@ -1903,9 +1930,10 @@ const bulkUpdateRemoteFunctions = Effect.fn("functions.deploy.bulkUpdateFunction api: ApiClient, projectRef: string, functions: ReadonlyArray, + apiMaxRetries: Option.Option, ) { return yield* Effect.gen(function* () { - const response = yield* rateLimitedRequest("bulk updating functions", () => + const response = yield* rateLimitedRequest("bulk updating functions", apiMaxRetries, () => api .executeRaw( operationDefinitions.v1BulkUpdateFunctions, @@ -1937,7 +1965,7 @@ const bulkUpdateRemoteFunctions = Effect.fn("functions.deploy.bulkUpdateFunction }).pipe( Effect.retry({ schedule: Schedule.exponential(Duration.seconds(1)), - times: 3, + times: cappedRetries(apiMaxRetries, 3), while: (error) => !(error instanceof FunctionsApiStatusError) || error.status >= 500, }), ); @@ -1948,7 +1976,9 @@ const upsertBundledFunction = Effect.fn("functions.deploy.upsertFunction")(funct projectRef: string, bundled: BundledFunction, exists: boolean, + apiMaxRetries: Option.Option, ) { + const retries = cappedRetries(apiMaxRetries, 3); const shouldUpdateRef = yield* Ref.make(exists); const upsert = Effect.gen(function* () { @@ -1971,14 +2001,20 @@ const upsertBundledFunction = Effect.fn("functions.deploy.upsertFunction")(funct slug: bundled.slug, name: bundled.slug, }; - const request = shouldUpdate - ? api.executeRaw(operationDefinitions.v1UpdateAFunction, { - ...updateInput, - function_slug: bundled.slug, - }) - : api.executeRaw(operationDefinitions.v1CreateAFunction, createInput); - const response = yield* request.pipe( - Effect.mapError((error) => mapTransportError(`failed to ${action} function`, error)), + const response = yield* cappedRequest( + `${shouldUpdate ? "updating" : "creating"} function ${bundled.slug}`, + apiMaxRetries, + (options) => + (shouldUpdate + ? api.executeRaw( + operationDefinitions.v1UpdateAFunction, + { ...updateInput, function_slug: bundled.slug }, + options, + ) + : api.executeRaw(operationDefinitions.v1CreateAFunction, createInput, options) + ).pipe( + Effect.mapError((error) => mapTransportError(`failed to ${action} function`, error)), + ), ); const expectedStatus = shouldUpdate ? 200 : 201; @@ -2013,8 +2049,19 @@ const upsertBundledFunction = Effect.fn("functions.deploy.upsertFunction")(funct return yield* upsert.pipe( Effect.retry({ schedule: Schedule.exponential(Duration.millis(500)), - times: 3, - while: (error) => !(error instanceof FunctionsApiStatusError && error.decode === true), + // A duplicated create still becomes an update when no retries are allowed. + times: Math.max(retries, 1), + while: (error) => + Ref.get(shouldUpdateRef).pipe( + Effect.map( + (shouldUpdate) => + !( + error instanceof FunctionsApiStatusError && + (error.decode === true || (Option.isSome(apiMaxRetries) && error.status === 429)) + ) && + (retries > 0 || shouldUpdate !== exists), + ), + ), }), ); }); @@ -2238,6 +2285,7 @@ const deployViaApi = Effect.fn("functions.deploy.viaApi")(function* ( configs: ReadonlyArray, api: ApiClient, jobs: number, + apiMaxRetries: Option.Option, ) { const output = yield* Output; const path = yield* Path.Path; @@ -2258,7 +2306,9 @@ const deployViaApi = Effect.fn("functions.deploy.viaApi")(function* ( } const remoteBySlug = enabled.some((config) => config.verifyJwt === undefined) - ? new Map((yield* listRemoteFunctions(api, projectRef)).map((fn) => [fn.slug, fn])) + ? new Map( + (yield* listRemoteFunctions(api, projectRef, apiMaxRetries)).map((fn) => [fn.slug, fn]), + ) : new Map(); const [singleConfig] = enabled; @@ -2271,6 +2321,7 @@ const deployViaApi = Effect.fn("functions.deploy.viaApi")(function* ( singleConfig, createSourceMetadata(path, projectRoot, singleConfig, remoteBySlug.get(singleConfig.slug)), false, + apiMaxRetries, ); return; } @@ -2293,6 +2344,7 @@ const deployViaApi = Effect.fn("functions.deploy.viaApi")(function* ( config, createSourceMetadata(path, projectRoot, config, remoteBySlug.get(config.slug)), true, + apiMaxRetries, ), ); }).pipe( @@ -2318,7 +2370,7 @@ const deployViaApi = Effect.fn("functions.deploy.viaApi")(function* ( return yield* Effect.fail(new AggregateError(causes, messages.join("\n"))); } - const updated = yield* bulkUpdateRemoteFunctions(api, projectRef, deployed).pipe( + const updated = yield* bulkUpdateRemoteFunctions(api, projectRef, deployed, apiMaxRetries).pipe( Effect.map(() => ({ success: true as const })), Effect.catch((error) => Effect.succeed({ success: false as const, error })), ); @@ -2338,6 +2390,7 @@ interface DeployViaDockerOptions { readonly functionsDir: string; readonly configs: ReadonlyArray; readonly api: ApiClient; + readonly apiMaxRetries: Option.Option; /** Already resolved (explicit flag > `SUPABASE_NETWORK_ID` > generated) — see the caller. */ readonly networkMode: string; readonly verbose?: boolean; @@ -2355,6 +2408,7 @@ const deployViaDocker = Effect.fn("functions.deploy.viaDocker")(function* ( functionsDir, configs, api, + apiMaxRetries, networkMode, verbose = false, styleEmphasis = (text: string) => text, @@ -2362,7 +2416,7 @@ const deployViaDocker = Effect.fn("functions.deploy.viaDocker")(function* ( } = options; const output = yield* Output; yield* Effect.annotateCurrentSpan({ "function.count": configs.length }); - const remoteFunctions = yield* listRemoteFunctions(api, projectRef); + const remoteFunctions = yield* listRemoteFunctions(api, projectRef, apiMaxRetries); const remoteBySlug = new Map(remoteFunctions.map((fn) => [fn.slug, fn])); const changed: BulkUpdateFunction[] = []; @@ -2399,13 +2453,19 @@ const deployViaDocker = Effect.fn("functions.deploy.viaDocker")(function* ( ); changed.push( yield* toBulkUpdateItem( - yield* upsertBundledFunction(api, projectRef, bundled, current !== undefined), + yield* upsertBundledFunction( + api, + projectRef, + bundled, + current !== undefined, + apiMaxRetries, + ), ), ); } if (changed.length > 1) { - yield* bulkUpdateRemoteFunctions(api, projectRef, changed); + yield* bulkUpdateRemoteFunctions(api, projectRef, changed, apiMaxRetries); } }); @@ -2413,8 +2473,9 @@ const findPruneCandidates = Effect.fn("functions.deploy.pruneCandidates")(functi projectRef: string, configs: ReadonlyArray, api: ApiClient, + apiMaxRetries: Option.Option, ) { - const remoteFunctions = yield* listRemoteFunctions(api, projectRef); + const remoteFunctions = yield* listRemoteFunctions(api, projectRef, apiMaxRetries); const localSlugs = new Set(configs.map((config) => config.slug)); return remoteFunctions .filter((remote) => remote.status !== "REMOVED" && !localSlugs.has(remote.slug)) @@ -2580,7 +2641,7 @@ export const deployFunctions = Effect.fn("functions.deploy")(function* < const toDeploy = configs.filter((config) => config.enabled).map((config) => config.slug); const toPrune = flags.prune - ? yield* findPruneCandidates(projectRef, configs, dependencies.api) + ? yield* findPruneCandidates(projectRef, configs, dependencies.api, dependencies.apiMaxRetries) : []; if (toDeploy.length > 0 || toPrune.length > 0) { yield* requireDeployConsent(toDeploy, toPrune); @@ -2592,6 +2653,7 @@ export const deployFunctions = Effect.fn("functions.deploy")(function* < configs, dependencies.api, jobs, + dependencies.apiMaxRetries, ).pipe( Effect.as(Option.none()), Effect.catchIf( @@ -2628,6 +2690,7 @@ export const deployFunctions = Effect.fn("functions.deploy")(function* < functionsDir: path.join(dependencies.projectRoot, SUPABASE_FUNCTIONS_DIR), configs, api: dependencies.api, + apiMaxRetries: dependencies.apiMaxRetries, networkMode, verbose: debugEnabled, styleEmphasis, diff --git a/apps/cli/src/shared/services/services.shared.ts b/apps/cli/src/shared/services/services.shared.ts index 297c675921..810f331e2b 100644 --- a/apps/cli/src/shared/services/services.shared.ts +++ b/apps/cli/src/shared/services/services.shared.ts @@ -267,6 +267,7 @@ export interface ServiceFetchConfig { readonly headers?: Readonly>; readonly api?: ApiClient; readonly tenantBaseUrlOverride?: string; + readonly maxRetries?: number; } /** @public */ @@ -445,12 +446,15 @@ const makeConfiguredApiClient = Effect.fn("Services.buildApiClient")(function* ( ) { return ( input.api ?? - (yield* makeApiClient({ - baseUrl: input.apiUrl, - accessToken: input.accessToken, - userAgent: input.userAgent, - headers: input.headers, - })) + (yield* makeApiClient( + { + baseUrl: input.apiUrl, + accessToken: input.accessToken, + userAgent: input.userAgent, + headers: input.headers, + }, + { retry: { maxRetries: input.maxRetries } }, + )) ); }); diff --git a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt index a6c3fdeac1..e139598268 100644 --- a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt +++ b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt @@ -1,4 +1,5 @@ AccessTokenRequiredError +ApiMaxRetriesError BackupListNetworkError BackupListUnexpectedStatusError BackupRestoreNetworkError diff --git a/apps/cli/src/shared/telemetry/event-catalog.ts b/apps/cli/src/shared/telemetry/event-catalog.ts index 3ffd96ba15..15fe7270d3 100644 --- a/apps/cli/src/shared/telemetry/event-catalog.ts +++ b/apps/cli/src/shared/telemetry/event-catalog.ts @@ -103,6 +103,7 @@ export const EnvSignalPresenceKeys: ReadonlyArray = [ export const EnvSignalValueKeys: ReadonlyArray = [ "AI_AGENT", "CURSOR_EXTENSION_HOST_ROLE", + "SUPABASE_API_MAX_RETRIES", "TERM", "TERM_PROGRAM", "TERM_PROGRAM_VERSION", diff --git a/apps/cli/tests/helpers/command-mocks.ts b/apps/cli/tests/helpers/command-mocks.ts index 1367aa9394..4ea20e684b 100644 --- a/apps/cli/tests/helpers/command-mocks.ts +++ b/apps/cli/tests/helpers/command-mocks.ts @@ -427,6 +427,7 @@ export function mockCommandSettings(opts: { readonly userAgent?: string; readonly supabaseHome?: string; readonly githubToken?: Option.Option>; + readonly apiMaxRetries?: Option.Option; }): Layer.Layer { return Layer.succeed(CommandSettings, { githubToken: opts.githubToken ?? Option.none(), @@ -444,6 +445,7 @@ export function mockCommandSettings(opts: { workdirEnvValue: opts.workdirEnvValue === undefined ? Option.none() : Option.some(opts.workdirEnvValue), userAgent: opts.userAgent ?? DEFAULT_USER_AGENT, + apiMaxRetries: opts.apiMaxRetries ?? Option.none(), }); } @@ -538,6 +540,7 @@ export interface MockCommandPlatformApiOpts { readonly apiUrl?: string; readonly userAgent?: string; readonly accessToken?: string; + readonly clientMaxRetries?: number; } export interface MockCommandPlatformApiResult { @@ -609,7 +612,7 @@ export function mockCommandPlatformApi( accessToken: opts.accessToken ?? VALID_TOKEN, userAgent: opts.userAgent ?? DEFAULT_USER_AGENT, }, - { retry: { maxRetries: 0 } }, + { retry: { maxRetries: opts.clientMaxRetries ?? 0 } }, ), ).pipe(Layer.provide(httpClientLayer)); diff --git a/apps/cli/tests/helpers/compute.ts b/apps/cli/tests/helpers/compute.ts index 03c9cc7aae..53286ac0de 100644 --- a/apps/cli/tests/helpers/compute.ts +++ b/apps/cli/tests/helpers/compute.ts @@ -406,6 +406,7 @@ const testCliConfigLayer = (workdir: string, explicitWorkdir: boolean) => explicitWorkdir, workdirEnvValue: Option.none(), userAgent: "supabase", + apiMaxRetries: Option.none(), }); }), ).pipe(Layer.provide(BunServices.layer)); From d20aae39527f11755c3fc206dd19777939ea10f8 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Mon, 2 Nov 2026 22:35:19 +0000 Subject: [PATCH 022/101] chore(stack): bump realtime to v2.151.5-r0 (#7450) This bumps realtime from v2.151.4-r0 to v2.151.5-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.151.5-r0 Planned after supabase/slim-services published v2.151.5-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index abc1c70e87..a35cb58fde 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.151.4 AS realtime +FROM supabase/realtime:v2.151.5 AS realtime FROM supabase/storage-api:v1.89.2 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index ea400911e4..6448b61c9c 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -443,23 +443,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.151.4", + upstreamVersion: "v2.151.5", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.151.4-r0@sha256:3625d9c0db8b313df88198935d0ef4a75ddc596d7cd99a068c942a131a9850f0", - upstreamImage: "supabase/realtime:v2.151.4", + "ghcr.io/supabase/cli/realtime:v2.151.5-r0@sha256:698aa3c3d06ca6b34eaa922908ae2837af94a584fa98f0d7ffce91f999f0a25e", + upstreamImage: "supabase/realtime:v2.151.5", natives: { "darwin-arm64": { - archive: "853c1f7b4a79f5b79f2c4e23a758ea1cb793430ef9fa9e1c2e6e9112fcf71978", - manifest: "74313b61677a9bd4eb2be32e924ecdd9c734fb87c569d7966d98981e664e36ba", + archive: "62dc6877b671302ffb12289bc25fa1e5c62e229fd19658c274704b8dab711f32", + manifest: "514da4e1e485377bc54a84fc425512d3c6a3c233659dd83d69f067bdfd65a4bc", }, "linux-amd64": { - archive: "6974df885b63b2036cd488dd55a3a939d0090779dcf546a813cc04838aca6020", - manifest: "83a84baff1806a316bf827bc399a2693542ae7e45ee55caff3c1ee2756e204ad", + archive: "552112db22de4c6d345dc05ee7d2a49db52d1db7fbf49152d15cd21bb086e56d", + manifest: "994353380903db72944e2d489ec0bd8050d32e71d213b76ad1a88a712317cbcb", }, "linux-arm64": { - archive: "57c69a13a73b30d49919e89c2fc98ddef93f37dd38dac9fa88f61fa5138a2ff5", - manifest: "13cceacab3812f848c54f163d2a677fc4a3de11b61b4a30a7a0c47acbc9fe4db", + archive: "f54fd08cb66f3891dc7ecbb5035734838b5241a864233dabc6d46a1d2b0175f3", + manifest: "646ca44473ddd1aaa7bcf511ce40bbb2a31867ee8326255320f057accf8c4b99", }, }, }, From 44d4566e2a9b6af7deb255c16074c0ea08588a17 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 02:41:03 +0000 Subject: [PATCH 023/101] chore(stack): bump realtime to v2.151.6-r0 (#7453) This bumps realtime from v2.151.5-r0 to v2.151.6-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.151.6-r0 Planned after supabase/slim-services published v2.151.6-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index a35cb58fde..452c635870 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.151.5 AS realtime +FROM supabase/realtime:v2.151.6 AS realtime FROM supabase/storage-api:v1.89.2 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 6448b61c9c..c1535890cb 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -443,23 +443,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.151.5", + upstreamVersion: "v2.151.6", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.151.5-r0@sha256:698aa3c3d06ca6b34eaa922908ae2837af94a584fa98f0d7ffce91f999f0a25e", - upstreamImage: "supabase/realtime:v2.151.5", + "ghcr.io/supabase/cli/realtime:v2.151.6-r0@sha256:1fbceab01330fcf679e2ccdff606cd153fb3fe7746621b3386aacb7a556cd4d7", + upstreamImage: "supabase/realtime:v2.151.6", natives: { "darwin-arm64": { - archive: "62dc6877b671302ffb12289bc25fa1e5c62e229fd19658c274704b8dab711f32", - manifest: "514da4e1e485377bc54a84fc425512d3c6a3c233659dd83d69f067bdfd65a4bc", + archive: "bcc0acbe1fd3b69cff41865911f1f6178ad65a59e88307663939466f829fd5cd", + manifest: "e65de0671f646011d8b2321bc797b6d2418a6d22b2c2c9b2b566e11bf2aec411", }, "linux-amd64": { - archive: "552112db22de4c6d345dc05ee7d2a49db52d1db7fbf49152d15cd21bb086e56d", - manifest: "994353380903db72944e2d489ec0bd8050d32e71d213b76ad1a88a712317cbcb", + archive: "9f7c11f8fd3b53712e42ec2845b055c9209d99a234501e56ef367911836cf52d", + manifest: "1fa535f146cdf926ec3713b3ae3b335e64e81d9c28ff526aee8bff2b2a6bf92a", }, "linux-arm64": { - archive: "f54fd08cb66f3891dc7ecbb5035734838b5241a864233dabc6d46a1d2b0175f3", - manifest: "646ca44473ddd1aaa7bcf511ce40bbb2a31867ee8326255320f057accf8c4b99", + archive: "8dc45e26af0f9fa8098fdaa4d6b4248768cc6524b95e12cd01045d535418b7d0", + manifest: "d1e088963fa237b63ff05982e7d932633f08d1274f0b7cb3653dc367bc8964f5", }, }, }, From efa454028855e9f702a44c51893c3f49e30a3e5d Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 02:49:17 +0000 Subject: [PATCH 024/101] chore(stack): bump postgres to 17.11.0.018-r0 (#7454) This bumps postgres from 17.11.0.017-r0 to 17.11.0.018-r0. https://github.com/supabase/slim-services/releases/tag/postgres-17.11.0.018-r0 Planned after supabase/slim-services published 17.11.0.018-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 452c635870..64a95ca105 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -5,7 +5,7 @@ # a catalog update. Everything else in this file (this comment, kong, pg14, the one-shot job # images) is hand- or Dependabot-managed. Keep a line between image lines: git conflicts on edits to # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. -FROM supabase/postgres:17.11.0.017 AS pg +FROM supabase/postgres:17.11.0.018 AS pg FROM supabase/postgres:15.19.0.017 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index c1535890cb..356c3693be 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -127,23 +127,23 @@ const definitions: Readonly> = { database: definition( "postgres", { - upstreamVersion: "17.11.0.017", + upstreamVersion: "17.11.0.018", revision: 0, image: - "ghcr.io/supabase/cli/postgres:17.11.0.017-r0@sha256:2ae0e78c6e87fb9cf0cd8ba6795a597198e45bd2ac6e6f63a4b0ece024a008ef", - upstreamImage: "supabase/postgres:17.11.0.017", + "ghcr.io/supabase/cli/postgres:17.11.0.018-r0@sha256:4f5cf9d03718b200df8c8e4735081eaa55a74d11dca848f5c8283203921e5751", + upstreamImage: "supabase/postgres:17.11.0.018", natives: { "darwin-arm64": { - archive: "543c9b10cee5efda3aa4819970f8760ea0cc52ad7a326e24ed52add97d665886", - manifest: "e058218a46629e2087c68baf5600722e51f694e1acff1062050110c94547a1ff", + archive: "5033846ff2dc56a992e9bf929d62d132bd2819656e040d4a359632121a9722e7", + manifest: "9e6d77b467111e6b6dd822ebf5b100214ca5837b563556e97227b93332ae87ce", }, "linux-amd64": { - archive: "ae6aa7604938c8b94d87769cff801a81ddd4dbf7b92dca1cfe31baa2de6af46c", - manifest: "8cc9731ff5011e630c142843c863e012dc726024dcb87d7e8e1d34931c2418fa", + archive: "80c80a56481e65491eaafe1609eedafe0b35ee2ff10773adfb9ab34e5ebfa8c5", + manifest: "56c4b951f5dd16870fca927e221c2c289d23bd42514027594fab13ede672d9b1", }, "linux-arm64": { - archive: "72e80f471e39d214cffeb5d3b695f09542af3fdd4b1b0b4b01e7135c0857a37a", - manifest: "27930c8c06808195d7520d179c62d06cf0c57f4788c5419298694288d13c9ece", + archive: "e066fcea166bf302ba24779d32ff22150905860fe7fb067c415cff7efe7e5f3e", + manifest: "42883bbb35d4f165706eabc4bf66002636eb500ea1ff55915c539e970f768339", }, }, }, From de6117ae3b63857a1a59ee805edc3dcfe9c902e3 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 02:50:01 +0000 Subject: [PATCH 025/101] chore(stack): add postgres 17.11.0.018-orioledb-r0 (#7456) This adds postgres 17.11.0.018-orioledb-r0 as a new release line. https://github.com/supabase/slim-services/releases/tag/postgres-17.11.0.018-orioledb-r0 Planned after supabase/slim-services published 17.11.0.018-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- packages/stack/src/Artifacts.ts | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 356c3693be..da445c9a00 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -388,6 +388,27 @@ const definitions: Readonly> = { }, }, }, + "17.11.0.018-orioledb": { + upstreamVersion: "17.11.0.018-orioledb", + revision: 0, + image: + "ghcr.io/supabase/cli/postgres:17.11.0.018-orioledb-r0@sha256:5f160baf2fc9eb544aafa8dd33231a611d5c725242f0ecb89c41a422f35e3335", + upstreamImage: "supabase/postgres:17.11.0.018-orioledb", + natives: { + "darwin-arm64": { + archive: "ea145998919ee892a0f1b7d52163ba4d3665acc492a033f250a0de76b44886fb", + manifest: "c096a92f791110c000932877e1b6f83455a82d2da8cf9d7c273cbbbc8952721f", + }, + "linux-amd64": { + archive: "eb715a98ed54059ca67fedf0c1bef6d8dce8e33d964d10531bce56dbe061459c", + manifest: "ce76378b7b29c3f47e16f70918e9357193eb229ef4cb30edb3a5c38b0fc3db03", + }, + "linux-arm64": { + archive: "b8077d9a167a8341444d3185dbcb3cf57d5768f179ab7082740ba64d75925feb", + manifest: "57cbddaca5c392325b570c4e1f8166dbfce32c8e31ab0ad49d7fd88ce634d1d7", + }, + }, + }, }, ), rest: definition( From c9473a797c52abd998a1f0d2b3a35650f42f86f8 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 02:50:43 +0000 Subject: [PATCH 026/101] chore(stack): bump postgres to 15.19.0.018-r0 (#7455) This bumps postgres from 15.19.0.017-r0 to 15.19.0.018-r0. https://github.com/supabase/slim-services/releases/tag/postgres-15.19.0.018-r0 Planned after supabase/slim-services published 17.11.0.018-orioledb-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 64a95ca105..b2358d250c 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -7,7 +7,7 @@ # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. FROM supabase/postgres:17.11.0.018 AS pg -FROM supabase/postgres:15.19.0.017 AS pg15 +FROM supabase/postgres:15.19.0.018 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). FROM supabase/postgres:14.1.0.89 AS pg14 # New always-on service alias: extend SERVICE_IMAGE_ALIASES in diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index da445c9a00..240bb4a696 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -157,24 +157,24 @@ const definitions: Readonly> = { "bin/psql", ], { - "15.19.0.017": { - upstreamVersion: "15.19.0.017", + "15.19.0.018": { + upstreamVersion: "15.19.0.018", revision: 0, image: - "ghcr.io/supabase/cli/postgres:15.19.0.017-r0@sha256:9faad1022a7b8a8c17108a4a38f41a124dadcb6a74248abde933bf9e3a6860c3", - upstreamImage: "supabase/postgres:15.19.0.017", + "ghcr.io/supabase/cli/postgres:15.19.0.018-r0@sha256:b0b4057a43543546d54d74434b562721af4bf9a7802b53a00158dcec7059bb4b", + upstreamImage: "supabase/postgres:15.19.0.018", natives: { "darwin-arm64": { - archive: "1b521f140a7890dfc657c677f1abe38f59897f7f239b4b13cb10b74cd35b6baf", - manifest: "55b693210d762465f0d1e2693cc832914e5dfbda2b8275a6067fb21ba90fe26e", + archive: "682bd39c23b0ef677e652f641f74862aa61b9a5472f51e29f5163b09876f6da4", + manifest: "27ad0dffd559ed1dc5e84ed368890dc1089d135d34a5591ce7b7640f5cf76aa6", }, "linux-amd64": { - archive: "09f2a1fbbe2ade0a286c96ee9f7b217951bc28820c5fc706cbf2a9fdf04476aa", - manifest: "2e2133d5b18d0291fe10349e3ac98e666c46b2adeb23e6cbb287d079b91fd9dc", + archive: "3191c152650e7284d4168e71ab91f45e31fb690153c58d8f2a425e03ea86b4e1", + manifest: "0301008324b5df8c885066105dd59f4faf41974be54bec7a285b7d6dcc71a535", }, "linux-arm64": { - archive: "7c434d619bc5621944afc65827390a30e993d949730d1fe33d967c66beb98fea", - manifest: "0f96b47e43910fd913d6354728904ea25944ac79f03a56234c336b37e51ed886", + archive: "21f3d647d844e19aa656802438992597f177d1b60545662e7244d6eb0be01248", + manifest: "04baea44897bd198c161460e0ae46bd304a349b0d8baf8885e158ec49aee8448", }, }, }, From b2cadc721bef31da5c0c3651d93902bf1653c7bd Mon Sep 17 00:00:00 2001 From: Bruno Kilian Date: Tue, 3 Nov 2026 09:17:03 +0000 Subject: [PATCH 027/101] feat(cli): add inspect db toast-sizes command (#6528) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Adds `supabase inspect db toast-sizes` — a new `inspect db` subcommand that lists user tables with TOAST storage, ordered by TOAST size (largest first). ## What changed - `apps/cli/src/commands/inspect/db/toast-sizes/` — 3-file subcommand (query, handler, command) - `apps/cli/docs/supabase/inspect/db-toast-sizes.md` — docs overlay with description and sample output - `apps/cli/src/commands/inspect/db/db.command.ts` — registers the new subcommand - `apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md` — updates active count (13 → 14) and query table - `apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts` — adds a test case for `toastSizesSpec` ## Why it matters TOAST (The Oversized-Attribute Storage Technique) is PostgreSQL's overflow storage for large values in `TEXT`, `JSONB`, and `bytea` columns. Autovacuum runs on TOAST tables **independently** from the main heap, so dead TOAST chunks accumulate silently. A high TOAST dead-tuple ratio increases I/O and inflates effective database size — but it doesn't appear in the existing `bloat` or `vacuum-stats` commands, which only cover the main heap. This command surfaces: - Heap vs. TOAST size split per table - Live/dead TOAST chunk counts and dead-chunk percentage - Last autovacuum/vacuum timestamps on the TOAST table It's a common source of invisible bloat, especially in tables with large JSONB columns. ## Testing - New integration test case in `inspect-specs.integration.test.ts` (count updated 13 → 14) - Query filters out internal Supabase schemas via the shared `$1` LIKE-escape parameter (consistent with `bloat`, `vacuum-stats`, `index-stats`, `table-stats`, `db-stats`) - Docs overlay at `apps/cli/docs/supabase/inspect/db-toast-sizes.md` follows the same format as existing overlays ## Docs A follow-up PR to `supabase/supabase` will add this command to the [Inspect guide](https://supabase.com/docs/guides/observability/inspect) under the **Disk Storage Analysis** section after this merges. --------- Co-authored-by: Claude Sonnet 4.6 Co-authored-by: avallete --- apps/cli-e2e/fixtures/pg/toast-sizes.json | 15 +++++ .../interactions.json | 1 + .../interactions.json | 1 + apps/cli-e2e/src/tests/inspect.e2e.test.ts | 1 + .../docs/supabase/inspect/db-toast-sizes.md | 13 ++++ .../src/commands/inspect/db/SIDE_EFFECTS.md | 5 +- .../cli/src/commands/inspect/db/db.command.ts | 2 + apps/cli/src/commands/inspect/db/db.layers.ts | 2 +- .../db/inspect-query.integration.test.ts | 2 +- .../src/commands/inspect/db/inspect-query.ts | 4 +- .../db/inspect-specs.integration.test.ts | 50 ++++++++++++++- .../db/toast-sizes/toast-sizes.command.ts | 23 +++++++ .../db/toast-sizes/toast-sizes.handler.ts | 4 ++ .../db/toast-sizes/toast-sizes.query.ts | 64 +++++++++++++++++++ .../commands/inspect/report/SIDE_EFFECTS.md | 8 +-- .../inspect/report/report.integration.test.ts | 20 +++--- .../commands/inspect/report/report.queries.ts | 4 +- .../report/report.queries.unit.test.ts | 3 +- 18 files changed, 198 insertions(+), 24 deletions(-) create mode 100644 apps/cli-e2e/fixtures/pg/toast-sizes.json create mode 100644 apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json create mode 100644 apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json create mode 100644 apps/cli/docs/supabase/inspect/db-toast-sizes.md create mode 100644 apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.command.ts create mode 100644 apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.handler.ts create mode 100644 apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts diff --git a/apps/cli-e2e/fixtures/pg/toast-sizes.json b/apps/cli-e2e/fixtures/pg/toast-sizes.json new file mode 100644 index 0000000000..296e33254f --- /dev/null +++ b/apps/cli-e2e/fixtures/pg/toast-sizes.json @@ -0,0 +1,15 @@ +{ + "columns": [ + "name", + "total_size", + "heap_size", + "toast_size", + "toast_live_chunks", + "toast_dead_chunks", + "toast_dead_pct", + "last_autovacuum", + "last_vacuum" + ], + "typeOids": [25, 25, 25, 25, 20, 20, 1700, 25, 25], + "rows": [["public.documents", "4200 MB", "800 MB", "3400 MB", "250000", "18000", "6.7", "2026-04-23 12:00", ""]] +} diff --git a/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json new file mode 100644 index 0000000000..fe51488c70 --- /dev/null +++ b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json @@ -0,0 +1 @@ +[] diff --git a/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json new file mode 100644 index 0000000000..fe51488c70 --- /dev/null +++ b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json @@ -0,0 +1 @@ +[] diff --git a/apps/cli-e2e/src/tests/inspect.e2e.test.ts b/apps/cli-e2e/src/tests/inspect.e2e.test.ts index 12f3e4d9d2..441b3888a3 100644 --- a/apps/cli-e2e/src/tests/inspect.e2e.test.ts +++ b/apps/cli-e2e/src/tests/inspect.e2e.test.ts @@ -29,6 +29,7 @@ const SUBCOMMANDS = [ { name: "vacuum-stats", fixtureKey: "vacuum-stats", assertValue: "public.events" }, { name: "table-stats", fixtureKey: "table-stats", assertValue: "public.orders" }, { name: "traffic-profile", fixtureKey: "traffic-profile", assertValue: "sessions" }, + { name: "toast-sizes", fixtureKey: "toast-sizes", assertValue: "public.documents" }, ] as const; describe("inspect:flags", () => { diff --git a/apps/cli/docs/supabase/inspect/db-toast-sizes.md b/apps/cli/docs/supabase/inspect/db-toast-sizes.md new file mode 100644 index 0000000000..2a0debf6e9 --- /dev/null +++ b/apps/cli/docs/supabase/inspect/db-toast-sizes.md @@ -0,0 +1,13 @@ +# db-toast-sizes + +This command displays TOAST table sizes and dead chunk counts for every ordinary user table that has a TOAST relation (materialized views and partitioned parents are not listed). When a row exceeds ~2 kB, Postgres compresses and/or moves its largest variable-length values (TEXT, JSONB, bytea) out-of-line into a companion TOAST table. TOAST Size includes the TOAST index; Total Size also includes the table's own indexes. Autovacuum runs on the TOAST table independently from the main heap, so it can accumulate dead chunks even when the parent table looks healthy by its own dead-tuple count. + +A table that appears fine by `vacuum-stats` or `bloat` alone can still have significant TOAST bloat that wastes disk space and slows reads. High `TOAST Dead %` values indicate that autovacuum is not keeping up with the TOAST table and a manual `VACUUM` may be needed. + +``` + TABLE │ TOTAL SIZE │ HEAP SIZE │ TOAST SIZE │ TOAST LIVE CHUNKS │ TOAST DEAD CHUNKS │ TOAST DEAD % │ LAST AUTOVACUUM │ LAST VACUUM +────────────────────────┼────────────┼───────────┼────────────┼───────────────────┼───────────────────┼──────────────┼────────────────────┼───────────── + public.documents │ 4350 MB │ 800 MB │ 3400 MB │ 250000 │ 18000 │ 6.7 │ 2024-03-01 04:12 │ + public.media_assets │ 905 MB │ 120 MB │ 770 MB │ 80000 │ 200 │ 0.2 │ 2024-03-02 01:45 │ + public.messages │ 365 MB │ 280 MB │ 60 MB │ 95000 │ 0 │ 0.0 │ 2024-03-02 03:10 │ +``` diff --git a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md index 4469c0119c..a2d64f1c8e 100644 --- a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md @@ -2,7 +2,7 @@ Target: selectable, no consent — see [Command target](../../../../docs/command-target.md) -Single shared side-effect document for all 13 active `inspect db` subcommands and +Single shared side-effect document for all 14 active `inspect db` subcommands and their 12 deprecated aliases. Every subcommand has the same surface — it resolves its target, runs one read-only `SELECT`, and renders the result as an ASCII table. They differ only in the SQL run and the columns rendered (see the per-subcommand `.query.ts`). @@ -57,7 +57,7 @@ Local targets and database URLs make none. ## Database Queries Each subcommand runs one read-only `SELECT` (the embedded `.sql`). The -5 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list; +6 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list; `db-stats` additionally takes `$2` = the database name. | Subcommand | SQL file | InternalSchemas param? | @@ -67,6 +67,7 @@ Each subcommand runs one read-only `SELECT` (the embedded `.sql`). The | bloat | bloat.sql | yes (`$1`) | | vacuum-stats | vacuum_stats.sql | yes (`$1`) | | table-stats | table_stats.sql | yes (`$1`) | +| toast-sizes | toast_sizes.sql | yes (`$1`) | | replication-slots | replication_slots.sql | no | | locks | locks.sql | no | | blocking | blocking.sql | no | diff --git a/apps/cli/src/commands/inspect/db/db.command.ts b/apps/cli/src/commands/inspect/db/db.command.ts index 396e32f831..726bdcba2d 100644 --- a/apps/cli/src/commands/inspect/db/db.command.ts +++ b/apps/cli/src/commands/inspect/db/db.command.ts @@ -19,6 +19,7 @@ import { inspectDbTableIndexSizesCommand } from "./table-index-sizes/table-index import { inspectDbTableRecordCountsCommand } from "./table-record-counts/table-record-counts.command.ts"; import { inspectDbTableSizesCommand } from "./table-sizes/table-sizes.command.ts"; import { inspectDbTableStatsCommand } from "./table-stats/table-stats.command.ts"; +import { inspectDbToastSizesCommand } from "./toast-sizes/toast-sizes.command.ts"; import { inspectDbTotalIndexSizeCommand } from "./total-index-size/total-index-size.command.ts"; import { inspectDbTotalTableSizesCommand } from "./total-table-sizes/total-table-sizes.command.ts"; import { inspectDbTrafficProfileCommand } from "./traffic-profile/traffic-profile.command.ts"; @@ -44,6 +45,7 @@ export const inspectDbCommand = Command.make("db").pipe( inspectDbVacuumStatsCommand, inspectDbTableStatsCommand, inspectDbTrafficProfileCommand, + inspectDbToastSizesCommand, inspectDbCacheHitCommand, inspectDbIndexUsageCommand, inspectDbTotalIndexSizeCommand.pipe(Command.unlisted), diff --git a/apps/cli/src/commands/inspect/db/db.layers.ts b/apps/cli/src/commands/inspect/db/db.layers.ts index 16b1f5f760..a5eba5fc8e 100644 --- a/apps/cli/src/commands/inspect/db/db.layers.ts +++ b/apps/cli/src/commands/inspect/db/db.layers.ts @@ -5,7 +5,7 @@ import { inspectBaseLayer } from "../inspect.layers.ts"; /** * The command-runtime path for a single `inspect db ` subcommand, appended to - * `["inspect", "db"]` so each of the 25 leaves — and each deprecated alias — records its own + * `["inspect", "db"]` so each of the 26 leaves — and each deprecated alias — records its own * name in `cli_command_executed`, not a shared "inspect db" event. */ export const inspectDbCommandPath = (leaf: string): ReadonlyArray => [ diff --git a/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts index 2585980738..81efeefbeb 100644 --- a/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts @@ -544,7 +544,7 @@ describe("inspect db through the real database-config resolver", () => { expect(asLines(requests)).toEqual([poolerRequest, loginRoleRequest]); expect(connection.connectCalls[0]?.cfg.host).toBe("aws-0-us-east-1.pooler.supabase.com"); expect(out.stdoutText).toContain("8192 kB"); - expect(INSPECT_DB_LEAVES).toHaveLength(25); + expect(INSPECT_DB_LEAVES).toHaveLength(26); expect(INSPECT_DB_LEAVES).toEqual(expect.arrayContaining(REMOVED_INSPECT_DB_LEAVES)); for (const command of RUNNER_INSPECT_DB_LEAVES) { assertPermissionDrift({ diff --git a/apps/cli/src/commands/inspect/db/inspect-query.ts b/apps/cli/src/commands/inspect/db/inspect-query.ts index 8020243eb0..7f4821e17b 100644 --- a/apps/cli/src/commands/inspect/db/inspect-query.ts +++ b/apps/cli/src/commands/inspect/db/inspect-query.ts @@ -44,8 +44,8 @@ export function inspectText(value: unknown): string { /** * The unwrapped text cell (no backtick code span): an empty/null value renders as the empty - * string. Only the `vacuum_stats` timestamp columns are written bare; every other string - * column is wrapped (use `inspectText`). + * string. Use it for timestamp columns (`vacuum_stats`, `toast_sizes`); wrap other string + * columns with `inspectText`. */ export function inspectPlainText(value: unknown): string { if (value === null || value === undefined) return ""; diff --git a/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts index e1e7f1d0d1..52c015ebce 100644 --- a/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts @@ -30,6 +30,7 @@ import { outliersSpec } from "./outliers/outliers.query.ts"; import { replicationSlotsSpec } from "./replication-slots/replication-slots.query.ts"; import { roleStatsSpec } from "./role-stats/role-stats.query.ts"; import { tableStatsSpec } from "./table-stats/table-stats.query.ts"; +import { toastSizesSpec } from "./toast-sizes/toast-sizes.query.ts"; import { trafficProfileSpec } from "./traffic-profile/traffic-profile.query.ts"; import { vacuumStatsSpec } from "./vacuum-stats/vacuum-stats.query.ts"; @@ -258,11 +259,27 @@ const cases: ReadonlyArray = [ }, expect: ["public", "100", "50", "12.0", "1:1 (Balanced)"], }, + { + spec: toastSizesSpec, + params: "schemas1", + row: { + name: "public.events", + total_size: "120 kB", + heap_size: "80 kB", + toast_size: "40 kB", + toast_live_chunks: 1200, + toast_dead_chunks: 80, + toast_dead_pct: "6.3", + last_autovacuum: "2025-01-15 03:00", + last_vacuum: "", + }, + expect: ["public.events", "120 kB", "40 kB", "1200", "80", "6.3", "2025-01-15 03:00"], + }, ]; describe("inspect db specs (per-subcommand correctness)", () => { - it("covers all 13 active subcommands", () => { - expect(cases).toHaveLength(13); + it("covers all 14 active subcommands", () => { + expect(cases).toHaveLength(14); }); for (const testCase of cases) { @@ -315,4 +332,33 @@ describe("inspect db specs (per-subcommand correctness)", () => { expect(cells[1]).toBe("``"); expect(cells[4]).toBe(""); }); + + it("projects toast-sizes columns in header order, leaving a never-vacuumed cell empty", () => { + const row: Record = { + name: "public.events", + total_size: "120 kB", + heap_size: "80 kB", + toast_size: "40 kB", + toast_live_chunks: 1200, + toast_dead_chunks: 80, + toast_dead_pct: "6.3", + last_autovacuum: "2025-01-15 03:00", + last_vacuum: null, + }; + const cells = toastSizesSpec.project(row, { + conn: LOCAL_CONN, + isLocal: true, + } satisfies ResolvedDbConfig); + expect(cells).toEqual([ + "public.events", + "120 kB", + "80 kB", + "40 kB", + "1200", + "80", + "6.3", + "2025-01-15 03:00", + "", + ]); + }); }); diff --git a/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.command.ts b/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.command.ts new file mode 100644 index 0000000000..937ec804a4 --- /dev/null +++ b/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.command.ts @@ -0,0 +1,23 @@ +import { Command } from "effect/cli"; +import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; +import { inspectDbToastSizes } from "./toast-sizes.handler.ts"; +import { + INSPECT_DB_FLAGS, + INSPECT_DB_PERMISSIONS, + inspectDbCommandHandler, + withInspectDbTargetPolicy, +} from "../inspect-db-command.ts"; +import { inspectDbRuntimeLayer } from "../db.layers.ts"; +import { cliConfigValuesLayer } from "../../../../config/cli-config-values.layer.ts"; +import { withCliConfigFlags } from "../../../../config/cli-config-flags.ts"; + +export const inspectDbToastSizesCommand = Command.make("toast-sizes", INSPECT_DB_FLAGS).pipe( + Command.withDescription("Show TOAST table sizes and dead chunk counts for user tables."), + Command.withShortDescription("Show TOAST table sizes and dead chunk counts"), + withPermissions(INSPECT_DB_PERMISSIONS), + Command.withHandler(inspectDbCommandHandler(inspectDbToastSizes)), + Command.provide(inspectDbRuntimeLayer("toast-sizes")), + Command.provide(cliConfigValuesLayer), + withCliConfigFlags(INSPECT_DB_FLAGS), + withInspectDbTargetPolicy, +); diff --git a/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.handler.ts b/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.handler.ts new file mode 100644 index 0000000000..68f14bfa2b --- /dev/null +++ b/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.handler.ts @@ -0,0 +1,4 @@ +import { makeInspectDbHandler } from "../inspect-query.ts"; +import { toastSizesSpec } from "./toast-sizes.query.ts"; + +export const inspectDbToastSizes = makeInspectDbHandler(toastSizesSpec, "inspect.db.toast-sizes"); diff --git a/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts b/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts new file mode 100644 index 0000000000..a4f0bd669e --- /dev/null +++ b/apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts @@ -0,0 +1,64 @@ +import { + inspectFloat1, + inspectInt, + inspectPlainText, + inspectText, + type InspectQuerySpec, +} from "../inspect-query.ts"; +import { INTERNAL_SCHEMAS, likeEscapeSchema } from "../inspect-schemas.ts"; + +// Ordinary tables only; materialized views and partitioned parents are out of scope. +const SQL = ` +SELECT + FORMAT('%I.%I', n.nspname, main.relname) AS name, + pg_size_pretty(pg_total_relation_size(main.oid)) AS total_size, + pg_size_pretty(pg_relation_size(main.oid)) AS heap_size, + pg_size_pretty(pg_total_relation_size(main.reltoastrelid)) AS toast_size, + COALESCE(ts.n_live_tup, 0) AS toast_live_chunks, + COALESCE(ts.n_dead_tup, 0) AS toast_dead_chunks, + COALESCE( + round(100.0 * ts.n_dead_tup / nullif(ts.n_live_tup + ts.n_dead_tup, 0), 1), + 0.0 + ) AS toast_dead_pct, + COALESCE(to_char(ts.last_autovacuum, 'YYYY-MM-DD HH24:MI'), '') AS last_autovacuum, + COALESCE(to_char(ts.last_vacuum, 'YYYY-MM-DD HH24:MI'), '') AS last_vacuum +FROM pg_class main +JOIN pg_namespace n ON n.oid = main.relnamespace +LEFT JOIN pg_stat_all_tables ts ON ts.relid = main.reltoastrelid +WHERE main.relkind = 'r' + AND main.reltoastrelid <> 0 + AND pg_total_relation_size(main.reltoastrelid) > 0 + AND NOT n.nspname LIKE ANY($1) +ORDER BY pg_total_relation_size(main.reltoastrelid) DESC, name`; + +/** + * `inspect db toast-sizes` — per-table TOAST footprint (including its index) and TOAST dead-chunk + * stats; Total Size also counts the parent's indexes, so it exceeds Heap + TOAST. + */ +export const toastSizesSpec: InspectQuerySpec = { + name: "toast-sizes", + sql: SQL, + params: () => [likeEscapeSchema(INTERNAL_SCHEMAS)], + headers: [ + "Table", + "Total Size", + "Heap Size", + "TOAST Size", + "TOAST Live Chunks", + "TOAST Dead Chunks", + "TOAST Dead %", + "Last Autovacuum", + "Last Vacuum", + ], + project: (row) => [ + inspectText(row["name"]), + inspectText(row["total_size"]), + inspectText(row["heap_size"]), + inspectText(row["toast_size"]), + inspectInt(row["toast_live_chunks"]), + inspectInt(row["toast_dead_chunks"]), + inspectFloat1(row["toast_dead_pct"]), + inspectPlainText(row["last_autovacuum"]), + inspectPlainText(row["last_vacuum"]), + ], +}; diff --git a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md index 320db759c7..c6db6f19af 100644 --- a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md @@ -16,7 +16,7 @@ validating those CSVs. | `/supabase/.temp/project-ref` | plain text | a remote project: resolves the ref (bare `--remote` with no `SUPABASE_PROJECT_ID`) and tells whether it is the linked project; with no selector, only to word the linked-checkout hint | | `/supabase/.temp/linked-project.json` | JSON | to name the project in the `Target:` banner and the linked-checkout hint, and as the parent project for `--remote=` | | `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` unset and the target is a remote project (or a branch is looked up) | -| `//.csv` ×14 | CSV | read back in-memory for rule evaluation | +| `//.csv` ×15 | CSV | read back in-memory for rule evaluation | | `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | remote project, when set; see `db push` | A **missing** `config.toml` is fine (defaults apply); a @@ -30,13 +30,13 @@ or an unknown rule key aborts the command before connecting. Rules and `env(VAR) | Path | Mode | When | | ------------------------------------------ | ---- | --------------------------------------------------------------- | | `//` (directory) | 0755 | always — created recursively | -| `//.csv` ×14 | 0644 | always — one CSV per inspect query (server-side `COPY ... CSV`) | +| `//.csv` ×15 | 0644 | always — one CSV per inspect query (server-side `COPY ... CSV`) | | `~/.supabase/telemetry.json` | — | always (telemetry flush) | -The 14 CSV basenames (underscored, matching the SQL filenames — **not** the +The 15 CSV basenames (underscored, matching the SQL filenames — **not** the `inspect db` command names): `bloat`, `blocking`, `calls`, `db_stats`, `index_stats`, `locks`, `long_running_queries`, `outliers`, `replication_slots`, -`role_stats`, `table_stats`, `traffic_profile`, `unused_indexes`, `vacuum_stats`. +`role_stats`, `table_stats`, `toast_sizes`, `traffic_profile`, `unused_indexes`, `vacuum_stats`. The date folder is **local-time** `YYYY-MM-DD`. A relative `--output-dir` resolves against the process CWD, not `--workdir`; an absolute path is used as-is. diff --git a/apps/cli/src/commands/inspect/report/report.integration.test.ts b/apps/cli/src/commands/inspect/report/report.integration.test.ts index 06aaf3feba..9d9a2eb5a3 100644 --- a/apps/cli/src/commands/inspect/report/report.integration.test.ts +++ b/apps/cli/src/commands/inspect/report/report.integration.test.ts @@ -293,11 +293,11 @@ describe("inspect report", () => { const base = yield* tempDir("supabase-report-out-"); yield* inspectReport(flags({ outputDir: base })); const { dir, files } = yield* dateFolderContents(base); - expect(files.length).toBe(14); + expect(files.length).toBe(15); expect(files).toContain("db_stats.csv"); expect(files).toContain("unused_indexes.csv"); expect(files).not.toContain("db-stats.csv"); - expect(connection.copiedSql.length).toBe(14); + expect(connection.copiedSql.length).toBe(15); expect( connection.copiedSql.every( (s) => s.startsWith("COPY (") && s.endsWith("TO STDOUT WITH CSV HEADER"), @@ -534,10 +534,10 @@ describe("inspect report", () => { expect.objectContaining({ type: "success", message: "inspect report" }), ); const data = out.messages.find((m) => m.type === "success")?.data; - expect(data?.files).toHaveLength(14); + expect(data?.files).toHaveLength(15); expect(typeof data?.outputDir).toBe("string"); expect(data?.rules).toHaveLength(13); - expect((yield* dateFolderContents(base)).files.length).toBe(14); + expect((yield* dateFolderContents(base)).files.length).toBe(15); expect(out.stderrText).toBe(""); }).pipe(Effect.provide(layer)); }); @@ -639,7 +639,7 @@ describe("inspect report", () => { const { layer } = setupReport({ csvs: DEFAULT_RULE_CSVS, cwd }); yield* inspectReport(flags({ outputDir: "reports" })).pipe(Effect.provide(layer)); const { files } = yield* dateFolderContents(path.join(cwd, "reports")); - expect(files.length).toBe(14); + expect(files.length).toBe(15); }).pipe(Effect.provide(BunServices.layer)), ); @@ -650,7 +650,7 @@ describe("inspect report", () => { const cwd = yield* tempDir("supabase-report-cwd-"); const { layer } = setupReport({ csvs: DEFAULT_RULE_CSVS, cwd }); yield* inspectReport(flags({ outputDir: base })).pipe(Effect.provide(layer)); - expect((yield* dateFolderContents(base)).files.length).toBe(14); + expect((yield* dateFolderContents(base)).files.length).toBe(15); expect((yield* fs.readDirectory(cwd)).length).toBe(0); }).pipe(Effect.provide(BunServices.layer)), ); @@ -706,8 +706,8 @@ describe("inspect report through the real database-config resolver", () => { `POST /v1/projects/${VALID_REF}/cli/login-role`, ]); const { files } = yield* dateFolderContents(base); - expect(files).toHaveLength(14); - expect(connection.copiedSql).toHaveLength(14); + expect(files).toHaveLength(15); + expect(connection.copiedSql).toHaveLength(15); assertPermissionDrift({ command: "inspect report", activeFlags: { remote: VALID_REF, "output-dir": true }, @@ -730,8 +730,8 @@ describe("inspect report through the real database-config resolver", () => { yield* stubManagementApi(requests, () => undefined); yield* inspectReport(flags({ outputDir: base })); const { files } = yield* dateFolderContents(base); - expect(files).toHaveLength(14); - expect(connection.copiedSql).toHaveLength(14); + expect(files).toHaveLength(15); + expect(connection.copiedSql).toHaveLength(15); expect(requests).toEqual([]); assertPermissionDrift({ command: "inspect report", diff --git a/apps/cli/src/commands/inspect/report/report.queries.ts b/apps/cli/src/commands/inspect/report/report.queries.ts index 6e508418c0..612bb5fd01 100644 --- a/apps/cli/src/commands/inspect/report/report.queries.ts +++ b/apps/cli/src/commands/inspect/report/report.queries.ts @@ -10,6 +10,7 @@ import { outliersSpec } from "../db/outliers/outliers.query.ts"; import { replicationSlotsSpec } from "../db/replication-slots/replication-slots.query.ts"; import { roleStatsSpec } from "../db/role-stats/role-stats.query.ts"; import { tableStatsSpec } from "../db/table-stats/table-stats.query.ts"; +import { toastSizesSpec } from "../db/toast-sizes/toast-sizes.query.ts"; import { trafficProfileSpec } from "../db/traffic-profile/traffic-profile.query.ts"; import { vacuumStatsSpec } from "../db/vacuum-stats/vacuum-stats.query.ts"; @@ -44,7 +45,7 @@ export interface ReportQuery { } /** - * The 14 report queries. Reuses the 13 `inspect db` specs' `.sql` verbatim + * The 15 report queries. Reuses the 14 `inspect db` specs' `.sql` verbatim * (byte-identical COPY input → byte-identical CSVs) plus the standalone * `unused_indexes` query. */ @@ -60,6 +61,7 @@ export const REPORT_QUERIES: ReadonlyArray = [ { fileName: "replication_slots", sql: replicationSlotsSpec.sql }, { fileName: "role_stats", sql: roleStatsSpec.sql }, { fileName: "table_stats", sql: tableStatsSpec.sql }, + { fileName: "toast_sizes", sql: toastSizesSpec.sql }, { fileName: "traffic_profile", sql: trafficProfileSpec.sql }, { fileName: "unused_indexes", sql: UNUSED_INDEXES_REPORT_SQL }, { fileName: "vacuum_stats", sql: vacuumStatsSpec.sql }, diff --git a/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts b/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts index d0d7fe6183..2c1c432861 100644 --- a/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts +++ b/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts @@ -38,7 +38,7 @@ describe("reportIgnoreSchemas", () => { }); describe("REPORT_QUERIES", () => { - it("has the 14 underscore CSV basenames", () => { + it("has the 15 underscore CSV basenames", () => { expect(REPORT_QUERIES.map((q) => q.fileName)).toEqual([ "bloat", "blocking", @@ -51,6 +51,7 @@ describe("REPORT_QUERIES", () => { "replication_slots", "role_stats", "table_stats", + "toast_sizes", "traffic_profile", "unused_indexes", "vacuum_stats", From ba71d228678b7498badb15cef05469bf93e8b52c Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 09:23:34 +0000 Subject: [PATCH 028/101] test(stack): name the failing psql call when Commands stderr is empty (#7457) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR Diagnostic only. The cause is still unknown, so the next failure will say which psql call lost its stderr and how that call exited. - `Commands.integration.test.ts › podman runs SQL from stdin, streams a dump, and returns client failure`: on Podman, psql exited non-zero but captured stderr was empty (`expected '' to contain 'missing_column'`). The vitest frame didn't show which of the two expected-failure calls fired, and stderr was shared across calls, so the report couldn't tell an attach that dropped output apart from a different exit path. Stdout and stderr are now reset before each expected psql failure, and both assertions carry the call label, the exit code and the captured stdout/stderr. ## ref spotted in: https://github.com/supabase/cli/actions/runs/41469215904/job/125358056261 (green rerun: https://github.com/supabase/cli/actions/runs/41469215904/job/125360621750) --- _Generated by [Claude Code](https://claude.ai/code/session_01MBEk1PhFmm5bGsyrcpQfsk)_ Co-authored-by: Claude --- .../stack/src/Commands.integration.test.ts | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/packages/stack/src/Commands.integration.test.ts b/packages/stack/src/Commands.integration.test.ts index 566f82d770..6adb9f54a6 100644 --- a/packages/stack/src/Commands.integration.test.ts +++ b/packages/stack/src/Commands.integration.test.ts @@ -159,14 +159,22 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(dump.jobId).not.toBe(sql.jobId); expect(yield* Ref.get(stdout)).toContain("COPY public.tool_story"); expect(yield* Ref.get(stdout)).toContain("streamed-row"); + const clientFailure = (label: string, exitCode: number) => + Effect.gen(function* () { + const [out, err] = [yield* Ref.get(stdout), yield* Ref.get(stderr)]; + return `${label} psql exited ${exitCode}\nstdout:\n${out}\nstderr:\n${err}`; + }); + yield* Ref.set(stdout, ""); + yield* Ref.set(stderr, ""); const failed = yield* runPostgres(runner, { postgres: postgres.psql({ pin: selection.database }), args: ["-X", "-v", "ON_ERROR_STOP=1", "-c", "SELECT missing_column FROM tool_story"], env, ...output, }); - expect(failed.exitCode).not.toBe(0); - expect(yield* Ref.get(stderr)).toContain("missing_column"); + const failedReport = yield* clientFailure("-c", failed.exitCode); + expect(failed.exitCode, failedReport).not.toBe(0); + expect(yield* Ref.get(stderr), failedReport).toContain("missing_column"); const rejected = yield* Effect.flip( runPostgres(runner, { postgres: postgres.psql({ pin: selection.database }), @@ -177,6 +185,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { }), ); expect(rejected.message).toContain("pgProve options require the pg_prove command"); + yield* Ref.set(stdout, ""); yield* Ref.set(stderr, ""); const early = yield* runPostgres(runner, { postgres: postgres.psql({ pin: selection.database }), @@ -189,8 +198,9 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { ), ...output, }); - expect(early.exitCode).not.toBe(0); - expect(yield* Ref.get(stderr)).toContain("missing_column"); + const earlyReport = yield* clientFailure("early-exit stdin", early.exitCode); + expect(early.exitCode, earlyReport).not.toBe(0); + expect(yield* Ref.get(stderr), earlyReport).toContain("missing_column"); const attached = yield* Deferred.make(); const job = yield* runPostgres(runner, { postgres: postgres.psql({ pin: selection.database }), From 93e8290a6bc2e1538e9fb82a28926878177e8a10 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 09:25:41 +0000 Subject: [PATCH 029/101] fix(stack): keep a concurrent traverse grant when restricting a directory (#7276) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR `restrictDirectoryToOwner` no longer chmods a directory that is already owner-only, so a concurrent opener can't strip an o+x grant with a stale mode. - `Ports.integration.test.ts › opening the port registry keeps the traverse bit a stepped-down database needs on its directory` failed with `expected 448 to be 449` (0700 instead of 0701). Every registry open does a stat followed by a chmod on `~/.supabase`. If another worker (here `Network.integration.test.ts`) read 0700 before the test set 0701, it wrote 0700 back afterwards. The same race can drop an o+x grant that a native start adds through `allowTraverse` on the state, cache or registry roots. The function now skips the chmod when the mode is already `0700 | (mode & o+x)`. An integration test injects the grant between the stat and the chmod to pin that behavior. ## ref spotted in: https://github.com/supabase/cli/actions/runs/39694591838/job/119548808927 --- _Generated by [Claude Code](https://claude.ai/code/session_011gUuCqGVVmA6V5MvJq7FgR)_ --------- Co-authored-by: Claude --- .../runtime/postgres-user.integration.test.ts | 53 ++++++++++++++++++- packages/stack/src/runtime/postgres-user.ts | 15 ++++-- 2 files changed, 64 insertions(+), 4 deletions(-) diff --git a/packages/stack/src/runtime/postgres-user.integration.test.ts b/packages/stack/src/runtime/postgres-user.integration.test.ts index bc6d75689c..6c62e468cd 100644 --- a/packages/stack/src/runtime/postgres-user.integration.test.ts +++ b/packages/stack/src/runtime/postgres-user.integration.test.ts @@ -1,6 +1,7 @@ import { NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Path } from "effect"; +import { Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; +import { systemError } from "effect/PlatformError"; // oxlint-disable-next-line effecttsgo/node-builtin-import -- the current process's own uid/gid make the chown calls below harmless no-ops. import { userInfo } from "node:os"; import { @@ -9,6 +10,7 @@ import { NativeRuntimeRootBase, nativeRuntimeRootPath, type PasswdEntry, + restrictDirectoryToOwner, } from "./postgres-user.ts"; /** @@ -141,3 +143,52 @@ describe("acquireNativeRuntimeRoot", () => { ).pipe(Effect.provide(NodeServices.layer)), ); }); + +// Windows has no POSIX mode bits. +describe.skipIf(process.platform === "win32")("restrictDirectoryToOwner", () => { + it.live("keeps a traverse grant another process adds after the mode was read", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "restrict-owner-race-" }); + yield* fs.chmod(directory, 0o700); + const grantAfterStat: FileSystem.FileSystem = { + ...fs, + stat: (target) => fs.stat(target).pipe(Effect.tap(() => fs.chmod(directory, 0o701))), + }; + + yield* restrictDirectoryToOwner(grantAfterStat, directory); + + expect((yield* fs.stat(directory)).mode & 0o777).toBe(0o701); + }), + ).pipe(Effect.provide(NodeServices.layer)), + ); + + it.live("still rejects a restricted directory another user owns", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "restrict-owner-foreign-" }); + yield* fs.chmod(directory, 0o701); + const ownedByAnotherUser: FileSystem.FileSystem = { + ...fs, + stat: (target) => + fs.stat(target).pipe(Effect.map((info) => ({ ...info, uid: Option.some(-1) }))), + chmod: () => + Effect.fail( + systemError({ + _tag: "PermissionDenied", + module: "FileSystem", + method: "chmod", + description: "not the owner", + }), + ), + }; + + const exit = yield* Effect.exit(restrictDirectoryToOwner(ownedByAnotherUser, directory)); + + expect(Exit.isFailure(exit)).toBe(true); + }), + ).pipe(Effect.provide(NodeServices.layer)), + ); +}); diff --git a/packages/stack/src/runtime/postgres-user.ts b/packages/stack/src/runtime/postgres-user.ts index ef8388dab5..7c57b81035 100644 --- a/packages/stack/src/runtime/postgres-user.ts +++ b/packages/stack/src/runtime/postgres-user.ts @@ -278,11 +278,20 @@ export const removeNativeSocketDirectory = Effect.fn("NativePostgresUser.removeS /** * Restricts a directory to its owner but keeps an existing traverse-only grant, because a * stepped-down PostgreSQL resolves bundle and data paths through the cache and state roots while it runs. + * An already-restricted directory is left unchanged, so a concurrent stat-then-chmod cannot write + * back a mode that predates another process's traverse grant. */ export const restrictDirectoryToOwner = (fs: FileSystem.FileSystem, directory: string) => - fs - .stat(directory) - .pipe(Effect.flatMap(({ mode }) => fs.chmod(directory, 0o700 | (mode & 0o001)))); + fs.stat(directory).pipe( + Effect.flatMap(({ mode, uid }) => { + const restricted = 0o700 | (mode & 0o001); + // chmod fails on another user's directory, so only an owned directory may skip it. + const owned = Option.getOrUndefined(uid) === process.getuid?.(); + return owned && (mode & 0o7777) === restricted + ? Effect.void + : fs.chmod(directory, restricted); + }), + ); const allowTraverse = Effect.fn("NativePostgresUser.allowTraverse")(function* ( user: PasswdEntry, From 46b5465f103b05f85cd67328a264fbfdd8467f97 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 09:30:51 +0000 Subject: [PATCH 030/101] ci(repo): post the sync resolver comment when one artifact is downloaded (#7295) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When the conflict resolver declines a develop-into-next merge, only the `sync-resolution` artifact exists. `download-artifact` extracts a lone matching artifact straight into `sync-artifacts/` instead of `sync-artifacts/sync-resolution/`, so the publish step found no `comment.md`. Because the resolve job succeeded, the failure fallback comment was skipped too. The job passed and the conflicting PR got no explanation, as seen on supabase/cli#7294. The publish step now falls back to `sync-artifacts/comment.md` when no per-artifact folder has one. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 --- .github/workflows/sync-branches.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/sync-branches.yml b/.github/workflows/sync-branches.yml index 5ab7797833..176beef600 100644 --- a/.github/workflows/sync-branches.yml +++ b/.github/workflows/sync-branches.yml @@ -364,6 +364,10 @@ jobs: break fi done + # download-artifact extracts a lone matching artifact into the path itself, not a subdirectory. + if [ -z "$stage" ] && [ -f "$ARTIFACTS/comment.md" ]; then + stage="$ARTIFACTS" + fi if [ "$RESOLVED" = true ]; then export REPO_DIR="$GITHUB_WORKSPACE" OUTPUT_DIR="$stage" HEAD_SHA if ! { From 6763aa45044b17c72f54e9e1bbb294676b6d9266 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 09:32:14 +0000 Subject: [PATCH 031/101] fix(stack): report the helper container state when its readiness times out (#7319) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR A database helper whose readiness times out now reports the container's state and log tail, so the next stall shows why it stalled. - `DockerDatabaseStorage.integration.test.ts › labels its volume with the configured test run`: `start --attach` printed no ready line within 30s, and stderr was empty. The error said only `Database helper did not become ready within 30 seconds`. That cannot tell apart a helper that never started, one that exited, and one that printed its ready line that the attach never relayed. On a readiness timeout, the helper start now reads the container's status, exit code and engine error (`inspect`) and its last 20 log lines, stdout and stderr (`logs`), each with its own 5s bound. It reads them before killing the attach, because `--rm` removes the helper once the attach ends. Both go into the terminal error. The span gets only structured attributes: each probe's outcome (`ok` / `timeout` / `error`), plus the container status and numeric exit code. The existing helper-timeout test now asserts on this output, and its fake fails `inspect` after the attach is killed. ## ref spotted in: - https://github.com/supabase/cli/actions/runs/39924018874/job/120315879910 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01EMySvvbM4B1zdmhw3pp1wA --------- Co-authored-by: Claude --- .../DockerDatabaseStorage.integration.test.ts | 34 ++++++++-- .../src/storage/DockerDatabaseStorage.ts | 67 ++++++++++++++++++- 2 files changed, 91 insertions(+), 10 deletions(-) diff --git a/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts b/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts index 3e7f050276..0e9417fd1d 100644 --- a/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts +++ b/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts @@ -426,7 +426,7 @@ describe.runIf(testEngine === "docker")( ).pipe(Effect.provide(NodeServices.layer)), ); - it.effect("reports a helper readiness timeout alongside Docker warnings", () => + it.effect("reports a helper readiness timeout with Docker warnings and the helper state", () => Effect.scoped( Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -435,6 +435,7 @@ describe.runIf(testEngine === "docker")( const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-helper-timeout-" }); const waiting = yield* Deferred.make(); const commands: string[][] = []; + let attachKilled = false; const warning = "WARNING: The requested image's platform does not match the host platform"; const spawner = ChildProcessSpawner.make((command) => @@ -444,22 +445,36 @@ describe.runIf(testEngine === "docker")( commands.push([...command.args]); const creating = command.args[0] === "create"; const attaching = command.args[0] === "start"; + const encode = (text: string) => Stream.succeed(new TextEncoder().encode(text)); + // `--rm` removes the helper once its attach is killed, so later inspection fails. + const removed = command.args[0] === "inspect" && attachKilled; return ChildProcessSpawner.makeHandle({ pid: ChildProcessSpawner.ProcessId(0), - exitCode: Effect.succeed(ChildProcessSpawner.ExitCode(0)), + exitCode: Effect.succeed(ChildProcessSpawner.ExitCode(removed ? 1 : 0)), isRunning: Effect.succeed(false), - kill: () => Effect.void, + kill: () => + Effect.sync(() => { + if (attaching) attachKilled = true; + }), stdin: Sink.drain, stdout: attaching ? Stream.fromEffect(Deferred.succeed(waiting, undefined)).pipe( Stream.drain, Stream.concat(Stream.never), ) - : Stream.empty, + : command.args[0] === "inspect" && !removed + ? encode("running\t0\t\n") + : command.args[0] === "logs" + ? encode("supabase-helper-ready\n") + : Stream.empty, // Docker prints the platform warning while creating the container, before start. stderr: creating - ? Stream.succeed(new TextEncoder().encode(`${warning}\n`)) - : Stream.empty, + ? encode(`${warning}\n`) + : removed + ? encode("Error: No such container\n") + : command.args[0] === "logs" + ? encode("helper stderr line\n") + : Stream.empty, all: Stream.empty, getInputFd: () => Sink.drain, getOutputFd: () => Stream.empty, @@ -495,7 +510,12 @@ describe.runIf(testEngine === "docker")( yield* TestClock.adjust("30 seconds"); const failure = yield* Fiber.join(operation); expect(failure.message).toBe( - `Database helper did not become ready within 30 seconds: ${warning}`, + [ + `Database helper did not become ready within 30 seconds: ${warning}`, + "helper state: running (exit 0)", + "helper logs: supabase-helper-ready", + "helper stderr line", + ].join("\n"), ); const create = commands.find((args) => args[0] === "create"); const name = create?.[create.indexOf("--name") + 1]; diff --git a/packages/stack/src/storage/DockerDatabaseStorage.ts b/packages/stack/src/storage/DockerDatabaseStorage.ts index c5a6c145fe..1f13c4f9d5 100644 --- a/packages/stack/src/storage/DockerDatabaseStorage.ts +++ b/packages/stack/src/storage/DockerDatabaseStorage.ts @@ -375,7 +375,10 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") ); const engineCommand = Effect.fn("DockerDatabaseStorage.engineCommand")( - (args: ReadonlyArray): Effect.Effect => + ( + args: ReadonlyArray, + output: "stdout" | "all" = "stdout", + ): Effect.Effect => Effect.scoped( Effect.gen(function* () { yield* Effect.annotateCurrentSpan({ @@ -415,7 +418,9 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") "engine", stderr.trim() || `Container engine exited with ${code}`, ); - return stdout.trim(); + return output === "all" + ? [stdout.trim(), stderr.trim()].filter((part) => part !== "").join("\n") + : stdout.trim(); }), ).pipe(Effect.mapError((cause) => errorFor("engine", cause))), ); @@ -638,6 +643,57 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") ) .pipe(Effect.mapError((cause) => errorFor("helper", cause))); const stderr = yield* stderrTail(child); + const diagnosticProbe = (command: Effect.Effect) => + command.pipe( + Effect.map((output) => ({ outcome: "ok" as const, output })), + Effect.timeoutOrElse({ + duration: "5 seconds", + orElse: () => + Effect.succeed({ + outcome: "timeout" as const, + output: "unavailable: timed out after 5 seconds", + }), + }), + Effect.catch((cause) => + Effect.succeed({ + outcome: "error" as const, + output: `unavailable: ${cause.message}`, + }), + ), + ); + const stalledHelperState = Effect.fn("DockerDatabaseStorage.stalledHelperState")( + function* () { + const [inspected, logs] = yield* Effect.all( + [ + diagnosticProbe( + engineCommand([ + "inspect", + "--format", + "{{.State.Status}}\t{{.State.ExitCode}}\t{{.State.Error}}", + name, + ]), + ), + // `logs` relays the helper's stderr on the engine's own stderr. + diagnosticProbe(engineCommand(["logs", "--tail", "20", name], "all")), + ], + { concurrency: "unbounded" }, + ); + yield* Effect.annotateCurrentSpan({ + "helper.inspect.outcome": inspected.outcome, + "helper.logs.outcome": logs.outcome, + }); + let state = inspected.output; + if (inspected.outcome === "ok") { + const [status = "", exitCode = "", engineError = ""] = inspected.output.split("\t"); + yield* Effect.annotateCurrentSpan({ + "helper.status": status, + "helper.exit_code": Number(exitCode), + }); + state = `${status} (exit ${exitCode})${engineError === "" ? "" : `: ${engineError}`}`; + } + return `helper state: ${state}\nhelper logs: ${logs.output || "(empty)"}`; + }, + ); return yield* child.stdout.pipe( Stream.decodeText, Stream.splitLines, @@ -650,6 +706,11 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") exit.value.value === "supabase-helper-ready" ? Effect.succeed(name) : Effect.gen(function* () { + const timedOut = + Exit.isFailure(exit) && + Option.exists(Cause.findErrorOption(exit.cause), Cause.isTimeoutError); + // Inspect before the kill: `--rm` removes the helper once its attach ends. + const state = timedOut ? yield* stalledHelperState() : ""; yield* child .kill({ killSignal: "SIGTERM", forceKillAfter: "5 seconds" }) .pipe(Effect.ignore); @@ -659,7 +720,7 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") ); // Docker prints warnings such as a platform mismatch on `create`, so keep that // output alongside what `start` wrote. - const tail = [createTail, diagnostic.trim()] + const tail = [createTail, diagnostic.trim(), state] .filter((part) => part !== "") .join("\n"); const reason = Exit.isFailure(exit) From a9855d6202b57c10163d99d8f7357d8cf9fdf284 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Tue, 3 Nov 2026 09:37:31 +0000 Subject: [PATCH 032/101] ci(repo): flag breaking changes in AI review (CLI-2904) (#7451) ## TL;DR flags changes that break released behavior in ai review and asks for a `!` title, plus `next` from `develop` ## ref - closes: CLI-2904 --- .github/ai-review/README.md | 6 ++++++ .github/ai-review/adjudicate-prompt.md | 10 ++++++++++ .github/ai-review/claude-review-prompt.md | 13 +++++++++++++ 3 files changed, 29 insertions(+) diff --git a/.github/ai-review/README.md b/.github/ai-review/README.md index 300372d67f..fb408be8a8 100644 --- a/.github/ai-review/README.md +++ b/.github/ai-review/README.md @@ -74,6 +74,12 @@ shown as `--remerge-diff`. Both sides were reviewed in their own PRs, so the prompts limit findings to what the resolution introduced; `pr.diff` stays the anchor for inline comments. +### Breaking changes + +Only the Claude prompt asks for `breaking-change` findings, because only its job +has `pr.json` with the title and branches. They post as `major` and ask for a +`!` title, plus a retarget to `next` on `develop`. + ## Once-per-PR semantics and manual re-runs New commits never re-trigger a review — `resolve.ts`'s dedup guard skips a diff --git a/.github/ai-review/adjudicate-prompt.md b/.github/ai-review/adjudicate-prompt.md index ef6951d1b0..f37a3e0085 100644 --- a/.github/ai-review/adjudicate-prompt.md +++ b/.github/ai-review/adjudicate-prompt.md @@ -71,6 +71,16 @@ when that file shows work on `next` the PR would fundamentally conflict with. If the file is absent, refute it. Keep the category `next-impact` and a severity no higher than `minor`; these findings are advisory and never block the PR. +### Breaking-change findings + +Verify each `breaking-change` finding against the behavior the PR changes: +refute it only when no released, non-experimental use fails or changes meaning, +when the change only rejects a value that never had a meaning, or when the +change matches the old `SIDE_EFFECTS.md`. The PR's intent, tests and +`SIDE_EFFECTS.md` edits are not counter-evidence. Keep the category +`breaking-change`, severity `major` and the request ending its `claim`, even +when merged with a Codex finding. + ### Merge into one deduplicated list - When a Claude finding and a Codex finding concern the same file/line/ diff --git a/.github/ai-review/claude-review-prompt.md b/.github/ai-review/claude-review-prompt.md index 79273c84e7..5ce275addf 100644 --- a/.github/ai-review/claude-review-prompt.md +++ b/.github/ai-review/claude-review-prompt.md @@ -76,6 +76,19 @@ higher than `minor`. These findings are advisory and never block the PR. Do not flag ordinary merge conflicts or style differences, and report nothing when the file does not exist. +## Breaking changes + +Only when `pr.json` has `baseRefName` `develop` or `next`, a `headRefName` other +than `next` or `sync/*`, and no `!` before the title's colon: report any change +that makes existing use of a released command, flag, config key, env var or exit +code fail or change meaning, for example removing a fallback so a command now +errors, even when that is the fix the PR set out to make. Use category +`breaking-change`, with severity `major`, and end the `claim` by asking for a +`type(scope)!:` title, a `BREAKING CHANGE:` line in the PR description and, on +`develop`, a retarget to `next`. Do not report internal code, tests, CI, +additive changes, experimental commands, newly rejected values that never had a +meaning, or fixes that match the old `SIDE_EFFECTS.md`. + ## Output Your final response must be ONLY the JSON object described by the provided From 4287d929066b9d5e53a6577aa0800490248559ea Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 09:46:23 +0000 Subject: [PATCH 033/101] fix(stack): name the stalled engine command in timeout errors (#7407) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR Engine command timeouts now name the stalled subcommand and the engine's latest stderr, so the next occurrence of this flake shows which call hung. - `Owner.logs.integration.test.ts › persists container output`: `startService` for Mailpit failed with `engine-timeout` and only `Operation timed out after '30s'`. The error didn't say which engine subcommand ran past the 30s default, so the cause couldn't be found from the logs. `Container.command` now fails with `Engine command '' did not finish within ; stderr: `. It keeps the `engine-timeout` / `image-pull` kinds. A regression test drives a stalled `start` with a controlled clock. ## ref spotted in: https://github.com/supabase/cli/actions/runs/40890624066/job/123506114202 (green rerun: https://github.com/supabase/cli/actions/runs/40890624066/job/123509245894) --- _Generated by [Claude Code](https://claude.ai/code/session_01MBEk1LaE2jUMLrpQ1WCiVy)_ --------- Co-authored-by: Claude Co-authored-by: Vaibhav <117663341+7ttp@users.noreply.github.com> --- .../src/runtime/Container.integration.test.ts | 61 +++++++++++++++++++ packages/stack/src/runtime/Container.ts | 46 +++++++++++--- 2 files changed, 99 insertions(+), 8 deletions(-) diff --git a/packages/stack/src/runtime/Container.integration.test.ts b/packages/stack/src/runtime/Container.integration.test.ts index 3cd82ba7b4..02cf48fa10 100644 --- a/packages/stack/src/runtime/Container.integration.test.ts +++ b/packages/stack/src/runtime/Container.integration.test.ts @@ -1549,6 +1549,34 @@ describe("container process adapter", { timeout: 120_000 }, () => { }).pipe(Effect.provide(NodeServices.layer)), ); + it.effect( + "names the stalled engine subcommand and its stderr when an engine command times out", + () => + Effect.gen(function* () { + const engine = yield* makeStalledStartSpawner("Error: waiting for network sandbox"); + const runtime = yield* makeContainerRuntime({ + target: containerTarget, + root: ".", + hostGateway: sharedHostGateway, + }).pipe(Effect.provide(engine.layer)); + const launch = yield* Effect.scoped( + runtime.launch({ image, stackId: "s".repeat(64), instanceId: "stalled-start", env: {} }), + ).pipe(Effect.provide(engine.layer), Effect.exit, Effect.forkChild); + yield* Deferred.await(engine.stderrRead); + yield* TestClock.adjust("30 seconds"); + const result = yield* Fiber.join(launch); + + const failure = Exit.isFailure(result) + ? Option.getOrUndefined(Cause.findErrorOption(result.cause)) + : undefined; + expect(failure instanceof ContainerLaunchError).toBe(true); + expect(failureKind(failure)).toBe("engine-timeout"); + expect(failure instanceof ContainerLaunchError ? failure.failure.message : undefined).toBe( + "Engine command 'start' did not finish within 30s; stderr: Error: waiting for network sandbox", + ); + }).pipe(Effect.provide(NodeServices.layer)), + ); + it.effect("removes the container a hung docker create may still create when interrupted", () => Effect.gen(function* () { const engine = yield* makeHangingCreateSpawner(); @@ -2202,6 +2230,39 @@ const successfulHandle = (stdout = "") => unref: Effect.succeed(Effect.void), }); +/** Answers every engine command except `start`, which writes `stderr` and never exits. */ +const makeStalledStartSpawner = Effect.fn("ContainerTest.stalledStartSpawner")(function* ( + stderr: string, +) { + const stderrRead = yield* Deferred.make(); + const spawner = ChildProcessSpawner.make((command) => { + if (!ChildProcess.isStandardCommand(command)) return Effect.die("unexpected piped command"); + if (command.args[0] === "info") return Effect.succeed(successfulHandle("fake-daemon-id")); + if (command.args[0] !== "start") return Effect.succeed(successfulHandle()); + // The second pull only happens once the reader has consumed the stderr line. + const stalledStderr = Stream.succeed(new TextEncoder().encode(stderr)).pipe( + Stream.concat(Stream.fromEffectDrain(Deferred.succeed(stderrRead, undefined))), + Stream.concat(Stream.never), + ); + return Effect.succeed( + ChildProcessSpawner.makeHandle({ + pid: ChildProcessSpawner.ProcessId(0), + exitCode: Effect.never, + isRunning: Effect.succeed(true), + kill: () => Effect.void, + stdin: Sink.drain, + stdout: Stream.empty, + stderr: stalledStderr, + all: Stream.empty, + getInputFd: () => Sink.drain, + getOutputFd: () => Stream.empty, + unref: Effect.succeed(Effect.void), + }), + ); + }); + return { stderrRead, layer: Layer.succeed(ChildProcessSpawner.ChildProcessSpawner, spawner) }; +}); + const makeHangingCreateSpawner = Effect.fn("ContainerTest.hangingCreateSpawner")(function* () { const commands: string[][] = []; const createStarted = yield* Deferred.make(); diff --git a/packages/stack/src/runtime/Container.ts b/packages/stack/src/runtime/Container.ts index 7e64d2ae34..77bbac5834 100644 --- a/packages/stack/src/runtime/Container.ts +++ b/packages/stack/src/runtime/Container.ts @@ -5,7 +5,7 @@ import { Crypto, Data, Deferred, - type Duration, + Duration, Effect, Exit, FileSystem, @@ -134,6 +134,18 @@ const engineKind = ( ? "port-allocation" : undefined; +/** Names the stalled engine subcommand and its latest stderr, never the full argv. */ +const commandTimedOut = ( + args: ReadonlyArray, + timeout: Duration.Input, + stderr: string, +): Cause.TimeoutError => { + const tail = stderr.trim().slice(-2000); + return new Cause.TimeoutError( + `Engine command '${args[0] ?? "command"}' did not finish within ${Duration.format(Duration.fromInputUnsafe(timeout))}${tail.length > 0 ? `; stderr: ${tail}` : ""}`, + ); +}; + const errorFor = (operation: string, cause: unknown): ContainerError => { const message = cause instanceof Error ? cause.message : String(cause); const kind = engineKind(operation, cause, message); @@ -527,19 +539,25 @@ export const makeContainerRuntime = (options: { "process.arg_count": args.length, "container.command": args[0] ?? "", }); + const stderrSoFar = yield* Ref.make(""); return yield* Effect.scoped( Effect.gen(function* () { const child = yield* spawner.spawn(command(args)); - const tail = (stream: typeof child.stdout) => + const tail = (stream: typeof child.stdout, progress?: Ref.Ref) => stream.pipe( Stream.decodeText, - Stream.runFold( + Stream.runFoldEffect( () => "", - (text, chunk) => (text + chunk).slice(-65536), + (text, chunk) => { + const next = (text + chunk).slice(-65536); + return progress === undefined + ? Effect.succeed(next) + : Ref.set(progress, next).pipe(Effect.as(next)); + }, ), ); const [stdout, stderr, code] = yield* Effect.all( - [tail(child.stdout), tail(child.stderr), child.exitCode], + [tail(child.stdout), tail(child.stderr, stderrSoFar), child.exitCode], { concurrency: "unbounded" }, ); yield* Effect.annotateCurrentSpan("process.exit_code", Number(code)); @@ -551,10 +569,22 @@ export const makeContainerRuntime = (options: { return stdout.trim(); }), ).pipe( - (effect) => - commandOptions.timeout === undefined + (effect) => { + const timeout = commandOptions.timeout; + return timeout === undefined ? effect - : effect.pipe(Effect.timeout(commandOptions.timeout)), + : effect.pipe( + Effect.timeoutOrElse({ + duration: timeout, + orElse: () => + Ref.get(stderrSoFar).pipe( + Effect.flatMap((stderr) => + Effect.fail(commandTimedOut(args, timeout, stderr)), + ), + ), + }), + ); + }, Effect.mapError((cause) => errorFor(args[0] ?? "command", cause)), ); }); From cf9c0e94e13d0ed9b2f30b314639437404bdfcb9 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 09:54:39 +0000 Subject: [PATCH 034/101] fix(stack): keep port registry reads open while a peer commits (#7342) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR The per-user port registry (`~/.supabase/ports.sqlite`) now opens in WAL mode, so a peer committing reservations no longer locks other connections out of opening or reading it. - `Network.integration.test.ts › retains dedicated assignments across network reopen` failed on Windows with `database is locked` while opening the registry. Under the rollback journal, a peer that commits back to back holds the lock nearly all the time, and a new connection's schema read keeps losing the race until the 2s busy timeout runs out. Parallel test files that scan for ports produce exactly that write pattern. In WAL mode readers never wait on a writer. Writers still serialize through `BEGIN IMMEDIATE`, as before. A new integration test opens and reads the registry while a peer holds its write lock: it fails on the old code with the same `database is locked` error and passes with this change. - Tradeoff: WAL needs shared memory, so a home directory on a network filesystem is not supported for this file. ## ref spotted in: https://github.com/supabase/cli/actions/runs/40438309049/job/121972299083 --- _Generated by [Claude Code](https://claude.ai/code/session_01MBEk1M3LvdWBHSeFmCJaGq)_ --------- Co-authored-by: Claude --- .../stack/destroy/SIDE_EFFECTS.md | 3 +- .../experimental/stack/start/SIDE_EFFECTS.md | 4 +- .../experimental/stack/stop/SIDE_EFFECTS.md | 3 +- packages/stack/README.md | 5 +- .../PortReservations.integration.test.ts | 31 +++++++ .../stack/src/namespace/PortReservations.ts | 84 ++++++++++++++----- 6 files changed, 104 insertions(+), 26 deletions(-) create mode 100644 packages/stack/src/namespace/PortReservations.integration.test.ts diff --git a/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md index 16b76c3d49..7915d66d55 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md @@ -75,7 +75,8 @@ routing/settings can read project configuration and profiles. Destruction releases this stack's rows in the per-user port registry at `/.supabase/ports.sqlite` (independent of `SUPABASE_HOME`, with no -override) only once cleanup is confirmed complete; a destroy that fails +override; WAL mode, with `ports.sqlite-wal` and `ports.sqlite-shm` alongside +while open) only once cleanup is confirmed complete; a destroy that fails keeps the stack's automatic reservations, and those ports remain unavailable to other stacks until a later destroy succeeds; configured ports are released with their listeners. A reservation whose owning diff --git a/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md index e77c42a8da..c5004d88f0 100644 --- a/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md @@ -64,7 +64,9 @@ secret key (`sb_secret_…`) or a JWT, values that themselves carry such a pair URL with a token), and URL userinfo. Public ports are reserved in one SQLite registry per OS user at `/.supabase/ports.sqlite`, -which `$SUPABASE_HOME` does not affect and which has no override. A stopped stack's automatic +which `$SUPABASE_HOME` does not affect and which has no override. The registry uses SQLite's WAL +mode, so `/.supabase` must be on a local filesystem, and `ports.sqlite-wal` and +`ports.sqlite-shm` sit next to it while a connection is open. A stopped stack's automatic ports stay reserved across every other stack's starts, including in a different state root, and starting it again reuses the same ports. A configured port is reserved only while its listener is open, so a stopped stack never blocks another project's configured port; a configured port takes diff --git a/apps/cli/src/commands/experimental/stack/stop/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/stop/SIDE_EFFECTS.md index 36eb5d7241..2ca3a0bcdb 100644 --- a/apps/cli/src/commands/experimental/stack/stop/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/experimental/stack/stop/SIDE_EFFECTS.md @@ -42,7 +42,8 @@ settings may read project configuration and the selected profile. Communicates only with local owner endpoints; no hosted API requests. Shutdown updates owned runtime resources but preserves persistent instance definitions, data and automatic port reservations in the per-user registry at `/.supabase/ports.sqlite` -(unaffected by `SUPABASE_HOME`); configured ports are released with their listeners. +(unaffected by `SUPABASE_HOME`; WAL mode, with `ports.sqlite-wal` and `ports.sqlite-shm` +alongside while open); configured ports are released with their listeners. Killing the owner process preserves every reservation, but another stack may take over a configured port while no owner runs. No caller-owned upload files are removed. diff --git a/packages/stack/README.md b/packages/stack/README.md index 21036cde44..f43b01bac5 100644 --- a/packages/stack/README.md +++ b/packages/stack/README.md @@ -138,7 +138,10 @@ stacks still relies only on the kernel refusing a second bind. A native backend' ports are reserved directly from 10000–19999, disjoint from the public auto range and below the OS ephemeral range on every supported platform; this is not configurable, and a public port pinned inside it is rejected. A host whose ephemeral range has been widened to overlap 10000–19999 -reintroduces the ephemeral-port race this reservation exists to avoid. +reintroduces the ephemeral-port race this reservation exists to avoid. The registry runs in SQLite's +WAL mode, so `/.supabase` must be on a local filesystem: WAL's shared memory does not +work over network mounts, and the registry location cannot be moved. While a connection is open, +`ports.sqlite-wal` and `ports.sqlite-shm` sit next to `ports.sqlite`. Native PostgreSQL receives the owner's `SSL_CERT_FILE` and `SSL_CERT_DIR` when they are non-empty. When `SSL_CERT_FILE` is unset or empty, it receives the first of `/etc/ssl/certs/ca-certificates.crt`, `/etc/pki/tls/certs/ca-bundle.crt`, `/etc/ssl/ca-bundle.pem`, and `/etc/ssl/cert.pem` that exists, if any. The `http` and `pg_net` extensions verify HTTPS certificates against these. On macOS, `/etc/ssl/cert.pem` does not include roots added to the Keychain; to trust those, set `SSL_CERT_FILE` to a bundle that also holds the public roots, such as a copy of `/etc/ssl/cert.pem` with those roots appended. When PostgreSQL runs as a separate user, that user must be able to read the owner's `SSL_CERT_FILE` and `SSL_CERT_DIR` paths, which are not chowned to it. The detached owner inherits the environment of the client that starts it and keeps it until it exits, through restarts and repeated starts, so set these before the owner starts, and stop the stack before starting it again after changing them. diff --git a/packages/stack/src/namespace/PortReservations.integration.test.ts b/packages/stack/src/namespace/PortReservations.integration.test.ts new file mode 100644 index 0000000000..94741482c1 --- /dev/null +++ b/packages/stack/src/namespace/PortReservations.integration.test.ts @@ -0,0 +1,31 @@ +import { NodeServices } from "@effect/platform-node"; +import { expect, it } from "@effect/vitest"; +import { Context, Effect, FileSystem, Layer, Path } from "effect"; +import * as PortReservations from "./PortReservations.ts"; +import { openDatabase } from "./drivers/Sqlite.ts"; + +const openRegistry = (directory: string) => + Layer.build(PortReservations.layerAt(directory)).pipe( + Effect.map((context) => Context.get(context, PortReservations.Service)), + ); + +it.live("opens and reads the registry while a peer connection holds its write lock", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "port-reservations-" }); + const registryFile = path.join(directory, "ports.sqlite"); + const first = yield* openRegistry(directory); + yield* first.reserve("/peer", "peer", "api", 54321); + + yield* openDatabase(registryFile, (peer) => { + peer.exec("PRAGMA busy_timeout = 2000"); + peer.exec("BEGIN EXCLUSIVE"); + }); + const second = yield* openRegistry(directory); + + expect(yield* second.find("/peer", "peer", "api")).toBe(54321); + }), + ).pipe(Effect.provide(NodeServices.layer)), +); diff --git a/packages/stack/src/namespace/PortReservations.ts b/packages/stack/src/namespace/PortReservations.ts index 3dff821b7d..93726d3e90 100644 --- a/packages/stack/src/namespace/PortReservations.ts +++ b/packages/stack/src/namespace/PortReservations.ts @@ -1,11 +1,20 @@ import { userInfo } from "node:os"; -import type { DatabaseSync } from "node:sqlite"; import { NodeServices } from "@effect/platform-node"; -import { Context, Effect, FileSystem, Layer, Path, Predicate, Scope, Stream } from "effect"; +import { + Context, + Effect, + FileSystem, + Layer, + Path, + Predicate, + Schedule, + Scope, + Stream, +} from "effect"; import { ChildProcess } from "effect/process"; import { namespaceError, type NamespaceError } from "./Capabilities.ts"; import { restrictDirectoryToOwner } from "../runtime/postgres-user.ts"; -import { errcode, openDatabase } from "./drivers/Sqlite.ts"; +import { errcode, isBusy, openDatabase } from "./drivers/Sqlite.ts"; import { lstatPath } from "./drivers/FileSystem.ts"; /** Identifies the stack that owns a port reservation. */ @@ -87,15 +96,39 @@ const isHolderRow = (value: unknown): value is HolderRow => Predicate.hasProperty(value, "stack_id") && typeof value.stack_id === "string"; -const connectionFor = (file: string): Effect.Effect => - openDatabase(file, (connection) => { - // Short and bounded: real contention between two of this user's processes is brief, and - // a blocking wait here is simpler than an async retry loop for a database this small. Set - // before the schema statement, so even first-time table creation waits out a racing peer - // instead of surfacing a spurious busy error. - connection.exec("PRAGMA busy_timeout = 2000"); - connection.exec(schema); +const connectionFor = Effect.fn("PortReservations.connectionFor")(function* (file: string) { + // Short and bounded: real contention between two of this user's processes is brief, and + // a blocking wait here is simpler than an async retry loop for a database this small. Set + // before the schema statement, so even first-time table creation waits out a racing peer + // instead of surfacing a spurious busy error. + const connection = yield* openDatabase(file, (connection) => + connection.exec("PRAGMA busy_timeout = 2000"), + ); + // WAL lets reads and opens proceed while a peer commits; under the rollback journal a peer + // that commits back to back can lock new readers out past the busy timeout. SQLite skips the + // busy timeout while a peer switches the file into WAL, so that one statement waits here. + let attempts = 0; + yield* Effect.try({ + try: () => { + attempts++; + connection.exec("PRAGMA journal_mode = WAL"); + }, + catch: (cause) => namespaceError("open", cause), + }).pipe( + Effect.retry({ + schedule: Schedule.spaced("10 millis").pipe(Schedule.upTo({ duration: "2 seconds" })), + while: isBusy, + }), + Effect.ensuring( + Effect.suspend(() => Effect.annotateCurrentSpan("sqlite.journal_mode.attempts", attempts)), + ), + ); + yield* Effect.try({ + try: () => connection.exec(schema), + catch: (cause) => namespaceError("open", cause), }); + return connection; +}); /** * Confirms `directory` (created 0700 if missing) is a real directory this uid owns, with the @@ -200,18 +233,11 @@ const resolvedHome: Effect.Effect = Effect.runSync( Effect.cached(resolvePasswdHome().pipe(Effect.provide(NodeServices.layer))), ); -/** - * One SQLite database per OS user at `/.supabase/ports.sqlite`, the only authority on - * which stack owns a public port across every state root on this machine. - */ -const make = Effect.fn("PortReservations.make")(function* (): Effect.fn.Return< - Interface, - NamespaceError, - FileSystem.FileSystem | Path.Path | Scope.Scope -> { +const make = Effect.fn("PortReservations.make")(function* ( + directory: string, +): Effect.fn.Return { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const directory = path.join(yield* resolvedHome, ".supabase"); yield* securePrivateDirectory(fs, directory); const connection = yield* connectionFor(path.join(directory, "ports.sqlite")); @@ -358,4 +384,18 @@ const make = Effect.fn("PortReservations.make")(function* (): Effect.fn.Return< return { find, reserve, reclaim, release, releaseStack }; }); -export const layer = Layer.effect(Service, make().pipe(Effect.map(Service.of))); +/** + * One SQLite database per OS user at `/.supabase/ports.sqlite`, the only authority on + * which stack owns a public port across every state root on this machine. + */ +export const layer = Layer.effect( + Service, + Effect.gen(function* () { + const path = yield* Path.Path; + return Service.of(yield* make(path.join(yield* resolvedHome, ".supabase"))); + }), +); + +/** Opens the registry in `directory` instead of the account's home, for isolated tests. */ +export const layerAt = (directory: string) => + Layer.effect(Service, make(directory).pipe(Effect.map(Service.of))); From 6840b8a77b5890affe1eb048edf6edb2c64041a8 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Tue, 3 Nov 2026 10:02:28 +0000 Subject: [PATCH 035/101] test(cli): use new api keys in live e2e (#7459) ## TL;DR fixes live e2e setup on projects with legacy api keys disabled by picking the publishable and secret keys the same way the cli does ## ref - spotted in the last 9 runs: https://github.com/supabase/cli/actions/workflows/live-e2e.yml - fixed now (tested): https://github.com/supabase/cli/actions/runs/41609496013 --- .../helpers/live-project.integration.test.ts | 109 +++++++++++++++++- apps/cli/tests/helpers/live-project.ts | 17 +-- 2 files changed, 114 insertions(+), 12 deletions(-) diff --git a/apps/cli/tests/helpers/live-project.integration.test.ts b/apps/cli/tests/helpers/live-project.integration.test.ts index 3d99a9c9fe..ebe30d76a9 100644 --- a/apps/cli/tests/helpers/live-project.integration.test.ts +++ b/apps/cli/tests/helpers/live-project.integration.test.ts @@ -7,7 +7,7 @@ import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { afterEach, describe, expect, it, vi } from "vitest"; import { cliConfigProviderLayer } from "../../src/shared/config/cli-config-provider.layer.ts"; -import { provisionLiveEnvironment } from "./live-project.ts"; +import { cleanupLiveEnvironment, provisionLiveEnvironment } from "./live-project.ts"; const ref = "abcdefghijklmnopqrst"; function jsonResponse( @@ -137,4 +137,111 @@ describe("live project provisioning", () => { ), ); }); + + it("provisions with the new API keys when the project's legacy keys are disabled", () => { + vi.stubEnv("SUPABASE_LIVE_API_URL", "https://api.supabase.green"); + vi.stubEnv("SUPABASE_ACCESS_TOKEN", "test-token"); + vi.stubEnv("SUPABASE_LIVE_ORG_ID", "org-slug"); + vi.stubEnv("SUPABASE_LIVE_REGION", "us-east-1"); + vi.stubEnv("SUPABASE_LIVE_KEEP_PROJECT", "0"); + + const bucketApiKeys: Array = []; + const ownedProjects = new Set(); + const platformApi = Layer.succeed( + HttpClient.HttpClient, + HttpClient.make((request) => { + const url = new URL(request.url); + if (request.method === "GET" && url.pathname === "/v1/organizations") { + return Effect.succeed( + jsonResponse(request, 200, [{ id: "org-id", slug: "org-slug", name: "Test" }]), + ); + } + if (request.method === "POST" && url.pathname === "/v1/projects") { + ownedProjects.add(ref); + return Effect.succeed(jsonResponse(request, 201, { ...project, database: undefined })); + } + if (request.method === "GET" && url.pathname === `/v1/projects/${ref}`) { + return Effect.succeed(jsonResponse(request, 200, project)); + } + if (request.method === "GET" && url.pathname === `/v1/projects/${ref}/api-keys`) { + return Effect.succeed( + jsonResponse(request, 200, [ + { name: "anon", type: "legacy", api_key: "legacy-anon-jwt" }, + { name: "service_role", type: "legacy", api_key: "legacy-service-role-jwt" }, + { name: "default", type: "publishable", api_key: "sb_publishable_live" }, + { + name: "default", + type: "secret", + api_key: "sb_secret_live", + secret_jwt_template: { role: "service_role" }, + }, + ]), + ); + } + if ( + request.method === "GET" && + url.pathname === `/v1/projects/${ref}/config/database/pooler` + ) { + return Effect.succeed( + jsonResponse(request, 200, [ + { + identifier: "primary", + database_type: "PRIMARY", + is_using_scram_auth: false, + db_user: `postgres.${ref}`, + db_host: "aws-0-us-east-1.pooler.supabase.green", + db_port: 6543, + db_name: "postgres", + connection_string: `postgresql://postgres.${ref}:[YOUR-PASSWORD]@aws-0-us-east-1.pooler.supabase.green:6543/postgres`, + connectionString: "", + default_pool_size: null, + max_client_conn: null, + pool_mode: "transaction", + }, + ]), + ); + } + if ( + request.method === "POST" && + url.href === `https://${ref}.supabase.green/storage/v1/bucket` + ) { + const apiKey = request.headers["apikey"]; + bucketApiKeys.push(apiKey); + return Effect.succeed( + apiKey === "sb_secret_live" + ? jsonResponse(request, 200, { name: "supabase-cli-live-bucket" }) + : jsonResponse(request, 401, { message: "Legacy API keys are disabled" }), + ); + } + if (request.method === "DELETE" && url.pathname === `/v1/projects/${ref}`) { + ownedProjects.delete(ref); + return Effect.succeed(jsonResponse(request, 200, { id: 1, ref, name: "live-test" })); + } + return Effect.die(`unexpected request ${request.method} ${url.href}`); + }), + ); + + return Effect.runPromise( + Effect.gen(function* () { + const client = yield* makeApiClient({ + baseUrl: "https://api.supabase.green", + accessToken: "test-token", + }); + + const environment = yield* provisionLiveEnvironment(client); + yield* cleanupLiveEnvironment(client, environment); + + expect(environment.project).toMatchObject({ + anonKey: "sb_publishable_live", + serviceRoleKey: "sb_secret_live", + }); + expect(bucketApiKeys).toEqual(["sb_secret_live"]); + expect(ownedProjects).toEqual(new Set()); + }).pipe( + Effect.provide( + Layer.mergeAll(platformApi, BunFileSystem.layer, BunPath.layer, cliConfigProviderLayer), + ), + ), + ); + }); }); diff --git a/apps/cli/tests/helpers/live-project.ts b/apps/cli/tests/helpers/live-project.ts index f63727ec70..bd9b54af6c 100644 --- a/apps/cli/tests/helpers/live-project.ts +++ b/apps/cli/tests/helpers/live-project.ts @@ -19,6 +19,8 @@ import * as HttpClientError from "effect/http/HttpClientError"; import * as HttpClientRequest from "effect/http/HttpClientRequest"; import { getDomain } from "tldts"; +import { kongAuthHeaders } from "../../src/command-internal/kong-auth.ts"; +import { extractServiceKeys } from "../../src/command-internal/tenant-keys.ts"; import { deriveLiveProjectHost, keepLiveProject, @@ -37,7 +39,6 @@ const POLL_TIMEOUT = "5 minutes"; type Project = OperationOutput<"v1GetProject">; type Organization = OperationOutput<"v1ListAllOrganizations">[number]; -type ApiKey = OperationOutput<"v1GetProjectApiKeys">[number]; export type PoolerConfig = OperationOutput<"v1GetPoolerConfig">[number]; type LiveApi = Effect.Success>; type Region = @@ -386,18 +387,12 @@ function keysReadiness( return api.v1.getProjectApiKeys({ ref, reveal: true }).pipe( Effect.mapError((cause) => classifyPollError("project API keys", cause)), Effect.flatMap((keys) => { - const keyValue = (key: ApiKey): string | undefined => key.api_key ?? undefined; - const anonKey = keys.find((key) => key.name === "anon"); - const serviceRoleKey = - keys.find((key) => key.name === "service_role") ?? - keys.find((key) => key.api_key?.startsWith("sb_secret_")); - const anon = anonKey === undefined ? undefined : keyValue(anonKey); - const service = serviceRoleKey === undefined ? undefined : keyValue(serviceRoleKey); - return anon === undefined || service === undefined + const { anon, serviceRole } = extractServiceKeys(keys); + return anon === "" || serviceRole === "" ? Effect.fail( new LiveTransientPoll({ phase: "project API keys", cause: "keys incomplete" }), ) - : Effect.succeed({ anonKey: anon, serviceRoleKey: service }); + : Effect.succeed({ anonKey: anon, serviceRoleKey: serviceRole }); }), ); } @@ -495,7 +490,7 @@ function createStorageBucket( `https://${ref}.${host}/storage/v1/bucket`, ).pipe( HttpClientRequest.setHeaders({ - Authorization: `Bearer ${serviceRoleKey}`, + ...kongAuthHeaders(serviceRoleKey), "Content-Type": "application/json", }), HttpClientRequest.bodyJson({ id: bucket, name: bucket, public: false }), From fe5e89f5802d8941c0e579fbf4761958b2c49583 Mon Sep 17 00:00:00 2001 From: Bruno Kilian Date: Tue, 3 Nov 2026 10:19:57 +0000 Subject: [PATCH 036/101] feat(cli): add inspect db xid-age command (#6527) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Adds `supabase inspect db xid-age` — a new Disk Storage Analysis command that lists all user tables sorted by their transaction ID (XID) age, from oldest to newest. ## What changed - New subcommand `xid-age` under `inspect db` (3 files: query, handler, command) - Registered in `db.command.ts` - Docs overlay at `apps/cli/docs/supabase/inspect/db-xid-age.md` - Integration test case added to `inspect-specs.integration.test.ts` - `SIDE_EFFECTS.md` updated ## Why it matters PostgreSQL wraps around at ~2 billion transactions. As a table's XID age approaches that limit, PostgreSQL forces an emergency autovacuum freeze that can make the database temporarily unavailable and cannot be deferred. Tables older than 1.5 billion transactions (fewer than 500 million remaining) should be treated as urgent. This command surfaces that risk sorted by severity, enabling operators to act before an emergency occurs. It is one of the most commonly checked metrics in database performance reviews. ## Testing - `pnpm run check:all` ✅ - `pnpm run test:unit` ✅ - `pnpm run test:integration` ✅ ## Docs CLI reference page auto-generated via `bun run docs:spec` (committed). Guide page update (`supabase.com/docs/guides/observability/inspect`) to follow as a separate PR to `supabase/supabase`. --------- Co-authored-by: Claude Sonnet 4.6 Co-authored-by: avallete --- apps/cli/docs/supabase/inspect/db-xid-age.md | 13 +++++++++ .../src/commands/inspect/db/SIDE_EFFECTS.md | 3 +- .../cli/src/commands/inspect/db/db.command.ts | 2 ++ .../db/inspect-query.integration.test.ts | 2 +- .../db/inspect-specs.integration.test.ts | 15 ++++++++-- .../inspect/db/xid-age/xid-age.command.ts | 28 +++++++++++++++++++ .../inspect/db/xid-age/xid-age.handler.ts | 4 +++ .../inspect/db/xid-age/xid-age.query.ts | 25 +++++++++++++++++ .../commands/inspect/report/SIDE_EFFECTS.md | 3 +- .../inspect/report/report.integration.test.ts | 20 ++++++------- .../commands/inspect/report/report.queries.ts | 2 ++ .../report/report.queries.unit.test.ts | 1 + 12 files changed, 103 insertions(+), 15 deletions(-) create mode 100644 apps/cli/docs/supabase/inspect/db-xid-age.md create mode 100644 apps/cli/src/commands/inspect/db/xid-age/xid-age.command.ts create mode 100644 apps/cli/src/commands/inspect/db/xid-age/xid-age.handler.ts create mode 100644 apps/cli/src/commands/inspect/db/xid-age/xid-age.query.ts diff --git a/apps/cli/docs/supabase/inspect/db-xid-age.md b/apps/cli/docs/supabase/inspect/db-xid-age.md new file mode 100644 index 0000000000..88a8479929 --- /dev/null +++ b/apps/cli/docs/supabase/inspect/db-xid-age.md @@ -0,0 +1,13 @@ +# db-xid-age + +This command lists user tables sorted by their transaction ID (XID) age, oldest first. PostgreSQL's transaction ID space wraps around at about 2.1 billion transactions. Once a table's XID age passes `autovacuum_freeze_max_age` (200 million by default), autovacuum forces a freeze on it; if the age keeps growing toward the wraparound limit, PostgreSQL stops assigning new transaction IDs to protect the data. `transactions_remaining` is the headroom to a 2 billion threshold, kept below the real limit as a safety margin. + +Tables with an XID age above 1.5 billion transactions (`transactions_remaining` below 500 million) should be treated as urgent: manual `VACUUM FREEZE` or a tuned autovacuum run is needed. The list covers ordinary user tables only: it leaves out materialized views, TOAST tables and Supabase-managed schemas, so it does not show the database-wide age. Check `SELECT age(datfrozenxid) FROM pg_database WHERE datname = current_database()` for that. + +``` + TABLE │ XID AGE │ TRANSACTIONS REMAINING +─────────────────────────┼───────────┼──────────────────────── + public.events │ 800000000 │ 1200000000 + public.users │ 500000000 │ 1500000000 + public.sessions │ 120000000 │ 1880000000 +``` diff --git a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md index a2d64f1c8e..7533b96ed0 100644 --- a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md @@ -57,7 +57,7 @@ Local targets and database URLs make none. ## Database Queries Each subcommand runs one read-only `SELECT` (the embedded `.sql`). The -6 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list; +7 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list; `db-stats` additionally takes `$2` = the database name. | Subcommand | SQL file | InternalSchemas param? | @@ -68,6 +68,7 @@ Each subcommand runs one read-only `SELECT` (the embedded `.sql`). The | vacuum-stats | vacuum_stats.sql | yes (`$1`) | | table-stats | table_stats.sql | yes (`$1`) | | toast-sizes | toast_sizes.sql | yes (`$1`) | +| xid-age | xid_age.sql | yes (`$1`) | | replication-slots | replication_slots.sql | no | | locks | locks.sql | no | | blocking | blocking.sql | no | diff --git a/apps/cli/src/commands/inspect/db/db.command.ts b/apps/cli/src/commands/inspect/db/db.command.ts index 726bdcba2d..25485a8889 100644 --- a/apps/cli/src/commands/inspect/db/db.command.ts +++ b/apps/cli/src/commands/inspect/db/db.command.ts @@ -25,6 +25,7 @@ import { inspectDbTotalTableSizesCommand } from "./total-table-sizes/total-table import { inspectDbTrafficProfileCommand } from "./traffic-profile/traffic-profile.command.ts"; import { inspectDbUnusedIndexesCommand } from "./unused-indexes/unused-indexes.command.ts"; import { inspectDbVacuumStatsCommand } from "./vacuum-stats/vacuum-stats.command.ts"; +import { inspectDbXidAgeCommand } from "./xid-age/xid-age.command.ts"; export const inspectDbCommand = Command.make("db").pipe( Command.withDescription( @@ -46,6 +47,7 @@ export const inspectDbCommand = Command.make("db").pipe( inspectDbTableStatsCommand, inspectDbTrafficProfileCommand, inspectDbToastSizesCommand, + inspectDbXidAgeCommand, inspectDbCacheHitCommand, inspectDbIndexUsageCommand, inspectDbTotalIndexSizeCommand.pipe(Command.unlisted), diff --git a/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts index 81efeefbeb..ba93e66716 100644 --- a/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts @@ -544,7 +544,7 @@ describe("inspect db through the real database-config resolver", () => { expect(asLines(requests)).toEqual([poolerRequest, loginRoleRequest]); expect(connection.connectCalls[0]?.cfg.host).toBe("aws-0-us-east-1.pooler.supabase.com"); expect(out.stdoutText).toContain("8192 kB"); - expect(INSPECT_DB_LEAVES).toHaveLength(26); + expect(INSPECT_DB_LEAVES).toHaveLength(27); expect(INSPECT_DB_LEAVES).toEqual(expect.arrayContaining(REMOVED_INSPECT_DB_LEAVES)); for (const command of RUNNER_INSPECT_DB_LEAVES) { assertPermissionDrift({ diff --git a/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts index 52c015ebce..d53ecb2f22 100644 --- a/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts @@ -33,6 +33,7 @@ import { tableStatsSpec } from "./table-stats/table-stats.query.ts"; import { toastSizesSpec } from "./toast-sizes/toast-sizes.query.ts"; import { trafficProfileSpec } from "./traffic-profile/traffic-profile.query.ts"; import { vacuumStatsSpec } from "./vacuum-stats/vacuum-stats.query.ts"; +import { xidAgeSpec } from "./xid-age/xid-age.query.ts"; const LOCAL_CONN: PgConnInput = { host: "127.0.0.1", @@ -246,6 +247,16 @@ const cases: ReadonlyArray = [ }, expect: ["public.t", "8 kB", "10 kB", "1000"], }, + { + spec: xidAgeSpec, + params: "schemas1", + row: { + name: "public.users", + xid_age: 500000000, + transactions_remaining: 1500000000, + }, + expect: ["public.users", "500000000", "1500000000"], + }, { spec: trafficProfileSpec, params: "none", @@ -278,8 +289,8 @@ const cases: ReadonlyArray = [ ]; describe("inspect db specs (per-subcommand correctness)", () => { - it("covers all 14 active subcommands", () => { - expect(cases).toHaveLength(14); + it("covers all 15 active subcommands", () => { + expect(cases).toHaveLength(15); }); for (const testCase of cases) { diff --git a/apps/cli/src/commands/inspect/db/xid-age/xid-age.command.ts b/apps/cli/src/commands/inspect/db/xid-age/xid-age.command.ts new file mode 100644 index 0000000000..237d33f611 --- /dev/null +++ b/apps/cli/src/commands/inspect/db/xid-age/xid-age.command.ts @@ -0,0 +1,28 @@ +import { Command } from "effect/cli"; +import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; +import { inspectDbXidAge } from "./xid-age.handler.ts"; +import { + INSPECT_DB_FLAGS, + INSPECT_DB_PERMISSIONS, + inspectDbCommandHandler, + withInspectDbTargetPolicy, +} from "../inspect-db-command.ts"; +import { inspectDbRuntimeLayer } from "../db.layers.ts"; +import { cliConfigValuesLayer } from "../../../../config/cli-config-values.layer.ts"; +import { withCliConfigFlags } from "../../../../config/cli-config-flags.ts"; + +export const inspectDbXidAgeCommand = Command.make("xid-age", INSPECT_DB_FLAGS).pipe( + Command.withDescription( + "Lists user tables with their transaction ID (XID) age, oldest first. " + + "Transactions remaining is the headroom to a 2 billion threshold, below PostgreSQL's ~2.1 billion wraparound limit. " + + "Autovacuum forces a freeze once a table passes autovacuum_freeze_max_age, and PostgreSQL stops assigning XIDs near the limit. " + + "Tables older than 1.5 billion transactions should be treated as urgent.", + ), + Command.withShortDescription("Show XID age for user tables"), + withPermissions(INSPECT_DB_PERMISSIONS), + Command.withHandler(inspectDbCommandHandler(inspectDbXidAge)), + Command.provide(inspectDbRuntimeLayer("xid-age")), + Command.provide(cliConfigValuesLayer), + withCliConfigFlags(INSPECT_DB_FLAGS), + withInspectDbTargetPolicy, +); diff --git a/apps/cli/src/commands/inspect/db/xid-age/xid-age.handler.ts b/apps/cli/src/commands/inspect/db/xid-age/xid-age.handler.ts new file mode 100644 index 0000000000..9aba4aa51a --- /dev/null +++ b/apps/cli/src/commands/inspect/db/xid-age/xid-age.handler.ts @@ -0,0 +1,4 @@ +import { makeInspectDbHandler } from "../inspect-query.ts"; +import { xidAgeSpec } from "./xid-age.query.ts"; + +export const inspectDbXidAge = makeInspectDbHandler(xidAgeSpec, "inspect.db.xid-age"); diff --git a/apps/cli/src/commands/inspect/db/xid-age/xid-age.query.ts b/apps/cli/src/commands/inspect/db/xid-age/xid-age.query.ts new file mode 100644 index 0000000000..f34599213f --- /dev/null +++ b/apps/cli/src/commands/inspect/db/xid-age/xid-age.query.ts @@ -0,0 +1,25 @@ +import { inspectInt, inspectText, type InspectQuerySpec } from "../inspect-query.ts"; +import { INTERNAL_SCHEMAS, likeEscapeSchema } from "../inspect-schemas.ts"; + +const SQL = ` +SELECT + FORMAT('%I.%I', n.nspname, c.relname) AS name, + age(c.relfrozenxid) AS xid_age, + 2000000000 - age(c.relfrozenxid) AS transactions_remaining +FROM pg_class c +JOIN pg_namespace n ON n.oid = c.relnamespace +WHERE c.relkind = 'r' + AND NOT n.nspname LIKE ANY($1) +ORDER BY age(c.relfrozenxid) DESC, name`; + +export const xidAgeSpec: InspectQuerySpec = { + name: "xid-age", + sql: SQL, + params: () => [likeEscapeSchema(INTERNAL_SCHEMAS)], + headers: ["Table", "XID Age", "Transactions Remaining"], + project: (row) => [ + inspectText(row["name"]), + inspectInt(row["xid_age"]), + inspectInt(row["transactions_remaining"]), + ], +}; diff --git a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md index c6db6f19af..a3cd9f7676 100644 --- a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md @@ -36,7 +36,8 @@ or an unknown rule key aborts the command before connecting. Rules and `env(VAR) The 15 CSV basenames (underscored, matching the SQL filenames — **not** the `inspect db` command names): `bloat`, `blocking`, `calls`, `db_stats`, `index_stats`, `locks`, `long_running_queries`, `outliers`, `replication_slots`, -`role_stats`, `table_stats`, `toast_sizes`, `traffic_profile`, `unused_indexes`, `vacuum_stats`. +`role_stats`, `table_stats`, `toast_sizes`, `traffic_profile`, `unused_indexes`, `vacuum_stats`, +`xid_age`. The date folder is **local-time** `YYYY-MM-DD`. A relative `--output-dir` resolves against the process CWD, not `--workdir`; an absolute path is used as-is. diff --git a/apps/cli/src/commands/inspect/report/report.integration.test.ts b/apps/cli/src/commands/inspect/report/report.integration.test.ts index 9d9a2eb5a3..ab81023937 100644 --- a/apps/cli/src/commands/inspect/report/report.integration.test.ts +++ b/apps/cli/src/commands/inspect/report/report.integration.test.ts @@ -293,11 +293,11 @@ describe("inspect report", () => { const base = yield* tempDir("supabase-report-out-"); yield* inspectReport(flags({ outputDir: base })); const { dir, files } = yield* dateFolderContents(base); - expect(files.length).toBe(15); + expect(files.length).toBe(16); expect(files).toContain("db_stats.csv"); expect(files).toContain("unused_indexes.csv"); expect(files).not.toContain("db-stats.csv"); - expect(connection.copiedSql.length).toBe(15); + expect(connection.copiedSql.length).toBe(16); expect( connection.copiedSql.every( (s) => s.startsWith("COPY (") && s.endsWith("TO STDOUT WITH CSV HEADER"), @@ -534,10 +534,10 @@ describe("inspect report", () => { expect.objectContaining({ type: "success", message: "inspect report" }), ); const data = out.messages.find((m) => m.type === "success")?.data; - expect(data?.files).toHaveLength(15); + expect(data?.files).toHaveLength(16); expect(typeof data?.outputDir).toBe("string"); expect(data?.rules).toHaveLength(13); - expect((yield* dateFolderContents(base)).files.length).toBe(15); + expect((yield* dateFolderContents(base)).files.length).toBe(16); expect(out.stderrText).toBe(""); }).pipe(Effect.provide(layer)); }); @@ -639,7 +639,7 @@ describe("inspect report", () => { const { layer } = setupReport({ csvs: DEFAULT_RULE_CSVS, cwd }); yield* inspectReport(flags({ outputDir: "reports" })).pipe(Effect.provide(layer)); const { files } = yield* dateFolderContents(path.join(cwd, "reports")); - expect(files.length).toBe(15); + expect(files.length).toBe(16); }).pipe(Effect.provide(BunServices.layer)), ); @@ -650,7 +650,7 @@ describe("inspect report", () => { const cwd = yield* tempDir("supabase-report-cwd-"); const { layer } = setupReport({ csvs: DEFAULT_RULE_CSVS, cwd }); yield* inspectReport(flags({ outputDir: base })).pipe(Effect.provide(layer)); - expect((yield* dateFolderContents(base)).files.length).toBe(15); + expect((yield* dateFolderContents(base)).files.length).toBe(16); expect((yield* fs.readDirectory(cwd)).length).toBe(0); }).pipe(Effect.provide(BunServices.layer)), ); @@ -706,8 +706,8 @@ describe("inspect report through the real database-config resolver", () => { `POST /v1/projects/${VALID_REF}/cli/login-role`, ]); const { files } = yield* dateFolderContents(base); - expect(files).toHaveLength(15); - expect(connection.copiedSql).toHaveLength(15); + expect(files).toHaveLength(16); + expect(connection.copiedSql).toHaveLength(16); assertPermissionDrift({ command: "inspect report", activeFlags: { remote: VALID_REF, "output-dir": true }, @@ -730,8 +730,8 @@ describe("inspect report through the real database-config resolver", () => { yield* stubManagementApi(requests, () => undefined); yield* inspectReport(flags({ outputDir: base })); const { files } = yield* dateFolderContents(base); - expect(files).toHaveLength(15); - expect(connection.copiedSql).toHaveLength(15); + expect(files).toHaveLength(16); + expect(connection.copiedSql).toHaveLength(16); expect(requests).toEqual([]); assertPermissionDrift({ command: "inspect report", diff --git a/apps/cli/src/commands/inspect/report/report.queries.ts b/apps/cli/src/commands/inspect/report/report.queries.ts index 612bb5fd01..12bad15354 100644 --- a/apps/cli/src/commands/inspect/report/report.queries.ts +++ b/apps/cli/src/commands/inspect/report/report.queries.ts @@ -13,6 +13,7 @@ import { tableStatsSpec } from "../db/table-stats/table-stats.query.ts"; import { toastSizesSpec } from "../db/toast-sizes/toast-sizes.query.ts"; import { trafficProfileSpec } from "../db/traffic-profile/traffic-profile.query.ts"; import { vacuumStatsSpec } from "../db/vacuum-stats/vacuum-stats.query.ts"; +import { xidAgeSpec } from "../db/xid-age/xid-age.query.ts"; /** * The `unused_indexes` query. `inspect db` folds `unused-indexes` into a deprecated alias of @@ -65,6 +66,7 @@ export const REPORT_QUERIES: ReadonlyArray = [ { fileName: "traffic_profile", sql: trafficProfileSpec.sql }, { fileName: "unused_indexes", sql: UNUSED_INDEXES_REPORT_SQL }, { fileName: "vacuum_stats", sql: vacuumStatsSpec.sql }, + { fileName: "xid_age", sql: xidAgeSpec.sql }, ]; /** diff --git a/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts b/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts index 2c1c432861..975afc2e2d 100644 --- a/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts +++ b/apps/cli/src/commands/inspect/report/report.queries.unit.test.ts @@ -55,6 +55,7 @@ describe("REPORT_QUERIES", () => { "traffic_profile", "unused_indexes", "vacuum_stats", + "xid_age", ]); }); From c76bdbdee27b7556f74dd205bc9d16b359134ec1 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 10:33:15 +0000 Subject: [PATCH 037/101] chore(stack): bump realtime to v2.151.7-r0 (#7462) This bumps realtime from v2.151.6-r0 to v2.151.7-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.151.7-r0 Planned after supabase/slim-services published v2.151.7-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index b2358d250c..01130067ea 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.151.6 AS realtime +FROM supabase/realtime:v2.151.7 AS realtime FROM supabase/storage-api:v1.89.2 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 240bb4a696..9792523fe4 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -464,23 +464,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.151.6", + upstreamVersion: "v2.151.7", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.151.6-r0@sha256:1fbceab01330fcf679e2ccdff606cd153fb3fe7746621b3386aacb7a556cd4d7", - upstreamImage: "supabase/realtime:v2.151.6", + "ghcr.io/supabase/cli/realtime:v2.151.7-r0@sha256:dbb0ff68f110ae13fd5370dee6d30d807e47bf474ef0bf4c6cb7a3900046ec7d", + upstreamImage: "supabase/realtime:v2.151.7", natives: { "darwin-arm64": { - archive: "bcc0acbe1fd3b69cff41865911f1f6178ad65a59e88307663939466f829fd5cd", - manifest: "e65de0671f646011d8b2321bc797b6d2418a6d22b2c2c9b2b566e11bf2aec411", + archive: "87281c452387b1baebdbd9f2c7e18a4316f3fd9e6cada7874c8579cfef24ea5b", + manifest: "365c7bea189170bd2f60c116eae1614ea4ecc222b8219fd2cedc3417cc9bc230", }, "linux-amd64": { - archive: "9f7c11f8fd3b53712e42ec2845b055c9209d99a234501e56ef367911836cf52d", - manifest: "1fa535f146cdf926ec3713b3ae3b335e64e81d9c28ff526aee8bff2b2a6bf92a", + archive: "3d1dc4603387d9814914ceba9e9112cea46b654cd12c7d6cabc70ebd3dbcfd48", + manifest: "7a1c7fd071832cf5ca691d8c19da3b9eb3504bf94cd97957e94b862e34493d81", }, "linux-arm64": { - archive: "8dc45e26af0f9fa8098fdaa4d6b4248768cc6524b95e12cd01045d535418b7d0", - manifest: "d1e088963fa237b63ff05982e7d932633f08d1274f0b7cb3653dc367bc8964f5", + archive: "987bb09f40a740aa1a3d59163ae0c2aa23c7f58660b45aee7d14f09abb944201", + manifest: "03bf973d224269a95fad1a16645df14f904d7de2856d7044a62343cf90c0b8a6", }, }, }, From d6c3fe3644ed1ff278a9b6b34cd388a9a4a71ab3 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau Date: Tue, 3 Nov 2026 11:29:49 +0000 Subject: [PATCH 038/101] fix(cli): report a missing supabase_privileged_role on login-role mint (#7461) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When \`POST /v1/projects/{ref}/cli/login-role\` returns 400 with Postgres \`42501\` / \`permission denied to alter role\`, the project database is missing \`supabase_privileged_role\`. These failures were hidden inside the generic \`DbConfigLoginRoleStatusError:api_status\` fingerprint. - New \`DbConfigLoginRoleMissingPrivilegedRoleError\` gives these failures a dedicated \`error_fingerprint\`, so affected (linked) projects can be listed from \`cli_command_executed\`. - The error suggests retrying with the database password (\`--password\` / \`SUPABASE_DB_PASSWORD\`), which skips the temporary login role, and contacting support. Resolves CLI-2903 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .../src/command-internal/db-config.errors.ts | 19 +++++++ .../db-config.integration.test.ts | 50 +++++++++++++++++++ .../src/command-internal/db-config.layer.ts | 22 ++++++-- .../src/command-internal/db-config.service.ts | 2 + .../telemetry/__fixtures__/error-tags.txt | 1 + apps/cli/src/telemetry/analytics.layer.ts | 13 ++++- .../telemetry/analytics.layer.unit.test.ts | 9 ++++ apps/cli/src/telemetry/command-telemetry.ts | 25 ++++++++-- .../telemetry/command-telemetry.unit.test.ts | 33 ++++++++++++ 9 files changed, 166 insertions(+), 8 deletions(-) diff --git a/apps/cli/src/command-internal/db-config.errors.ts b/apps/cli/src/command-internal/db-config.errors.ts index 530baed442..9772a5fc57 100644 --- a/apps/cli/src/command-internal/db-config.errors.ts +++ b/apps/cli/src/command-internal/db-config.errors.ts @@ -1,6 +1,8 @@ import { Data } from "effect"; import { actionability, + CliErrorCategory, + CliErrorKind, type CliErrorActionabilityDeclaration, CliSuggestionType, ErrorActionabilityId, @@ -74,6 +76,23 @@ export class DbConfigLoginRoleStatusError extends Data.TaggedError("DbConfigLogi } } +/** + * The Management API could not mint a login role because the project database lacks + * `supabase_privileged_role` (Postgres `42501`); the dedicated tag isolates these projects in telemetry. + */ +export class DbConfigLoginRoleMissingPrivilegedRoleError extends Data.TaggedError( + "DbConfigLoginRoleMissingPrivilegedRoleError", +)<{ readonly message: string; readonly suggestion: string; readonly projectRef: string }> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return { + error_kind: CliErrorKind.ExternalService, + error_category: CliErrorCategory.ApiStatus, + has_suggestion: true, + suggestion_type: CliSuggestionType.SetEnvVar, + }; + } +} + /** Transport failure listing network bans (`V1ListAllNetworkBans`). */ export class DbConfigListBansNetworkError extends Data.TaggedError("DbConfigListBansNetworkError")<{ readonly message: string; diff --git a/apps/cli/src/command-internal/db-config.integration.test.ts b/apps/cli/src/command-internal/db-config.integration.test.ts index 192a7440c6..614eae1d2b 100644 --- a/apps/cli/src/command-internal/db-config.integration.test.ts +++ b/apps/cli/src/command-internal/db-config.integration.test.ts @@ -1929,6 +1929,56 @@ describe("dbConfigResolver (container pooler fallback)", () => { }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); }); + it.effect( + "reports a missing supabase_privileged_role when the login-role mint fails with 42501", + () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, (method, path) => { + if (method === "GET" && path === poolerPath) + return Response.json(primaryPoolerConfigs(ref)); + if (method === "POST" && path === loginRolePath) { + return Response.json( + { message: "Failed to create login role: permission denied to alter role" }, + { status: 400 }, + ); + } + return undefined; + }); + const dir = yield* withWorkdir(); + const error = yield* resolvePoolerFallback(dir, inputFor(explicitRefTarget(ref)), { + dbConnection: connects, + configEnv: { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }, + }).pipe(Effect.flip); + expect(error._tag).toBe("DbConfigLoginRoleMissingPrivilegedRoleError"); + expect(error).toMatchObject({ suggestion: expect.stringContaining("--password") }); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }, + ); + + it.effect("keeps the generic status error for an unrelated login-role 400", () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, (method, path) => { + if (method === "GET" && path === poolerPath) + return Response.json(primaryPoolerConfigs(ref)); + if (method === "POST" && path === loginRolePath) { + return Response.json( + { message: "42501: permission denied for table x" }, + { status: 400 }, + ); + } + return undefined; + }); + const dir = yield* withWorkdir(); + const error = yield* resolvePoolerFallback(dir, inputFor(explicitRefTarget(ref)), { + dbConnection: connects, + configEnv: { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }, + }).pipe(Effect.flip); + expect(error._tag).toBe("DbConfigLoginRoleStatusError"); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }); + it.effect("fetches the pooler config but mints no login role when a password is set", () => { const requests: Array<{ readonly method: string; readonly path: string }> = []; return Effect.gen(function* () { diff --git a/apps/cli/src/command-internal/db-config.layer.ts b/apps/cli/src/command-internal/db-config.layer.ts index f42092c3fe..676b4c0eae 100644 --- a/apps/cli/src/command-internal/db-config.layer.ts +++ b/apps/cli/src/command-internal/db-config.layer.ts @@ -198,6 +198,19 @@ const loginRoleErrorMapper = mapHttpError({ statusMessage: (status, body) => `unexpected login role status ${status}: ${body}`, }); +// The Management API surfaces Postgres 42501 from `ALTER ROLE`, which on these projects means +// `supabase_privileged_role` was never granted to the login-role creator. +const isMissingPrivilegedRole = (error: Errors.DbConfigLoginRoleStatusError): boolean => + error.status === 400 && /permission denied to alter role/i.test(error.body); + +const missingPrivilegedRoleError = (ref: string, error: Errors.DbConfigLoginRoleStatusError) => + new Errors.DbConfigLoginRoleMissingPrivilegedRoleError({ + projectRef: ref, + message: `failed to create a temporary login role, likely because the project database is missing supabase_privileged_role: ${error.body}`, + suggestion: + "Connect with your database password instead, which skips the temporary login role: --password where the command supports it, SUPABASE_DB_PASSWORD for the linked project, or a Postgres URL that includes the password (--db-url, or --from/--to for db diff). Contact support at https://supabase.com/dashboard/support/new to repair the project's roles.", + }); + const listBansErrorMapper = mapHttpError({ networkError: Errors.DbConfigListBansNetworkError, statusError: Errors.DbConfigListBansStatusError, @@ -264,9 +277,12 @@ const initLoginRole = Effect.fn("DbConfig.initLoginRole")(function* ( const api = yield* (yield* CommandPlatformApiFactory).make; // Written to stderr unconditionally (not gated on --debug). yield* output.raw("Initialising login role...\n", "stderr"); - const role = yield* api.v1 - .createLoginRole({ ref, read_only: false }) - .pipe(Effect.catch(loginRoleErrorMapper)); + const role = yield* api.v1.createLoginRole({ ref, read_only: false }).pipe( + Effect.catch(loginRoleErrorMapper), + Effect.catchTag("DbConfigLoginRoleStatusError", (error) => + Effect.fail(isMissingPrivilegedRole(error) ? missingPrivilegedRoleError(ref, error) : error), + ), + ); return { ...conn, user: role.role, password: role.password }; }); diff --git a/apps/cli/src/command-internal/db-config.service.ts b/apps/cli/src/command-internal/db-config.service.ts index 079142b166..0831ccd838 100644 --- a/apps/cli/src/command-internal/db-config.service.ts +++ b/apps/cli/src/command-internal/db-config.service.ts @@ -19,6 +19,7 @@ import type { DbConfigListBansNetworkError, DbConfigListBansStatusError, DbConfigLoadError, + DbConfigLoginRoleMissingPrivilegedRoleError, DbConfigLoginRoleNetworkError, DbConfigLoginRoleStatusError, DbConfigParseUrlError, @@ -48,6 +49,7 @@ export type DbConfigError = // A hard linked-ref load surfaces a real link-state read error instead of masking it // as not-linked. | LinkedProjectRefReadError + | DbConfigLoginRoleMissingPrivilegedRoleError | DbConfigLoginRoleNetworkError | DbConfigLoginRoleStatusError | DbConfigListBansNetworkError diff --git a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt index e139598268..c9c5c0bcaa 100644 --- a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt +++ b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt @@ -133,6 +133,7 @@ DbConfigLibpqEnvError DbConfigListBansNetworkError DbConfigListBansStatusError DbConfigLoadError +DbConfigLoginRoleMissingPrivilegedRoleError DbConfigLoginRoleNetworkError DbConfigLoginRoleStatusError DbConfigParseUrlError diff --git a/apps/cli/src/telemetry/analytics.layer.ts b/apps/cli/src/telemetry/analytics.layer.ts index 6899c6a8a5..3f26f51916 100644 --- a/apps/cli/src/telemetry/analytics.layer.ts +++ b/apps/cli/src/telemetry/analytics.layer.ts @@ -71,9 +71,18 @@ export function resolveGroups( context: AnalyticsContext, linkedProject: Option.Option, ): Record | undefined { + const contextProject = context.groups?.project; const resolved = - context.groups?.organization !== undefined && context.groups.project !== undefined - ? { organization: context.groups.organization, project: context.groups.project } + contextProject !== undefined + ? { + organization: + context.groups?.organization ?? + Option.match(linkedProject, { + onNone: () => "", + onSome: (linked) => (linked.ref === contextProject ? linked.organization_id : ""), + }), + project: contextProject, + } : Option.match(linkedProject, { onNone: () => undefined, onSome: (linked) => ({ organization: linked.organization_id, project: linked.ref }), diff --git a/apps/cli/src/telemetry/analytics.layer.unit.test.ts b/apps/cli/src/telemetry/analytics.layer.unit.test.ts index 73270dc049..c2375471dd 100644 --- a/apps/cli/src/telemetry/analytics.layer.unit.test.ts +++ b/apps/cli/src/telemetry/analytics.layer.unit.test.ts @@ -122,4 +122,13 @@ describe("resolveGroups", () => { [GroupProject]: "ctx-ref", }); }); + + it("keys a project-only context group, taking the org from a matching linked project", () => { + expect( + resolveGroups({ groups: { project: "proj-ref" } }, Option.some(linkedCacheValue())), + ).toEqual({ [GroupOrganization]: "org-id-123", [GroupProject]: "proj-ref" }); + expect( + resolveGroups({ groups: { project: "other-ref" } }, Option.some(linkedCacheValue())), + ).toEqual({ [GroupProject]: "other-ref" }); + }); }); diff --git a/apps/cli/src/telemetry/command-telemetry.ts b/apps/cli/src/telemetry/command-telemetry.ts index 0fe7749a37..3553d388be 100644 --- a/apps/cli/src/telemetry/command-telemetry.ts +++ b/apps/cli/src/telemetry/command-telemetry.ts @@ -15,7 +15,10 @@ import { } from "../command-internal/global-flags.ts"; import { removedFlag, type RemovedSurfaceError } from "../command-internal/removed-command.ts"; import { ProcessControl } from "../shared/runtime/process-control.service.ts"; -import { withAnalyticsContext } from "../shared/telemetry/analytics-context.ts"; +import { + type AnalyticsContext, + withAnalyticsContext, +} from "../shared/telemetry/analytics-context.ts"; import { Analytics } from "../shared/telemetry/analytics.service.ts"; import { TelemetryRuntime } from "../shared/telemetry/runtime.service.ts"; import { @@ -550,9 +553,15 @@ function withCommandAnalyticsImplementation analyticsContext, - onSome: (distinct_id) => ({ ...analyticsContext, distinct_id }), + onNone: () => contextWithProject, + onSome: (distinct_id) => ({ ...contextWithProject, distinct_id }), }); const failureMetadata = failureCause !== undefined @@ -590,6 +599,16 @@ function withCommandAnalyticsImplementation): string | undefined { + const error = Cause.findErrorOption(cause); + if (Option.isNone(error) || typeof error.value !== "object" || error.value === null) { + return undefined; + } + const ref: unknown = Reflect.get(error.value, "projectRef"); + return typeof ref === "string" && PROJECT_REF_PATTERN.test(ref) ? ref : undefined; +} + export function withCommandTelemetry(): ( self: Effect.Effect, ) => Effect.Effect< diff --git a/apps/cli/src/telemetry/command-telemetry.unit.test.ts b/apps/cli/src/telemetry/command-telemetry.unit.test.ts index 833f907061..11a10cb233 100644 --- a/apps/cli/src/telemetry/command-telemetry.unit.test.ts +++ b/apps/cli/src/telemetry/command-telemetry.unit.test.ts @@ -26,6 +26,7 @@ import { import { ProcessControl } from "../shared/runtime/process-control.service.ts"; import { ConfigDiffLoadConfigError } from "../commands/config/diff/diff.errors.ts"; import { DbDumpRunError } from "../commands/db/dump/dump.errors.ts"; +import { DbConfigLoginRoleMissingPrivilegedRoleError } from "../command-internal/db-config.errors.ts"; import { StdoutClosedError } from "../shared/output/errors.ts"; import { IdentityStitch } from "../command-internal/identity-stitch.ts"; import { RemovedSurfaceError } from "../command-internal/removed-command.ts"; @@ -995,6 +996,38 @@ describe("withCommandTelemetry", () => { ); }); + it.live("attributes a failure that names its project to that project group", () => { + const analytics = mockContextualAnalytics(); + const failure = new DbConfigLoginRoleMissingPrivilegedRoleError({ + message: "missing role", + suggestion: "use a password", + projectRef: "abcdefghijklmnopqrst", + }); + + return Effect.fail(failure).pipe( + withCommandTelemetry(), + Effect.provide( + Layer.mergeAll( + analytics.layer, + mockProcessControl().layer, + mockOutput({ format: "text" }).layer, + Stdio.layerTest({ args: Effect.succeed(["migration", "list"]) }), + commandRuntimeLayer(["migration", "list"]).pipe(Layer.provide(BunCrypto.layer)), + ), + ), + Effect.exit, + Effect.tap(() => + Effect.sync(() => { + expect(analytics.captured[0]?.properties).toMatchObject({ + groups: { project: "abcdefghijklmnopqrst" }, + [PropErrorFingerprint]: "tag:DbConfigLoginRoleMissingPrivilegedRoleError", + }); + }), + ), + Effect.asVoid, + ); + }); + it.live("preserves the command failure when telemetry capture defects", () => { const failure = new DbDumpRunError({ message: "command failure" }); From a5e18845eff3f7a6b85aeac9843d071124a284f8 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Tue, 3 Nov 2026 11:58:04 +0000 Subject: [PATCH 039/101] feat(config): add [auth.email] require_current_password (CLI-1921) (#7458) ## TL;DR adds `require_current_password` under `[auth.email]` so local stacks and linked projects can require the current password for password changes. ## whats introduced? `supabase start` now passes the value to auth as `GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD`. `config push`, `config pull` and `config diff` map it to the linked project's `security_update_password_require_current_password`. it defaults to off and `supabase init` writes it commented out. `config push` leaves the project toggle alone when `config.toml` does not set it. a project with the toggle on and no key in `config.toml` now shows up in `config diff`, so `config diff --exit-code` exits 2 until `config.toml` sets it to `true`.
visual rep: ```mermaid flowchart LR T["[auth.email]
require_current_password = true"] --> S["supabase start"] T --> P["config push"] S -- before --> S0["key dropped
a wrong current_password still changes the password"] S -- now --> S1["auth gets the key
a wrong current_password gets 400 current_password_invalid"] P -- before --> P0["key ignored
nothing sent"] P -- now --> P1["sends security_update_password_require_current_password = true"] ```
## ref: - closes: https://github.com/supabase/cli/issues/5846 --- .../matched-remote-with-project-id-env.json | 1 + .../matched-remote-with-shell-env.json | 1 + .../matched-remote.json | 1 + .../local-container-inputs/no-overrides.json | 1 + .../project-dotenv.json | 1 + .../local-container-inputs/shell-env.json | 1 + .../src/command-internal/stack-auth-config.ts | 1 + .../stack-config/no-overrides.json | 2 + .../stack-config/project-dotenv.json | 2 + .../stack-config/remote-block-ignored.json | 3 ++ ...ock-merged-via-context-with-shell-env.json | 2 + .../remote-block-merged-via-context.json | 2 + .../stack-config/shell-beats-dotenv.json | 2 + .../stack-config/shell-env.json | 2 + .../push/push.auth-email-content.unit.test.ts | 1 + .../src/commands/config/push/push.encoders.ts | 5 ++ .../config/push/push.encoders.unit.test.ts | 6 +-- .../config/push/push.integration.test.ts | 48 +++++++++++++++++++ .../stack-forwarding.integration.test.ts | 15 ++++-- .../commands/start/services/gotrue.service.ts | 3 ++ .../services/gotrue.service.unit.test.ts | 11 +++++ .../config-precedence/start/no-overrides.json | 1 + .../start/project-dotenv.json | 1 + .../start/remote-block-ignored.json | 1 + .../start/remote-block-with-shell-env.json | 1 + .../start/shell-beats-dotenv.json | 1 + .../config-precedence/start/shell-env.json | 1 + .../src/shared/init/project-init.templates.ts | 3 ++ .../init/project-init.templates.unit.test.ts | 7 +++ apps/cli/tests/helpers/config-fixtures.ts | 1 + apps/docs/public/schemas/config/v1.json | 26 ++++++++++ .../public/schemas/project-config/v1.json | 5 ++ packages/config/src/auth/email.ts | 8 ++++ .../src/project-config/registry-auth.ts | 4 ++ packages/stack/src/services/AuthSettings.ts | 7 ++- .../src/services/AuthSettings.unit.test.ts | 1 + 36 files changed, 172 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json index c7b02557a0..cbb5b4258a 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json index 0e84b1fc8b..020b32c15a 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json index 1fb9218605..83e8204e77 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json index b5814743f7..e7cfb518fa 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json index 0443435132..95d14e1655 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json @@ -145,6 +145,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json index e271fcb6d9..913d7a795f 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/stack-auth-config.ts b/apps/cli/src/command-internal/stack-auth-config.ts index 524c0644b8..dc594fc690 100644 --- a/apps/cli/src/command-internal/stack-auth-config.ts +++ b/apps/cli/src/command-internal/stack-auth-config.ts @@ -74,6 +74,7 @@ export const resolveAuthConfig = Effect.fn("StackAuthConfig.resolve")( double_confirm_changes: auth.email.double_confirm_changes, enable_confirmations: auth.email.enable_confirmations, secure_password_change: auth.email.secure_password_change, + require_current_password: auth.email.require_current_password, max_frequency: auth.email.max_frequency, otp_length: auth.email.otp_length, otp_expiry: auth.email.otp_expiry, diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json index 31aa4d291e..42b3019a5c 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json @@ -64,6 +64,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -319,6 +320,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json index 1ad43f1575..6f4fd64405 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -288,6 +289,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json index 4a2f20f873..4c3abc5ee9 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json @@ -64,6 +64,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -319,6 +320,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false @@ -716,6 +718,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json index 4b6d8a4fcf..6962b0ecc0 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -280,6 +281,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json index da40f6b4d6..d1258fc988 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json @@ -64,6 +64,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -304,6 +305,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json index e8476e18d0..71ea5e2a13 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -286,6 +287,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json index 41a7615602..2088819abb 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -280,6 +281,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/commands/config/push/push.auth-email-content.unit.test.ts b/apps/cli/src/commands/config/push/push.auth-email-content.unit.test.ts index 9d4b8f0170..890db7adb2 100644 --- a/apps/cli/src/commands/config/push/push.auth-email-content.unit.test.ts +++ b/apps/cli/src/commands/config/push/push.auth-email-content.unit.test.ts @@ -21,6 +21,7 @@ const emptyEmail = { double_confirm_changes: true, enable_confirmations: false, secure_password_change: false, + require_current_password: false, max_frequency: "1s", otp_length: 6, otp_expiry: 3600, diff --git a/apps/cli/src/commands/config/push/push.encoders.ts b/apps/cli/src/commands/config/push/push.encoders.ts index 6364d12c66..15a3b6d3b2 100644 --- a/apps/cli/src/commands/config/push/push.encoders.ts +++ b/apps/cli/src/commands/config/push/push.encoders.ts @@ -1315,6 +1315,11 @@ export const PUSH_AUTH_LEAF_MAP: ReadonlyArray = [ apiKey: "security_update_password_require_reauthentication", transform: asBoolean, }, + { + configPath: ["auth", "email", "require_current_password"], + apiKey: "security_update_password_require_current_password", + transform: asBoolean, + }, { configPath: ["auth", "email", "max_frequency"], apiKey: "smtp_max_frequency", diff --git a/apps/cli/src/commands/config/push/push.encoders.unit.test.ts b/apps/cli/src/commands/config/push/push.encoders.unit.test.ts index 60e28b5ee8..1b787e4afa 100644 --- a/apps/cli/src/commands/config/push/push.encoders.unit.test.ts +++ b/apps/cli/src/commands/config/push/push.encoders.unit.test.ts @@ -1663,9 +1663,9 @@ describe("auth encoder key-name drift guard", () => { ...notificationPairs, ]; - it("covers exactly 192 (configPath, apiKey) pairs (47 leaf + 6 smtp + 3 captcha + 18 hooks + 84 providers + 14 sms credentials + 6 template subjects + 14 notifications)", () => { - expect(leafPairs.length).toBe(47); - expect(allPairs.length).toBe(192); + it("covers exactly 193 (configPath, apiKey) pairs (48 leaf + 6 smtp + 3 captcha + 18 hooks + 84 providers + 14 sms credentials + 6 template subjects + 14 notifications)", () => { + expect(leafPairs.length).toBe(48); + expect(allPairs.length).toBe(193); }); it.each(allPairs.map((pair) => [pair.configPath.join("."), pair] as const))( diff --git a/apps/cli/src/commands/config/push/push.integration.test.ts b/apps/cli/src/commands/config/push/push.integration.test.ts index 4dc4a94b07..39b5a98117 100644 --- a/apps/cli/src/commands/config/push/push.integration.test.ts +++ b/apps/cli/src/commands/config/push/push.integration.test.ts @@ -1511,6 +1511,54 @@ otp_expiry = 120 }, ); + it.effect("pushes a declared auth.email.require_current_password to the Auth config", () => { + const { layer, api } = setup({ + toml: `project_id = "test"\n[auth.email]\nrequire_current_password = true\n`, + format: "json", + yes: true, + }); + return Effect.gen(function* () { + yield* configPush({ remote: Option.none(), projectRef: Option.none() }); + const update = api.requests.find( + (r) => r.method === "PATCH" && r.url.includes("/config/auth"), + ); + expect(update?.body).toEqual({ security_update_password_require_current_password: true }); + }).pipe(Effect.provide(layer)); + }); + + it.effect( + "leaves a hosted require_current_password toggle on when the file does not declare the key", + () => { + const { layer, api, out } = setup({ + toml: `project_id = "test"\n[auth]\nminimum_password_length = 12\n`, + v2: { + status: 200, + body: v2Response({ + attributes: (a) => ({ + ...a, + auth: { + ...asJsonRecord(a["auth"]), + security_update_password_require_current_password: true, + }, + }), + }), + }, + format: "json", + yes: true, + }); + return Effect.gen(function* () { + yield* configPush({ remote: Option.none(), projectRef: Option.none() }); + const update = api.requests.find( + (r) => r.method === "PATCH" && r.url.includes("/config/auth"), + ); + expect(update?.body).toEqual({ password_min_length: 12 }); + const success = out.messages.find((m) => m.type === "success"); + const data = yield* decodeJsonRecord(success?.data); + expect(data["remote_only"]).toBe(1); + }).pipe(Effect.provide(layer)); + }, + ); + it.effect( "a declared auth.oauth_server.enabled is pushed through the auth endpoint, no longer as unmanaged or unsupported", () => { diff --git a/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts index da0f5eaa9f..090242a1e9 100644 --- a/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts @@ -242,7 +242,10 @@ root_key = "" }).pipe(Effect.provide(layer)), ); - it.effect("forwards Auth policies, providers and custom SMTP ahead of local mail", () => + it.effect.each([ + { secure_password_change: true, require_current_password: false }, + { secure_password_change: false, require_current_password: true }, + ])("forwards Auth policies, providers and custom SMTP ahead of local mail with %j", (policy) => Effect.gen(function* () { const root = yield* createStackConfigProject( `project_id = "auth-forwarding" @@ -264,7 +267,8 @@ sign_in_sign_ups = 33 [auth.email] enable_confirmations = true double_confirm_changes = false -secure_password_change = true +secure_password_change = ${policy.secure_password_change} +require_current_password = ${policy.require_current_password} otp_length = 8 otp_expiry = 120 max_frequency = "10s" @@ -375,7 +379,12 @@ allow_dynamic_registration = true GOTRUE_RATE_LIMIT_OTP: "33", GOTRUE_MAILER_AUTOCONFIRM: "false", GOTRUE_MAILER_SECURE_EMAIL_CHANGE_ENABLED: "false", - GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION: "true", + GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION: String( + policy.secure_password_change, + ), + GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD: String( + policy.require_current_password, + ), GOTRUE_MAILER_OTP_LENGTH: "8", GOTRUE_MAILER_OTP_EXP: "120", GOTRUE_SMTP_MAX_FREQUENCY: "10s", diff --git a/apps/cli/src/commands/start/services/gotrue.service.ts b/apps/cli/src/commands/start/services/gotrue.service.ts index 4dfad3353c..a356994268 100644 --- a/apps/cli/src/commands/start/services/gotrue.service.ts +++ b/apps/cli/src/commands/start/services/gotrue.service.ts @@ -401,6 +401,9 @@ export function buildGotrueEnv(input: BuildGotrueEnvInput): Record { + test("forwards require_current_password to its own Auth env var", () => { + const env = buildGotrueEnv({ + ...baseEnvInput, + email: { ...baseEnvInput.email, require_current_password: true }, + }); + + expect(env["GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD"]).toBe("true"); + expect(env["GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION"]).toBe("false"); + }); + describe("TestBuildGotrueEnv parity", () => { test("uses auth scoped external url and absolute mailer verify urls", () => { const env = buildGotrueEnv({ diff --git a/apps/cli/src/commands/start/testdata/config-precedence/start/no-overrides.json b/apps/cli/src/commands/start/testdata/config-precedence/start/no-overrides.json index f9db4d565d..2e5a1ca49d 100644 --- a/apps/cli/src/commands/start/testdata/config-precedence/start/no-overrides.json +++ b/apps/cli/src/commands/start/testdata/config-precedence/start/no-overrides.json @@ -99,6 +99,7 @@ "GOTRUE_SECURITY_MANUAL_LINKING_ENABLED": "false", "GOTRUE_SECURITY_REFRESH_TOKEN_REUSE_INTERVAL": "10", "GOTRUE_SECURITY_REFRESH_TOKEN_ROTATION_ENABLED": "true", + "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD": "false", "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION": "false", "GOTRUE_SITE_URL": "http://base.example.com", "GOTRUE_SMS_AUTOCONFIRM": "true", diff --git a/apps/cli/src/commands/start/testdata/config-precedence/start/project-dotenv.json b/apps/cli/src/commands/start/testdata/config-precedence/start/project-dotenv.json index cf0dcec1ff..9a3c6a98e2 100644 --- a/apps/cli/src/commands/start/testdata/config-precedence/start/project-dotenv.json +++ b/apps/cli/src/commands/start/testdata/config-precedence/start/project-dotenv.json @@ -96,6 +96,7 @@ "GOTRUE_SECURITY_MANUAL_LINKING_ENABLED": "false", "GOTRUE_SECURITY_REFRESH_TOKEN_REUSE_INTERVAL": "10", "GOTRUE_SECURITY_REFRESH_TOKEN_ROTATION_ENABLED": "true", + "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD": "false", "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION": "false", "GOTRUE_SITE_URL": "http://dotenv.example.com", "GOTRUE_SMS_AUTOCONFIRM": "true", diff --git a/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-ignored.json b/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-ignored.json index f9db4d565d..2e5a1ca49d 100644 --- a/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-ignored.json +++ b/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-ignored.json @@ -99,6 +99,7 @@ "GOTRUE_SECURITY_MANUAL_LINKING_ENABLED": "false", "GOTRUE_SECURITY_REFRESH_TOKEN_REUSE_INTERVAL": "10", "GOTRUE_SECURITY_REFRESH_TOKEN_ROTATION_ENABLED": "true", + "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD": "false", "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION": "false", "GOTRUE_SITE_URL": "http://base.example.com", "GOTRUE_SMS_AUTOCONFIRM": "true", diff --git a/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-with-shell-env.json b/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-with-shell-env.json index 8a46c6a9f0..99d2b30283 100644 --- a/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-with-shell-env.json +++ b/apps/cli/src/commands/start/testdata/config-precedence/start/remote-block-with-shell-env.json @@ -96,6 +96,7 @@ "GOTRUE_SECURITY_MANUAL_LINKING_ENABLED": "false", "GOTRUE_SECURITY_REFRESH_TOKEN_REUSE_INTERVAL": "10", "GOTRUE_SECURITY_REFRESH_TOKEN_ROTATION_ENABLED": "true", + "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD": "false", "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION": "false", "GOTRUE_SITE_URL": "http://shell.example.com", "GOTRUE_SMS_AUTOCONFIRM": "true", diff --git a/apps/cli/src/commands/start/testdata/config-precedence/start/shell-beats-dotenv.json b/apps/cli/src/commands/start/testdata/config-precedence/start/shell-beats-dotenv.json index 0ffd236e72..123262d12b 100644 --- a/apps/cli/src/commands/start/testdata/config-precedence/start/shell-beats-dotenv.json +++ b/apps/cli/src/commands/start/testdata/config-precedence/start/shell-beats-dotenv.json @@ -96,6 +96,7 @@ "GOTRUE_SECURITY_MANUAL_LINKING_ENABLED": "false", "GOTRUE_SECURITY_REFRESH_TOKEN_REUSE_INTERVAL": "10", "GOTRUE_SECURITY_REFRESH_TOKEN_ROTATION_ENABLED": "true", + "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD": "false", "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION": "false", "GOTRUE_SITE_URL": "http://shell.example.com", "GOTRUE_SMS_AUTOCONFIRM": "true", diff --git a/apps/cli/src/commands/start/testdata/config-precedence/start/shell-env.json b/apps/cli/src/commands/start/testdata/config-precedence/start/shell-env.json index 8a46c6a9f0..99d2b30283 100644 --- a/apps/cli/src/commands/start/testdata/config-precedence/start/shell-env.json +++ b/apps/cli/src/commands/start/testdata/config-precedence/start/shell-env.json @@ -96,6 +96,7 @@ "GOTRUE_SECURITY_MANUAL_LINKING_ENABLED": "false", "GOTRUE_SECURITY_REFRESH_TOKEN_REUSE_INTERVAL": "10", "GOTRUE_SECURITY_REFRESH_TOKEN_ROTATION_ENABLED": "true", + "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD": "false", "GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION": "false", "GOTRUE_SITE_URL": "http://shell.example.com", "GOTRUE_SMS_AUTOCONFIRM": "true", diff --git a/apps/cli/src/shared/init/project-init.templates.ts b/apps/cli/src/shared/init/project-init.templates.ts index 9a3a5e8f43..d0ca2992fc 100644 --- a/apps/cli/src/shared/init/project-init.templates.ts +++ b/apps/cli/src/shared/init/project-init.templates.ts @@ -231,6 +231,9 @@ double_confirm_changes = true enable_confirmations = false # If enabled, users will need to reauthenticate or have logged in recently to change their password. secure_password_change = false +# If enabled, users will need to provide their current password to change their password, unless +# they signed in with a one-time password, magic link, or recovery link. +# require_current_password = true # Controls the minimum amount of time that must pass before sending another signup confirmation or password reset email. max_frequency = "1s" # Number of characters used in the email OTP. diff --git a/apps/cli/src/shared/init/project-init.templates.unit.test.ts b/apps/cli/src/shared/init/project-init.templates.unit.test.ts index e4e199fc26..a5520e5e50 100644 --- a/apps/cli/src/shared/init/project-init.templates.unit.test.ts +++ b/apps/cli/src/shared/init/project-init.templates.unit.test.ts @@ -96,6 +96,12 @@ const NATIVE_SMS_SIGNUP_COMMENT = `# Allow/disallow the phone provider. When fal # To block new signups only, set enable_signup = false under [auth] instead. enabled = false`; +/* Only the native template documents `require_current_password`. */ +const NATIVE_REQUIRE_CURRENT_PASSWORD = `secure_password_change = false +# If enabled, users will need to provide their current password to change their password, unless +# they signed in with a one-time password, magic link, or recovery link. +# require_current_password = true`; + const renderExpectedNativeEject = renderExpectedInitOutput.pipe( Effect.map((eject) => eject @@ -107,6 +113,7 @@ const renderExpectedNativeEject = renderExpectedInitOutput.pipe( .replace(SCAFFOLD_SIGNUP_COMMENT, NATIVE_SIGNUP_COMMENT) .replace(SCAFFOLD_EMAIL_SIGNUP_COMMENT, NATIVE_EMAIL_SIGNUP_COMMENT) .replace(SCAFFOLD_SMS_SIGNUP_COMMENT, NATIVE_SMS_SIGNUP_COMMENT) + .replace("secure_password_change = false", NATIVE_REQUIRE_CURRENT_PASSWORD) .replace("rows returns from", "rows returned from") .replace("(SIWS, EIP-4361)", "(SIWS)") .replace( diff --git a/apps/cli/tests/helpers/config-fixtures.ts b/apps/cli/tests/helpers/config-fixtures.ts index b8605c097d..d619e7916e 100644 --- a/apps/cli/tests/helpers/config-fixtures.ts +++ b/apps/cli/tests/helpers/config-fixtures.ts @@ -64,6 +64,7 @@ export function v2ProjectConfigResponse( mailer_secure_email_change_enabled: true, mailer_autoconfirm: true, security_update_password_require_reauthentication: false, + security_update_password_require_current_password: false, mailer_otp_length: 6, mailer_otp_exp: 3600, smtp_max_frequency: 1, diff --git a/apps/docs/public/schemas/config/v1.json b/apps/docs/public/schemas/config/v1.json index 49d9416cf2..4a7fe39a3d 100644 --- a/apps/docs/public/schemas/config/v1.json +++ b/apps/docs/public/schemas/config/v1.json @@ -909,6 +909,19 @@ } ] }, + "require_current_password": { + "description": "If enabled, users will need to provide their current password to change their password, unless they signed in with a one-time password, magic link, or recovery link.", + "default": false, + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "string", + "pattern": "^env\\((.*)\\)$" + } + ] + }, "max_frequency": { "type": "string", "description": "Controls the minimum amount of time that must pass before sending another signup confirmation or password reset email.", @@ -5488,6 +5501,19 @@ } ] }, + "require_current_password": { + "description": "If enabled, users will need to provide their current password to change their password, unless they signed in with a one-time password, magic link, or recovery link.", + "default": false, + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "string", + "pattern": "^env\\((.*)\\)$" + } + ] + }, "max_frequency": { "type": "string", "description": "Controls the minimum amount of time that must pass before sending another signup confirmation or password reset email.", diff --git a/apps/docs/public/schemas/project-config/v1.json b/apps/docs/public/schemas/project-config/v1.json index ed0875f833..258c28fb1e 100644 --- a/apps/docs/public/schemas/project-config/v1.json +++ b/apps/docs/public/schemas/project-config/v1.json @@ -382,6 +382,11 @@ "description": "If enabled, users will need to reauthenticate or have logged in recently to change their password.", "default": false }, + "require_current_password": { + "type": "boolean", + "description": "If enabled, users will need to provide their current password to change their password, unless they signed in with a one-time password, magic link, or recovery link.", + "default": false + }, "max_frequency": { "type": "string", "description": "Controls the minimum amount of time that must pass before sending another signup confirmation or password reset email.", diff --git a/packages/config/src/auth/email.ts b/packages/config/src/auth/email.ts index 771cfbaec5..5cfc21498c 100644 --- a/packages/config/src/auth/email.ts +++ b/packages/config/src/auth/email.ts @@ -15,6 +15,7 @@ const defaultEnableSignup = true; const defaultDoubleConfirmChanges = true; const defaultEnableConfirmations = false; const defaultSecurePasswordChange = false; +const defaultRequireCurrentPassword = false; const defaultMaxFrequency = "1s"; const defaultOtpLength = 6; const defaultOtpExpiry = 3600; @@ -112,6 +113,13 @@ export const email = Schema.Struct({ tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultSecurePasswordChange))), + require_current_password: Schema.Boolean.annotate({ + default: defaultRequireCurrentPassword, + description: + "If enabled, users will need to provide their current password to change their password, unless they signed in with a one-time password, magic link, or recovery link.", + tags, + links: [links.auth], + }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultRequireCurrentPassword))), max_frequency: Schema.String.annotate({ default: defaultMaxFrequency, description: diff --git a/packages/config/src/project-config/registry-auth.ts b/packages/config/src/project-config/registry-auth.ts index d87785978c..f1f13cbc2b 100644 --- a/packages/config/src/project-config/registry-auth.ts +++ b/packages/config/src/project-config/registry-auth.ts @@ -601,6 +601,10 @@ const emailBaseRows: ReadonlyArray = [ ["auth", "email", "secure_password_change"], "security_update_password_require_reauthentication", ), + boolRow( + ["auth", "email", "require_current_password"], + "security_update_password_require_current_password", + ), uintRow(["auth", "email", "otp_length"], "mailer_otp_length"), uintRow(["auth", "email", "otp_expiry"], "mailer_otp_exp"), secondsDurationRow(["auth", "email", "max_frequency"], "smtp_max_frequency"), diff --git a/packages/stack/src/services/AuthSettings.ts b/packages/stack/src/services/AuthSettings.ts index 9bf72c2e0d..2f0a5b3f78 100644 --- a/packages/stack/src/services/AuthSettings.ts +++ b/packages/stack/src/services/AuthSettings.ts @@ -1,4 +1,4 @@ -import { Schema } from "effect"; +import { Effect, Schema } from "effect"; const Hook = Schema.Struct({ enabled: Schema.Boolean, @@ -62,6 +62,10 @@ const Email = Schema.Struct({ double_confirm_changes: Schema.Boolean, enable_confirmations: Schema.Boolean, secure_password_change: Schema.Boolean, + /* Stacks saved before this key omit it. */ + require_current_password: Schema.Boolean.pipe( + Schema.withDecodingDefaultKey(Effect.succeed(false)), + ), max_frequency: Schema.String, otp_length: Schema.Finite, otp_expiry: Schema.Finite, @@ -193,6 +197,7 @@ export const settingsEnvironment = ( put("GOTRUE_MAILER_SECURE_EMAIL_CHANGE_ENABLED", email.double_confirm_changes); put("GOTRUE_MAILER_AUTOCONFIRM", !email.enable_confirmations); put("GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_REAUTHENTICATION", email.secure_password_change); + put("GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD", email.require_current_password); put("GOTRUE_MAILER_OTP_LENGTH", email.otp_length); put("GOTRUE_MAILER_OTP_EXP", email.otp_expiry); put("GOTRUE_SMTP_MAX_FREQUENCY", email.max_frequency); diff --git a/packages/stack/src/services/AuthSettings.unit.test.ts b/packages/stack/src/services/AuthSettings.unit.test.ts index 66caf0f533..6714738d59 100644 --- a/packages/stack/src/services/AuthSettings.unit.test.ts +++ b/packages/stack/src/services/AuthSettings.unit.test.ts @@ -95,6 +95,7 @@ describe("Auth settings environment", () => { expect(env["GOTRUE_EXTERNAL_CUSTOM_REDIRECT_URI"]).toBe("https://issuer.example/callback"); expect(env["GOTRUE_EXTERNAL_CUSTOM_SECRET"]).toBe(""); expect(env["GOTRUE_PASSKEY_ENABLED"]).toBe("true"); + expect(env["GOTRUE_SECURITY_UPDATE_PASSWORD_REQUIRE_CURRENT_PASSWORD"]).toBe("false"); expect(env["GOTRUE_WEBAUTHN_RP_ORIGINS"]).toBe("https://example.test,https://app.example.test"); expect(env["GOTRUE_MAILER_NOTIFICATIONS_EMAIL_CHANGED_ENABLED"]).toBe("true"); expect(env["GOTRUE_MAILER_SUBJECTS_EMAIL_CHANGED_NOTIFICATION"]).toBe("changed"); From 00c8a5b055002444266738121a4ce3cfce26061b Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 3 Nov 2026 13:11:03 +0000 Subject: [PATCH 040/101] feat(cli): point init's config.toml at the hosted schema and trust it in VS Code (CLI-2870) (#7464) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary The config schema is served from `https://supabase.com/schemas/config/v1.json` now (supabase/supabase#52509), but nothing a fresh project gets points an editor at it. - `supabase init` writes `#:schema https://supabase.com/schemas/config/v1.json` as the first line of `config.toml`. Taplo and Tombi read that directive to pick the schema, and it pins the version for when a v2 comes along. The URL comes from `CLI_CONFIG_SCHEMA_URL`, so it moves with the published schema - The VS Code settings written by `init` (and `functions new`, which shares the helper) now include `"json.schemaDownload.trustedDomains": { "https://supabase.com/schemas/": true }`. VS Code only downloads `$schema` URLs from trusted domains and supabase.com isn't on its default list, so a JSON config whose `$schema` points at our schema shows "Location ... is untrusted" and gets no completion until the user trusts it. VS Code merges object settings across scopes, so this adds to its default list rather than replacing it, and the key is a prefix match that covers the config, project-config and lock schemas - Merging into an existing `.vscode/settings.json` now goes key by key through nested objects, with our values still winning on conflicts. Before, each top-level key replaced the whole value, so an existing `trustedDomains` map or `[typescript]` block lost the user's own entries. Arrays and scalars are still replaced like before Older CLIs read the `#:schema` line as a plain comment, so nothing changes for them. ## Linked issue Closes CLI-2870 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- .../commands/functions/new/SIDE_EFFECTS.md | 3 +- apps/cli/src/commands/init/SIDE_EFFECTS.md | 4 +- .../commands/init/init.integration.test.ts | 43 +++++++++++++++++++ .../src/shared/init/project-init.templates.ts | 7 ++- .../init/project-init.templates.unit.test.ts | 19 +++++++- apps/cli/src/shared/init/project-init.ts | 25 ++++++++--- .../init/testdata/templates/config.toml | 1 + .../templates/vscode-settings.json.golden | 3 ++ 8 files changed, 96 insertions(+), 9 deletions(-) diff --git a/apps/cli/src/commands/functions/new/SIDE_EFFECTS.md b/apps/cli/src/commands/functions/new/SIDE_EFFECTS.md index 4ca3f8e4d7..581c53fa95 100644 --- a/apps/cli/src/commands/functions/new/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/functions/new/SIDE_EFFECTS.md @@ -9,7 +9,7 @@ | `/supabase/config.toml` | TOML | best-effort when resolving template values, detecting existing `[functions.]` declarations, and scanning declared function slugs | | `/supabase/functions/*/index.ts` | TypeScript | when checking whether this is the first local function | | `/.vscode/extensions.json` | JSONC | when merging VS Code recommendations into an existing file | -| `/.vscode/settings.json` | JSONC | when merging Deno settings into an existing file | +| `/.vscode/settings.json` | JSONC | when merging Deno settings into an existing file (nested objects merge key by key, template values win) | | `/telemetry.json` | JSON | when present, before post-run telemetry state is refreshed | ## Files Written @@ -80,5 +80,6 @@ Emits a structured success result event with `path`, `function_name`, and `auth` - The `[functions.]` config section is **appended** (`O_APPEND` semantics, `flag: "a"`), never rewritten, so the existing file is left byte-for-byte untouched and a partial write cannot truncate it. - Existing-declaration detection scans the raw `config.toml` text (`^\s*\[functions\.\]\s*$`) rather than a parsed config map. For all well-formed configs the two approaches agree. - IDE settings scaffolding (`.vscode`, `.idea`) only runs in `--output-format text`; json / stream-json runs are payload-only. +- The VS Code settings come from the shared `init` helper, so `.vscode/settings.json` also includes `json.schemaDownload.trustedDomains` trusting `https://supabase.com/schemas/`. - No Management API requests are made; all behavior is local filesystem work plus telemetry flush. - A non-existent `--workdir`/`SUPABASE_WORKDIR` now fails before any directory or file is created (CLI-2285) — previously a typo'd `--workdir` could scaffold a fresh `supabase/functions/…` tree (plus a new `config.toml`) at the wrong path. diff --git a/apps/cli/src/commands/init/SIDE_EFFECTS.md b/apps/cli/src/commands/init/SIDE_EFFECTS.md index 9e09b1a070..28b0725419 100644 --- a/apps/cli/src/commands/init/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/init/SIDE_EFFECTS.md @@ -84,6 +84,7 @@ Try rerunning the command with --debug to troubleshoot the error. - Uses the invocation cwd directly and does not recurse upward looking for an existing project. - The `--force` flag overwrites an existing `supabase/config.toml`. +- `supabase/config.toml` starts with a `#:schema https://supabase.com/schemas/config/v1.json` directive so Taplo and Tombi validate it against the hosted schema. - The `--use-orioledb` flag writes the OrioleDB version to `db.orioledb_version`; it does not require `--experimental`. - `SUPABASE_EXPERIMENTAL_STACK=1` opts the new project into the experimental stack backend: the written config includes `[experimental] stack = true` and omits the Docker-era default ports @@ -92,6 +93,7 @@ Try rerunning the command with --debug to troubleshoot the error. established template. - The `--interactive` / `-i` flag enables IDE settings prompts (only effective in TTY). - The `--with-vscode-settings` and `--with-vscode-workspace` flags are hidden backward-compat aliases for the same VS Code helper and both write `.vscode/settings.json` and `.vscode/extensions.json`. +- `.vscode/settings.json` includes `json.schemaDownload.trustedDomains` trusting `https://supabase.com/schemas/`, so VS Code loads the hosted config schema without an untrusted-location warning. - The `--with-intellij-settings` flag is a hidden backward-compat alias for generating `.idea/deno.xml`. -- An existing `.vscode/settings.json` / `.vscode/extensions.json` is parsed tolerantly through a JSONC boundary that strips line/block comments and trailing commas, then the template is merged on top (template keys win). An empty file is treated as absent and the template is written verbatim. A non-empty file that is not valid JSON(C) aborts the command with `InitParseSettingsError` and is left untouched rather than being overwritten. +- An existing `.vscode/settings.json` / `.vscode/extensions.json` is parsed tolerantly through a JSONC boundary that strips line/block comments and trailing commas, then the template is merged on top: nested objects are merged key by key with template values winning on conflicts, and arrays and scalars from the template replace the existing value. An empty file is treated as absent and the template is written verbatim. A non-empty file that is not valid JSON(C) aborts the command with `InitParseSettingsError` and is left untouched rather than being overwritten. - No authentication required - purely local file creation. diff --git a/apps/cli/src/commands/init/init.integration.test.ts b/apps/cli/src/commands/init/init.integration.test.ts index 3c4ed5ed74..8a47a91fbe 100644 --- a/apps/cli/src/commands/init/init.integration.test.ts +++ b/apps/cli/src/commands/init/init.integration.test.ts @@ -33,6 +33,10 @@ const decodeOrioleDbConfig = Schema.decodeUnknownEffect( }), ); +const decodeJsonObject = Schema.decodeUnknownEffect( + Schema.fromJsonString(Schema.Record(Schema.String, Schema.Unknown)), +); + const readTextFile = (...segments: Array) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -142,6 +146,7 @@ describe("init", () => { }).pipe(Effect.provide(layer)); const content = yield* readTextFile(tempDir, "supabase", "config.toml"); + expect(content.split("\n")[0]).toBe("#:schema https://supabase.com/schemas/config/v1.json"); expect(content).toContain("major_version = 17"); expect(content).toContain("port = 54321"); expect(content).not.toContain("stack = true"); @@ -348,6 +353,44 @@ describe("init", () => { }); }); + it.effect("merges the VS Code settings into an existing file without losing user entries", () => { + const tempDir = tempRoot.current; + + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const settingsPath = path.join(tempDir, ".vscode", "settings.json"); + yield* fs.makeDirectory(path.dirname(settingsPath), { recursive: true }); + yield* fs.writeFileString( + settingsPath, + `{ + // user preferences + "editor.fontSize": 14, + "json.schemaDownload.trustedDomains": { "https://example.com/": true }, + "[typescript]": { "editor.tabSize": 4 }, +} +`, + ); + const { layer } = setup(tempDir); + + yield* init({ ...BASE_INIT_FLAGS, interactive: false, withVscodeSettings: true }).pipe( + Effect.provide(layer), + ); + + expect(yield* decodeJsonObject(yield* readTextFile(settingsPath))).toMatchObject({ + "editor.fontSize": 14, + "json.schemaDownload.trustedDomains": { + "https://example.com/": true, + "https://supabase.com/schemas/": true, + }, + "[typescript]": { + "editor.tabSize": 4, + "editor.defaultFormatter": "denoland.vscode-deno", + }, + }); + }).pipe(Effect.provide(BunServices.layer)); + }); + it.effect("writes the stack-opt-in template when SUPABASE_EXPERIMENTAL_STACK=1", () => { const tempDir = tempRoot.current; diff --git a/apps/cli/src/shared/init/project-init.templates.ts b/apps/cli/src/shared/init/project-init.templates.ts index d0ca2992fc..a11639e381 100644 --- a/apps/cli/src/shared/init/project-init.templates.ts +++ b/apps/cli/src/shared/init/project-init.templates.ts @@ -1,7 +1,9 @@ +import { CLI_CONFIG_SCHEMA_URL } from "@supabase/config"; import { orioledbVersions, postgresMajor } from "@supabase/stack/internal/artifacts"; import { postgresVersionCompare } from "../../command-internal/db-bootstrap/postgres.service.ts"; -const CONFIG_TEMPLATE_RAW = `# For detailed configuration reference documentation, visit: +const CONFIG_TEMPLATE_RAW = `#:schema ${CLI_CONFIG_SCHEMA_URL} +# For detailed configuration reference documentation, visit: # https://supabase.com/docs/guides/local-development/cli/config # A string used to distinguish different Supabase projects on the same host. Defaults to the # working directory name when running \`supabase init\`. @@ -465,6 +467,9 @@ export const VSCODE_SETTINGS_TEMPLATE = `{ ], "[typescript]": { "editor.defaultFormatter": "denoland.vscode-deno" + }, + "json.schemaDownload.trustedDomains": { + "https://supabase.com/schemas/": true } } `; diff --git a/apps/cli/src/shared/init/project-init.templates.unit.test.ts b/apps/cli/src/shared/init/project-init.templates.unit.test.ts index a5520e5e50..4570252e5c 100644 --- a/apps/cli/src/shared/init/project-init.templates.unit.test.ts +++ b/apps/cli/src/shared/init/project-init.templates.unit.test.ts @@ -1,6 +1,7 @@ import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Path } from "effect"; +import { Effect, FileSystem, Path, Predicate } from "effect"; +import { CLI_CONFIG_SCHEMA_URL, PROJECT_CONFIG_SCHEMA_URL } from "@supabase/config"; import { applyConfigEdits, loadCliConfig, type ConfigEdit } from "@supabase/config/internal"; import { orioledbVersions, postgresMajor } from "@supabase/stack/internal/artifacts"; import { postgresVersionCompare } from "../../command-internal/db-bootstrap/postgres.service.ts"; @@ -226,6 +227,22 @@ describe("project init templates", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it("trusts the hosted schema URLs in the VS Code settings", () => { + const settings: unknown = JSON.parse(VSCODE_SETTINGS_TEMPLATE); + const trustedDomains = Predicate.isObject(settings) + ? settings["json.schemaDownload.trustedDomains"] + : undefined; + const trusted = Predicate.isObject(trustedDomains) + ? Object.entries(trustedDomains) + .filter(([, enabled]) => enabled === true) + .map(([prefix]) => prefix) + : []; + + for (const url of [CLI_CONFIG_SCHEMA_URL, PROJECT_CONFIG_SCHEMA_URL]) { + expect(trusted.some((prefix) => url.startsWith(prefix))).toBe(true); + } + }); + it.effect("matches the IntelliJ scaffold", () => Effect.gen(function* () { expect(INTELLIJ_DENO_TEMPLATE).toBe(yield* readVendoredTemplate("idea-deno.xml")); diff --git a/apps/cli/src/shared/init/project-init.ts b/apps/cli/src/shared/init/project-init.ts index c707def433..224b5f8d11 100644 --- a/apps/cli/src/shared/init/project-init.ts +++ b/apps/cli/src/shared/init/project-init.ts @@ -1,4 +1,4 @@ -import { Effect, FileSystem, Path, Schema } from "effect"; +import { Effect, FileSystem, Path, Predicate, Schema } from "effect"; import { promptYesNo } from "../../command-internal/prompt-yes-no.ts"; import { sanitizeProjectId } from "../config/project-id.ts"; import { Output } from "../output/output.service.ts"; @@ -144,6 +144,21 @@ function writeJsonFile(pathname: string, contents: Record) { }); } +function mergeJsonObjects( + existing: Readonly>, + template: Readonly>, +): Record { + const merged: Record = { ...existing }; + for (const [key, value] of Object.entries(template)) { + const current = merged[key]; + merged[key] = + Predicate.isObject(current) && Predicate.isObject(value) + ? mergeJsonObjects(current, value) + : value; + } + return merged; +} + function updateJsonFile(pathname: string, template: string) { return Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -159,10 +174,10 @@ function updateJsonFile(pathname: string, template: string) { return; } - const merged = { - ...(yield* parseJsonObject(pathname, existing)), - ...(yield* parseJsonObject(pathname, template)), - }; + const merged = mergeJsonObjects( + yield* parseJsonObject(pathname, existing), + yield* parseJsonObject(pathname, template), + ); yield* writeJsonFile(pathname, merged); }); } diff --git a/apps/cli/src/shared/init/testdata/templates/config.toml b/apps/cli/src/shared/init/testdata/templates/config.toml index 9862194971..560346d2bd 100644 --- a/apps/cli/src/shared/init/testdata/templates/config.toml +++ b/apps/cli/src/shared/init/testdata/templates/config.toml @@ -1,3 +1,4 @@ +#:schema https://supabase.com/schemas/config/v1.json # For detailed configuration reference documentation, visit: # https://supabase.com/docs/guides/local-development/cli/config # A string used to distinguish different Supabase projects on the same host. Defaults to the diff --git a/apps/cli/src/shared/init/testdata/templates/vscode-settings.json.golden b/apps/cli/src/shared/init/testdata/templates/vscode-settings.json.golden index af62c23f88..d527025809 100644 --- a/apps/cli/src/shared/init/testdata/templates/vscode-settings.json.golden +++ b/apps/cli/src/shared/init/testdata/templates/vscode-settings.json.golden @@ -20,5 +20,8 @@ ], "[typescript]": { "editor.defaultFormatter": "denoland.vscode-deno" + }, + "json.schemaDownload.trustedDomains": { + "https://supabase.com/schemas/": true } } From ffc2d556b43758edfea19f1226f830cbca9714d7 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 13:21:24 +0000 Subject: [PATCH 041/101] chore(api): sync Management API OpenAPI spec (#7466) This PR was automatically created to sync the generated `@supabase/api` package with the latest Management API OpenAPI document. Changes were detected in the upstream OpenAPI documents exposed by `https://api.supabase.com/api/v1-json` and `https://api.supabase.com/api/v2-json`. Co-authored-by: jgoux <1443499+jgoux@users.noreply.github.com> --- packages/api/src/generated/contracts.ts | 10 ++++++++++ packages/api/src/generated/openapi.json | 18 ++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/packages/api/src/generated/contracts.ts b/packages/api/src/generated/contracts.ts index 62cd187ef5..3cda8dd741 100644 --- a/packages/api/src/generated/contracts.ts +++ b/packages/api/src/generated/contracts.ts @@ -199,6 +199,7 @@ export const BranchResponse_Output = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -309,6 +310,7 @@ export const V1ProjectWithDatabaseResponse_Output = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -1094,6 +1096,7 @@ export const V1CreateABranchOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -1352,6 +1355,7 @@ export const V1CreateAProjectOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -2530,6 +2534,7 @@ export const V1GetABranchOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -2584,6 +2589,7 @@ export const V1GetABranchConfigOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -2956,6 +2962,7 @@ export const V1GetAllProjectsForOrganizationOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -5004,6 +5011,7 @@ export const V1GetProjectOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -7681,6 +7689,7 @@ export const V1UpdateABranchConfigOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]), OpenString, ]), @@ -13953,6 +13962,7 @@ export const V2ListOrganizationProjectsOutput = Schema.Struct({ "RESTARTING", "PAUSE_FAILED", "RESIZING", + "PROMOTING", ]).annotate({ description: "Project status" }), OpenString, ]), diff --git a/packages/api/src/generated/openapi.json b/packages/api/src/generated/openapi.json index 42344c6444..15b13f4053 100644 --- a/packages/api/src/generated/openapi.json +++ b/packages/api/src/generated/openapi.json @@ -31888,7 +31888,8 @@ "RESTORE_FAILED", "RESTARTING", "PAUSE_FAILED", - "RESIZING" + "RESIZING", + "PROMOTING" ] }, "db_host": { @@ -32059,7 +32060,8 @@ "RESTORE_FAILED", "RESTARTING", "PAUSE_FAILED", - "RESIZING" + "RESIZING", + "PROMOTING" ] } }, @@ -32176,7 +32178,8 @@ "RESTORE_FAILED", "RESTARTING", "PAUSE_FAILED", - "RESIZING" + "RESIZING", + "PROMOTING" ] }, "database": { @@ -32437,7 +32440,8 @@ "RESTORE_FAILED", "RESTARTING", "PAUSE_FAILED", - "RESIZING" + "RESIZING", + "PROMOTING" ] } }, @@ -41479,7 +41483,8 @@ "RESTORE_FAILED", "RESTARTING", "PAUSE_FAILED", - "RESIZING" + "RESIZING", + "PROMOTING" ] }, "inserted_at": { @@ -45877,7 +45882,8 @@ "RESTORE_FAILED", "RESTARTING", "PAUSE_FAILED", - "RESIZING" + "RESIZING", + "PROMOTING" ], "description": "Project status" }, From 41723cea14b0faada998c5732552e99078a55262 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 3 Nov 2026 13:23:35 +0000 Subject: [PATCH 042/101] fix(cli): treat edge runtime exit 15 as a stop in functions serve (CLI-2908) (#7463) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Running `supabase stop` (or `docker stop` on the container) while `functions serve` is up makes serve fail with `error running container supabase_edge_runtime_: exit 15` and exit 1, as if the edge runtime had crashed. Edge runtime exits with the raw signal number on SIGTERM, so a stopped container reports exit 15, not the 143 a shell would give. Serve only treated 129, 130, 131 and 143 as the container being stopped from outside, so exit 15 fell through to `EdgeRuntimeContainerCrashedError`. `supabase stop` stops every container at once and only removes them after they've all stopped, so serve's follow-up inspect nearly always finds the edge runtime container still there with exit 15. - Exit 15 now counts as a stop. Serve prints `Edge Runtime container stopped (exit 15). Stopped serving supabase/functions` and exits 0. - Serve's `cli_command_executed` event gets two new properties, so we can see how sessions end and split real crashes by exit code: - `serve_end_reason`: `signal`, `container_stopped`, `container_exited`, `container_removed` or `container_crashed` - `edge_runtime_exit_code`: the container's exit code, when there is one - The same values are annotated on the serve trace as `functions.serve.end_reason` and `container.exit_code`. - `SIDE_EFFECTS.md` and `docs/analytics.md` cover both. ## Linked issue CLI-2908 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- apps/cli/docs/analytics.md | 9 + .../commands/functions/serve/SIDE_EFFECTS.md | 40 +-- .../functions/serve/serve.integration.test.ts | 324 +++++++++++++++--- apps/cli/src/shared/functions/serve.errors.ts | 2 + apps/cli/src/shared/functions/serve.ts | 64 +++- .../cli/src/shared/telemetry/event-catalog.ts | 4 + .../telemetry/command-telemetry-attributes.ts | 10 +- 7 files changed, 382 insertions(+), 71 deletions(-) diff --git a/apps/cli/docs/analytics.md b/apps/cli/docs/analytics.md index fe808b4684..a557223755 100644 --- a/apps/cli/docs/analytics.md +++ b/apps/cli/docs/analytics.md @@ -101,6 +101,15 @@ Local-runtime commands also carry properties that split error rates by runtime: - `orioledb` (boolean) once the command resolves local database config, `true` when the project selects an OrioleDB image (`db.orioledb_version` on a 15/17 project). It describes local config only; for `--remote` targets it says nothing about the remote database +- `serve_end_reason` on the Docker-runtime `functions serve`, set once its final outcome is known: + `signal` (the CLI received a shutdown signal), `container_stopped` (an external supervisor + stopped the Edge Runtime container, e.g. `supabase stop`), `container_exited` (the container + exited on its own with code 0), `container_removed` (the container disappeared), or + `container_crashed` (the container exited with a crash code, which also fails the command). A + restart records nothing; only the final end does +- `edge_runtime_exit_code` (number) alongside `serve_end_reason` when the container's exit code is + known: its stop code for `container_stopped`, `0` for `container_exited`, its crash code for + `container_crashed` Flag capture is intentionally conservative: `flags` records only flags changed on the command line, and redacts each value unless it is boolean, a choice flag, or listed in the command's diff --git a/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md b/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md index 7ca04c2328..a3b71f0f8b 100644 --- a/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md @@ -62,27 +62,27 @@ back to local keys. No scheme/host validation is performed on the discovered URL ## Exit Codes -| Code | Condition | -| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `0` | clean shutdown after `SIGINT` or `SIGTERM` | -| `0` | the edge-runtime container stops on its own with exit code `0` | -| `0` | the edge-runtime container is torn down by an external supervisor — exit `129`/`130`/`131`/`143` (`SIGHUP`/`SIGINT`/`SIGQUIT`/`SIGTERM`), e.g. `supabase stop` run in another terminal | -| `0` | the edge-runtime container is already gone by the time a follow-up `docker container inspect` runs after the log stream ended | -| `0` | the edge-runtime container is force-killed (exit `137`, no `State.OOMKilled`) and pruned before the re-inspect that follows a non-OOM `137` completes, e.g. by `supabase stop` | -| `0` | an edge-runtime startup failure or log-stream failure lands within the shutdown grace period (~50ms) of a `SIGINT`/`SIGTERM` | -| `1` | local DB container is not running, or the Docker daemon is unreachable (surfaces from the DB inspect as `failed to inspect service: …` plus the Docker Desktop install suggestion) | -| `1` | invalid inspect flag combination, or a `Config.Validate` failure anywhere in `config.toml` (not just project/auth config) | -| `1` | an enabled function has `import_map.json` but no `deno.json*` or configured import map — startup fails before the container is created | -| `1` | env file, signing key, import map, or function bind resolution failure | -| `1` | edge-runtime container startup, log streaming, or restart loop failure — including the edge-runtime container crashing with any exit code other than `0` or `129`/`130`/`131`/`143` | -| `1` | the edge-runtime container is killed for exceeding its memory limit — `State.OOMKilled`, typically with exit `137` | -| `1` | the edge-runtime container is killed from outside the CLI and is still present after a re-inspect — exit `137` without `State.OOMKilled` | +| Code | Condition | +| ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `0` | clean shutdown after `SIGINT` or `SIGTERM` | +| `0` | the edge-runtime container stops on its own with exit code `0` | +| `0` | the edge-runtime container is torn down by an external supervisor — exit `129`/`130`/`131`/`143` (`SIGHUP`/`SIGINT`/`SIGQUIT`/`SIGTERM`) or `15` (edge-runtime's own `SIGTERM` exit), e.g. `supabase stop` run in another terminal | +| `0` | the edge-runtime container is already gone by the time a follow-up `docker container inspect` runs after the log stream ended | +| `0` | the edge-runtime container is force-killed (exit `137`, no `State.OOMKilled`) and pruned before the re-inspect that follows a non-OOM `137` completes, e.g. by `supabase stop` | +| `0` | an edge-runtime startup failure or log-stream failure lands within the shutdown grace period (~50ms) of a `SIGINT`/`SIGTERM` | +| `1` | local DB container is not running, or the Docker daemon is unreachable (surfaces from the DB inspect as `failed to inspect service: …` plus the Docker Desktop install suggestion) | +| `1` | invalid inspect flag combination, or a `Config.Validate` failure anywhere in `config.toml` (not just project/auth config) | +| `1` | an enabled function has `import_map.json` but no `deno.json*` or configured import map — startup fails before the container is created | +| `1` | env file, signing key, import map, or function bind resolution failure | +| `1` | edge-runtime container startup, log streaming, or restart loop failure — including the edge-runtime container crashing with any exit code other than `0`, `15`, or `129`/`130`/`131`/`143` | +| `1` | the edge-runtime container is killed for exceeding its memory limit — `State.OOMKilled`, typically with exit `137` | +| `1` | the edge-runtime container is killed from outside the CLI and is still present after a re-inspect — exit `137` without `State.OOMKilled` | ## Telemetry Events Fired -| Event | When | Notable properties / groups | -| ---------------------- | ------------------------------------------ | ----------------------------------- | -| `cli_command_executed` | post-run, success or failure (via wrapper) | `exit_code`, `duration_ms`, `flags` | +| Event | When | Notable properties / groups | +| ---------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- | +| `cli_command_executed` | post-run, success or failure (via wrapper) | `exit_code`, `duration_ms`, `flags`, `serve_end_reason`, `edge_runtime_exit_code` | ## Output @@ -97,7 +97,7 @@ Writes lifecycle text to stderr / stdout while the command is running: - in stack mode, `--- chunk(s) lost ---` on stderr when the stack could not persist Functions output - `Stopped serving supabase/functions` on a user-initiated shutdown (`SIGINT`/`SIGTERM`) - `Edge Runtime exited (code 0). Stopped serving supabase/functions` when the container stops on its own with exit code `0` -- `Edge Runtime container stopped (exit ). Stopped serving supabase/functions` when an external supervisor tears the container down (exit `129`/`130`/`131`/`143`) +- `Edge Runtime container stopped (exit ). Stopped serving supabase/functions` when an external supervisor tears the container down (exit `129`/`130`/`131`/`143`, or `15` from edge-runtime's own `SIGTERM` exit) - `Edge Runtime container is no longer available. Stopped serving supabase/functions` when the container is already gone by the time a follow-up inspect runs ### `--output-format json` @@ -156,7 +156,7 @@ per-function `import_map.json` is not read and does not fail the command. - Config, project dotenv discovery, and function discovery all resolve from `` with no ancestor search, so they can never disagree. - Before each container (re)start, resolves the edge-runtime image through the same registry-candidate pull-with-retry every native `functions` Docker path uses: `docker image inspect ` (ECR, then GHCR, then Docker Hub) to check the local cache, then `docker pull ` with 2 retries (4s/8s backoff) on a miss, after `assertLocalDbRunning` — resolving it earlier would hijack the down-daemon error message that DB-inspect step is responsible for producing. - Runs the full `Config.Validate` pipeline (`resolveLocalConfigValues`, same one `start`/`stop`/`status` use) on every startup/restart, before `assertLocalDbRunning` — an invalid config now fails `serve` up front even for fields this command never otherwise reads (e.g. a bad `db.major_version` or malformed auth hook). -- A container that stops on its own with exit code `0`, or that is torn down by an external supervisor (exit `129`/`130`/`131`/`143`, e.g. `supabase stop` in another terminal), or that is already gone by the time a follow-up inspect runs, all end the command successfully — each prints its own distinct line (see Output above) rather than the user-initiated `Stopped serving …` line, so scrollback can tell "I stopped it" from "the runtime walked out" or "a supervisor tore it down". In a `functions serve &` CI step this means a runtime that exits on its own does not fail the step; the distinct message is the only signal, and a downstream failure otherwise only surfaces later as connection-refused. Exit `137` (SIGKILL) fails the command: `State.OOMKilled` separates a container that hit its memory limit, which reports the memory-allocation remediation, from one killed by something the CLI cannot identify. A non-OOM `137` gets one re-inspect first — if `supabase stop` force-killed a runtime that ignored `SIGTERM` and the follow-up prune has already removed the container, the command still ends successfully instead of failing on a container that is already gone. Any other non-zero container exit fails the command; the error message includes the container id. Only a watched-file change restarts the container itself — none of these outcomes ever restart it. +- A container that stops on its own with exit code `0`, or that is torn down by an external supervisor (exit `129`/`130`/`131`/`143`, or `15` from edge-runtime's own `SIGTERM` exit, e.g. `supabase stop` or `docker stop` in another terminal), or that is already gone by the time a follow-up inspect runs, all end the command successfully — each prints its own distinct line (see Output above) rather than the user-initiated `Stopped serving …` line, so scrollback can tell "I stopped it" from "the runtime walked out" or "a supervisor tore it down". In a `functions serve &` CI step this means a runtime that exits on its own does not fail the step; the distinct message is the only signal, and a downstream failure otherwise only surfaces later as connection-refused. Exit `137` (SIGKILL) fails the command: `State.OOMKilled` separates a container that hit its memory limit, which reports the memory-allocation remediation, from one killed by something the CLI cannot identify. A non-OOM `137` gets one re-inspect first — if `supabase stop` force-killed a runtime that ignored `SIGTERM` and the follow-up prune has already removed the container, the command still ends successfully instead of failing on a container that is already gone. Any other non-zero container exit fails the command; the error message includes the container id. Only a watched-file change restarts the container itself — none of these outcomes ever restart it. - A `docker logs -f` re-attach (the daemon can close the stream while the container keeps running) resumes with `--since ` instead of replaying the full log history, and is capped at 5 consecutive re-attaches that forward no new output; exceeding the cap fails the command with a tagged error instead of looping forever. - On the log-stream path, the Docker-daemon-unreachable classification comes from a follow-up `docker container inspect` failure, not from `docker logs -f`'s own stderr text. That same follow-up inspect's `State.OOMKilled` also drives the memory-limit remediation on this path, so a `docker logs -f` failure over a container killed for exceeding its memory limit reports the same suggestion as the clean-exit case above, instead of a bare unattributable failure. - The worker bootstrap template (`serve.main.ts`) is bundled into a single self-contained module with `jose` and the local path/status helpers inlined, so the edge-runtime worker boots without any network access (supabase/supabase#45570). The bundle is embedded at build time for shipped binaries and produced on demand (esbuild) when running from source. It is delivered into the created (not yet started) container as a `docker cp` stdin tar archive at `/root/index.ts` — never a single-file host bind mount, which materializes as an empty directory on daemons that cannot see the client's filesystem (remote `DOCKER_HOST`/Docker-context daemons, podman machines) and breaks bring-up with edge-runtime's "failed to determine entrypoint" (supabase/cli#6254). Only this bootstrap template is daemon-independent: user function sources, import maps, static files, and the multiline-env script directory (present only when an env value contains a newline) still arrive by host bind mounts, so they require a daemon that can see the project directory. diff --git a/apps/cli/src/commands/functions/serve/serve.integration.test.ts b/apps/cli/src/commands/functions/serve/serve.integration.test.ts index 1b1019f1b8..007bf50bfa 100644 --- a/apps/cli/src/commands/functions/serve/serve.integration.test.ts +++ b/apps/cli/src/commands/functions/serve/serve.integration.test.ts @@ -6,6 +6,7 @@ import { describe, expect, it } from "@effect/vitest"; import { CliConfigParseError } from "@supabase/config"; import { Cause, + Clock, Deferred, Duration, Effect, @@ -16,11 +17,13 @@ import { Predicate, PubSub, Queue, + Ref, Schema, Sink, Stream, } from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; +import { TestClock } from "effect/testing"; import { beforeEach, vi } from "vitest"; import { @@ -52,6 +55,10 @@ import { RuntimeInfo } from "../../../shared/runtime/runtime-info.service.ts"; import { dockerfileServiceImage } from "../../../shared/services/dockerfile-images.ts"; import { slimImagesEnabled } from "../../../shared/services/slim-images.ts"; import { getRegistryImageUrl } from "../../../command-internal/docker-registry.ts"; +import { + CommandTelemetryAttributes, + type CommandTelemetryAttributeValues, +} from "../../../telemetry/command-telemetry-attributes.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { DockerLogsStreamError, @@ -295,9 +302,30 @@ function mockQueuedProcessControl() { }; } -function mockFileWatcher(expectedPaths: ReadonlyArray = []) { - const pubsub = Effect.runSync(PubSub.unbounded>({ replay: 8 })); +function mockCommandTelemetry() { + const records = Ref.makeUnsafe>([]); + + return { + provide: Effect.provideService(CommandTelemetryAttributes, { + record: (values: CommandTelemetryAttributeValues) => + Ref.update(records, (current) => [...current, values]), + }), + recorded: Ref.get(records), + }; +} + +// `replay: false` delivers an event only to streams already subscribed, so a restart's +// re-subscription cannot see it again. +function mockFileWatcher( + expectedPaths: ReadonlyArray = [], + { replay = true }: { readonly replay?: boolean } = {}, +) { + const pubsub = Effect.runSync( + PubSub.unbounded>({ replay: replay ? 8 : 0 }), + ); const expectedWatch = Deferred.makeUnsafe(); + const expectedSubscription = Deferred.makeUnsafe(); + const subscribedPaths = new Set(); const watchCalls: Array<{ path: string; ignore?: ReadonlyArray; @@ -321,7 +349,19 @@ function mockFileWatcher(expectedPaths: ReadonlyArray = []) { ) { Deferred.doneUnsafe(expectedWatch, Exit.void); } - return Stream.fromPubSub(pubsub); + return Stream.unwrap( + PubSub.subscribe(pubsub).pipe( + Effect.tap(() => + Effect.sync(() => { + subscribedPaths.add(path); + if (expectedPaths.every((expectedPath) => subscribedPaths.has(expectedPath))) { + Deferred.doneUnsafe(expectedSubscription, Exit.void); + } + }), + ), + Effect.map(Stream.fromSubscription), + ), + ); }, }), ), @@ -332,6 +372,7 @@ function mockFileWatcher(expectedPaths: ReadonlyArray = []) { return watchCalls; }, awaitExpectedWatch: Deferred.await(expectedWatch), + awaitExpectedSubscription: Deferred.await(expectedSubscription), }; } @@ -489,6 +530,42 @@ function serveWithTimers(flags: FunctionsServeFlags, timers: FunctionsServeTimer }); } +// Opens when serve starts a sleep of `period`, so a test acts inside that window instead of +// racing it. +function sleepProbe(period: Duration.Duration, window: string) { + const opened = Deferred.makeUnsafe(); + + return { + period, + provide: (effect: Effect.Effect) => + Clock.clockWith((clock) => + Effect.provideService(effect, Clock.Clock, { + currentTimeMillisUnsafe: () => clock.currentTimeMillisUnsafe(), + currentTimeMillis: clock.currentTimeMillis, + currentTimeNanosUnsafe: () => clock.currentTimeNanosUnsafe(), + currentTimeNanos: clock.currentTimeNanos, + monotonicTimeNanosUnsafe: () => clock.monotonicTimeNanosUnsafe(), + monotonicTimeNanos: clock.monotonicTimeNanos, + sleep: (duration) => + Duration.equals(duration, period) + ? Deferred.succeed(opened, undefined).pipe(Effect.andThen(clock.sleep(duration))) + : clock.sleep(duration), + }), + ), + awaitOpened: Effect.fnUntraced(function* (serving: Fiber.Fiber) { + const didOpen = yield* Effect.raceFirst( + Deferred.await(opened).pipe(Effect.as(true)), + Fiber.await(serving).pipe(Effect.as(false)), + ); + expect(didOpen, `serve ended before the ${window} opened`).toBe(true); + }), + }; +} + +function shutdownGraceProbe() { + return sleepProbe(Duration.millis(2_345), "shutdown grace window"); +} + const writeCliConfig = Effect.fnUntraced(function* (content: string) { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; @@ -2023,6 +2100,7 @@ describe("functions serve integration", () => { it.effect("stops serving cleanly on a process signal", () => { const processControl = mockQueuedProcessControl(); const childSpawner = mockDockerLogSpawner([{ pending: true }]); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { const created = yield* Deferred.make(); @@ -2057,6 +2135,7 @@ describe("functions serve integration", () => { const { layer, out } = setupServe({ processControl, childSpawner }); const fiber = yield* functionsServe(baseFlags()).pipe( Effect.provide(layer), + telemetry.provide, Effect.forkChild({ startImmediately: true }), ); @@ -2071,6 +2150,7 @@ describe("functions serve integration", () => { .replaceAll("\u001b[22m", "") .replaceAll("\\", "/"), ).toContain("Stopped serving supabase/functions\n"); + expect(yield* telemetry.recorded).toStrictEqual([{ serve_end_reason: "signal" }]); }); }); @@ -2305,30 +2385,25 @@ describe("functions serve integration", () => { () => { deployMockState.runHandler = baseDockerRunHandler(); const processControl = mockQueuedProcessControl(); + const grace = shutdownGraceProbe(); + const childSpawner = mockDockerLogSpawner([ + { exitCode: 1, stderr: "docker logs killed by signal" }, + ]); return Effect.gen(function* () { - const scope = yield* Effect.scope; - // Delays the signal past the log-stream failure so only the grace window, not a - // same-tick race, can produce a clean exit. A generous injected grace period keeps - // this margin independent of the real clock. - const childSpawner = mockDockerLogSpawner([ - { - exitCode: 1, - stderr: "docker logs killed by signal", - onSpawn: () => - Effect.sleep(Duration.millis(15)).pipe( - Effect.andThen(Effect.sync(() => processControl.signal("SIGINT"))), - Effect.forkIn(scope), - Effect.asVoid, - ), - }, - ]); yield* writeHelloFunction; const { layer, out } = setupServe({ processControl, childSpawner }); const fiber = yield* serveWithTimers(baseFlags(), { - shutdownSignalGracePeriod: Duration.seconds(2), - }).pipe(Effect.provide(layer), Effect.forkChild({ startImmediately: true })); + shutdownSignalGracePeriod: grace.period, + }).pipe( + Effect.provide(layer), + grace.provide, + Effect.forkChild({ startImmediately: true }), + ); + + yield* grace.awaitOpened(fiber); + processControl.signal("SIGINT"); const exit = yield* Fiber.await(fiber); expect(Exit.isSuccess(exit)).toBe(true); @@ -2346,26 +2421,20 @@ describe("functions serve integration", () => { "downgrades a startup failure to a clean shutdown when the signal lands within the grace window", () => { const processControl = mockQueuedProcessControl(); + const telemetry = mockCommandTelemetry(); + const grace = shutdownGraceProbe(); return Effect.gen(function* () { - const scope = yield* Effect.scope; - // Delays the signal past the startup failure so only the grace window, not a - // same-tick race, can produce a clean exit. A generous injected grace period keeps - // this margin independent of the real clock. deployMockState.runHandler = (command, args) => { if (command !== "docker") { throw new Error(`unexpected process: ${command}`); } if (args[0] === "container" && args[1] === "inspect") { - return Effect.sleep(Duration.millis(15)).pipe( - Effect.andThen(Effect.sync(() => processControl.signal("SIGINT"))), - Effect.forkIn(scope), - Effect.as({ - exitCode: 1, - stdout: "", - stderr: "Error: No such container: supabase_db_test-project", - }), - ); + return { + exitCode: 1, + stdout: "", + stderr: "Error: No such container: supabase_db_test-project", + }; } throw new Error(`unexpected docker args: ${args.join(" ")}`); }; @@ -2373,8 +2442,16 @@ describe("functions serve integration", () => { const { layer, out } = setupServe({ processControl }); const fiber = yield* serveWithTimers(baseFlags(), { - shutdownSignalGracePeriod: Duration.seconds(2), - }).pipe(Effect.provide(layer), Effect.forkChild({ startImmediately: true })); + shutdownSignalGracePeriod: grace.period, + }).pipe( + Effect.provide(layer), + telemetry.provide, + grace.provide, + Effect.forkChild({ startImmediately: true }), + ); + + yield* grace.awaitOpened(fiber); + processControl.signal("SIGINT"); const exit = yield* Fiber.await(fiber); expect(Exit.isSuccess(exit)).toBe(true); @@ -2384,6 +2461,7 @@ describe("functions serve integration", () => { .replaceAll("\u001b[22m", "") .replaceAll("\\", "/"), ).toContain("Stopped serving supabase/functions\n"); + expect(yield* telemetry.recorded).toStrictEqual([{ serve_end_reason: "signal" }]); }); }, ); @@ -2398,12 +2476,17 @@ describe("functions serve integration", () => { { exitCode: 0 }, inspectStateBehavior(false, 1), ]); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { yield* writeHelloFunction; const { layer } = setupServe({ childSpawner }); - const error = yield* functionsServe(baseFlags()).pipe(Effect.provide(layer), Effect.flip); + const error = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + Effect.flip, + ); expect(error).toBeInstanceOf(EdgeRuntimeContainerCrashedError); if (error instanceof EdgeRuntimeContainerCrashedError) { @@ -2414,28 +2497,75 @@ describe("functions serve integration", () => { // and specifically not `unknown`. expect(error[ErrorActionabilityId]).toEqual(actionability.runtimeCrash); } + expect(yield* telemetry.recorded).toStrictEqual([ + { serve_end_reason: "container_crashed", edge_runtime_exit_code: 1 }, + ]); }); }, ); - it.effect( - "ends the session successfully, with a distinct message, when a supervisor tears the container down (exit 143)", - () => { + it.effect.each([143, 15])( + "ends the session successfully, with a distinct message, when a supervisor tears the container down (exit %d)", + (exitCode) => { deployMockState.runHandler = baseDockerRunHandler(); const childSpawner = mockDockerLogSpawner([ { exitCode: 0 }, - inspectStateBehavior(false, 143), + inspectStateBehavior(false, exitCode), ]); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { yield* writeHelloFunction; const { layer, out } = setupServe({ childSpawner }); - const exit = yield* functionsServe(baseFlags()).pipe(Effect.provide(layer), Effect.exit); + const exit = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + Effect.exit, + ); expect(Exit.isSuccess(exit)).toBe(true); - expect(out.stdoutText).toContain("Edge Runtime container stopped (exit 143)."); + expect(out.stdoutText).toContain(`Edge Runtime container stopped (exit ${exitCode}).`); expect(out.stdoutText).toContain("Stopped serving"); + expect(yield* telemetry.recorded).toStrictEqual([ + { serve_end_reason: "container_stopped", edge_runtime_exit_code: exitCode }, + ]); + }); + }, + ); + + it.live( + "records the shutdown signal, not a crash, when a signal lands within the grace window of a container crash", + () => { + deployMockState.runHandler = baseDockerRunHandler(); + const processControl = mockQueuedProcessControl(); + const telemetry = mockCommandTelemetry(); + const grace = shutdownGraceProbe(); + const childSpawner = mockDockerLogSpawner([ + { exitCode: 0 }, + inspectStateBehavior(false, 1), + ]); + + return Effect.gen(function* () { + yield* writeHelloFunction; + + const { layer } = setupServe({ processControl, childSpawner }); + const fiber = yield* serveWithTimers(baseFlags(), { + shutdownSignalGracePeriod: grace.period, + }).pipe( + Effect.provide(layer), + telemetry.provide, + grace.provide, + Effect.forkChild({ startImmediately: true }), + ); + + yield* grace.awaitOpened(fiber); + processControl.signal("SIGINT"); + + const exit = yield* Fiber.await(fiber); + expect(Exit.isSuccess(exit)).toBe(true); + expect(containerInspectCalls(childSpawner)).toHaveLength(1); + expect(yield* telemetry.recorded).toStrictEqual([{ serve_end_reason: "signal" }]); }); }, ); @@ -2570,16 +2700,24 @@ describe("functions serve integration", () => { { exitCode: 0 }, inspectStateBehavior(false, 0), ]); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { yield* writeHelloFunction; const { layer, out } = setupServe({ childSpawner }); - const exit = yield* functionsServe(baseFlags()).pipe(Effect.provide(layer), Effect.exit); + const exit = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + Effect.exit, + ); expect(Exit.isSuccess(exit)).toBe(true); expect(out.stdoutText).toContain("Edge Runtime exited (code 0)."); expect(out.stdoutText).toContain("Stopped serving"); + expect(yield* telemetry.recorded).toStrictEqual([ + { serve_end_reason: "container_exited", edge_runtime_exit_code: 0 }, + ]); }); }, ); @@ -2596,20 +2734,72 @@ describe("functions serve integration", () => { "Error response from daemon: No such container: supabase_edge_runtime_test-project", }, ]); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { yield* writeHelloFunction; const { layer, out } = setupServe({ childSpawner }); - const exit = yield* functionsServe(baseFlags()).pipe(Effect.provide(layer), Effect.exit); + const exit = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + Effect.exit, + ); expect(Exit.isSuccess(exit)).toBe(true); expect(out.stdoutText).toContain("Edge Runtime container is no longer available."); expect(out.stdoutText).toContain("Stopped serving"); + expect(yield* telemetry.recorded).toStrictEqual([ + { serve_end_reason: "container_removed" }, + ]); }); }, ); + it.effect("records only the final end, not the watched-file restart before it", () => { + deployMockState.runHandler = baseDockerRunHandler(); + const childSpawner = mockDockerLogSpawner([ + { pending: true }, + { exitCode: 0 }, + inspectStateBehavior(false, 15), + ]); + const telemetry = mockCommandTelemetry(); + const restartDebounce = sleepProbe(Duration.millis(500), "restart debounce window"); + + return Effect.gen(function* () { + const path = yield* Path.Path; + yield* writeHelloFunction; + const functionsDir = path.join(tempRoot.current, "supabase", "functions"); + const fileWatcher = mockFileWatcher([functionsDir], { replay: false }); + + const { layer, out } = setupServe({ fileWatcher, childSpawner }); + const fiber = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + restartDebounce.provide, + Effect.forkChild({ startImmediately: true }), + ); + + yield* fileWatcher.awaitExpectedSubscription; + fileWatcher.emit([{ path: path.join(functionsDir, "hello", "index.ts"), type: "update" }]); + yield* restartDebounce.awaitOpened(fiber); + yield* TestClock.adjust(Duration.millis(500)); + + const exit = yield* Fiber.await(fiber); + expect(Exit.isSuccess(exit)).toBe(true); + expect( + deployMockState.runCalls.filter( + (call) => call.command === "docker" && call.args[0] === "create", + ), + ).toHaveLength(2); + expect(fileWatcher.watchCalls.filter((call) => call.path === functionsDir)).toHaveLength(2); + expect(out.stdoutText).toContain("Edge Runtime container stopped (exit 15)."); + expect(yield* telemetry.recorded).toStrictEqual([ + { serve_end_reason: "container_stopped", edge_runtime_exit_code: 15 }, + ]); + }).pipe(Effect.provide(BunServices.layer)); + }); + it.live( "re-attaches instead of reporting a graceful exit when docker logs exits 0 but the container is still running", () => { @@ -2685,6 +2875,7 @@ describe("functions serve integration", () => { const childSpawner = mockDockerLogSpawner( Array.from({ length: 6 }, () => reattachPair).flat(), ); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { yield* writeHelloFunction; @@ -2692,7 +2883,7 @@ describe("functions serve integration", () => { const { layer } = setupServe({ childSpawner }); const error = yield* serveWithTimers(baseFlags(), { dockerLogRetryDelay: Duration.millis(1), - }).pipe(Effect.provide(layer), Effect.flip); + }).pipe(Effect.provide(layer), telemetry.provide, Effect.flip); expect(error).toBeInstanceOf(EdgeRuntimeLogStreamLostError); if (error instanceof EdgeRuntimeLogStreamLostError) { @@ -2701,6 +2892,7 @@ describe("functions serve integration", () => { expect(error.message).toContain("5 times"); } expect(childSpawner.maxLiveHandles).toBe(1); + expect(yield* telemetry.recorded).toStrictEqual([]); }); }, ); @@ -2742,12 +2934,17 @@ describe("functions serve integration", () => { { exitCode: 1, stderr: "docker logs connection reset" }, inspectStateBehavior(false, 137, true), ]); + const telemetry = mockCommandTelemetry(); return Effect.gen(function* () { yield* writeHelloFunction; const { layer } = setupServe({ childSpawner }); - const error = yield* functionsServe(baseFlags()).pipe(Effect.provide(layer), Effect.flip); + const error = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + Effect.flip, + ); expect(error).toBeInstanceOf(DockerLogsStreamError); if (error instanceof DockerLogsStreamError) { @@ -2759,6 +2956,39 @@ describe("functions serve integration", () => { fingerprint_suffix: "out_of_memory", }); } + expect(yield* telemetry.recorded).toStrictEqual([ + { serve_end_reason: "container_crashed", edge_runtime_exit_code: 137 }, + ]); + }); + }, + ); + + it.live.each([0, 15])( + "fails with the log-stream error without recording a crash when the stopped container exited with %d", + (containerExitCode) => { + deployMockState.runHandler = baseDockerRunHandler(); + const childSpawner = mockDockerLogSpawner([ + { exitCode: 1, stderr: "docker logs connection reset" }, + inspectStateBehavior(false, containerExitCode), + ]); + const telemetry = mockCommandTelemetry(); + + return Effect.gen(function* () { + yield* writeHelloFunction; + + const { layer } = setupServe({ childSpawner }); + const error = yield* functionsServe(baseFlags()).pipe( + Effect.provide(layer), + telemetry.provide, + Effect.flip, + ); + + expect(error).toBeInstanceOf(DockerLogsStreamError); + if (error instanceof DockerLogsStreamError) { + expect(error.message).toBe("docker logs connection reset"); + expect(error.oomKilled).toBe(false); + } + expect(yield* telemetry.recorded).toStrictEqual([]); }); }, ); diff --git a/apps/cli/src/shared/functions/serve.errors.ts b/apps/cli/src/shared/functions/serve.errors.ts index 730c99e211..1718f43c7a 100644 --- a/apps/cli/src/shared/functions/serve.errors.ts +++ b/apps/cli/src/shared/functions/serve.errors.ts @@ -60,6 +60,8 @@ export class DockerLogsStreamError extends Data.TaggedError("DockerLogsStreamErr * False when that inspect failed and there is no container state to read. */ readonly oomKilled: boolean; + /** The stopped container's exit code from the follow-up inspect, absent when that inspect failed. */ + readonly containerExitCode?: number; }> { get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { if (this.oomKilled) { diff --git a/apps/cli/src/shared/functions/serve.ts b/apps/cli/src/shared/functions/serve.ts index 727f0b920c..dc4be8ed44 100644 --- a/apps/cli/src/shared/functions/serve.ts +++ b/apps/cli/src/shared/functions/serve.ts @@ -94,6 +94,8 @@ import { import { CliConfigKeys } from "../../config/cli-config-keys.ts"; import { resolveConfigSubtree } from "../../config/cli-config-subtree.ts"; import { CliConfigValues } from "../../config/cli-config-values.service.ts"; +import { recordCommandTelemetry } from "../../telemetry/command-telemetry-attributes.ts"; +import { PropEdgeRuntimeExitCode, PropServeEndReason } from "../telemetry/event-catalog.ts"; const dockerRuntimeServerPort = 8081; const dockerRuntimeInspectorPort = 8083; @@ -128,6 +130,7 @@ const shutdownSignalGracePeriod = Duration.millis(50); // not a self-raised crash signal; 137 is excluded because it needs `OOMKilled` to tell a // memory-limit kill from a kill the CLI cannot attribute. const externalTerminationExitCodes = new Set([ + 15, // SIGTERM as edge-runtime reports it: the raw signal number, not 128 + 15 129, // SIGHUP 130, // SIGINT 131, // SIGQUIT @@ -1205,6 +1208,42 @@ type ServeStartOutcome = Data.TaggedEnum<{ }>; const ServeStartOutcome = Data.taggedEnum(); +/** How a `functions serve` session ended, as recorded on the command's telemetry event. */ +export type ServeEndReason = + | "signal" + | "container_stopped" + | "container_exited" + | "container_removed" + | "container_crashed"; + +const recordServeEnd = (reason: ServeEndReason, exitCode?: number) => + Effect.andThen( + Effect.annotateCurrentSpan({ + "functions.serve.end_reason": reason, + ...(exitCode === undefined ? {} : { "container.exit_code": exitCode }), + }), + recordCommandTelemetry({ + [PropServeEndReason]: reason, + ...(exitCode === undefined ? {} : { [PropEdgeRuntimeExitCode]: exitCode }), + }), + ); + +const recordContainerLogsEnd = (reason: ContainerLogsEndReason) => + ContainerLogsEndReason.$match(reason, { + ContainerExited: () => recordServeEnd("container_exited", 0), + SupervisorTerminated: ({ exitCode }) => recordServeEnd("container_stopped", exitCode), + ContainerGone: () => recordServeEnd("container_removed"), + }); + +// A log-stream failure over a clean or supervisor stop is the stream's failure, not a crash. +const crashedContainerExitCode = (error: unknown): number | undefined => { + if (error instanceof EdgeRuntimeContainerCrashedError) return error.exitCode; + const exitCode = error instanceof DockerLogsStreamError ? error.containerExitCode : undefined; + return exitCode !== undefined && exitCode !== 0 && !externalTerminationExitCodes.has(exitCode) + ? exitCode + : undefined; +}; + type Spawner = ChildProcessSpawner.ChildProcessSpawner["Service"]; /** @@ -1368,6 +1407,7 @@ const streamContainerLogs = Effect.fn("functions.serve.streamContainerLogs")(fun stderr: trimmedStderr, daemonDown: false, oomKilled: inspected.success.oomKilled, + containerExitCode: inspected.success.exitCode, }); } }); @@ -2047,7 +2087,11 @@ export const serveFunctions = Effect.fn("functions.serve")(function* ( ); const keepServing = yield* ServeStartOutcome.$match(startOutcome, { - Shutdown: () => writeStoppedServingMessage().pipe(Effect.as(false)), + Shutdown: () => + recordServeEnd("signal").pipe( + Effect.andThen(writeStoppedServingMessage()), + Effect.as(false), + ), Started: ({ started }) => Effect.gen(function* () { const outcome = yield* withShutdownGrace( @@ -2068,6 +2112,12 @@ export const serveFunctions = Effect.fn("functions.serve")(function* ( gracePeriod, () => ServeSessionOutcome.Shutdown(), ).pipe( + Effect.tapError((error) => { + const exitCode = crashedContainerExitCode(error); + return exitCode === undefined + ? Effect.void + : recordServeEnd("container_crashed", exitCode); + }), Effect.ensuring( bestEffortRemoveContainer(started.containerId).pipe( Effect.ensuring(started.cleanup), @@ -2077,8 +2127,16 @@ export const serveFunctions = Effect.fn("functions.serve")(function* ( return yield* ServeSessionOutcome.$match(outcome, { Restart: () => Effect.succeed(true), - Shutdown: () => writeStoppedServingMessage().pipe(Effect.as(false)), - Exited: ({ reason }) => writeContainerEndedMessage(reason).pipe(Effect.as(false)), + Shutdown: () => + recordServeEnd("signal").pipe( + Effect.andThen(writeStoppedServingMessage()), + Effect.as(false), + ), + Exited: ({ reason }) => + recordContainerLogsEnd(reason).pipe( + Effect.andThen(writeContainerEndedMessage(reason)), + Effect.as(false), + ), }); }), }); diff --git a/apps/cli/src/shared/telemetry/event-catalog.ts b/apps/cli/src/shared/telemetry/event-catalog.ts index 15fe7270d3..5586717a33 100644 --- a/apps/cli/src/shared/telemetry/event-catalog.ts +++ b/apps/cli/src/shared/telemetry/event-catalog.ts @@ -45,6 +45,10 @@ export const PropStackBackend = "stack_backend"; export const PropOrioleDb = "orioledb"; // Set only once a stack-backend command resolves its stack's runtime. export const PropStackRuntime = "stack_runtime"; +// Set only once `functions serve` on the Docker runtime reaches its final outcome. +export const PropServeEndReason = "serve_end_reason"; +// Set only when the Edge Runtime container's exit code is known at that outcome. +export const PropEdgeRuntimeExitCode = "edge_runtime_exit_code"; // Reserved for a closed workflow label; nothing emits it until a closed // vocabulary is agreed (tests assert its absence on failure events). export const PropWorkflow = "workflow"; diff --git a/apps/cli/src/telemetry/command-telemetry-attributes.ts b/apps/cli/src/telemetry/command-telemetry-attributes.ts index 9b4f2f71ec..5f0f77f9b0 100644 --- a/apps/cli/src/telemetry/command-telemetry-attributes.ts +++ b/apps/cli/src/telemetry/command-telemetry-attributes.ts @@ -1,11 +1,19 @@ import { Context, Effect } from "effect"; import type { StackRuntime } from "../command-internal/stack-runtime.ts"; -import { PropOrioleDb, PropStackRuntime } from "../shared/telemetry/event-catalog.ts"; +import type { ServeEndReason } from "../shared/functions/serve.ts"; +import { + PropEdgeRuntimeExitCode, + PropOrioleDb, + PropServeEndReason, + PropStackRuntime, +} from "../shared/telemetry/event-catalog.ts"; /** Properties that code below a command handler adds to its `cli_command_executed` event. */ export interface CommandTelemetryAttributeValues { readonly [PropOrioleDb]?: boolean; readonly [PropStackRuntime]?: StackRuntime; + readonly [PropServeEndReason]?: ServeEndReason; + readonly [PropEdgeRuntimeExitCode]?: number; } /** Command-scoped attribute sink; `withCommandTelemetry` provides one, otherwise records are dropped. */ From 497bf42f5ee71458d5345fdb6168b1abcf69c242 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 3 Nov 2026 13:51:44 +0000 Subject: [PATCH 043/101] ci(release): fix recurring release failures since the v3 cut (CLI-2913) (#7465) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Most `Release` runs since the v3 cut have failed, and the `next` channel has quietly stopped releasing. There are four separate causes, plus two smaller release-notes failures. **`next` and `develop` tag the same commits.** Between majors `next` has no commits of its own, so the sync fast-forwards it onto `develop` and every fix or feature releases on both channels from one SHA. semantic-release reads channel notes per commit and gives every tag on a commit the first note ([semantic-release#4073](https://github.com/semantic-release/semantic-release/issues/4073)), so `v3.1.0-next.1` reads as a beta release. Every `next` run since has recomputed 3.1.0-next.1, published nothing and gone green. The same shared commits break beta's release-notes backfill: its note seeding looked for notes on the annotated tag object (which never has one), force-wrote a note for every tag, and on a shared commit the `next` note won, so backfilling `v3.1.0-beta.2` computed `v3.1.0-beta.1`. - A new `next-ahead` job compares `next` with `develop`. A `next` push with no commits beyond `develop` skips `Test next` and the release, and a `channel=next` dispatch in that state is refused up front. The plan checks again after the tests and the stable-release wait, so a major cut landing in between can't make `next` tag a commit `develop` also tags. Once a breaking change lands, `next` releases from its own commits again. Until then `supabase@next` stays on 3.1.0-next.1 and `supabase@beta` carries the newest code. - The backfill drops other channels' prerelease tags (and non-CLI tags) in its temporary clone and rebuilds one channel note per tagged commit from the tag names, instead of reading or overwriting notes. **The setup-cli v3 smoke test races npm.** setup-cli v3 installs through npm. npm lists a new version a few minutes after publish, and the registry CDN then caches each packument for five minutes, separately per region and per `Accept` header. One run failed with a 404 several seconds after npm had already recorded the publish. A single wait job on one runner can't cover the macOS, Windows and Alpine runners, so: - setup-cli v3 is pinned to v3.2.0, and each v3 job first runs `.github/scripts/wait-for-npm-version.sh`, which polls `npm view` and the abbreviated packument from that runner for both `supabase` and the runner's platform package, for up to 15 minutes. - Since v3.1.0, setup-cli falls back to the GitHub release whenever the npm install fails, so the v3 jobs now fail unless `install-source` is `npm`. One extra ubuntu job forces the fallback and checks it still works. **The platform-package wait is too short.** `publish.ts` waits for every platform package to become installable before publishing the umbrella. That has taken anywhere from 6 to more than 15 minutes, and two releases timed out at the 15-minute limit. The ceiling is now 30 minutes and the `publish` job timeout 60. It's a poll, so a quick npm costs nothing extra. **A beta planned during a stable release misses the stable tag.** `main` pushes its tag only after the npm publish, so a `develop` push in that window computed a stale beta version (`3.0.0-beta.18` was planned while 3.0.0 was publishing). The `develop` and `next` plans now wait, for up to 120 minutes, until the `publish` job of any in-flight `Release` run on `main` has finished. A shared concurrency group would have let a pending `develop` run cancel a pending `main` release. **A forced maintenance version gets no release notes.** The 2.122.1 maintenance release was dispatched with an explicit `version`, and its only commits were `ci:` commits, so the backfill's semantic-release dry run computed no release and left the GitHub release body empty. The backfill now gives semantic-release an extra commit analyzer that returns the release type implied by the tag's own version, so commit analysis still wins when it computes more and a forced version still gets its notes. **Proposing release notes fails to mint a token.** `propose-release-notes.yml` asked for an `issues: read` permission the releaser app doesn't have, so token creation returned 422. The request is dropped; the notes agent reads public issues without it. This supersedes #7439. These are release-infra paths, so `v2.x` needs the setup-cli, platform-wait, backfill and propose-release-notes changes before its next maintenance release; `v2.x` has no `next` channel, so the `next-ahead` gate doesn't apply there. Closes CLI-2913, CLI-2914, CLI-2915, CLI-2916, CLI-2917 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- .gitattributes | 3 + .github/MAINTAINERS.md | 1 + .github/scripts/wait-for-npm-version.sh | 35 ++++ .github/workflows/propose-release-notes.yml | 1 - .github/workflows/release-shared.yml | 2 +- .github/workflows/release.yml | 121 ++++++++++-- .github/workflows/setup-cli-smoke-test.yml | 109 ++++++++++- apps/cli/docs/release-process.md | 15 +- .../backfill-release-notes.e2e.test.ts | 180 ++++++++++++++++++ apps/cli/scripts/backfill-release-notes.ts | 82 +++++--- apps/cli/scripts/npm-registry.ts | 2 +- apps/cli/scripts/npm-registry.unit.test.ts | 2 +- apps/cli/scripts/release-channels.ts | 30 +++ .../cli/scripts/release-channels.unit.test.ts | 44 +++++ ...-release-branches-and-maintenance-lines.md | 6 +- 15 files changed, 573 insertions(+), 60 deletions(-) create mode 100755 .github/scripts/wait-for-npm-version.sh create mode 100644 apps/cli/scripts/backfill-release-notes.e2e.test.ts diff --git a/.gitattributes b/.gitattributes index 19f3e8920a..017dd6d47b 100644 --- a/.gitattributes +++ b/.gitattributes @@ -10,6 +10,9 @@ patches/*.patch text eol=lf # can't leave a trailing `\r` on every entry and make every tag mismatch. apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt text eol=lf +# The Windows smoke-test runs this under Git Bash; a CRLF checkout breaks `set -euo pipefail`. +.github/scripts/wait-for-npm-version.sh text eol=lf + # Generated artifacts. `linguist-generated` keeps these out of GitHub's language # statistics and collapses them in pull request diffs -- they are build output to # be regenerated, not code to review, and together they outweigh the hand-written diff --git a/.github/MAINTAINERS.md b/.github/MAINTAINERS.md index 25c044e909..50ec58f4c5 100644 --- a/.github/MAINTAINERS.md +++ b/.github/MAINTAINERS.md @@ -187,6 +187,7 @@ Fast-forward PRs are the deploy PR (`develop` → `main`), the major cut (`next` → `develop`), and the two sync PRs. **Dispatch guards.** `release.yml` refuses `channel=next` off `next`, +`channel=next` while `next` has no commits beyond `develop`, `channel=maintenance` off `v.x`, `beta`/`stable` on `next` or `v.x`, a non-dry-run `stable` outside `main`, and a `version` that is not `X.Y.Z[-prerelease]`. diff --git a/.github/scripts/wait-for-npm-version.sh b/.github/scripts/wait-for-npm-version.sh new file mode 100755 index 0000000000..dfb6e07ad7 --- /dev/null +++ b/.github/scripts/wait-for-npm-version.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Usage: wait-for-npm-version.sh ... +# +# npm lists a new version minutes after publish, and the registry CDN caches each packument per +# region and Accept header, so both the npm client and the abbreviated packument must serve it. +set -euo pipefail + +if [[ $# -lt 2 ]]; then + echo "usage: $0 ..." >&2 + exit 2 +fi + +version="$1" +shift +timeout="${WAIT_TIMEOUT_SECONDS:-900}" + +npm_serves_version() { + local pkg="$1" + npm view "${pkg}@${version}" version --prefer-online --fetch-timeout=30000 --fetch-retries=0 > /dev/null && + curl -fsSL --max-time 30 -H 'Accept: application/vnd.npm.install-v1+json' "https://registry.npmjs.org/${pkg//\//%2f}" | + jq -e --arg v "$version" '.versions | has($v)' > /dev/null +} + +deadline=$((SECONDS + timeout)) +for pkg in "$@"; do + while true; do + if (( SECONDS >= deadline )); then + echo "::error::npm does not serve ${pkg}@${version} after ${timeout} seconds" + exit 1 + fi + npm_serves_version "$pkg" && break + echo "Waiting for npm to serve ${pkg}@${version}..." + sleep 15 + done +done diff --git a/.github/workflows/propose-release-notes.yml b/.github/workflows/propose-release-notes.yml index 3ba3ad70fc..8ddc40d29c 100644 --- a/.github/workflows/propose-release-notes.yml +++ b/.github/workflows/propose-release-notes.yml @@ -62,7 +62,6 @@ jobs: client-id: ${{ vars.GH_APP_CLIENT_ID }} private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} permission-contents: write - permission-issues: read permission-pull-requests: write - uses: useblacksmith/checkout@6fd481652155169ed4d2f25ebaf97464f685175f # v1 diff --git a/.github/workflows/release-shared.yml b/.github/workflows/release-shared.yml index 32bde58beb..add707ada2 100644 --- a/.github/workflows/release-shared.yml +++ b/.github/workflows/release-shared.yml @@ -275,7 +275,7 @@ jobs: # count as self-hosted from sigstore's POV, so this job must stay on a github-hosted runner # (it's short and not compute-bound, so the cost is negligible). runs-on: ubuntu-latest - timeout-minutes: 45 + timeout-minutes: 60 env: CHANNEL: ${{ inputs.channel }} NPM_TAG: ${{ inputs.npm_tag }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1a1ce8040a..de02c7ed9f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -47,9 +47,36 @@ permissions: contents: read jobs: + # semantic-release reads channel notes per commit (semantic-release/semantic-release#4073), so + # releasing `next` on a commit develop already tagged misreads both tags. + next-ahead: + name: Check next has its own commits + if: github.ref_name == 'next' + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + outputs: + ahead: ${{ steps.compare.outputs.ahead }} + steps: + - id: compare + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + SHA: ${{ github.sha }} + run: | + set -euo pipefail + ahead_by="$(gh api "repos/${REPO}/compare/develop...${SHA}" --jq .ahead_by)" + if [[ "$ahead_by" -gt 0 ]]; then + echo "ahead=true" >> "$GITHUB_OUTPUT" + else + echo "ahead=false" >> "$GITHUB_OUTPUT" + fi + test-next: name: Test next - if: github.event_name == 'push' && github.ref_name == 'next' + needs: next-ahead + if: github.event_name == 'push' && github.ref_name == 'next' && needs.next-ahead.outputs.ahead == 'true' permissions: contents: read actions: read @@ -61,12 +88,20 @@ jobs: plan: name: Plan release - needs: test-next - # A skipped test-next (every non-next trigger) makes the implicit success() false, so the - # result is checked explicitly. - if: ${{ !cancelled() && contains(fromJSON('["success","skipped"]'), needs.test-next.result) }} + needs: [next-ahead, test-next] + # Skipped next-ahead and test-next jobs (every non-next trigger) make the implicit success() + # false, so their results are checked explicitly. A next push with no commits beyond develop + # plans nothing. + if: >- + ${{ !cancelled() && + contains(fromJSON('["success","skipped"]'), needs.next-ahead.result) && + contains(fromJSON('["success","skipped"]'), needs.test-next.result) && + !(github.event_name == 'push' && github.ref_name == 'next' && needs.next-ahead.outputs.ahead != 'true') }} runs-on: ubuntu-latest - timeout-minutes: 15 + timeout-minutes: 135 + permissions: + contents: read + actions: read outputs: should_release: ${{ steps.compute.outputs.should_release }} version: ${{ steps.compute.outputs.version }} @@ -78,6 +113,59 @@ jobs: dry_run: ${{ steps.compute.outputs.dry_run }} channel: ${{ steps.compute.outputs.channel }} steps: + - name: Refuse a next dispatch with no commits beyond develop + if: ${{ github.event_name == 'workflow_dispatch' && github.ref_name == 'next' && needs.next-ahead.outputs.ahead != 'true' }} + run: | + echo "::error::next has no commits beyond develop, so a next release would tag a commit develop already released" + exit 1 + # The stable tag is pushed by main's publish job, so a prerelease planned before that job + # finishes misses it. The rest of the run (brew, scoop, smoke tests) does not affect the plan. + - name: Wait for an in-flight stable release + if: ${{ (github.ref_name == 'develop' || github.ref_name == 'next') && (github.event_name == 'push' || inputs.version == '') }} + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + run: | + set -euo pipefail + deadline=$((SECONDS + 7200)) + while true; do + pending=0 + if run_ids="$(gh run list --repo "$REPO" --workflow release.yml --branch main --json databaseId,status --jq '.[] | select(.status != "completed") | .databaseId')"; then + for run_id in $run_ids; do + publish_status="$(gh api "repos/${REPO}/actions/runs/${run_id}/jobs" --paginate --jq '.jobs[] | select(.name == "Release / publish") | .status')" || publish_status=unknown + if [[ "$publish_status" != completed ]]; then + pending=$((pending + 1)) + fi + done + else + echo "::warning::Could not list Release runs on main; retrying." + pending=1 + fi + if [[ "$pending" -eq 0 ]]; then + break + fi + if (( SECONDS >= deadline )); then + echo "::error::The publish job of a Release run on main is still in progress after 120 minutes. Re-run this job once it finishes." + exit 1 + fi + echo "Waiting for the publish job of ${pending} Release run(s) on main to finish..." + sleep 30 + done + # A major cut can fast-forward develop onto next while the tests and the wait above run. + - id: next-still-ahead + if: ${{ github.ref_name == 'next' }} + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + SHA: ${{ github.sha }} + run: | + set -euo pipefail + ahead_by="$(gh api "repos/${REPO}/compare/develop...${SHA}" --jq .ahead_by)" + if [[ "$ahead_by" -gt 0 ]]; then + echo "ahead=true" >> "$GITHUB_OUTPUT" + else + echo "ahead=false" >> "$GITHUB_OUTPUT" + fi # semantic-release runs `git push --dry-run HEAD:` as part of verifyAuth even in # `dry_run: true` mode, so the token needs push access to the protected develop/main # branches — the default GITHUB_TOKEN doesn't have it. @@ -110,6 +198,7 @@ jobs: DISPATCH_CHANNEL: ${{ inputs.channel }} DISPATCH_VERSION: ${{ inputs.version }} DISPATCH_DRY_RUN: ${{ inputs.dry_run }} + NEXT_AHEAD: ${{ steps.next-still-ahead.outputs.ahead }} SR_PUBLISHED: ${{ steps.semantic-release.outputs.new_release_published }} SR_VERSION: ${{ steps.semantic-release.outputs.new_release_version }} run: | @@ -126,6 +215,15 @@ jobs: next) natural=next ;; *) [[ "$REF_NAME" =~ $maint_re ]] && natural=maintenance ;; esac + if [[ "$REF_NAME" == next && "$NEXT_AHEAD" != true ]]; then + if [[ "$EVENT" == "workflow_dispatch" ]]; then + echo "::error::next has no commits beyond develop, so a next release would tag a commit develop already released" + exit 1 + fi + echo "::notice::next has no commits beyond develop, so there is nothing to release" + echo "should_release=false" >> "$GITHUB_OUTPUT" + exit 0 + fi if [[ "$EVENT" == "workflow_dispatch" ]]; then channel="$DISPATCH_CHANNEL" dry_run="$DISPATCH_DRY_RUN" @@ -394,13 +492,14 @@ jobs: SLACK_RELEASE_WEBHOOK: ${{ secrets.SLACK_RELEASE_WEBHOOK }} # Reports a broken release on every channel. `failure()` evaluates against the full `needs` - # chain, so this fires whenever `test-next`, `plan` or `release` (including the reusable - # release-shared workflow) fails. Skipped jobs don't count as failures. Dry runs are excluded - # via the dispatch input directly, so even a plan job that dies before recording its `dry_run` - # output stays quiet. A failed `test-next` skips `plan`, so the channel falls back to the ref. + # chain, so this fires whenever `next-ahead`, `test-next`, `plan` or `release` (including the + # reusable release-shared workflow) fails. Skipped jobs don't count as failures. Dry runs are + # excluded via the dispatch input directly, so even a plan job that dies before recording its + # `dry_run` output stays quiet. A failed `next-ahead` or `test-next` skips `plan`, so the + # channel falls back to the ref. notify-slack-failure: name: Notify Slack (failure) - needs: [test-next, plan, release] + needs: [next-ahead, test-next, plan, release] if: ${{ failure() && !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }} uses: ./.github/workflows/slack-notify.yml with: diff --git a/.github/workflows/setup-cli-smoke-test.yml b/.github/workflows/setup-cli-smoke-test.yml index 5e00dad37b..7250265fcd 100644 --- a/.github/workflows/setup-cli-smoke-test.yml +++ b/.github/workflows/setup-cli-smoke-test.yml @@ -1,9 +1,11 @@ name: setup-cli Smoke Test # Smoke test for the `supabase/setup-cli` GitHub Action against a specific published CLI -# version. Runs automatically after every beta release and also manually via workflow_dispatch -# when debugging setup-cli regressions. Catches regressions like supabase/setup-cli#427, where -# musl libc + archive layout changes silently broke the action on Alpine. +# version. Runs automatically after every published release and also manually via +# workflow_dispatch when debugging setup-cli regressions. Catches regressions like +# supabase/setup-cli#427, where musl libc + archive layout changes silently broke the action on +# Alpine. v3 installs from npm: each v3 runner first waits until npm serves the version, and the +# job fails if setup-cli fell back to the GitHub release. A separate job forces that fallback. on: workflow_call: @@ -39,7 +41,7 @@ jobs: - v2 - v3 runs-on: ${{ matrix.runner }} - timeout-minutes: 10 + timeout-minutes: 25 env: VERSION: ${{ inputs.version }} steps: @@ -53,9 +55,24 @@ jobs: uses: supabase/setup-cli@afb1b15109756ea5cf9d8985a359d9095235ca2b # v2.1.2 with: version: ${{ inputs.version }} + - name: Checkout the npm wait script + if: matrix.major-version == 'v3' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + path: .smoke-tooling + sparse-checkout: .github/scripts + persist-credentials: false + - name: Wait for npm to serve the version + if: matrix.major-version == 'v3' + shell: bash + run: | + set -euo pipefail + platform="$(node -p 'process.platform + "-" + process.arch')" + bash .smoke-tooling/.github/scripts/wait-for-npm-version.sh "${VERSION#v}" supabase "@supabase/cli-${platform/win32/windows}" - name: Install Supabase CLI via setup-cli@v3 + id: setup-cli-v3 if: matrix.major-version == 'v3' - uses: supabase/setup-cli@45a513f8c64c0bc8e0e3dfe572b5c95be85f6359 # v3.0.1 + uses: supabase/setup-cli@689adcfb80d947100a1eab127289b97d1fb07b13 # v3.2.0 with: version: ${{ inputs.version }} - name: Verify supabase --version matches the expected version @@ -72,6 +89,17 @@ jobs: echo "Version mismatch: expected ${expected}, got ${actual}" >&2 exit 1 fi + - name: Verify setup-cli installed from npm + if: matrix.major-version == 'v3' + shell: bash + env: + INSTALL_SOURCE: ${{ steps.setup-cli-v3.outputs.install-source }} + run: | + set -euo pipefail + if [ "${INSTALL_SOURCE}" != "npm" ]; then + echo "::error::setup-cli fell back to ${INSTALL_SOURCE} because installing supabase@${VERSION#v} from npm failed; check the setup-cli step log" + exit 1 + fi # A separate job because GitHub Actions only honours `container:` on Linux runners, and mixing # container + non-container entries in one matrix is fragile when the value is empty. The @@ -87,7 +115,7 @@ jobs: - v2 - v3 runs-on: ubuntu-latest - timeout-minutes: 10 + timeout-minutes: 25 # `node:20-alpine` ships a musl-linked Node, required for JavaScript-based actions to launch # here — the runner's mounted glibc Node won't execute inside an Alpine container. container: @@ -99,7 +127,7 @@ jobs: VERSION: ${{ inputs.version }} steps: - name: Install Alpine prerequisites - run: apk add --no-cache bash curl tar + run: apk add --no-cache bash curl git jq tar - name: Install Supabase CLI via setup-cli@v1 if: matrix.major-version == 'v1' uses: supabase/setup-cli@1dedf2c611547ede7232d26866dd3c56ab903bbb # v1.7.3 @@ -110,9 +138,64 @@ jobs: uses: supabase/setup-cli@afb1b15109756ea5cf9d8985a359d9095235ca2b # v2.1.2 with: version: ${{ inputs.version }} + - name: Checkout the npm wait script + if: matrix.major-version == 'v3' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + path: .smoke-tooling + sparse-checkout: .github/scripts + persist-credentials: false + - name: Wait for npm to serve the version + if: matrix.major-version == 'v3' + shell: bash + run: | + set -euo pipefail + platform="$(node -p 'process.platform + "-" + process.arch')" + bash .smoke-tooling/.github/scripts/wait-for-npm-version.sh "${VERSION#v}" supabase "@supabase/cli-${platform}-musl" - name: Install Supabase CLI via setup-cli@v3 + id: setup-cli-v3 + if: matrix.major-version == 'v3' + uses: supabase/setup-cli@689adcfb80d947100a1eab127289b97d1fb07b13 # v3.2.0 + with: + version: ${{ inputs.version }} + - name: Verify supabase --version matches the expected version + shell: bash + run: | + set -euo pipefail + # `supabase --version` prints the version without the leading `v`, + # while release tags / setup-cli inputs include it, so strip it + # before comparing. + expected="${VERSION#v}" + actual="$(supabase --version | tr -d '\r' | head -n1)" + echo "supabase --version: ${actual}" + if [ "${actual}" != "${expected}" ]; then + echo "Version mismatch: expected ${expected}, got ${actual}" >&2 + exit 1 + fi + - name: Verify setup-cli installed from npm if: matrix.major-version == 'v3' - uses: supabase/setup-cli@45a513f8c64c0bc8e0e3dfe572b5c95be85f6359 # v3.0.1 + shell: bash + env: + INSTALL_SOURCE: ${{ steps.setup-cli-v3.outputs.install-source }} + run: | + set -euo pipefail + if [ "${INSTALL_SOURCE}" != "npm" ]; then + echo "::error::setup-cli fell back to ${INSTALL_SOURCE} because installing supabase@${VERSION#v} from npm failed; check the setup-cli step log" + exit 1 + fi + + setup-cli-smoke-test-github-release: + name: setup-cli v3 GitHub release fallback (ubuntu-latest) + runs-on: ubuntu-latest + timeout-minutes: 10 + env: + VERSION: ${{ inputs.version }} + # An npm executable that always fails sends setup-cli down its GitHub release fallback. + SUPABASE_SETUP_CLI_NPM: "false" + steps: + - name: Install Supabase CLI via setup-cli@v3 + id: setup-cli-v3 + uses: supabase/setup-cli@689adcfb80d947100a1eab127289b97d1fb07b13 # v3.2.0 with: version: ${{ inputs.version }} - name: Verify supabase --version matches the expected version @@ -129,3 +212,13 @@ jobs: echo "Version mismatch: expected ${expected}, got ${actual}" >&2 exit 1 fi + - name: Verify setup-cli installed from the GitHub release + shell: bash + env: + INSTALL_SOURCE: ${{ steps.setup-cli-v3.outputs.install-source }} + run: | + set -euo pipefail + if [ "${INSTALL_SOURCE}" != "github-release" ]; then + echo "::error::setup-cli installed from ${INSTALL_SOURCE} instead of falling back to the GitHub release with npm disabled" + exit 1 + fi diff --git a/apps/cli/docs/release-process.md b/apps/cli/docs/release-process.md index 55cc60a409..7ade1efbf7 100644 --- a/apps/cli/docs/release-process.md +++ b/apps/cli/docs/release-process.md @@ -213,7 +213,7 @@ Production releases live in a single `[.github/workflows/release.yml](../../../. | next | push: next (after `Test next` passes) | `next` | none | prerelease | `X.Y.Z-next.N` | | maintenance | manual dispatch on a `v.x` ref | `v.stable` | `supabase@` / `supabase-v-stable` | published, never latest | `N.Y.Z` | -`beta` auto-publishes on every merge to `develop` (CLI). `stable` auto-publishes after a develop→main fast-forward, which itself happens when the weekly `[deploy.yml](../../../.github/workflows/deploy.yml)` cron PR is approved (the FF push to `main` re-fires `release.yml` via the `push: branches: [main]` trigger). `next` auto-publishes on every push to the `next` branch, and `maintenance` is dispatched by hand (see [Maintenance lines](#maintenance-lines)). +`beta` auto-publishes on every merge to `develop` (CLI). `stable` auto-publishes after a develop→main fast-forward, which itself happens when the weekly `[deploy.yml](../../../.github/workflows/deploy.yml)` cron PR is approved (the FF push to `main` re-fires `release.yml` via the `push: branches: [main]` trigger). `next` auto-publishes on every push to the `next` branch that carries commits beyond `develop` (see [The `next` channel](#the-next-channel)), and `maintenance` is dispatched by hand (see [Maintenance lines](#maintenance-lines)). Versions are computed by `cycjimmy/semantic-release-action` from the first line of each squash commit (the PR title). Conventional commit titles use `feat:` → minor, `fix:`, `perf:`, or `revert:` → patch, and `!` after the type or scope (`feat!:` / `fix(scope)!:`) → major. Commit bodies, including `BREAKING CHANGE` notes, are ignored for version calculation. The `release` config lives in `apps/cli/package.json`: `develop` is a `beta` prerelease branch, `next` is a `next` prerelease branch, `main` emits plain `X.Y.Z`, and a wildcard entry `v+([0-9]).x` turns every `v.x` branch into a maintenance line on the `v.stable` channel (the entry is ignored while no such branch exists). @@ -245,7 +245,7 @@ flowchart LR flowchart TD pushDev[push: develop] --> plan pushMain[push: main] --> plan - pushNext[push: next] --> testNext["Test next
test.yml (reusable)"] + pushNext[push: next] -->|"only with commits beyond develop"| testNext["Test next
test.yml (reusable)"] testNext -->|passes| plan pr["approved PR
develop→main, next→develop, sync/*"] --> ff["fast-forward.yml
checks + git push via App token"] ff --> pushMain @@ -281,11 +281,14 @@ gh workflow run release.yml \ Auto-trigger paths leave `version` empty: semantic-release computes it from commits since the last tag. A dispatch with an empty `version` does the same for the dispatched ref, and fails if the channel does not match the ref or if semantic-release computes no release. +The `develop` and `next` plans wait, up to 120 minutes, until the `publish` job of any in-flight `Release` run on `main` has finished before reading tags: that job pushes the stable tag after the npm publish, so a prerelease planned earlier would miss it and compute a stale version. The rest of the `main` run (Homebrew, Scoop, smoke tests) does not delay the plan. If the wait times out, the plan job fails; re-run it once that job has finished. + The plan job refuses a dispatch when: - `channel=next` is not on the `next` ref, or `channel=maintenance` is not on a `v.x` ref; - `next` or a `v.x` ref is dispatched with `beta` or `stable` (those refs release only their own channel); - `channel=stable` is not a dry run and the ref is not `main` (any other ref, including `hotfix/*`, may only dry-run); +- `channel=next` is dispatched while `next` has no commits beyond `develop` (see [The `next` channel](#the-next-channel)); - a non-empty `version` is not semver (no `v` prefix, no build metadata) or does not match the channel: `next` needs `X.Y.Z-next.N`, `beta` needs `X.Y.Z-beta.N`, and `stable` and `maintenance` need plain `X.Y.Z`. ### Hotfix release flow @@ -315,9 +318,11 @@ Do not use `workflow_dispatch dry_run=false` as a hotfix path. A stable release `next` is one long-lived branch reused for every future major. Breaking PRs target `next` instead of `develop`, with a `type(scope)!:` title; they use the same merge queue, required checks, and review as `develop`. The `Lint Pull Request` check fails any `!` title whose base is not `next`, with one exception: the `next` → `develop` cut PR. -Every push to `next` first runs the full `test.yml` suite (the `Test next` job in `release.yml`). Only when it passes does the pipeline plan and publish `X.Y.Z-next.N` to the npm `next` dist-tag and a GitHub prerelease. `next` has no Homebrew, Scoop, or Linear publication. It does get the raw semantic-release changelog on its GitHub Release (`backfill-release-notes`); the LLM-proposed notes PR is stable-only, and `apply-release-notes` still accepts `-next` tags for a manual proposal. +Every push to `next` that carries commits beyond `develop` first runs the full `test.yml` suite (the `Test next` job in `release.yml`). Only when it passes does the pipeline plan and publish `X.Y.Z-next.N` to the npm `next` dist-tag and a GitHub prerelease. `next` has no Homebrew, Scoop, or Linear publication. It does get the raw semantic-release changelog on its GitHub Release (`backfill-release-notes`); the LLM-proposed notes PR is stable-only, and `apply-release-notes` still accepts `-next` tags for a manual proposal. + +`next` publishes use the `next` prerelease identifier so versions never collide with `beta` on npm after a cut. After a stable release, semantic-release on `next` computes past it on the following push with commits beyond `develop`. The npm `next` dist-tag is not moved by the stable release, because publishing is OIDC-only and `npm dist-tag add` needs a token the pipeline does not hold; `supabase@next` catches up on the next release from `next`, which needs commits beyond `develop`. A release can be re-cut by dispatching `release.yml` with `channel=next` on the `next` ref. -`next` publishes use the `next` prerelease identifier so versions never collide with `beta` on npm after a cut. After a stable release, semantic-release on `next` computes past it on the following push. The npm `next` dist-tag is not moved by the stable release, because publishing is OIDC-only and `npm dist-tag add` needs a token the pipeline does not hold; `supabase@next` catches up on the next push to `next`. A release can be re-cut by dispatching `release.yml` with `channel=next` on the `next` ref. +A push to `next` with no commits beyond `develop` skips `Test next` and the release. That is the usual state between majors, because the sync fast-forwards `next` onto `develop`. semantic-release reads channel notes per commit ([semantic-release#4073](https://github.com/semantic-release/semantic-release/issues/4073)), so a `next` tag on a commit `develop` already tagged misreads both tags. `supabase@next` stays on the last `next` release until a breaking change lands on `next`, and `supabase@beta` carries the newest code meanwhile. A `channel=next` dispatch is refused in that state. The nightly live e2e run covers `main`, `develop`, and `next`. @@ -529,7 +534,7 @@ Both updaters run automatically from `release-shared.yml`'s `publish-homebrew` a Once the channels are live, two reusable workflows run automatically (last in `release-shared.yml`, non-gating — by the time they run the artifacts are already published, so a failure surfaces as a red post-release signal rather than blocking distribution): -- `[setup-cli-smoke-test.yml](../../../.github/workflows/setup-cli-smoke-test.yml)` (`setup-cli-smoke` job) — installs the released version through `supabase/setup-cli` v1 and v2 (the GitHub Release download path) and v3 (the npm package) on Linux, macOS, Windows, and Alpine. +- `[setup-cli-smoke-test.yml](../../../.github/workflows/setup-cli-smoke-test.yml)` (`setup-cli-smoke` job) — installs the released version through `supabase/setup-cli` v1 and v2 (the GitHub Release download path) and v3 (the npm package) on Linux, macOS, Windows, and Alpine. v3 waits on each runner until npm serves the version (up to 15 minutes) and fails unless setup-cli installed from npm; one ubuntu job forces the GitHub release fallback. - `[verify-install-channels.yml](../../../.github/workflows/verify-install-channels.yml)` (`verify-install-channels` job) — runs a **real** `brew install` (macOS **and** Linux, so both the `on_macos` and `on_linux` stanzas of the formula are exercised), `scoop install`, and `curl|bash` install of the **published** install script (fetched from the release asset, not the repo checkout) against the just-published Homebrew tap, Scoop bucket, and GitHub Release. Each leg then asserts `supabase --version` matches and runs `supabase --help` to exercise the actual command tree. brew, scoop, and the install script each verify the published `sha256`/`hash` against the downloaded tarball, so this is the signal that would have caught CLI v2.107.0 (where the brew/scoop manifests shipped checksums that did not match the release tarballs and every `brew install` / `scoop install` failed). It only runs for `beta`/`stable`/`maintenance` (the channels that publish brew/scoop) and can be dispatched manually against any already-published version via the Actions tab. ### Code signing (macOS) diff --git a/apps/cli/scripts/backfill-release-notes.e2e.test.ts b/apps/cli/scripts/backfill-release-notes.e2e.test.ts new file mode 100644 index 0000000000..3323fcc37c --- /dev/null +++ b/apps/cli/scripts/backfill-release-notes.e2e.test.ts @@ -0,0 +1,180 @@ +import { BunFileSystem, BunPath } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Layer, Path, Schema } from "effect"; +import { fileURLToPath } from "node:url"; + +const scriptPath = fileURLToPath(new URL("./backfill-release-notes.ts", import.meta.url)); +const cliPackagePath = fileURLToPath(new URL("../package.json", import.meta.url)); +const analyzerPath = fileURLToPath(new URL("./analyze-commits-title.js", import.meta.url)); +const platformLayer = Layer.mergeAll(BunFileSystem.layer, BunPath.layer); + +const decodeCliPackage = Schema.decodeUnknownEffect( + Schema.fromJsonString(Schema.Struct({ release: Schema.Unknown })), +); +const encodePackage = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown)); + +const isolatedGit = { + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_CONFIG_NOSYSTEM: "1", + GIT_AUTHOR_NAME: "seed", + GIT_AUTHOR_EMAIL: "seed@example.com", + GIT_COMMITTER_NAME: "seed", + GIT_COMMITTER_EMAIL: "seed@example.com", +}; + +const spawn = (argv: ReadonlyArray, cwd: string, env: Record) => + Effect.acquireUseRelease( + Effect.sync(() => Bun.spawn([...argv], { cwd, env, stdout: "pipe", stderr: "pipe" })), + (child) => + Effect.gen(function* () { + const [exitCode, stdout, stderr] = yield* Effect.all( + [ + Effect.promise(() => child.exited), + Effect.promise(() => new Response(child.stdout).text()), + Effect.promise(() => new Response(child.stderr).text()), + ], + { concurrency: "unbounded" }, + ); + return { exitCode, stdout, stderr }; + }), + (child) => + child.exitCode === null && child.signalCode === null + ? Effect.sync(() => child.kill()).pipe(Effect.andThen(Effect.promise(() => child.exited))) + : Effect.void, + ); + +interface SeedHistory { + readonly git: (...args: ReadonlyArray) => Effect.Effect; + readonly commit: (message: string) => Effect.Effect; + readonly release: (tag: string, channels: string) => Effect.Effect; +} + +/** + * A working repository on `main` with an `origin` bare remote, `v1.0.0` on its first commit and + * the history `seed` adds on top. Every branch and tag is pushed, and every release also gets + * the per-tag channel note ref the release pipeline pushes. + */ +const repositoryWithHistory = (seed: (history: SeedHistory) => Effect.Effect) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "backfill-release-notes-" }); + const origin = path.join(root, "origin.git"); + const work = path.join(root, "work"); + const env = { ...process.env, ...isolatedGit }; + + const git = (cwd: string, ...args: ReadonlyArray) => + Effect.gen(function* () { + const result = yield* spawn(["git", ...args], cwd, env); + expect(result.exitCode, result.stderr).toBe(0); + return result.stdout; + }); + const commit = (message: string) => + git(work, "commit", "--quiet", "--allow-empty", "-m", message); + const release = (tag: string, channels: string) => + Effect.gen(function* () { + yield* git(work, "tag", "-a", tag, "-m", `Release ${tag}`); + yield* git( + work, + "notes", + `--ref=semantic-release-${tag}`, + "add", + "-m", + `{"channels":${channels}}`, + `${tag}^{commit}`, + ); + }); + + const cliPackage = yield* decodeCliPackage(yield* fs.readFileString(cliPackagePath)); + const packageJson = yield* encodePackage({ + name: "supabase", + version: "0.0.0-automated", + repository: { type: "git", url: "https://github.com/supabase/cli" }, + release: cliPackage.release, + }); + yield* fs.makeDirectory(path.join(work, "apps/cli/scripts"), { recursive: true }); + yield* fs.writeFileString(path.join(work, "apps/cli/package.json"), `${packageJson}\n`); + yield* fs.copyFile(analyzerPath, path.join(work, "apps/cli/scripts/analyze-commits-title.js")); + + yield* git(root, "init", "--quiet", "--bare", "--initial-branch=main", origin); + yield* git(work, "init", "--quiet", "--initial-branch=main"); + yield* git(work, "remote", "add", "origin", origin); + yield* git(work, "add", "-A"); + yield* commit("feat: initial"); + yield* release("v1.0.0", "[null]"); + + yield* seed({ git: (...args) => git(work, ...args), commit, release }); + + yield* git( + work, + "push", + "--quiet", + "origin", + "refs/heads/*:refs/heads/*", + "refs/tags/*:refs/tags/*", + "refs/notes/*:refs/notes/*", + ); + + const bunExecutable = Bun.which("bun"); + if (!bunExecutable) throw new Error("Bun executable not found"); + const { FORCE_COLOR: _forceColor, NO_COLOR: _noColor, ...inherited } = process.env; + + return { + backfill: (tag: string) => + spawn([bunExecutable, scriptPath, "--tag", tag], work, { ...inherited, ...isolatedGit }), + }; + }); + +/** + * `develop` and `next` share the commit that carries both `v1.1.0-beta.1` and `v1.1.0-next.1` + * (`next` is fast-forwarded onto `develop`), and `develop` moves on to `v1.1.0-beta.2`. + */ +const sharedBetaAndNextHistory = repositoryWithHistory(({ git, commit, release }) => + Effect.gen(function* () { + yield* git("switch", "--quiet", "--create", "develop"); + yield* commit("feat: add shared thing"); + yield* release("v1.1.0-beta.1", '["beta"]'); + yield* release("v1.1.0-next.1", '["next"]'); + yield* git("branch", "next"); + yield* commit("fix: patch beta"); + yield* release("v1.1.0-beta.2", '["beta"]'); + }), +); + +const forcedMaintenanceHistory = repositoryWithHistory(({ git, commit, release }) => + Effect.gen(function* () { + yield* commit("feat!: drop the legacy flag"); + yield* release("v2.0.0", "[null]"); + yield* git("switch", "--quiet", "--create", "v1.x", "v1.0.0"); + yield* commit("ci: pin the runner image"); + yield* release("v1.0.1", '["v1.stable"]'); + }), +); + +describe("backfill-release-notes.ts on a repository whose beta and next tags share a commit", () => { + it.live("backfills the next beta from the beta line instead of the next channel's tags", () => + Effect.gen(function* () { + const repository = yield* sharedBetaAndNextHistory; + + const { exitCode, stdout, stderr } = yield* repository.backfill("v1.1.0-beta.2"); + + expect(exitCode, stderr).toBe(0); + expect(stdout, stderr).toContain("compare/v1.1.0-beta.1...v1.1.0-beta.2"); + expect(stdout, stderr).toContain("patch beta"); + expect(stdout, stderr).not.toContain("add shared thing"); + }).pipe(Effect.provide(platformLayer)), + ); +}); + +describe("backfill-release-notes.ts on a maintenance line", () => { + it.live("backfills a forced maintenance release whose commits imply no release", () => + Effect.gen(function* () { + const repository = yield* forcedMaintenanceHistory; + + const { exitCode, stdout, stderr } = yield* repository.backfill("v1.0.1"); + + expect(exitCode, stderr).toBe(0); + expect(stdout, stderr).toContain("compare/v1.0.0...v1.0.1"); + }).pipe(Effect.provide(platformLayer)), + ); +}); diff --git a/apps/cli/scripts/backfill-release-notes.ts b/apps/cli/scripts/backfill-release-notes.ts index e5b81aed56..265b9b9874 100644 --- a/apps/cli/scripts/backfill-release-notes.ts +++ b/apps/cli/scripts/backfill-release-notes.ts @@ -20,11 +20,16 @@ // Without it, raw markdown notes are printed to stdout. import { BunFileSystem, BunPath } from "@effect/platform-bun"; import { $ } from "bun"; -import { Console, Data, Effect, FileSystem, Layer, Path, Schema } from "effect"; +import { Array as Arr, Console, Data, Effect, FileSystem, Layer, Path, Schema } from "effect"; import process from "node:process"; import { parseArgs } from "node:util"; -import semanticRelease from "semantic-release"; -import { channelForTag, releaseBranchForTag } from "./release-channels.ts"; +import semanticRelease, { type AnalyzeCommitsContext } from "semantic-release"; +import { + channelForTag, + isInChannelHistory, + releaseBranchForTag, + releaseTypeBetween, +} from "./release-channels.ts"; class JsonParseFailed extends Data.TaggedError("JsonParseFailed")<{ readonly cause: unknown }> {} @@ -151,19 +156,6 @@ const main = Effect.gen(function* () { $`git -C ${repoRoot} remote get-url origin`.text(), )).trim(); - // Notes refs aren't fetched by `git clone`. Pull from the source repo first - // (network-free), then from origin so even tags whose notes haven't been - // sync'd locally yet are available. - yield* shell(() => - $`git -C ${clone} fetch --no-tags --quiet ${repoRoot} +refs/notes/*:refs/notes/*` - .nothrow() - .quiet(), - ); - yield* shell(() => - $`git -C ${clone} fetch --no-tags --quiet ${originUrl} +refs/notes/*:refs/notes/*` - .nothrow() - .quiet(), - ); // One fetch per branch: git aborts a multi-ref fetch when any named ref is missing, and `next` // may not exist. The `v*.x` glob matches zero or more branches without failing. for (const branchSpec of ["main", "develop", "next", "v*.x"]) { @@ -207,6 +199,13 @@ const main = Effect.gen(function* () { for (const ct of coincidentTags) { yield* shell(() => $`git -C ${clone} tag -d ${ct}`.quiet().nothrow()); } + const targetChannel = channelForTag(tag); + const tagsOutsideChannel = (yield* shell(() => $`git -C ${clone} tag --list`.text())) + .split("\n") + .filter((t) => t && !isInChannelHistory(t, targetChannel)); + if (tagsOutsideChannel.length > 0) { + yield* shell(() => $`git -C ${clone} tag -d ${tagsOutsideChannel}`.quiet()); + } yield* shell(() => $`git -C ${clone} checkout -B ${branch} ${sha} --quiet`); // The clone only carries refs/heads/$BRANCH locally; seed the other @@ -224,23 +223,34 @@ const main = Effect.gen(function* () { } } - // semantic-release's notes reader returns channels=[null] for any tag - // missing an annotation, and the prerelease filter drops those tags - // entirely — walking lastRelease back far enough to drag unrelated commits - // into the changelog. Seed a channel note for every reachable tag that - // lacks one, inferring the channel from the tag name. + // semantic-release reads one channel note per commit (semantic-release/semantic-release#4073), + // so the clone keeps only this channel's prereleases and rebuilds one note per tagged + // commit from tag names. const mergedTagsOut = yield* shell(() => $`git -C ${clone} tag --merged HEAD --sort=v:refname`.text(), ); const mergedTags = mergedTagsOut.split("\n").filter((t) => t && t !== tag); - for (const prevTag of mergedTags) { - const noteCheck = yield* shell(() => - $`git -C ${clone} notes --ref semantic-release show ${prevTag}`.nothrow().quiet(), + const taggedCommits = (yield* shell(() => + $`git -C ${clone} rev-parse ${mergedTags.map((t) => `refs/tags/${t}^{commit}`)}`.text(), + )) + .split("\n") + .filter(Boolean); + if (taggedCommits.length !== mergedTags.length) { + yield* Console.error( + `Resolved ${taggedCommits.length} commits for ${mergedTags.length} tags merged into ${tag}`, ); - if (noteCheck.exitCode === 0) continue; - const payload = yield* encodeJson({ channels: [channelForTag(prevTag)] }); + return yield* Effect.sync(() => process.exit(1)); + } + const channelByCommit = new Map>(); + for (const [prevTag, commit] of Arr.zip(mergedTags, taggedCommits)) { + if (prevTag.includes("-") || !channelByCommit.has(commit)) { + channelByCommit.set(commit, channelForTag(prevTag)); + } + } + for (const [commit, channel] of channelByCommit) { + const payload = yield* encodeJson({ channels: [channel] }); yield* shell(() => - $`git -C ${clone} notes --ref semantic-release add -f -m ${payload} ${prevTag}^{commit}`.quiet(), + $`git -C ${clone} notes --ref semantic-release add -f -m ${payload} ${commit}`.quiet(), ); } @@ -280,10 +290,25 @@ const main = Effect.gen(function* () { delete childEnv[key]; } + const expected = tag.replace(/^v/, ""); + // A dispatch `version` can force a release its commits do not imply (e.g. only `ci:` commits). + const releaseFloor = { + analyzeCommits: (_config: unknown, context: AnalyzeCommitsContext) => + releaseTypeBetween(context.lastRelease.version, expected), + }; + const configuredPlugins = Array.isArray(rootPkg.release?.plugins) + ? rootPkg.release.plugins + : []; + const result = yield* Effect.tryPromise({ try: () => semanticRelease( - { dryRun: true, noCi: true, repositoryUrl: repoUrl }, + { + dryRun: true, + noCi: true, + repositoryUrl: repoUrl, + plugins: [...configuredPlugins, releaseFloor], + }, { cwd: path.join(clone, "apps/cli"), env: childEnv, @@ -299,7 +324,6 @@ const main = Effect.gen(function* () { return yield* Effect.sync(() => process.exit(1)); } - const expected = tag.replace(/^v/, ""); if (result.nextRelease.version !== expected) { yield* Console.error( `semantic-release computed v${result.nextRelease.version} but expected ${tag}; ` + diff --git a/apps/cli/scripts/npm-registry.ts b/apps/cli/scripts/npm-registry.ts index 3759b8858c..2f4863be96 100644 --- a/apps/cli/scripts/npm-registry.ts +++ b/apps/cli/scripts/npm-registry.ts @@ -2,7 +2,7 @@ import { Data, Effect, Ref, Schedule, Schema } from "effect"; import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/http"; const POLL_INTERVAL = "10 seconds"; -const POLL_BUDGET = "15 minutes"; +const POLL_BUDGET = "30 minutes"; const PROBE_TIMEOUT = "30 seconds"; const RETRYABLE_CLIENT_STATUSES = new Set([404, 408, 429]); diff --git a/apps/cli/scripts/npm-registry.unit.test.ts b/apps/cli/scripts/npm-registry.unit.test.ts index f2639a2822..56923cd397 100644 --- a/apps/cli/scripts/npm-registry.unit.test.ts +++ b/apps/cli/scripts/npm-registry.unit.test.ts @@ -108,7 +108,7 @@ describe("waitUntilInstallable", () => { onInstallable: (pkg) => Effect.sync(() => void installable.push(pkg.name)), }).pipe(Effect.flip, Effect.forkChild({ startImmediately: true })); - yield* TestClock.adjust("14 minutes"); + yield* TestClock.adjust("29 minutes"); expect(fiber.pollUnsafe()).toBeUndefined(); yield* TestClock.adjust("2 minutes"); diff --git a/apps/cli/scripts/release-channels.ts b/apps/cli/scripts/release-channels.ts index bf3746c553..b3c31384e3 100644 --- a/apps/cli/scripts/release-channels.ts +++ b/apps/cli/scripts/release-channels.ts @@ -1,3 +1,5 @@ +import { semver } from "bun"; + const MAINTENANCE_TAG = /^v[1-9]\d*\.stable$/; const PUBLISHABLE_TAGS = new Set(["latest", "alpha", "beta", "next"]); @@ -38,3 +40,31 @@ export function channelForTag(tag: string): "beta" | "next" | "alpha" | "latest" if (tag.includes("-alpha.")) return "alpha"; return "latest"; } + +/** Whether `tag` is a CLI release tag in `channel`'s history: stable tags always, prereleases only on their own channel. */ +export function isInChannelHistory( + tag: string, + channel: ReturnType, +): boolean { + return /^v\d/.test(tag) && (!tag.includes("-") || channelForTag(tag) === channel); +} + +function versionBase(version: string): readonly [number, number, number] | undefined { + const match = /^(\d+)\.(\d+)\.(\d+)/.exec(version); + return match ? [Number(match[1]), Number(match[2]), Number(match[3])] : undefined; +} + +/** The smallest release type that takes `lastVersion` to `version`, or null unless `version` is later by SemVer precedence. */ +export function releaseTypeBetween( + lastVersion: string | undefined, + version: string, +): "major" | "minor" | "patch" | null { + if (lastVersion === undefined || semver.order(version, lastVersion) !== 1) return null; + const last = versionBase(lastVersion); + const next = versionBase(version); + if (!last || !next) return null; + + if (next[0] !== last[0]) return "major"; + if (next[1] !== last[1]) return "minor"; + return "patch"; +} diff --git a/apps/cli/scripts/release-channels.unit.test.ts b/apps/cli/scripts/release-channels.unit.test.ts index 5a157f5517..439c21c027 100644 --- a/apps/cli/scripts/release-channels.unit.test.ts +++ b/apps/cli/scripts/release-channels.unit.test.ts @@ -3,8 +3,10 @@ import { channelForTag, homebrewClassName, homebrewConflictsLine, + isInChannelHistory, isPublishableNpmTag, releaseBranchForTag, + releaseTypeBetween, } from "./release-channels.ts"; describe("isPublishableNpmTag", () => { @@ -72,3 +74,45 @@ describe("channelForTag", () => { expect(channelForTag(tag)).toBe(expected); }); }); + +describe("isInChannelHistory", () => { + test.each([ + ["v3.1.0-beta.1", "beta", true], + ["v3.0.0", "beta", true], + ["v3.1.0-next.1", "beta", false], + ["v2.99.0-alpha.1", "beta", false], + ["v3.1.0-next.1", "next", true], + ["v3.0.0", "next", true], + ["v3.1.0-beta.1", "next", false], + ["v2.99.0-alpha.1", "next", false], + ["v3.0.0", "latest", true], + ["v2.151.1", "latest", true], + ["v3.1.0-beta.1", "latest", false], + ["v3.1.0-next.1", "latest", false], + ["v2.99.0-alpha.1", "latest", false], + ["config-v1.2.0", "latest", false], + ["apps/cli-go/pkg/v1.0.0", "latest", false], + ["config-v1.2.0", "beta", false], + ] as const)("%s in the %s history -> %s", (tag, channel, expected) => { + expect(isInChannelHistory(tag, channel)).toBe(expected); + }); +}); + +describe("releaseTypeBetween", () => { + test.each([ + ["2.122.0", "2.122.1", "patch"], + ["2.122.1", "2.123.0", "minor"], + ["2.123.4", "3.0.0", "major"], + ["3.1.0-beta.4", "3.1.0-beta.5", "patch"], + ["3.0.0", "3.1.0-beta.1", "minor"], + ["2.151.0", "3.0.0-beta.1", "major"], + [undefined, "3.0.0", null], + ["2.122.1", "2.122.1", null], + ["2.122.1", "2.122.0", null], + ["3.1.0", "2.151.0", null], + ["3.1.0-beta.5", "3.1.0-beta.4", null], + ["3.1.0", "3.1.0-beta.1", null], + ] as const)("%s -> %s requires %s", (lastVersion, version, expected) => { + expect(releaseTypeBetween(lastVersion, version)).toBe(expected); + }); +}); diff --git a/docs/adr/0028-release-branches-and-maintenance-lines.md b/docs/adr/0028-release-branches-and-maintenance-lines.md index 8a56bd3c3d..fb59b90e2e 100644 --- a/docs/adr/0028-release-branches-and-maintenance-lines.md +++ b/docs/adr/0028-release-branches-and-maintenance-lines.md @@ -18,7 +18,7 @@ Add two release channels and the branches behind them. `develop` and `main` are | `next` | Breaking-change integration, tracks `develop` | `X.Y.Z-next.N` | `next` | none | prerelease | | `v.x` | Maintenance for a past major | `N.Y.Z` | `v.stable` | `supabase@` / `supabase-v-stable` | published, never latest | -- **`next`** is one long-lived branch reused for every major. Breaking PRs target it with a `type(scope)!:` title, enforced by the PR title lint. Every push runs the full test suite before publishing. The prerelease identifier is `next`, not `beta`, so versions do not collide on npm after a cut. Publishing stays OIDC-only, so the stable release does not move the `next` dist-tag; `supabase@next` catches up on the next push to `next`. +- **`next`** is one long-lived branch reused for every major. Breaking PRs target it with a `type(scope)!:` title, enforced by the PR title lint. Every push that carries commits beyond `develop` runs the full test suite before publishing; a push without any releases nothing, because semantic-release reads channel notes per commit ([semantic-release#4073](https://github.com/semantic-release/semantic-release/issues/4073)) and cannot tell a `next` tag from a `beta` tag on the same commit. The prerelease identifier is `next`, not `beta`, so versions do not collide on npm after a cut. Publishing stays OIDC-only, so the stable release does not move the `next` dist-tag; `supabase@next` catches up on the next release from `next`. - **`develop` → `next`** is kept in sync on every push, and `main` → `develop` after every stable release. A clean merge is pushed directly. A conflict opens a PR from `sync/-into-` (at the source tip) into the target, and approving it fast-forwards the target to the PR head. - **Fast-forward on approval** applies to the deploy PR (`develop` → `main`), the major cut (`next` → `develop`), and sync PRs. The approver must have write access; the approved commit must be the PR head with the required checks green; the target must fast-forward (for a sync PR that is stale because the target or the source moved, a clean merge lands directly, as for a clean sync; only a new conflict or a target that keeps moving sends it back for re-approval); and a major version bump on the deploy PR needs the `release-major` label. The merge button is blocked on these PRs because the repository is squash-only and squashing destroys the ancestry. - **`v.x`** is created by hand from the last stable `N.*` tag at the cut of the next major. It is released by dispatching `release.yml` on that ref with `channel=maintenance`; the workflow, scripts, and live e2e tests come from the frozen commit. A wildcard maintenance entry in the semantic-release config covers every such branch. Maintenance releases publish `v.stable` on npm and are never marked latest. @@ -48,8 +48,8 @@ The operational runbook, including the major-cut steps, is [`apps/cli/docs/relea ### Negative - Release-infrastructure fixes must be cherry-picked to each active `v.x` by hand; Dependabot targets `develop` only. -- The `next` dist-tag can lag behind `latest` until the next push to `next`. -- Every `develop` push triggers a sync and, once `next` exists, a `next` test run and release. +- The `next` dist-tag can lag behind `latest` until the next release from `next`. +- Every `develop` push triggers a sync into `next`; `next` runs its tests and releases only when it carries commits beyond `develop`. - Rulesets for `next` and `v*.x` and the `release-major` label are manual setup. - `supabase@` and `supabase` both install `bin/supabase`, so Homebrew can link only one. - Homebrew and Scoop for `next` are not published. From 96eab1d048ffe4ea4dfd7035f6430217ca463631 Mon Sep 17 00:00:00 2001 From: Julien Goux Date: Tue, 3 Nov 2026 14:19:19 +0000 Subject: [PATCH 044/101] ci(release): attest GitHub Release assets with build provenance (#7467) Adds a build provenance attestation for every GitHub Release asset, created in the `publish` job right before the release is made immutable. - `actions/attest` signs the versioned tarballs, zips, deb/rpm/apk packages, `checksums.txt` and the `install` script. The unversioned `supabase__.tar.gz` aliases are byte-identical copies, so the same attestations cover them by digest. - `attestations: write` is granted to the `publish` job and to both callers of `release-shared.yml` (`release.yml` and `release-smoke-test.yml`), since a reusable workflow cannot exceed its caller's permissions. Anyone can now verify a downloaded asset with `gh attestation verify --repo supabase/cli`. Once a release ships with attestations, the aqua registry entry for `supabase/cli` can add `github_artifact_attestations` with `signer_workflow: supabase/cli/.github/workflows/release-shared.yml`, which makes mise and aqua verify each download against it. This touches release infrastructure, so it needs a cherry-pick to `v2.x`. --- .github/workflows/release-shared.yml | 11 +++++++++++ .github/workflows/release-smoke-test.yml | 1 + .github/workflows/release.yml | 1 + 3 files changed, 13 insertions(+) diff --git a/.github/workflows/release-shared.yml b/.github/workflows/release-shared.yml index add707ada2..45c2b4d04d 100644 --- a/.github/workflows/release-shared.yml +++ b/.github/workflows/release-shared.yml @@ -283,6 +283,7 @@ jobs: LINEAR_CLI_STABLE_RELEASE_ACCESS_KEY: ${{ secrets.LINEAR_CLI_STABLE_RELEASE_ACCESS_KEY }} LINEAR_CLI_BETA_RELEASE_ACCESS_KEY: ${{ secrets.LINEAR_CLI_BETA_RELEASE_ACCESS_KEY }} permissions: + attestations: write contents: write id-token: write steps: @@ -409,6 +410,16 @@ jobs: GH_TOKEN: ${{ github.token }} run: bun apps/cli/scripts/upload-release-assets.ts verify --version "${VERSION}" + # Lets installers such as aqua and mise verify each asset with `gh attestation verify`; the + # unversioned tarball aliases share their versioned twin's digest, so they are covered too. + - name: Attest release assets + uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2 + with: + subject-path: | + dist/supabase_${{ inputs.version }}_* + dist/checksums.txt + install + - name: Publish GitHub Release (immutable) env: GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/release-smoke-test.yml b/.github/workflows/release-smoke-test.yml index 0aa4ff6b17..d4521493ab 100644 --- a/.github/workflows/release-smoke-test.yml +++ b/.github/workflows/release-smoke-test.yml @@ -35,6 +35,7 @@ jobs: # dry_run here, but GitHub validates nested-workflow permissions at startup. permissions: actions: read + attestations: write contents: write id-token: write pull-requests: write diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index de02c7ed9f..6f32d098ac 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -341,6 +341,7 @@ jobs: # those granted to the calling job, so this must be declared here too. permissions: actions: read + attestations: write contents: write id-token: write pull-requests: write From f4212ae975bdec7fe9af30e2490a0532b0971b58 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 15:36:29 +0000 Subject: [PATCH 045/101] chore(stack): bump realtime to v2.152.0-r0 (#7470) This bumps realtime from v2.151.7-r0 to v2.152.0-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.152.0-r0 Planned after supabase/slim-services published v2.152.0-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 01130067ea..55712c1730 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.151.7 AS realtime +FROM supabase/realtime:v2.152.0 AS realtime FROM supabase/storage-api:v1.89.2 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 9792523fe4..ac01c3c090 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -464,23 +464,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.151.7", + upstreamVersion: "v2.152.0", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.151.7-r0@sha256:dbb0ff68f110ae13fd5370dee6d30d807e47bf474ef0bf4c6cb7a3900046ec7d", - upstreamImage: "supabase/realtime:v2.151.7", + "ghcr.io/supabase/cli/realtime:v2.152.0-r0@sha256:c02e01a456e601fb3fbbbfd0e1f4d98a3908d7a09feabedb5dca3fae25489689", + upstreamImage: "supabase/realtime:v2.152.0", natives: { "darwin-arm64": { - archive: "87281c452387b1baebdbd9f2c7e18a4316f3fd9e6cada7874c8579cfef24ea5b", - manifest: "365c7bea189170bd2f60c116eae1614ea4ecc222b8219fd2cedc3417cc9bc230", + archive: "508fb18501ea90e6caa865b3c57dbc823962b78d8d5d452bac0764b5bb83da3f", + manifest: "43a511b780eaf4ead230a71e49aac1d6f4014ad860227355b72c29fc3a72f97d", }, "linux-amd64": { - archive: "3d1dc4603387d9814914ceba9e9112cea46b654cd12c7d6cabc70ebd3dbcfd48", - manifest: "7a1c7fd071832cf5ca691d8c19da3b9eb3504bf94cd97957e94b862e34493d81", + archive: "cdce982bfde19044932d67d35878706c30e8b666bd455661af26ec7c21627c15", + manifest: "d059496942a05a92156c53317bcb114591d608543a0e8ada603a391bab835a4f", }, "linux-arm64": { - archive: "987bb09f40a740aa1a3d59163ae0c2aa23c7f58660b45aee7d14f09abb944201", - manifest: "03bf973d224269a95fad1a16645df14f904d7de2856d7044a62343cf90c0b8a6", + archive: "227fea5401b473ec6314e16a73618216a9cdfe5fa76a85e135f5043881f51ecf", + manifest: "93c8c13894a4bf398ceda81f75762600e49dff783c2cb97411b5c1340ea34b65", }, }, }, From 0a245f236c9afd3324e38c3302d6fc9845b777c0 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 3 Nov 2026 15:39:51 +0000 Subject: [PATCH 046/101] feat(cli): record which config key failed in config-load telemetry (CLI-2912) (#7468) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary A failed `cli_command_executed` event shows that a command failed on config (`error_category: invalid_config`, `error_fingerprint: tag:StartConfigLoadError`), but not which key caused it. So we can't tell how many config-load failures come from the removed `[inbucket]` section and how many from everything else. This adds a `config_key` property to those events. - `CliConfigValues.load` and `ResolvedCliConfig.get` record the key through the existing `recordCommandTelemetry` sink when they fail. It has to be recorded at load time because command wrappers such as `StartConfigLoadError` keep only the message and drop the original error - The key comes from three errors: - `CliConfigRemovedKeyError`: every removed key, sorted and comma-joined - `CliConfigValueError`: its path - `CliConfigParseError`: the first issue path, when the cause is a schema error - Only key names are recorded, and only from a fixed list built from the CLI's own config schema and registry, plus the four removed keys. Values, file paths, remote names and other user-chosen names never reach the event. A user-chosen segment collapses to its nearest known parent, so `functions.my-fn.verify_jwt` records `functions` and a custom `auth.external.` records `auth.external` - `withCommandTelemetry` only adds `config_key` when the event's `error_category` is `invalid_config`, so a config failure that a command swallowed can't label a success or an unrelated failure - `apps/cli/docs/analytics.md` documents the new property The list of removed keys is kept in the CLI rather than exported from `@supabase/config`, so this change stays out of that package's release train. ## Linked issue Closes CLI-2912 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- apps/cli/docs/analytics.md | 6 +- .../functions/new/new.integration.test.ts | 52 +++- apps/cli/src/config/cli-config-failure-key.ts | 105 +++++++++ .../cli-config-failure-key.unit.test.ts | 222 ++++++++++++++++++ .../cli-config-values.integration.test.ts | 131 +++++++++++ .../cli/src/config/cli-config-values.layer.ts | 23 +- .../cli/src/shared/telemetry/event-catalog.ts | 2 + .../telemetry/command-telemetry-attributes.ts | 2 + apps/cli/src/telemetry/command-telemetry.ts | 10 +- .../telemetry/command-telemetry.unit.test.ts | 91 +++++++ 10 files changed, 637 insertions(+), 7 deletions(-) create mode 100644 apps/cli/src/config/cli-config-failure-key.ts create mode 100644 apps/cli/src/config/cli-config-failure-key.unit.test.ts diff --git a/apps/cli/docs/analytics.md b/apps/cli/docs/analytics.md index a557223755..f56f6bc8c3 100644 --- a/apps/cli/docs/analytics.md +++ b/apps/cli/docs/analytics.md @@ -78,8 +78,12 @@ sanitized error classification: - `has_suggestion` - `suggestion_type` - `suggested_command` (only when the remediation is an allowlisted command) +- `config_key` (only on `invalid_config` failures from config resolution: the failing config key + path, such as `inbucket` or `auth.email.smtp.port`, comma-joined when several removed keys fail; + keys only, never values, file paths or user-chosen names, which collapse to their nearest known + parent key) -These values come exclusively from the closed taxonomy in +Apart from `config_key`, these values come exclusively from the closed taxonomy in `src/shared/telemetry/error-actionability.ts` — never from raw error text — and the KPI query semantics (strict recovery, repeat errors, internal/unknown bug rate) are documented there in `CliErrorActionabilityMetricDefinitions`. diff --git a/apps/cli/src/commands/functions/new/new.integration.test.ts b/apps/cli/src/commands/functions/new/new.integration.test.ts index eec34a04f2..0463816039 100644 --- a/apps/cli/src/commands/functions/new/new.integration.test.ts +++ b/apps/cli/src/commands/functions/new/new.integration.test.ts @@ -1,6 +1,6 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, Exit, FileSystem, Layer, Path } from "effect"; +import { Effect, Exit, FileSystem, Layer, Path, Stdio } from "effect"; import { mockCommandSettings, @@ -8,8 +8,16 @@ import { useTempWorkdir, withConfigEnv, } from "../../../../tests/helpers/command-mocks.ts"; -import { mockOutput, mockStdin, mockTty } from "../../../../tests/helpers/mocks.ts"; +import { + mockContextualAnalytics, + mockOutput, + mockProcessControl, + mockStdin, + mockTty, +} from "../../../../tests/helpers/mocks.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; +import { commandRuntimeLayer } from "../../../shared/runtime/command-runtime.layer.ts"; +import { withCommandTelemetry } from "../../../telemetry/command-telemetry.ts"; import { YesFlag } from "../../../command-internal/global-flags.ts"; import { CliConfigValueError } from "../../../config/cli-config.errors.ts"; import { configValuesLayer } from "../../../../tests/helpers/config-values-layer.ts"; @@ -192,6 +200,44 @@ describe("functions new integration", () => { }).pipe(Effect.provide(layer), (body) => withConfigEnv({ SUPABASE_API_PORT: "abc" }, body)); }); + it.effect("records the failing config key on the failed command event", () => { + const { layer, workdir } = setup(); + const analytics = mockContextualAnalytics(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(workdir, "supabase", "config.toml"), + "[inbucket]\nenabled = true\n", + ); + + const exit = yield* functionsNew({ functionName: "hello", auth: "apikey" }).pipe( + withCommandTelemetry(), + Effect.exit, + ); + + expect(Exit.isFailure(exit)).toBe(true); + expect(analytics.captured).toHaveLength(1); + expect(analytics.captured[0]?.properties).toMatchObject({ + command: "functions new", + exit_code: 1, + error_category: "invalid_config", + config_key: "inbucket", + }); + }).pipe( + Effect.provide( + Layer.mergeAll( + layer, + analytics.layer, + mockProcessControl().layer, + Stdio.layerTest({ args: Effect.succeed(["functions", "new", "hello"]) }), + commandRuntimeLayer(["functions", "new"]).pipe(Layer.provide(BunCrypto.layer)), + ), + ), + ); + }); + it.effect("warns and skips the config append when the function is already declared", () => { const { layer, out, workdir } = setup(); return Effect.gen(function* () { diff --git a/apps/cli/src/config/cli-config-failure-key.ts b/apps/cli/src/config/cli-config-failure-key.ts new file mode 100644 index 0000000000..97b169e43f --- /dev/null +++ b/apps/cli/src/config/cli-config-failure-key.ts @@ -0,0 +1,105 @@ +import { + CliConfigSchema, + type CliConfigParseError, + type CliConfigRemovedKeyError, +} from "@supabase/config"; +import { Predicate, Schema, type SchemaIssue } from "effect"; + +import { cliConfigDocumentOnlyPaths, cliConfigRegistry } from "./cli-config-keys.ts"; +import type { CliConfigValueError } from "./cli-config.errors.ts"; + +const REMOVED_KEY_PATHS: ReadonlyArray = [ + "inbucket", + "experimental.orioledb_version", + "auth.external.linkedin", + "auth.external.slack", +]; + +const withAncestors = (path: string): ReadonlyArray => { + const segments = path.split("."); + return segments.map((_, index) => segments.slice(0, index + 1).join(".")); +}; + +const KNOWN_PATHS: ReadonlySet = new Set( + [ + ...Object.keys(CliConfigSchema.fields), + ...cliConfigRegistry.keys.map((key) => key.path), + ...cliConfigRegistry.families.map((family) => family.prefix), + ...cliConfigDocumentOnlyPaths, + ].flatMap(withAncestors), +); + +const taggedAs = + (tag: E["_tag"]) => + (error: unknown): error is E => + Predicate.isTagged(error, tag); + +const isRemovedKeyError = taggedAs("CliConfigRemovedKeyError"); +const isParseError = taggedAs("CliConfigParseError"); +const isValueError = taggedAs("CliConfigValueError"); + +const removedKeyPath = (displayKey: string): string | undefined => { + const path = displayKey.replaceAll(/[[\]]/g, ""); + return REMOVED_KEY_PATHS.find( + (canonical) => + path === canonical || (path.startsWith("remotes.") && path.endsWith(`.${canonical}`)), + ); +}; + +const knownPrefix = (segments: ReadonlyArray): string | undefined => { + for (let length = segments.length; length > 0; length--) { + const candidate = segments.slice(0, length).join("."); + if (KNOWN_PATHS.has(candidate)) return candidate; + } + return undefined; +}; + +/** Remote blocks decode as one record, so their issue paths start with the remote name instead of `remotes`. */ +const decodedAsRemoteBlocks = (issue: SchemaIssue.Issue): boolean => + issue._tag === "Composite" && + issue.ast._tag === "Objects" && + issue.ast.propertySignatures.length === 0; + +const firstIssuePath = ( + issue: SchemaIssue.Issue, + path: ReadonlyArray, +): ReadonlyArray | undefined => { + switch (issue._tag) { + case "Pointer": + return firstIssuePath(issue.issue, [...path, ...issue.path.map(String)]); + case "Filter": + case "Encoding": + return firstIssuePath(issue.issue, path); + case "Composite": + case "AnyOf": + for (const nested of issue.issues) { + const found = firstIssuePath(nested, path); + if (found !== undefined) return found; + } + return undefined; + default: + return path.length > 0 ? path : undefined; + } +}; + +/** The source-owned config key path a config load failure names, never a value, file path or user-chosen name. */ +export const cliConfigFailureKey = (error: unknown): string | undefined => { + if (isRemovedKeyError(error)) { + const matched = new Set(); + for (const { key } of error.removedKeys) { + const canonical = removedKeyPath(key); + if (canonical !== undefined) matched.add(canonical); + } + return matched.size === 0 ? undefined : [...matched].sort().join(","); + } + if (isValueError(error)) { + return knownPrefix(error.path.split(".")); + } + if (isParseError(error) && Schema.isSchemaError(error.cause)) { + const { issue } = error.cause; + const path = firstIssuePath(issue, []); + if (path === undefined) return undefined; + return knownPrefix(decodedAsRemoteBlocks(issue) ? path.slice(1) : path); + } + return undefined; +}; diff --git a/apps/cli/src/config/cli-config-failure-key.unit.test.ts b/apps/cli/src/config/cli-config-failure-key.unit.test.ts new file mode 100644 index 0000000000..0c916099e1 --- /dev/null +++ b/apps/cli/src/config/cli-config-failure-key.unit.test.ts @@ -0,0 +1,222 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { CliConfigRemovedKeyError } from "@supabase/config"; +import { Effect, Option, Schema } from "effect"; + +import { withConfigEnv } from "../../tests/helpers/command-mocks.ts"; +import { definedEnv } from "../../tests/helpers/config-env-pins.ts"; +import { configValuesLayer } from "../../tests/helpers/config-values-layer.ts"; +import { createStackConfigProject } from "../../tests/helpers/stack-config.ts"; +import { cliConfigFailureKey } from "./cli-config-failure-key.ts"; +import { CliConfigValueError } from "./cli-config.errors.ts"; +import { CliConfigValues } from "./cli-config-values.service.ts"; + +const SECRET = "sk_live_4eC39Hq"; + +const REMOTE_REF = "abcdefghijklmnopqrst"; + +const loadFailure = (config: string) => + Effect.gen(function* () { + const workdir = yield* createStackConfigProject(config, { + prefix: "supabase-config-failure-key-", + }); + return yield* CliConfigValues.use((values) => + values.load({ workdir, projectRef: Option.none() }), + ).pipe(Effect.provide(configValuesLayer()), Effect.flip); + }).pipe( + Effect.provide(BunServices.layer), + (effect) => withConfigEnv(definedEnv({}), effect), + Effect.scoped, + ); + +const remoteDbPort = (remote: string) => + `[remotes."${remote}"]\nproject_id = "${REMOTE_REF}"\n[remotes."${remote}".db]\nport = "x"\n`; + +const remoteDbSection = (remote: string) => + `[remotes."${remote}"]\nproject_id = "${REMOTE_REF}"\ndb = "invalid"\n`; + +describe("cliConfigFailureKey for a removed key", () => { + it.live.each([ + { + name: "the [inbucket] table", + config: "[inbucket]\nenabled = true\n", + key: "inbucket", + }, + { + name: "experimental.orioledb_version", + config: '[experimental]\norioledb_version = "15.1.0.150"\n', + key: "experimental.orioledb_version", + }, + { + name: "the linkedin provider", + config: "[auth.external.linkedin]\nenabled = true\n", + key: "auth.external.linkedin", + }, + { + name: "the slack provider", + config: "[auth.external.slack]\nenabled = true\n", + key: "auth.external.slack", + }, + { + name: "a remote block whose name contains dots and a secret", + config: `[remotes."prod.${SECRET}.example.com".inbucket]\nenabled = true\n`, + key: "inbucket", + }, + { + name: "a removed provider in a remote block", + config: "[remotes.staging.auth.external.slack]\nenabled = true\n", + key: "auth.external.slack", + }, + { + name: "several keys, joined sorted without repeats", + config: [ + "[inbucket]", + "enabled = true", + "[experimental]", + 'orioledb_version = "15.1.0.150"', + "[auth.external.slack]", + "enabled = true", + "[remotes.staging.inbucket]", + "enabled = true", + ].join("\n"), + key: "auth.external.slack,experimental.orioledb_version,inbucket", + }, + ])("names $name", ({ config, key }) => + Effect.gen(function* () { + const error = yield* loadFailure(config); + + expect(error._tag).toBe("CliConfigRemovedKeyError"); + expect(cliConfigFailureKey(error)).toBe(key); + }), + ); + + it("drops removed keys outside the source-owned vocabulary", () => { + const removedKeys = (...keys: ReadonlyArray) => + new CliConfigRemovedKeyError({ + path: "supabase/config.toml", + format: "toml", + removedKeys: keys.map((key) => ({ key, replacement: "[replacement]" })), + }); + + expect(cliConfigFailureKey(removedKeys(`[${SECRET}]`, "[inbucket]"))).toBe("inbucket"); + expect(cliConfigFailureKey(removedKeys(`[${SECRET}]`))).toBeUndefined(); + }); +}); + +describe("cliConfigFailureKey for an invalid value", () => { + it.live.each([ + { + name: "a registry key", + config: "[db]\nport = 70000\n", + key: "db.port", + }, + { + name: "a provider the source names", + config: "[auth.external.github]\nclient_id = 5\n", + key: "auth.external.github.client_id", + }, + { + name: "a user-chosen provider name, collapsed to its family", + config: "[auth.external.my_provider]\nclient_id = 5\n", + key: "auth.external", + }, + { + name: "a user-chosen provider name containing dots and a secret", + config: `[auth.external."corp.${SECRET}.sso"]\nclient_id = 5\n`, + key: "auth.external", + }, + ])("names $name", ({ config, key }) => + Effect.gen(function* () { + const error = yield* loadFailure(config); + + expect(error._tag).toBe("CliConfigValueError"); + expect(cliConfigFailureKey(error)).toBe(key); + }), + ); + + it("names nothing for a path outside the source-owned vocabulary", () => { + const error = new CliConfigValueError({ + path: `vendor_tables.${SECRET}.password`, + tier: "config", + message: "invalid", + }); + + expect(cliConfigFailureKey(error)).toBeUndefined(); + }); +}); + +describe("cliConfigFailureKey for a config document that does not decode", () => { + it.live.each([ + { + name: "a section table written as an array", + config: "[[auth.hook.custom_access_token]]\n", + key: "auth.hook.custom_access_token", + }, + { + name: "a user-named bucket, collapsed to its section", + config: `[storage.buckets."${SECRET}.assets"]\npublic = "yes"\n`, + key: "storage", + }, + { + name: "a user-named function, collapsed to functions", + config: '[functions.my-fn]\nverify_jwt = "x"\n', + key: "functions", + }, + { + name: "a key inside a remote block", + config: remoteDbPort("staging"), + key: "db.port", + }, + { + name: "a key inside a remote named after a config section", + config: remoteDbPort("auth"), + key: "db.port", + }, + { + name: "a section inside a remote named after a config section", + config: remoteDbSection("auth"), + key: "db", + }, + ])("names $name", ({ config, key }) => + Effect.gen(function* () { + const error = yield* loadFailure(config); + + expect(error._tag).toBe("CliConfigParseError"); + expect(Schema.isSchemaError("cause" in error ? error.cause : undefined)).toBe(true); + expect(cliConfigFailureKey(error)).toBe(key); + }), + ); + + it.live("names the key without the dotted, secret-looking remote name", () => + Effect.gen(function* () { + const error = yield* loadFailure(remoteDbPort(`prod.${SECRET}.example.com`)); + + expect(error._tag).toBe("CliConfigParseError"); + expect(Schema.isSchemaError("cause" in error ? error.cause : undefined)).toBe(true); + const key = cliConfigFailureKey(error); + expect(key).toBe("db.port"); + expect(key).not.toContain(SECRET); + }), + ); + + it.live("names nothing for a TOML syntax error", () => + Effect.gen(function* () { + const error = yield* loadFailure("[db\nport = "); + + expect(error._tag).toBe("CliConfigParseError"); + expect(Schema.isSchemaError("cause" in error ? error.cause : undefined)).toBe(false); + expect(cliConfigFailureKey(error)).toBeUndefined(); + }), + ); +}); + +describe("cliConfigFailureKey for a failure unrelated to config", () => { + it.each([ + { name: "a plain Error", failure: new Error("db.port is invalid") }, + { name: "a string", failure: "db.port" }, + { name: "undefined", failure: undefined }, + { name: "a lookalike with another tag", failure: { _tag: "SomeOtherError", path: "db.port" } }, + ])("names nothing for $name", ({ failure }) => { + expect(cliConfigFailureKey(failure)).toBeUndefined(); + }); +}); diff --git a/apps/cli/src/config/cli-config-values.integration.test.ts b/apps/cli/src/config/cli-config-values.integration.test.ts index fa591823be..5b2661c8f5 100644 --- a/apps/cli/src/config/cli-config-values.integration.test.ts +++ b/apps/cli/src/config/cli-config-values.integration.test.ts @@ -17,6 +17,10 @@ import { loadStackConfig } from "../command-internal/stack-config.ts"; import { CliEnvNameSource } from "../shared/config/cli-config-provider.layer.ts"; import { DebugLogger } from "../shared/output/debug-logger.service.ts"; import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; +import { + CommandTelemetryAttributes, + type CommandTelemetryAttributeValues, +} from "../telemetry/command-telemetry-attributes.ts"; import { CliConfigFlagInputs, makeCliConfigFlagInputs, @@ -2027,3 +2031,130 @@ project_id = "${OTHER}" }).pipe(Effect.provide(BunServices.layer), Effect.scoped), ); }); + +describe("CliConfigValues failure telemetry", () => { + const withCommandRun = ( + body: (telemetry: { + readonly records: ReadonlyArray; + readonly configKey: () => string | undefined; + }) => Effect.Effect, + ) => { + const records: Array = []; + let attributes: CommandTelemetryAttributeValues = {}; + const sink = { + record: (values: CommandTelemetryAttributeValues) => + Effect.sync(() => { + records.push(values); + attributes = { ...attributes, ...values }; + }), + }; + return body({ records, configKey: () => attributes.config_key }).pipe( + Effect.provide(configValuesLayer()), + Effect.provideService(CommandTelemetryAttributes, sink), + ); + }; + + const load = (workdir: string) => + CliConfigValues.use((values) => values.load({ workdir, projectRef: Option.none() })); + + const loadTolerating = (workdir: string) => + CliConfigValues.use((values) => + values.load({ workdir, projectRef: Option.none(), tolerateInvalid: true }), + ); + + it.live("records nothing for a successful load and lookup with no earlier failure", () => + withCommandRun(({ records }) => + Effect.gen(function* () { + const root = yield* project("[db]\nport = 54400\n"); + + const resolvedConfig = yield* load(root); + const port = yield* resolvedConfig.get(CliConfigKeys.db.port); + + expect(port.value).toBe(54400); + expect(records).toEqual([]); + }), + ).pipe(Effect.provide(BunServices.layer), (effect) => withShell({}, effect), Effect.scoped), + ); + + it.live("records the key of a tolerated invalid value when it is read", () => + withCommandRun(({ records }) => + Effect.gen(function* () { + const root = yield* project('[db]\nport = "not-a-port"\n'); + + const resolvedConfig = yield* loadTolerating(root); + expect(records).toEqual([]); + + const error = yield* resolvedConfig.get(CliConfigKeys.db.port).pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "CliConfigValueError", path: "db.port" }); + expect(records).toEqual([{ config_key: "db.port" }]); + }), + ).pipe(Effect.provide(BunServices.layer), (effect) => withShell({}, effect), Effect.scoped), + ); + + it.live("records no key when the load failure names no config key", () => + withCommandRun(({ records }) => + Effect.gen(function* () { + const root = yield* project("[db\nport = "); + + const error = yield* load(root).pipe(Effect.flip); + + expect(error._tag).toBe("CliConfigParseError"); + expect(records).toEqual([]); + }), + ).pipe(Effect.provide(BunServices.layer), (effect) => withShell({}, effect), Effect.scoped), + ); + + it.live("records no key for a keyless load failure after a recovered keyed lookup failure", () => + withCommandRun(({ configKey }) => + Effect.gen(function* () { + const invalid = yield* project('[db]\nport = "not-a-port"\n'); + const malformed = yield* project("[db\nport = "); + + const resolvedConfig = yield* loadTolerating(invalid); + const lookup = yield* resolvedConfig.get(CliConfigKeys.db.port).pipe(Effect.flip); + expect(lookup).toMatchObject({ _tag: "CliConfigValueError", path: "db.port" }); + expect(configKey()).toBe("db.port"); + + const error = yield* load(malformed).pipe(Effect.flip); + + expect(error._tag).toBe("CliConfigParseError"); + expect(configKey()).toBeUndefined(); + }), + ).pipe(Effect.provide(BunServices.layer), (effect) => withShell({}, effect), Effect.scoped), + ); + + it.live("clears the key of a recovered lookup failure when a later load succeeds", () => + withCommandRun(({ configKey }) => + Effect.gen(function* () { + const invalid = yield* project('[db]\nport = "not-a-port"\n'); + const valid = yield* project("[db]\nport = 54400\n"); + + const resolvedConfig = yield* loadTolerating(invalid); + const lookup = yield* resolvedConfig.get(CliConfigKeys.db.port).pipe(Effect.flip); + expect(lookup).toMatchObject({ _tag: "CliConfigValueError", path: "db.port" }); + expect(configKey()).toBe("db.port"); + + yield* load(valid); + + expect(configKey()).toBeUndefined(); + }), + ).pipe(Effect.provide(BunServices.layer), (effect) => withShell({}, effect), Effect.scoped), + ); + + it.live("clears the key of a recovered lookup failure when a later lookup succeeds", () => + withCommandRun(({ configKey }) => + Effect.gen(function* () { + const root = yield* project('[db]\nport = "not-a-port"\n'); + + const resolvedConfig = yield* loadTolerating(root); + yield* resolvedConfig.get(CliConfigKeys.db.port).pipe(Effect.flip); + expect(configKey()).toBe("db.port"); + + yield* resolvedConfig.get(CliConfigKeys.db.majorVersion); + + expect(configKey()).toBeUndefined(); + }), + ).pipe(Effect.provide(BunServices.layer), (effect) => withShell({}, effect), Effect.scoped), + ); +}); diff --git a/apps/cli/src/config/cli-config-values.layer.ts b/apps/cli/src/config/cli-config-values.layer.ts index e9ac913c49..9fb8ce705a 100644 --- a/apps/cli/src/config/cli-config-values.layer.ts +++ b/apps/cli/src/config/cli-config-values.layer.ts @@ -16,6 +16,7 @@ import { Layer, Option, Path, + Ref, Result, } from "effect"; @@ -26,6 +27,8 @@ import { CliConfigLoadError } from "../shared/config/cli-config-load.errors.ts"; import { DebugLogger } from "../shared/output/debug-logger.service.ts"; import { escapeControlCharacters } from "../shared/output/escape-control-characters.ts"; import { Output } from "../shared/output/output.service.ts"; +import { PropConfigKey } from "../shared/telemetry/event-catalog.ts"; +import { recordCommandTelemetry } from "../telemetry/command-telemetry-attributes.ts"; import { CLI_CONFIG_FAMILIES, CLI_CONFIG_REMOVED_ENV_NAMES } from "./cli-config-key-annotations.ts"; import { cloneDocumentRecord, @@ -37,6 +40,7 @@ import { sameDocumentValue, setDocumentValue, } from "./cli-config-document.ts"; +import { cliConfigFailureKey } from "./cli-config-failure-key.ts"; import { CliConfigFlagInputs, cliConfigFlagConflictError } from "./cli-config-flags.ts"; import { decodingFailedMessage, @@ -220,6 +224,21 @@ export const cliConfigValuesLayer = Layer.effect( return output.raw(line, "stderr"); }; + // The latest config load or lookup decides `config_key`, so a recovered failure cannot label + // a later one; only changes are recorded. + const recordedFailureKey = yield* Ref.make(undefined); + const settleFailureKey = (key: string | undefined) => + Ref.getAndSet(recordedFailureKey, key).pipe( + Effect.flatMap((previous) => + previous === key ? Effect.void : recordCommandTelemetry({ [PropConfigKey]: key }), + ), + ); + const tracksConfigFailureKey = (self: Effect.Effect) => + self.pipe( + Effect.tapError((error) => settleFailureKey(cliConfigFailureKey(error))), + Effect.tap(() => settleFailureKey(undefined)), + ); + const parsedDocuments = yield* Cache.makeWith( (workdir: string) => withPlatform(parseCliConfigDocumentFile(workdir, { search: false })), { @@ -534,7 +553,7 @@ export const cliConfigValuesLayer = Layer.effect( const picked = pickCliConfigKey(key, sources); if (Result.isFailure(picked)) return yield* picked.failure; return picked.success; - }); + }).pipe(tracksConfigFailureKey); for (const message of overrideWarnings) yield* warnOnce(message); if (Option.isSome(debugLogger)) { @@ -600,7 +619,7 @@ export const cliConfigValuesLayer = Layer.effect( tolerateUnreadableLinkedRef: target.tolerateUnreadableLinkedRef === true, tolerateInvalid: target.tolerateInvalid === true, }), - ), + ).pipe(tracksConfigFailureKey), writeThrough: (write) => Effect.ensuring(write, invalidate), invalidate, }); diff --git a/apps/cli/src/shared/telemetry/event-catalog.ts b/apps/cli/src/shared/telemetry/event-catalog.ts index 5586717a33..cfd281e979 100644 --- a/apps/cli/src/shared/telemetry/event-catalog.ts +++ b/apps/cli/src/shared/telemetry/event-catalog.ts @@ -45,6 +45,8 @@ export const PropStackBackend = "stack_backend"; export const PropOrioleDb = "orioledb"; // Set only once a stack-backend command resolves its stack's runtime. export const PropStackRuntime = "stack_runtime"; +// Set only on failures classified `invalid_config` whose config key is known. +export const PropConfigKey = "config_key"; // Set only once `functions serve` on the Docker runtime reaches its final outcome. export const PropServeEndReason = "serve_end_reason"; // Set only when the Edge Runtime container's exit code is known at that outcome. diff --git a/apps/cli/src/telemetry/command-telemetry-attributes.ts b/apps/cli/src/telemetry/command-telemetry-attributes.ts index 5f0f77f9b0..58a1ef7ecc 100644 --- a/apps/cli/src/telemetry/command-telemetry-attributes.ts +++ b/apps/cli/src/telemetry/command-telemetry-attributes.ts @@ -2,6 +2,7 @@ import { Context, Effect } from "effect"; import type { StackRuntime } from "../command-internal/stack-runtime.ts"; import type { ServeEndReason } from "../shared/functions/serve.ts"; import { + PropConfigKey, PropEdgeRuntimeExitCode, PropOrioleDb, PropServeEndReason, @@ -10,6 +11,7 @@ import { /** Properties that code below a command handler adds to its `cli_command_executed` event. */ export interface CommandTelemetryAttributeValues { + readonly [PropConfigKey]?: string; readonly [PropOrioleDb]?: boolean; readonly [PropStackRuntime]?: StackRuntime; readonly [PropServeEndReason]?: ServeEndReason; diff --git a/apps/cli/src/telemetry/command-telemetry.ts b/apps/cli/src/telemetry/command-telemetry.ts index 3553d388be..dd4c16b7c7 100644 --- a/apps/cli/src/telemetry/command-telemetry.ts +++ b/apps/cli/src/telemetry/command-telemetry.ts @@ -22,6 +22,7 @@ import { import { Analytics } from "../shared/telemetry/analytics.service.ts"; import { TelemetryRuntime } from "../shared/telemetry/runtime.service.ts"; import { + CliErrorCategory, type CliErrorActionability, classifyCliErrorActionability, unknownProcessControlledFailureActionability, @@ -30,7 +31,9 @@ import { DbAdvisorsFailOnError } from "../commands/db/advisors/advisors.errors.t import { DbLintFailOnError } from "../commands/db/lint/lint.errors.ts"; import { EventCommandExecuted, + PropConfigKey, PropDurationMs, + PropErrorCategory, PropExitCode, PropOutputFormat, PropStackBackend, @@ -513,7 +516,8 @@ function withCommandAnalyticsImplementation { ); }); + describe("config_key", () => { + const capturedEvent = (body: Effect.Effect) => { + const analytics = mockContextualAnalytics(); + + return body.pipe( + withCommandTelemetry(), + Effect.provide( + Layer.mergeAll( + analytics.layer, + mockProcessControl().layer, + mockOutput({ format: "text" }).layer, + Stdio.layerTest({ args: Effect.succeed(["db", "dump", "--local"]) }), + commandRuntimeLayer(["db", "dump"]).pipe(Layer.provide(BunCrypto.layer)), + ), + ), + Effect.exit, + Effect.map((exit) => ({ exit, properties: analytics.captured[0]?.properties })), + ); + }; + + it.live("emits the recorded config key on an invalid_config failure", () => + Effect.gen(function* () { + const { exit, properties } = yield* capturedEvent( + recordCommandTelemetry({ [PropConfigKey]: "db.port" }).pipe( + Effect.andThen( + Effect.fail( + new CliConfigValueError({ path: "db.port", tier: "config", message: "invalid" }), + ), + ), + ), + ); + + expect(Exit.isFailure(exit)).toBe(true); + expect(properties).toMatchObject({ + exit_code: 1, + error_category: "invalid_config", + [PropConfigKey]: "db.port", + }); + }), + ); + + it.live("omits a config key that a later config outcome cleared", () => + Effect.gen(function* () { + const { properties } = yield* capturedEvent( + recordCommandTelemetry({ [PropConfigKey]: "db.port" }).pipe( + Effect.andThen(recordCommandTelemetry({ [PropConfigKey]: undefined })), + Effect.andThen( + Effect.fail( + new CliConfigValueError({ path: "db.port", tier: "config", message: "invalid" }), + ), + ), + ), + ); + + expect(properties).toMatchObject({ exit_code: 1, error_category: "invalid_config" }); + expect(properties).not.toHaveProperty(PropConfigKey); + }), + ); + + it.live("omits the recorded config key from a failure in another category", () => + Effect.gen(function* () { + const { properties } = yield* capturedEvent( + recordCommandTelemetry({ [PropConfigKey]: "db.port", [PropOrioleDb]: true }).pipe( + Effect.andThen(Effect.fail(new DbDumpRunError({ message: "dump failed" }))), + ), + ); + + expect(properties).toMatchObject({ + exit_code: 1, + error_category: "db_connection", + [PropOrioleDb]: true, + }); + expect(properties).not.toHaveProperty(PropConfigKey); + }), + ); + + it.live("omits the recorded config key from a successful run", () => + Effect.gen(function* () { + const { exit, properties } = yield* capturedEvent( + recordCommandTelemetry({ [PropConfigKey]: "db.port", [PropOrioleDb]: true }), + ); + + expect(Exit.isSuccess(exit)).toBe(true); + expect(properties).toMatchObject({ exit_code: 0, [PropOrioleDb]: true }); + expect(properties).not.toHaveProperty(PropConfigKey); + }), + ); + }); + it.live("reports machine output formats emitted through the text layer", () => { const analytics = mockContextualAnalytics(); From 8f9903e3e9385fcacc65be3cb862767f9dc6995b Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 19:35:08 +0000 Subject: [PATCH 047/101] chore(stack): bump storage to v1.89.3-r0 (#7474) This bumps storage from v1.89.2-r0 to v1.89.3-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.89.3-r0 Planned after supabase/slim-services published v1.89.3-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 55712c1730..839d2bbf47 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.152.0 AS realtime -FROM supabase/storage-api:v1.89.2 AS storage +FROM supabase/storage-api:v1.89.3 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index ac01c3c090..5982319fe4 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -490,23 +490,23 @@ const definitions: Readonly> = { storage: definition( "storage", { - upstreamVersion: "v1.89.2", + upstreamVersion: "v1.89.3", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.89.2-r0@sha256:faba36d1b165a52af782ffa5f6e2ad7ca0160a0b52125cf622d7d8991b0898a4", - upstreamImage: "supabase/storage-api:v1.89.2", + "ghcr.io/supabase/cli/storage:v1.89.3-r0@sha256:f6f292f18903acb42fd0529cf045f5298477091616b1a80290b065cf80d50bdf", + upstreamImage: "supabase/storage-api:v1.89.3", natives: { "darwin-arm64": { - archive: "ce36b600d0a9fa77aeb7d8a6e67cfa147f2b55687c86547d0f2d19551add6899", - manifest: "bdd31e46912193868f9c66ac5c21582c44679279f65c4804721a9fe9625bb21e", + archive: "915294a2f92495e1b1ee8b4016fe1bf2b313fc27c081a47fd810bc031cef4010", + manifest: "1cc6c3618008f1cb3b27d406bef028d81e9f534cb2c311f4e7f002ee938077c5", }, "linux-amd64": { - archive: "5312dba58da9adede90b880546a2d7e629217f3e8f89ce3f2907965b4548fd5b", - manifest: "4f83527b182bc00256cbdcd5b3326abe85652e570568b948493bbb899b03b5b5", + archive: "8f06cdae036493ef04f4a5afaab45d0ca6944335908bd9d60dfa7c8c893148ca", + manifest: "9d6bd5652aa2c36508a3e4b4b0da23d85a694e0c1a93abceb3af90dba02b7fae", }, "linux-arm64": { - archive: "00ec8132ed5e39c4cae5ffbf767e4553e7e8e7fdb1dca4eb8539f4e9161b4ed2", - manifest: "75d7bf69670a58ec1dffa64b7b6cbf8a386364db796845ffbfb7d9eb9eb24534", + archive: "ac4d24f7c3b350fde7ef352066169f435ed94918daa2052eefb3781b7bded441", + manifest: "95d757f0f53678761d6f91523a093174591440d986569c3d38fa71482d652604", }, }, }, From b66f0f3fd08f6e6def9a22e556856107cec6d2d4 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Tue, 3 Nov 2026 21:33:57 +0000 Subject: [PATCH 048/101] chore(stack): bump storage to v1.89.4-r0 (#7477) This bumps storage from v1.89.3-r0 to v1.89.4-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.89.4-r0 Planned after supabase/slim-services published v1.89.4-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 839d2bbf47..b5951a4328 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.152.0 AS realtime -FROM supabase/storage-api:v1.89.3 AS storage +FROM supabase/storage-api:v1.89.4 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 5982319fe4..ce42070359 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -490,23 +490,23 @@ const definitions: Readonly> = { storage: definition( "storage", { - upstreamVersion: "v1.89.3", + upstreamVersion: "v1.89.4", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.89.3-r0@sha256:f6f292f18903acb42fd0529cf045f5298477091616b1a80290b065cf80d50bdf", - upstreamImage: "supabase/storage-api:v1.89.3", + "ghcr.io/supabase/cli/storage:v1.89.4-r0@sha256:3692dcb984770167ff7ab132d73ce1f1172f34e5932c7cc6cf9045899a584d13", + upstreamImage: "supabase/storage-api:v1.89.4", natives: { "darwin-arm64": { - archive: "915294a2f92495e1b1ee8b4016fe1bf2b313fc27c081a47fd810bc031cef4010", - manifest: "1cc6c3618008f1cb3b27d406bef028d81e9f534cb2c311f4e7f002ee938077c5", + archive: "cdc4320f8d78d94b06343c44e9c82ceb9739c1e049b479155516c67666e8c03a", + manifest: "f1cc0138912f249264ea2e31be96f40a57c29bc48808b6bce4ff0f3ec4597b7c", }, "linux-amd64": { - archive: "8f06cdae036493ef04f4a5afaab45d0ca6944335908bd9d60dfa7c8c893148ca", - manifest: "9d6bd5652aa2c36508a3e4b4b0da23d85a694e0c1a93abceb3af90dba02b7fae", + archive: "320c0f3152187d9ceea34cdd91dfbad52c8326dd19d351a879829eddc097cb39", + manifest: "0efba0fc139dbf4b04b8cb06af31c62cf1987f54528006c227a2bfd77ddf939f", }, "linux-arm64": { - archive: "ac4d24f7c3b350fde7ef352066169f435ed94918daa2052eefb3781b7bded441", - manifest: "95d757f0f53678761d6f91523a093174591440d986569c3d38fa71482d652604", + archive: "2c5863c891e35678064259cbb606579a48359b27c2ad4deff69714db1233dd10", + manifest: "b30e3f8135e4da938c59dea06c9d7bb901876ba31d69e7bdf1552753a46486d3", }, }, }, From c94ecf316791df62c60061e67ab46387bcaa5ecf Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Tue, 3 Nov 2026 21:44:51 +0000 Subject: [PATCH 049/101] fix(cli): stop hanging on a closed output pipe (CLI-2507) (#7469) ## TL;DR stops the cli spinning forever when the reader of its output exits early ## whats improved/fixed? piping text output into a reader that exits early, like `supabase projects list | true`, left the cli spinning at full cpu and never exiting, and `--debug` did the same progress lines and `--debug` logs write to the pipe directly, so bun marks the stream failed without the output writer noticing, and the next write waits for a `drain` that never comes ## how? every stdio write, from the output writer and the debug trace spans alike, now fails right away on an already failed stream, so the existing closed pipe handling ends the run with exit 0 each write also runs without yielding, so the late `error` event from a direct write cannot slip past that check ## ref: - closes: https://github.com/supabase/cli/issues/6846 - broken in: https://github.com/supabase/cli/pull/6293 - extends: https://github.com/supabase/cli/pull/7175 --- apps/cli/src/shared/cli/run.e2e.test.ts | 18 +++- apps/cli/src/shared/output/output.layer.ts | 33 +++++++- .../shared/output/output.layer.unit.test.ts | 82 ++++++++++++++++++- .../shared/telemetry/trace-export.layer.ts | 9 +- .../telemetry/trace-export.unit.test.ts | 14 +++- apps/cli/tests/helpers/cli.ts | 20 +++-- apps/cli/tests/helpers/mocks.ts | 20 +++++ 7 files changed, 172 insertions(+), 24 deletions(-) diff --git a/apps/cli/src/shared/cli/run.e2e.test.ts b/apps/cli/src/shared/cli/run.e2e.test.ts index 2729952aff..9525e9ea51 100644 --- a/apps/cli/src/shared/cli/run.e2e.test.ts +++ b/apps/cli/src/shared/cli/run.e2e.test.ts @@ -132,12 +132,12 @@ describe("CLI upgrade notice (#5853)", () => { }); /** Only a real pipe whose reader has exited reproduces the broken-pipe write. */ -describe("CLI output to a closed stdout", () => { +describe("CLI output to a closed pipe", () => { it.live.each(["text", "json"])("ends quietly with exit 0 in %s output", (format) => Effect.gen(function* () { const { exitCode, stderr, timedOutAfterMs } = yield* runSupabaseEffect( ["gen", "signing-key", "--algorithm", "ES256", "--output-format", format], - { closeStdout: true }, + { closeStream: "stdout" }, ); expect(timedOutAfterMs).toBeUndefined(); expect(exitCode).toBe(0); @@ -155,7 +155,7 @@ describe("CLI output to a closed stdout", () => { ])("keeps exit 1 for $name whose JSON error cannot be written", ({ args }) => Effect.gen(function* () { const { exitCode, stderr, timedOutAfterMs } = yield* runSupabaseEffect(args, { - closeStdout: true, + closeStream: "stdout", env: { SUPABASE_ACCESS_TOKEN: undefined }, }); expect(timedOutAfterMs).toBeUndefined(); @@ -163,4 +163,16 @@ describe("CLI output to a closed stdout", () => { expect(stderr).not.toContain("StdoutClosedError"); }), ); + + it.live("finishes with exit 0 once a direct `--debug` write met a closed stderr", () => + Effect.gen(function* () { + const { exitCode, stdout, timedOutAfterMs } = yield* runSupabaseEffect( + ["--debug", "gen", "signing-key", "--algorithm", "ES256"], + { closeStream: "stderr" }, + ); + expect(timedOutAfterMs).toBeUndefined(); + expect(exitCode).toBe(0); + expect(stdout).toContain('"alg":"ES256"'); + }), + ); }); diff --git a/apps/cli/src/shared/output/output.layer.ts b/apps/cli/src/shared/output/output.layer.ts index a848a05a76..6703f93178 100644 --- a/apps/cli/src/shared/output/output.layer.ts +++ b/apps/cli/src/shared/output/output.layer.ts @@ -22,6 +22,7 @@ import { Layer, Option, PlatformError, + References, Schema, Stdio, Stream, @@ -89,8 +90,33 @@ const isBrokenPipe = (error: PlatformError.PlatformError): boolean => { ); }; -// Keyed by the Stdio service so every output layer built on the same process streams sees a -// stream close, including the layer that renders the run's final error. +/* A direct write (clack's progress lines, `--debug`) can meet the closed pipe first and leave the + process stream errored, and the sink would then wait forever for `drain`. Bun sets `errored` a + tick after that write, so nothing may yield between this check and the sink's `error` listener. */ +export const writeStdio = ( + stdio: typeof Stdio.Stdio.Service, + stream: "stdout" | "stderr", + chunk: string | Uint8Array, +) => + Effect.suspend(() => { + const errored = process[stream].errored; + if (errored !== null) { + return Effect.fail( + PlatformError.systemError({ + module: "Stdio", + method: stream, + _tag: "Unknown", + cause: errored, + }), + ); + } + return Stream.make(chunk).pipe( + Stream.run(stream === "stderr" ? stdio.stderr() : stdio.stdout()), + ); + }).pipe(Effect.provideService(References.PreventSchedulerYield, true)); + +/* Keyed by the Stdio service so output layers sharing one skip a stream any of them saw close. + Layers on another Stdio (run.ts builds the final-error layer on its own) see it via `errored`. */ const closedStreams = new WeakMap>(); /** @@ -108,8 +134,7 @@ const stdioWriter = (stdio: typeof Stdio.Stdio.Service) => { Effect.suspend(() => closed.has(stream) ? Effect.void - : Stream.make(chunk).pipe( - Stream.run(stream === "stderr" ? stdio.stderr() : stdio.stdout()), + : writeStdio(stdio, stream, chunk).pipe( Effect.catchIf(isBrokenPipe, () => Effect.sync(() => closed.add(stream)).pipe( Effect.andThen( diff --git a/apps/cli/src/shared/output/output.layer.unit.test.ts b/apps/cli/src/shared/output/output.layer.unit.test.ts index a2dd3c9ec9..d648802385 100644 --- a/apps/cli/src/shared/output/output.layer.unit.test.ts +++ b/apps/cli/src/shared/output/output.layer.unit.test.ts @@ -10,6 +10,7 @@ import { Layer, Option, PlatformError, + Scheduler, Schema, Sink, Stdio, @@ -17,7 +18,7 @@ import { } from "effect"; import { TestClock } from "effect/testing"; import { CONTEXT_CANCELED_MESSAGE, NonInteractiveError, StdoutClosedError } from "./errors.ts"; -import { mockTty } from "../../../tests/helpers/mocks.ts"; +import { epipe, mockTty, overrideStreamErrored } from "../../../tests/helpers/mocks.ts"; import { announceTarget, CommandTarget, @@ -846,6 +847,85 @@ describe("Output", () => { }, ); + it.effect("drops diagnostics once a direct write broke stderr", () => { + const stdio = mockStdio(); + return Effect.gen(function* () { + yield* overrideStreamErrored(process.stderr, epipe); + const out = yield* Output; + yield* out.raw("diagnostic\n", "stderr"); + yield* out.raw("payload\n"); + + expect(stdio.stderr).toEqual([]); + expect(stdio.stdout).toEqual(["payload\n"]); + }).pipe(provideLayer(stdio.layer)); + }); + + it.effect("ends the command once a direct write broke stdout", () => { + const stdio = mockStdio(); + return Effect.gen(function* () { + yield* overrideStreamErrored(process.stdout, epipe); + const out = yield* Output; + yield* out.raw("diagnostic\n", "stderr"); + const exit = yield* Effect.exit(out.raw("payload\n")); + + expect(Exit.isFailure(exit) && Cause.squash(exit.cause)).toBeInstanceOf(StdoutClosedError); + expect(stdio.stderr).toEqual(["diagnostic\n"]); + expect(stdio.stdout).toEqual([]); + }).pipe(provideLayer(stdio.layer)); + }); + + it.effect("never yields between the stream check and the write", () => { + let inWindow = false; + let windows = 0; + let yieldChecks = 0; + let suspensions = 0; + const stdio = Layer.succeed( + Stdio.Stdio, + Stdio.make({ + args: Effect.succeed([]), + stdin: Stream.empty, + stdout: () => + Sink.forEach((_item: string | Uint8Array) => + Effect.sync(() => { + inWindow = false; + }), + ), + stderr: () => Sink.drain, + }), + ); + return Effect.gen(function* () { + yield* overrideStreamErrored(process.stdout, () => { + inWindow = true; + windows++; + /* Bun delivers the pending error in a microtask; it runs inside the window only if the + write suspends there. */ + queueMicrotask(() => { + if (inWindow) suspensions++; + }); + return null; + }); + const scheduler = yield* Scheduler.Scheduler; + const out = yield* Output; + yield* out.raw("payload\n").pipe( + Effect.provideService(Scheduler.Scheduler, { + executionMode: scheduler.executionMode, + shouldYield: (fiber) => { + if (inWindow) yieldChecks++; + return scheduler.shouldYield(fiber); + }, + makeDispatcher: () => scheduler.makeDispatcher(), + }), + ); + + expect({ windows, inWindow, yieldChecks, suspensions }).toEqual({ + windows: 1, + inWindow: false, + yieldChecks: 0, + suspensions: 0, + }); + }).pipe(provideLayer(stdio)); + }); + it.effect("keeps writing stdout once the stderr reader has closed", () => { const stdio = brokenStdio("stderr", "EPIPE"); return Effect.gen(function* () { diff --git a/apps/cli/src/shared/telemetry/trace-export.layer.ts b/apps/cli/src/shared/telemetry/trace-export.layer.ts index 3144736c7d..926101cac3 100644 --- a/apps/cli/src/shared/telemetry/trace-export.layer.ts +++ b/apps/cli/src/shared/telemetry/trace-export.layer.ts @@ -15,12 +15,12 @@ import { Redacted, Scope, Stdio, - Stream, Tracer, } from "effect"; import { Headers, HttpTraceContext } from "effect/http"; import { CLI_VERSION } from "../cli/version.ts"; import { CliSettings } from "../config/cli-settings.service.ts"; +import { writeStdio } from "../output/output.layer.ts"; import { RuntimeInfo } from "../runtime/runtime-info.service.ts"; import { makeDebugConsoleExporter } from "./exporters/debug-console.ts"; import { detectCi } from "./runtime.layer.ts"; @@ -33,10 +33,7 @@ const OTLP_TRACES_PATH = "/v1/traces"; /** Tracing never stops a command: a setup problem prints one warning and the run continues untraced. */ const warnTracingDisabled = (reason: string) => Effect.flatMap(Stdio.Stdio, (stdio) => - Stream.make(`Warning: tracing disabled: ${reason}\n`).pipe( - Stream.run(stdio.stderr()), - Effect.ignore, - ), + writeStdio(stdio, "stderr", `Warning: tracing disabled: ${reason}\n`).pipe(Effect.ignore), ); type TraceSink = @@ -226,7 +223,7 @@ const debugConsoleTracer = Effect.fnUntraced(function* ( ) { const stdio = yield* Stdio.Stdio; const exportSpan = makeDebugConsoleExporter( - (line) => Stream.make(line).pipe(Stream.run(stdio.stderr()), Effect.asVoid), + (line) => writeStdio(stdio, "stderr", line), yield* makeTraceSanitizer, root, ); diff --git a/apps/cli/src/shared/telemetry/trace-export.unit.test.ts b/apps/cli/src/shared/telemetry/trace-export.unit.test.ts index a9e3971517..83ca6a8887 100644 --- a/apps/cli/src/shared/telemetry/trace-export.unit.test.ts +++ b/apps/cli/src/shared/telemetry/trace-export.unit.test.ts @@ -11,7 +11,7 @@ import { Stdio, } from "effect"; import { BunServices } from "@effect/platform-bun"; -import { mockRuntimeInfo } from "../../../tests/helpers/mocks.ts"; +import { epipe, mockRuntimeInfo, overrideStreamErrored } from "../../../tests/helpers/mocks.ts"; import { CliSettings } from "../config/cli-settings.service.ts"; import { resolveTraceSettings, withDebugConsole, withTraceExport } from "./trace-export.layer.ts"; @@ -246,6 +246,18 @@ describe("withDebugConsole", () => { }), ); + it.effect("skips span lines once a direct write broke stderr", () => + Effect.gen(function* () { + yield* overrideStreamErrored(process.stderr, epipe); + const output = yield* runWithDebugConsole( + { debug: "1" }, + Effect.void.pipe(Effect.withSpan("Probe.span")), + ); + + expect(output).toBe(""); + }), + ); + it.effect("prints nothing unless a debug setting is exactly 1", () => Effect.gen(function* () { const output = yield* runWithDebugConsole( diff --git a/apps/cli/tests/helpers/cli.ts b/apps/cli/tests/helpers/cli.ts index e373e336c9..d4f30c6882 100644 --- a/apps/cli/tests/helpers/cli.ts +++ b/apps/cli/tests/helpers/cli.ts @@ -568,8 +568,8 @@ export interface SpawnSupabaseOptions { readonly cleanupProcessGroupOnClose?: boolean; /** Maximum time to wait for the process to exit before force-killing it. */ readonly exitTimeoutMs?: number; - /** Close the read end of stdout at spawn, like a `| head` reader that has already exited. */ - readonly closeStdout?: boolean; + /** Close the read end of this stream at spawn, like a `| head` reader that has already exited. */ + readonly closeStream?: "stdout" | "stderr"; } export interface SpawnedSupabase { @@ -598,15 +598,17 @@ const isBrokenPipe = (error: { readonly cause?: unknown }) => Predicate.hasProperty(error.cause, "code") && error.cause.code === "EPIPE"; /* - * The Effect spawner only hands out stdout as a stream it reads itself, so a pipe whose reader - * has exited needs a relay: it runs the CLI on a real pipe, closes the read end at once, and + * The Effect spawner only hands out stdout and stderr as streams it reads itself, so a pipe whose + * reader has exited needs a relay: it runs the CLI on a real pipe, closes the read end at once, and * mirrors the CLI's exit code. It ignores group signals so the CLI's own exit wins. */ -const closedStdoutRelay = (argvJson: string) => +const closedStreamRelay = (argvJson: string, stream: "stdout" | "stderr") => [ `const { spawn } = require("node:child_process");`, - `const child = spawn(process.execPath, ${argvJson}, { stdio: ["inherit", "pipe", "inherit"] });`, - `child.stdout.destroy();`, + `const stdio = ["inherit", "inherit", "inherit"];`, + `stdio[${stream === "stdout" ? 1 : 2}] = "pipe";`, + `const child = spawn(process.execPath, ${argvJson}, { stdio });`, + `child.${stream}.destroy();`, `for (const signal of ["SIGINT", "SIGTERM", "SIGHUP"]) process.on(signal, () => {});`, `child.on("exit", (code) => process.exit(code ?? 1));`, ].join("\n"); @@ -672,8 +674,8 @@ const spawnSupabaseProgram = Effect.fnUntraced(function* ( // build artifact without needing platform wrapper packages. env["SUPABASE_CLI_BINARY_OVERRIDE"] = BINARY_PATH; const shimArgs = [SHIM_PATH, ...args]; - const commandArgs = options?.closeStdout - ? ["-e", closedStdoutRelay(yield* encodeArgv(shimArgs))] + const commandArgs = options?.closeStream + ? ["-e", closedStreamRelay(yield* encodeArgv(shimArgs), options.closeStream)] : shimArgs; const command = ChildProcess.make("node", commandArgs, { cwd: options?.cwd, diff --git a/apps/cli/tests/helpers/mocks.ts b/apps/cli/tests/helpers/mocks.ts index eec45e65cc..1c74788fb0 100644 --- a/apps/cli/tests/helpers/mocks.ts +++ b/apps/cli/tests/helpers/mocks.ts @@ -142,6 +142,26 @@ export function mockTty( }); } +/** + * Stands in for a direct write that left a real process stream `errored`, until the scope closes. + * A configurable own getter, unlike a spy on Bun's non-configurable inherited one, is removed again. + */ +export const overrideStreamErrored = (stream: NodeJS.WriteStream, errored: () => Error | null) => + Effect.acquireRelease( + Effect.sync(() => { + const own = Object.getOwnPropertyDescriptor(stream, "errored"); + Object.defineProperty(stream, "errored", { configurable: true, get: errored }); + return own; + }), + (own) => + Effect.sync(() => { + if (own === undefined) Reflect.deleteProperty(stream, "errored"); + else Object.defineProperty(stream, "errored", own); + }), + ); + +export const epipe = () => Object.assign(new Error("EPIPE"), { code: "EPIPE" }); + export function mockRuntimeInfo( opts: { cwd?: string; From 2e1be9bcae8379e2b5901c989cf091ec424a7d1e Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Wed, 4 Nov 2026 00:52:54 +0000 Subject: [PATCH 050/101] chore(stack): bump storage to v1.89.6-r0 (#7478) This bumps storage from v1.89.4-r0 to v1.89.6-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.89.6-r0 Planned after supabase/slim-services published v1.89.6-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index b5951a4328..bdd909195d 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.152.0 AS realtime -FROM supabase/storage-api:v1.89.4 AS storage +FROM supabase/storage-api:v1.89.6 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index ce42070359..2647c6921a 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -490,23 +490,23 @@ const definitions: Readonly> = { storage: definition( "storage", { - upstreamVersion: "v1.89.4", + upstreamVersion: "v1.89.6", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.89.4-r0@sha256:3692dcb984770167ff7ab132d73ce1f1172f34e5932c7cc6cf9045899a584d13", - upstreamImage: "supabase/storage-api:v1.89.4", + "ghcr.io/supabase/cli/storage:v1.89.6-r0@sha256:7df85c3d3fb33bfa922a5cf3e577acfaa96eea039656f1621518a5f5b7ad3552", + upstreamImage: "supabase/storage-api:v1.89.6", natives: { "darwin-arm64": { - archive: "cdc4320f8d78d94b06343c44e9c82ceb9739c1e049b479155516c67666e8c03a", - manifest: "f1cc0138912f249264ea2e31be96f40a57c29bc48808b6bce4ff0f3ec4597b7c", + archive: "991ec3cfd952f0225cd90bea3c40026de33a52747251993026129f9a32c9dc4a", + manifest: "76d34f8fb539505ddb120f877c852d177819a0893dcadb6fdbf0d495f5e43a75", }, "linux-amd64": { - archive: "320c0f3152187d9ceea34cdd91dfbad52c8326dd19d351a879829eddc097cb39", - manifest: "0efba0fc139dbf4b04b8cb06af31c62cf1987f54528006c227a2bfd77ddf939f", + archive: "ec41e1851a8c4695b9be2f5c347cce2454d9c7330608c010f2260fb809960810", + manifest: "31d6ca129f11640fbfa99eae4d90bfa0939ff8754c8219f667c81a54a33340bf", }, "linux-arm64": { - archive: "2c5863c891e35678064259cbb606579a48359b27c2ad4deff69714db1233dd10", - manifest: "b30e3f8135e4da938c59dea06c9d7bb901876ba31d69e7bdf1552753a46486d3", + archive: "8493bc98cd3a0bb271131facb3027d0440169340e7ee7dd6aa42b00b1c5aa415", + manifest: "b29221838412940cd78e50481b4e390850db20807e00125daace2bdbf41485c5", }, }, }, From a952b3ae025a0c9ac5ccb66ee7827ef3f4a856b0 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Wed, 4 Nov 2026 01:33:48 +0000 Subject: [PATCH 051/101] chore(stack): bump realtime to v2.152.1-r0 (#7479) This bumps realtime from v2.152.0-r0 to v2.152.1-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.152.1-r0 Planned after supabase/slim-services published v2.152.1-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index bdd909195d..617846cc6e 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.152.0 AS realtime +FROM supabase/realtime:v2.152.1 AS realtime FROM supabase/storage-api:v1.89.6 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 2647c6921a..f06e85e17a 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -464,23 +464,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.152.0", + upstreamVersion: "v2.152.1", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.152.0-r0@sha256:c02e01a456e601fb3fbbbfd0e1f4d98a3908d7a09feabedb5dca3fae25489689", - upstreamImage: "supabase/realtime:v2.152.0", + "ghcr.io/supabase/cli/realtime:v2.152.1-r0@sha256:696d78c8df95104b132a61903f9f6b67c092a28c28879a68ea7cb31f48f937df", + upstreamImage: "supabase/realtime:v2.152.1", natives: { "darwin-arm64": { - archive: "508fb18501ea90e6caa865b3c57dbc823962b78d8d5d452bac0764b5bb83da3f", - manifest: "43a511b780eaf4ead230a71e49aac1d6f4014ad860227355b72c29fc3a72f97d", + archive: "bd2ac20bcdc5108c2eadae51a0932051d7ed6bb59c82f5cb4ca1a2303e4f8823", + manifest: "9171366df5c4d737013db04065aa8d81ff7506aabe17d1d278b1d0578c0243e4", }, "linux-amd64": { - archive: "cdce982bfde19044932d67d35878706c30e8b666bd455661af26ec7c21627c15", - manifest: "d059496942a05a92156c53317bcb114591d608543a0e8ada603a391bab835a4f", + archive: "37fc6f220e9573c5a8ddd56c96f045082a3bede86ea47871b5ee79a5c52fa09b", + manifest: "4a5aee173be984d6f92b6dfd0009d5944fce8cac2fe067d15a165a7154e82c9f", }, "linux-arm64": { - archive: "227fea5401b473ec6314e16a73618216a9cdfe5fa76a85e135f5043881f51ecf", - manifest: "93c8c13894a4bf398ceda81f75762600e49dff783c2cb97411b5c1340ea34b65", + archive: "d5f0e5b6a85fc1c617519562d85567b32455213adf796b39554cbafedb8f82a3", + manifest: "eb5ac58a81b72472f99dfa3923ecf01e80eaa961b57da3cb97c815ac998d5487", }, }, }, From d2e9c6d7c840f0dc8f5d3657b712e7954facd49f Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Wed, 4 Nov 2026 04:33:20 +0000 Subject: [PATCH 052/101] chore(stack): bump realtime to v2.153.0-r0 (#7480) This bumps realtime from v2.152.1-r0 to v2.153.0-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.153.0-r0 Planned after supabase/slim-services published v2.153.0-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 617846cc6e..e747601354 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.152.1 AS realtime +FROM supabase/realtime:v2.153.0 AS realtime FROM supabase/storage-api:v1.89.6 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index f06e85e17a..abc721a05e 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -464,23 +464,23 @@ const definitions: Readonly> = { realtime: definition( "realtime", { - upstreamVersion: "v2.152.1", + upstreamVersion: "v2.153.0", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.152.1-r0@sha256:696d78c8df95104b132a61903f9f6b67c092a28c28879a68ea7cb31f48f937df", - upstreamImage: "supabase/realtime:v2.152.1", + "ghcr.io/supabase/cli/realtime:v2.153.0-r0@sha256:c95b297a49de6124069cc613f15ba4b3b19aa6d1c0ae481ce628f1bc70b88c35", + upstreamImage: "supabase/realtime:v2.153.0", natives: { "darwin-arm64": { - archive: "bd2ac20bcdc5108c2eadae51a0932051d7ed6bb59c82f5cb4ca1a2303e4f8823", - manifest: "9171366df5c4d737013db04065aa8d81ff7506aabe17d1d278b1d0578c0243e4", + archive: "4ba248b99f0b8d3d4d76a7141f1cce7cc5aa3594db75124114467b3a9d79898c", + manifest: "4d39eed9884f0920327a6fb27606306555273175d82698446f90d1081a37355e", }, "linux-amd64": { - archive: "37fc6f220e9573c5a8ddd56c96f045082a3bede86ea47871b5ee79a5c52fa09b", - manifest: "4a5aee173be984d6f92b6dfd0009d5944fce8cac2fe067d15a165a7154e82c9f", + archive: "eb5e825bf1e86f9acbfc89268d963ab81595ade87f48874f01c245af1afc6466", + manifest: "46d48668da03022586c7c281df6c967b6e0bfc2ece7a2d867db1d7fcba0bd7f2", }, "linux-arm64": { - archive: "d5f0e5b6a85fc1c617519562d85567b32455213adf796b39554cbafedb8f82a3", - manifest: "eb5ac58a81b72472f99dfa3923ecf01e80eaa961b57da3cb97c815ac998d5487", + archive: "a14365555d5f7b7e8790014a331829075bd111798f68304a8eb207cf9b259760", + manifest: "f85f8db173fedde7b33a24ae2eaef7f21066054334b03a2752ff8c619822a807", }, }, }, From 2cbc0bbb91ea0a0c006f5a56fef0ddf753d032c4 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Wed, 4 Nov 2026 09:22:09 +0000 Subject: [PATCH 053/101] chore(api): sync Management API OpenAPI spec (#7484) This PR was automatically created to sync the generated `@supabase/api` package with the latest Management API OpenAPI document. Changes were detected in the upstream OpenAPI documents exposed by `https://api.supabase.com/api/v1-json` and `https://api.supabase.com/api/v2-json`. Co-authored-by: jgoux <1443499+jgoux@users.noreply.github.com> --- packages/api/src/generated/contracts.ts | 1 + packages/api/src/generated/openapi.json | 10 ++++++++++ 2 files changed, 11 insertions(+) diff --git a/packages/api/src/generated/contracts.ts b/packages/api/src/generated/contracts.ts index 3cda8dd741..7874b5c479 100644 --- a/packages/api/src/generated/contracts.ts +++ b/packages/api/src/generated/contracts.ts @@ -4791,6 +4791,7 @@ export const V1GetPostgresUpgradeEligibilityOutput = Schema.Struct({ Schema.Struct({ type: Schema.Literal("ltree_reindex_required") }), Schema.Struct({ type: Schema.Literal("operator_estimator_gate") }), Schema.Struct({ type: Schema.Literal("btree_gist_nan_reindex") }), + Schema.Struct({ type: Schema.Literal("collation_reindex_required") }), ], { mode: "oneOf" }, ), diff --git a/packages/api/src/generated/openapi.json b/packages/api/src/generated/openapi.json index 15b13f4053..df755e7326 100644 --- a/packages/api/src/generated/openapi.json +++ b/packages/api/src/generated/openapi.json @@ -34464,6 +34464,16 @@ } }, "required": ["type"] + }, + { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["collation_reindex_required"] + } + }, + "required": ["type"] } ] } From 6890ba8c1428f192edb5ed1291952b4cd059e899 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Wed, 4 Nov 2026 09:46:06 +0000 Subject: [PATCH 054/101] fix(cli): drop old service versions when `link` switches projects (CLI-2924) (#7476) ## TL;DR stops `supabase link` carrying the old project's service versions into the new one ## whats hurting? relinking from project A to B kept A's `gotrue-version`, `rest-version`, `storage-version` and `storage-migration` whenever B's checks were skipped, failed or came back empty, so `supabase start` ran A's Auth, PostgREST and Storage while linked to B since #7388 every switch with a token that cannot read secret keys hits this ## now fixed by? `link` reads the saved `project-ref` first and drops those files unless it already names B, so B gets its own versions or the CLI defaults, never A's relinking the same project keeps its versions
visual rep ```mermaid %%{init: {"flowchart": {"wrappingWidth": 400}}}%% flowchart TD a["linked to A, .temp holds A's versions"] --> l["supabase link B
with B's checks skipped, failed or empty"] l -- "before" --> b1["A's version files stay
project-ref saved as B"] b1 --> b2["supabase start runs A's
Auth, PostgREST and Storage"] l -- "now" --> n1["saved project-ref is not B
A's version files removed"] n1 --> n2["nothing written for B
project-ref saved as B"] n2 --> n3["supabase start runs
the CLI defaults"] classDef t fill:transparent classDef bad fill:transparent,stroke-dasharray:4 3 class a,l,b1,n1,n2,n3 t class b2 bad ```
## ref: - spotted while working on: https://github.com/supabase/cli/pull/7388 --- .../command-internal/link-services-core.ts | 17 ++- .../src/commands/bootstrap/SIDE_EFFECTS.md | 3 +- apps/cli/src/commands/link/SIDE_EFFECTS.md | 10 +- .../commands/link/link.integration.test.ts | 124 +++++++++++++++++- 4 files changed, 146 insertions(+), 8 deletions(-) diff --git a/apps/cli/src/command-internal/link-services-core.ts b/apps/cli/src/command-internal/link-services-core.ts index 7cbd5e28b6..37bf717237 100644 --- a/apps/cli/src/command-internal/link-services-core.ts +++ b/apps/cli/src/command-internal/link-services-core.ts @@ -44,7 +44,9 @@ export interface LinkServicesInput { * storage-version}`. Each probe is independently best-effort — an * unreachable service never fails the caller. Does not write `project-ref`, * the linked-project cache, or fire `cli_project_linked`; the `link` command - * owns those, and `bootstrap` calls this directly to skip them. + * owns those, and `bootstrap` calls this directly to skip them. Unless the saved + * `project-ref` is `ref`, the files above except `pooler-url` are removed first, + * since their readers do no ref check. */ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( input: LinkServicesInput, @@ -58,6 +60,19 @@ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( const state = yield* LegacyProjectState; const { workdir } = input; + const refChanged = yield* state.readProjectRef(workdir).pipe( + Effect.map((linked) => !Option.contains(linked, input.ref)), + Effect.orElseSucceed(() => true), + ); + yield* Effect.annotateCurrentSpan("link.ref_changed", refChanged); + if (refChanged) { + yield* Effect.forEach( + ["storageMigration", "restVersion", "gotrueVersion", "storageVersion"] as const, + (file) => state.remove(workdir, file).pipe(Effect.ignore), + { discard: true }, + ); + } + yield* linkStorageMigration(api, input.ref, state, workdir); yield* linkPooler({ api, ref: input.ref, skipPooler: input.skipPooler, state, workdir }); if (input.serviceKey.length === 0) return; diff --git a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md index 3dfd263281..b3c44513b1 100644 --- a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md @@ -16,6 +16,7 @@ command — see Notes). | `/.env.example` | dotenv | optional; merged into the generated `.env` | | `/supabase/{.env..local,.env.local,.env.,.env}` | dotenv | step I (`loadProjectEnvValues`), before config.toml validation and again inside `checkDbToml`; `` is `SUPABASE_ENV` (default `development`), `.env.local` is skipped when `SUPABASE_ENV=test`; first of the 4 files (in this order) to set a key wins, and this `supabase/` directory tier beats the workdir-root tier below — feeds config.toml `env(VAR)` expansion and the push step's `SUPABASE_YES` auto-confirm default | | `/{.env..local,.env.local,.env.,.env}` | dotenv | same read as above; lower-precedence fallback tier, only consulted for a key none of the `supabase/` directory's 4 files above already set | +| `/supabase/.temp/project-ref` | plain text | link-services step, to tell whether it names the new project | | `/supabase/config.toml` | TOML | native push step (embedded defaults used when absent) | | `/supabase/.temp/pooler-url` | plain text | native push step's connection resolution, only when the direct `db..:5432` host is unreachable (IPv4-only network) — `resolveLinkedConn` falls back through the saved pooler URL `link.LinkServices` wrote in the earlier link-services step | | `/supabase/migrations/` | directory | native push step, when `[db.migrations].enabled` (default true) | @@ -31,7 +32,7 @@ command — see Notes). | `/supabase/config.toml` | TOML | blank/`scratch` path only (via `initProject`) | | `/