diff --git a/.gitattributes b/.gitattributes index 19f3e8920a..017dd6d47b 100644 --- a/.gitattributes +++ b/.gitattributes @@ -10,6 +10,9 @@ patches/*.patch text eol=lf # can't leave a trailing `\r` on every entry and make every tag mismatch. apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt text eol=lf +# The Windows smoke-test runs this under Git Bash; a CRLF checkout breaks `set -euo pipefail`. +.github/scripts/wait-for-npm-version.sh text eol=lf + # Generated artifacts. `linguist-generated` keeps these out of GitHub's language # statistics and collapses them in pull request diffs -- they are build output to # be regenerated, not code to review, and together they outweigh the hand-written diff --git a/.github/MAINTAINERS.md b/.github/MAINTAINERS.md index 25c044e909..50ec58f4c5 100644 --- a/.github/MAINTAINERS.md +++ b/.github/MAINTAINERS.md @@ -187,6 +187,7 @@ Fast-forward PRs are the deploy PR (`develop` → `main`), the major cut (`next` → `develop`), and the two sync PRs. **Dispatch guards.** `release.yml` refuses `channel=next` off `next`, +`channel=next` while `next` has no commits beyond `develop`, `channel=maintenance` off `v.x`, `beta`/`stable` on `next` or `v.x`, a non-dry-run `stable` outside `main`, and a `version` that is not `X.Y.Z[-prerelease]`. diff --git a/.github/ai-review/README.md b/.github/ai-review/README.md index 300372d67f..fb408be8a8 100644 --- a/.github/ai-review/README.md +++ b/.github/ai-review/README.md @@ -74,6 +74,12 @@ shown as `--remerge-diff`. Both sides were reviewed in their own PRs, so the prompts limit findings to what the resolution introduced; `pr.diff` stays the anchor for inline comments. +### Breaking changes + +Only the Claude prompt asks for `breaking-change` findings, because only its job +has `pr.json` with the title and branches. They post as `major` and ask for a +`!` title, plus a retarget to `next` on `develop`. + ## Once-per-PR semantics and manual re-runs New commits never re-trigger a review — `resolve.ts`'s dedup guard skips a diff --git a/.github/ai-review/adjudicate-prompt.md b/.github/ai-review/adjudicate-prompt.md index ef6951d1b0..f37a3e0085 100644 --- a/.github/ai-review/adjudicate-prompt.md +++ b/.github/ai-review/adjudicate-prompt.md @@ -71,6 +71,16 @@ when that file shows work on `next` the PR would fundamentally conflict with. If the file is absent, refute it. Keep the category `next-impact` and a severity no higher than `minor`; these findings are advisory and never block the PR. +### Breaking-change findings + +Verify each `breaking-change` finding against the behavior the PR changes: +refute it only when no released, non-experimental use fails or changes meaning, +when the change only rejects a value that never had a meaning, or when the +change matches the old `SIDE_EFFECTS.md`. The PR's intent, tests and +`SIDE_EFFECTS.md` edits are not counter-evidence. Keep the category +`breaking-change`, severity `major` and the request ending its `claim`, even +when merged with a Codex finding. + ### Merge into one deduplicated list - When a Claude finding and a Codex finding concern the same file/line/ diff --git a/.github/ai-review/claude-review-prompt.md b/.github/ai-review/claude-review-prompt.md index 79273c84e7..5ce275addf 100644 --- a/.github/ai-review/claude-review-prompt.md +++ b/.github/ai-review/claude-review-prompt.md @@ -76,6 +76,19 @@ higher than `minor`. These findings are advisory and never block the PR. Do not flag ordinary merge conflicts or style differences, and report nothing when the file does not exist. +## Breaking changes + +Only when `pr.json` has `baseRefName` `develop` or `next`, a `headRefName` other +than `next` or `sync/*`, and no `!` before the title's colon: report any change +that makes existing use of a released command, flag, config key, env var or exit +code fail or change meaning, for example removing a fallback so a command now +errors, even when that is the fix the PR set out to make. Use category +`breaking-change`, with severity `major`, and end the `claim` by asking for a +`type(scope)!:` title, a `BREAKING CHANGE:` line in the PR description and, on +`develop`, a retarget to `next`. Do not report internal code, tests, CI, +additive changes, experimental commands, newly rejected values that never had a +meaning, or fixes that match the old `SIDE_EFFECTS.md`. + ## Output Your final response must be ONLY the JSON object described by the provided diff --git a/.github/scripts/wait-for-npm-version.sh b/.github/scripts/wait-for-npm-version.sh new file mode 100755 index 0000000000..dfb6e07ad7 --- /dev/null +++ b/.github/scripts/wait-for-npm-version.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Usage: wait-for-npm-version.sh ... +# +# npm lists a new version minutes after publish, and the registry CDN caches each packument per +# region and Accept header, so both the npm client and the abbreviated packument must serve it. +set -euo pipefail + +if [[ $# -lt 2 ]]; then + echo "usage: $0 ..." >&2 + exit 2 +fi + +version="$1" +shift +timeout="${WAIT_TIMEOUT_SECONDS:-900}" + +npm_serves_version() { + local pkg="$1" + npm view "${pkg}@${version}" version --prefer-online --fetch-timeout=30000 --fetch-retries=0 > /dev/null && + curl -fsSL --max-time 30 -H 'Accept: application/vnd.npm.install-v1+json' "https://registry.npmjs.org/${pkg//\//%2f}" | + jq -e --arg v "$version" '.versions | has($v)' > /dev/null +} + +deadline=$((SECONDS + timeout)) +for pkg in "$@"; do + while true; do + if (( SECONDS >= deadline )); then + echo "::error::npm does not serve ${pkg}@${version} after ${timeout} seconds" + exit 1 + fi + npm_serves_version "$pkg" && break + echo "Waiting for npm to serve ${pkg}@${version}..." + sleep 15 + done +done diff --git a/.github/workflows/propose-release-notes.yml b/.github/workflows/propose-release-notes.yml index 3ba3ad70fc..8ddc40d29c 100644 --- a/.github/workflows/propose-release-notes.yml +++ b/.github/workflows/propose-release-notes.yml @@ -62,7 +62,6 @@ jobs: client-id: ${{ vars.GH_APP_CLIENT_ID }} private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} permission-contents: write - permission-issues: read permission-pull-requests: write - uses: useblacksmith/checkout@6fd481652155169ed4d2f25ebaf97464f685175f # v1 diff --git a/.github/workflows/release-shared.yml b/.github/workflows/release-shared.yml index 32bde58beb..45c2b4d04d 100644 --- a/.github/workflows/release-shared.yml +++ b/.github/workflows/release-shared.yml @@ -275,7 +275,7 @@ jobs: # count as self-hosted from sigstore's POV, so this job must stay on a github-hosted runner # (it's short and not compute-bound, so the cost is negligible). runs-on: ubuntu-latest - timeout-minutes: 45 + timeout-minutes: 60 env: CHANNEL: ${{ inputs.channel }} NPM_TAG: ${{ inputs.npm_tag }} @@ -283,6 +283,7 @@ jobs: LINEAR_CLI_STABLE_RELEASE_ACCESS_KEY: ${{ secrets.LINEAR_CLI_STABLE_RELEASE_ACCESS_KEY }} LINEAR_CLI_BETA_RELEASE_ACCESS_KEY: ${{ secrets.LINEAR_CLI_BETA_RELEASE_ACCESS_KEY }} permissions: + attestations: write contents: write id-token: write steps: @@ -409,6 +410,16 @@ jobs: GH_TOKEN: ${{ github.token }} run: bun apps/cli/scripts/upload-release-assets.ts verify --version "${VERSION}" + # Lets installers such as aqua and mise verify each asset with `gh attestation verify`; the + # unversioned tarball aliases share their versioned twin's digest, so they are covered too. + - name: Attest release assets + uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2 + with: + subject-path: | + dist/supabase_${{ inputs.version }}_* + dist/checksums.txt + install + - name: Publish GitHub Release (immutable) env: GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/release-smoke-test.yml b/.github/workflows/release-smoke-test.yml index 0aa4ff6b17..d4521493ab 100644 --- a/.github/workflows/release-smoke-test.yml +++ b/.github/workflows/release-smoke-test.yml @@ -35,6 +35,7 @@ jobs: # dry_run here, but GitHub validates nested-workflow permissions at startup. permissions: actions: read + attestations: write contents: write id-token: write pull-requests: write diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1a1ce8040a..6f32d098ac 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -47,9 +47,36 @@ permissions: contents: read jobs: + # semantic-release reads channel notes per commit (semantic-release/semantic-release#4073), so + # releasing `next` on a commit develop already tagged misreads both tags. + next-ahead: + name: Check next has its own commits + if: github.ref_name == 'next' + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + outputs: + ahead: ${{ steps.compare.outputs.ahead }} + steps: + - id: compare + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + SHA: ${{ github.sha }} + run: | + set -euo pipefail + ahead_by="$(gh api "repos/${REPO}/compare/develop...${SHA}" --jq .ahead_by)" + if [[ "$ahead_by" -gt 0 ]]; then + echo "ahead=true" >> "$GITHUB_OUTPUT" + else + echo "ahead=false" >> "$GITHUB_OUTPUT" + fi + test-next: name: Test next - if: github.event_name == 'push' && github.ref_name == 'next' + needs: next-ahead + if: github.event_name == 'push' && github.ref_name == 'next' && needs.next-ahead.outputs.ahead == 'true' permissions: contents: read actions: read @@ -61,12 +88,20 @@ jobs: plan: name: Plan release - needs: test-next - # A skipped test-next (every non-next trigger) makes the implicit success() false, so the - # result is checked explicitly. - if: ${{ !cancelled() && contains(fromJSON('["success","skipped"]'), needs.test-next.result) }} + needs: [next-ahead, test-next] + # Skipped next-ahead and test-next jobs (every non-next trigger) make the implicit success() + # false, so their results are checked explicitly. A next push with no commits beyond develop + # plans nothing. + if: >- + ${{ !cancelled() && + contains(fromJSON('["success","skipped"]'), needs.next-ahead.result) && + contains(fromJSON('["success","skipped"]'), needs.test-next.result) && + !(github.event_name == 'push' && github.ref_name == 'next' && needs.next-ahead.outputs.ahead != 'true') }} runs-on: ubuntu-latest - timeout-minutes: 15 + timeout-minutes: 135 + permissions: + contents: read + actions: read outputs: should_release: ${{ steps.compute.outputs.should_release }} version: ${{ steps.compute.outputs.version }} @@ -78,6 +113,59 @@ jobs: dry_run: ${{ steps.compute.outputs.dry_run }} channel: ${{ steps.compute.outputs.channel }} steps: + - name: Refuse a next dispatch with no commits beyond develop + if: ${{ github.event_name == 'workflow_dispatch' && github.ref_name == 'next' && needs.next-ahead.outputs.ahead != 'true' }} + run: | + echo "::error::next has no commits beyond develop, so a next release would tag a commit develop already released" + exit 1 + # The stable tag is pushed by main's publish job, so a prerelease planned before that job + # finishes misses it. The rest of the run (brew, scoop, smoke tests) does not affect the plan. + - name: Wait for an in-flight stable release + if: ${{ (github.ref_name == 'develop' || github.ref_name == 'next') && (github.event_name == 'push' || inputs.version == '') }} + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + run: | + set -euo pipefail + deadline=$((SECONDS + 7200)) + while true; do + pending=0 + if run_ids="$(gh run list --repo "$REPO" --workflow release.yml --branch main --json databaseId,status --jq '.[] | select(.status != "completed") | .databaseId')"; then + for run_id in $run_ids; do + publish_status="$(gh api "repos/${REPO}/actions/runs/${run_id}/jobs" --paginate --jq '.jobs[] | select(.name == "Release / publish") | .status')" || publish_status=unknown + if [[ "$publish_status" != completed ]]; then + pending=$((pending + 1)) + fi + done + else + echo "::warning::Could not list Release runs on main; retrying." + pending=1 + fi + if [[ "$pending" -eq 0 ]]; then + break + fi + if (( SECONDS >= deadline )); then + echo "::error::The publish job of a Release run on main is still in progress after 120 minutes. Re-run this job once it finishes." + exit 1 + fi + echo "Waiting for the publish job of ${pending} Release run(s) on main to finish..." + sleep 30 + done + # A major cut can fast-forward develop onto next while the tests and the wait above run. + - id: next-still-ahead + if: ${{ github.ref_name == 'next' }} + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + SHA: ${{ github.sha }} + run: | + set -euo pipefail + ahead_by="$(gh api "repos/${REPO}/compare/develop...${SHA}" --jq .ahead_by)" + if [[ "$ahead_by" -gt 0 ]]; then + echo "ahead=true" >> "$GITHUB_OUTPUT" + else + echo "ahead=false" >> "$GITHUB_OUTPUT" + fi # semantic-release runs `git push --dry-run HEAD:` as part of verifyAuth even in # `dry_run: true` mode, so the token needs push access to the protected develop/main # branches — the default GITHUB_TOKEN doesn't have it. @@ -110,6 +198,7 @@ jobs: DISPATCH_CHANNEL: ${{ inputs.channel }} DISPATCH_VERSION: ${{ inputs.version }} DISPATCH_DRY_RUN: ${{ inputs.dry_run }} + NEXT_AHEAD: ${{ steps.next-still-ahead.outputs.ahead }} SR_PUBLISHED: ${{ steps.semantic-release.outputs.new_release_published }} SR_VERSION: ${{ steps.semantic-release.outputs.new_release_version }} run: | @@ -126,6 +215,15 @@ jobs: next) natural=next ;; *) [[ "$REF_NAME" =~ $maint_re ]] && natural=maintenance ;; esac + if [[ "$REF_NAME" == next && "$NEXT_AHEAD" != true ]]; then + if [[ "$EVENT" == "workflow_dispatch" ]]; then + echo "::error::next has no commits beyond develop, so a next release would tag a commit develop already released" + exit 1 + fi + echo "::notice::next has no commits beyond develop, so there is nothing to release" + echo "should_release=false" >> "$GITHUB_OUTPUT" + exit 0 + fi if [[ "$EVENT" == "workflow_dispatch" ]]; then channel="$DISPATCH_CHANNEL" dry_run="$DISPATCH_DRY_RUN" @@ -243,6 +341,7 @@ jobs: # those granted to the calling job, so this must be declared here too. permissions: actions: read + attestations: write contents: write id-token: write pull-requests: write @@ -394,13 +493,14 @@ jobs: SLACK_RELEASE_WEBHOOK: ${{ secrets.SLACK_RELEASE_WEBHOOK }} # Reports a broken release on every channel. `failure()` evaluates against the full `needs` - # chain, so this fires whenever `test-next`, `plan` or `release` (including the reusable - # release-shared workflow) fails. Skipped jobs don't count as failures. Dry runs are excluded - # via the dispatch input directly, so even a plan job that dies before recording its `dry_run` - # output stays quiet. A failed `test-next` skips `plan`, so the channel falls back to the ref. + # chain, so this fires whenever `next-ahead`, `test-next`, `plan` or `release` (including the + # reusable release-shared workflow) fails. Skipped jobs don't count as failures. Dry runs are + # excluded via the dispatch input directly, so even a plan job that dies before recording its + # `dry_run` output stays quiet. A failed `next-ahead` or `test-next` skips `plan`, so the + # channel falls back to the ref. notify-slack-failure: name: Notify Slack (failure) - needs: [test-next, plan, release] + needs: [next-ahead, test-next, plan, release] if: ${{ failure() && !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }} uses: ./.github/workflows/slack-notify.yml with: diff --git a/.github/workflows/setup-cli-smoke-test.yml b/.github/workflows/setup-cli-smoke-test.yml index 5e00dad37b..7250265fcd 100644 --- a/.github/workflows/setup-cli-smoke-test.yml +++ b/.github/workflows/setup-cli-smoke-test.yml @@ -1,9 +1,11 @@ name: setup-cli Smoke Test # Smoke test for the `supabase/setup-cli` GitHub Action against a specific published CLI -# version. Runs automatically after every beta release and also manually via workflow_dispatch -# when debugging setup-cli regressions. Catches regressions like supabase/setup-cli#427, where -# musl libc + archive layout changes silently broke the action on Alpine. +# version. Runs automatically after every published release and also manually via +# workflow_dispatch when debugging setup-cli regressions. Catches regressions like +# supabase/setup-cli#427, where musl libc + archive layout changes silently broke the action on +# Alpine. v3 installs from npm: each v3 runner first waits until npm serves the version, and the +# job fails if setup-cli fell back to the GitHub release. A separate job forces that fallback. on: workflow_call: @@ -39,7 +41,7 @@ jobs: - v2 - v3 runs-on: ${{ matrix.runner }} - timeout-minutes: 10 + timeout-minutes: 25 env: VERSION: ${{ inputs.version }} steps: @@ -53,9 +55,24 @@ jobs: uses: supabase/setup-cli@afb1b15109756ea5cf9d8985a359d9095235ca2b # v2.1.2 with: version: ${{ inputs.version }} + - name: Checkout the npm wait script + if: matrix.major-version == 'v3' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + path: .smoke-tooling + sparse-checkout: .github/scripts + persist-credentials: false + - name: Wait for npm to serve the version + if: matrix.major-version == 'v3' + shell: bash + run: | + set -euo pipefail + platform="$(node -p 'process.platform + "-" + process.arch')" + bash .smoke-tooling/.github/scripts/wait-for-npm-version.sh "${VERSION#v}" supabase "@supabase/cli-${platform/win32/windows}" - name: Install Supabase CLI via setup-cli@v3 + id: setup-cli-v3 if: matrix.major-version == 'v3' - uses: supabase/setup-cli@45a513f8c64c0bc8e0e3dfe572b5c95be85f6359 # v3.0.1 + uses: supabase/setup-cli@689adcfb80d947100a1eab127289b97d1fb07b13 # v3.2.0 with: version: ${{ inputs.version }} - name: Verify supabase --version matches the expected version @@ -72,6 +89,17 @@ jobs: echo "Version mismatch: expected ${expected}, got ${actual}" >&2 exit 1 fi + - name: Verify setup-cli installed from npm + if: matrix.major-version == 'v3' + shell: bash + env: + INSTALL_SOURCE: ${{ steps.setup-cli-v3.outputs.install-source }} + run: | + set -euo pipefail + if [ "${INSTALL_SOURCE}" != "npm" ]; then + echo "::error::setup-cli fell back to ${INSTALL_SOURCE} because installing supabase@${VERSION#v} from npm failed; check the setup-cli step log" + exit 1 + fi # A separate job because GitHub Actions only honours `container:` on Linux runners, and mixing # container + non-container entries in one matrix is fragile when the value is empty. The @@ -87,7 +115,7 @@ jobs: - v2 - v3 runs-on: ubuntu-latest - timeout-minutes: 10 + timeout-minutes: 25 # `node:20-alpine` ships a musl-linked Node, required for JavaScript-based actions to launch # here — the runner's mounted glibc Node won't execute inside an Alpine container. container: @@ -99,7 +127,7 @@ jobs: VERSION: ${{ inputs.version }} steps: - name: Install Alpine prerequisites - run: apk add --no-cache bash curl tar + run: apk add --no-cache bash curl git jq tar - name: Install Supabase CLI via setup-cli@v1 if: matrix.major-version == 'v1' uses: supabase/setup-cli@1dedf2c611547ede7232d26866dd3c56ab903bbb # v1.7.3 @@ -110,9 +138,64 @@ jobs: uses: supabase/setup-cli@afb1b15109756ea5cf9d8985a359d9095235ca2b # v2.1.2 with: version: ${{ inputs.version }} + - name: Checkout the npm wait script + if: matrix.major-version == 'v3' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + path: .smoke-tooling + sparse-checkout: .github/scripts + persist-credentials: false + - name: Wait for npm to serve the version + if: matrix.major-version == 'v3' + shell: bash + run: | + set -euo pipefail + platform="$(node -p 'process.platform + "-" + process.arch')" + bash .smoke-tooling/.github/scripts/wait-for-npm-version.sh "${VERSION#v}" supabase "@supabase/cli-${platform}-musl" - name: Install Supabase CLI via setup-cli@v3 + id: setup-cli-v3 + if: matrix.major-version == 'v3' + uses: supabase/setup-cli@689adcfb80d947100a1eab127289b97d1fb07b13 # v3.2.0 + with: + version: ${{ inputs.version }} + - name: Verify supabase --version matches the expected version + shell: bash + run: | + set -euo pipefail + # `supabase --version` prints the version without the leading `v`, + # while release tags / setup-cli inputs include it, so strip it + # before comparing. + expected="${VERSION#v}" + actual="$(supabase --version | tr -d '\r' | head -n1)" + echo "supabase --version: ${actual}" + if [ "${actual}" != "${expected}" ]; then + echo "Version mismatch: expected ${expected}, got ${actual}" >&2 + exit 1 + fi + - name: Verify setup-cli installed from npm if: matrix.major-version == 'v3' - uses: supabase/setup-cli@45a513f8c64c0bc8e0e3dfe572b5c95be85f6359 # v3.0.1 + shell: bash + env: + INSTALL_SOURCE: ${{ steps.setup-cli-v3.outputs.install-source }} + run: | + set -euo pipefail + if [ "${INSTALL_SOURCE}" != "npm" ]; then + echo "::error::setup-cli fell back to ${INSTALL_SOURCE} because installing supabase@${VERSION#v} from npm failed; check the setup-cli step log" + exit 1 + fi + + setup-cli-smoke-test-github-release: + name: setup-cli v3 GitHub release fallback (ubuntu-latest) + runs-on: ubuntu-latest + timeout-minutes: 10 + env: + VERSION: ${{ inputs.version }} + # An npm executable that always fails sends setup-cli down its GitHub release fallback. + SUPABASE_SETUP_CLI_NPM: "false" + steps: + - name: Install Supabase CLI via setup-cli@v3 + id: setup-cli-v3 + uses: supabase/setup-cli@689adcfb80d947100a1eab127289b97d1fb07b13 # v3.2.0 with: version: ${{ inputs.version }} - name: Verify supabase --version matches the expected version @@ -129,3 +212,13 @@ jobs: echo "Version mismatch: expected ${expected}, got ${actual}" >&2 exit 1 fi + - name: Verify setup-cli installed from the GitHub release + shell: bash + env: + INSTALL_SOURCE: ${{ steps.setup-cli-v3.outputs.install-source }} + run: | + set -euo pipefail + if [ "${INSTALL_SOURCE}" != "github-release" ]; then + echo "::error::setup-cli installed from ${INSTALL_SOURCE} instead of falling back to the GitHub release with npm disabled" + exit 1 + fi diff --git a/.github/workflows/sync-branches.yml b/.github/workflows/sync-branches.yml index 5ab7797833..176beef600 100644 --- a/.github/workflows/sync-branches.yml +++ b/.github/workflows/sync-branches.yml @@ -364,6 +364,10 @@ jobs: break fi done + # download-artifact extracts a lone matching artifact into the path itself, not a subdirectory. + if [ -z "$stage" ] && [ -f "$ARTIFACTS/comment.md" ]; then + stage="$ARTIFACTS" + fi if [ "$RESOLVED" = true ]; then export REPO_DIR="$GITHUB_WORKSPACE" OUTPUT_DIR="$stage" HEAD_SHA if ! { diff --git a/.oxlintrc.effect.json b/.oxlintrc.effect.json deleted file mode 100644 index d3209edd72..0000000000 --- a/.oxlintrc.effect.json +++ /dev/null @@ -1,25 +0,0 @@ -{ - "$schema": "./node_modules/@effect/tsgo/oxlint-schema.json", - "extends": ["./.oxlintrc.json", "./node_modules/@effect/tsgo/oxlint-presets/recommended.json"], - "options": { - "denyWarnings": true - }, - "ignorePatterns": [ - "**", - "!packages/api/**", - "!packages/config/**", - "!packages/stack/**", - "!apps/cli-e2e/**", - "!apps/cli/**", - "!apps/docs/**", - "!packages/cli-test-helpers/**", - "!tools/**", - // Last match wins across the whole list: keep bare re-exclusions after every - // `!` entry that would otherwise re-include them. - "apps/cli/src/shared/cli/bin.ts", - "apps/cli/src/shared/compute/stacks/**", - "apps/cli/src/shared/feedback/database.types.ts", - "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts", - "packages/api/src/generated/contracts.ts" - ] -} diff --git a/.oxlintrc.json b/.oxlintrc.json index 51bff7da6a..f48e4f2a84 100644 --- a/.oxlintrc.json +++ b/.oxlintrc.json @@ -1,15 +1,28 @@ { - "$schema": "./node_modules/oxlint/configuration_schema.json", + "$schema": "./node_modules/@effect/tsgo/oxlint-schema.json", + "extends": ["./node_modules/@effect/tsgo/oxlint-presets/recommended.json"], "options": { "typeAware": true, "denyWarnings": true }, "ignorePatterns": [ + /* Reference submodules, populated by `repos:install`. */ ".repos", + /* Generated from the OpenAPI spec by `packages/api/scripts/generate.ts`. */ + "packages/api/src/generated/contracts.ts", // Generated by `supabase gen types` (see `gen:feedback-types` in apps/cli). "apps/cli/src/shared/feedback/database.types.ts", + /* Standalone Node shim built with `bun build --target node` that does not use Effect. */ + "apps/cli/src/shared/cli/bin.ts", + // Templates that run inside the user's deployed container, not in the CLI process. + "apps/cli/src/shared/compute/stacks", + /* Standalone CI automation, not Effect code. `github-scripts-ci.yml` type-checks and tests it. */ + ".github/scripts", + /* Static JSON fixtures served by the e2e replay server. */ "apps/cli-e2e/fixtures", + /* Test fixtures and golden snapshots. */ "**/testdata", + /* Build, coverage and docs output. */ "**/dist", "**/coverage", "**/.next", @@ -51,11 +64,7 @@ "excludeFiles": [ "apps/cli/src/**/*.test.ts", "apps/cli/src/shared/config/cli-config-provider.layer.ts", - "apps/cli/src/shared/config/cli-config-env.ts", - // Standalone node shim built with `bun build --target node`; it cannot import effect. - "apps/cli/src/shared/cli/bin.ts", - // Templates that run inside the user's deployed container, not in the CLI process. - "apps/cli/src/shared/compute/stacks/**" + "apps/cli/src/shared/config/cli-config-env.ts" ], "rules": { "no-restricted-properties": [ @@ -78,15 +87,6 @@ ] } }, - { - // `.github/scripts` is the only `bun:test` consumer; `@types/bun`'s shared `Matchers` - // types `.rejects.toThrow(...)` as returning `void`, even though it must be awaited at - // runtime. Removing the `await` would make the assertion racy. - "files": [".github/scripts/**"], - "rules": { - "typescript/await-thenable": "off" - } - }, { // The namespace owns paths, leases, artifact pins, and port reservations; everywhere // in the repo, outside `src/namespace/**` itself, reaches that state through @@ -118,8 +118,6 @@ "apps/cli/src/**/*.test.ts", "apps/cli/src/shared/config/cli-config-provider.layer.ts", "apps/cli/src/shared/config/cli-config-env.ts", - "apps/cli/src/shared/cli/bin.ts", - "apps/cli/src/shared/compute/stacks/**", "apps/cli/src/cli/complete.ts", "apps/cli/src/command-internal/colors.ts", "apps/cli/src/command-internal/db-config.parse.ts", @@ -574,6 +572,13 @@ } ] } + }, + { + /* The Docker Engine probe needs `node:http`'s `socketPath` for unix and npipe sockets. */ + "files": ["apps/cli/src/command-internal/db-bootstrap/local-db-running.ts"], + "rules": { + "effecttsgo/node-builtin-import": "off" + } } ] } diff --git a/AGENTS.md b/AGENTS.md index 3afcece062..ad5787f493 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ Bun monorepo with workspaces under `apps/` and `packages/`. `pnpm` is the packag Use an existing TypeScript/Bun workspace, especially `packages/api`, as the package-structure reference. Published `apps/cli` and `packages/config` are not private; `apps/docs` and -`packages/cli-*` have their own shapes. Generic lint, format, and unused-code tooling is +`packages/cli-*` have their own shapes. Lint, format, and unused-code tooling is root-owned. ### Config Naming Vocabulary @@ -35,8 +35,10 @@ all TypeScript runtime code in Effect. Promise-returning APIs are allowed only a for consumers that do not use Effect. The skill is authoritative for Effect coding practices when repository instructions conflict. -Effect linting uses oxlint via `.oxlintrc.effect.json`; run `pnpm lint:effect:check` or -`pnpm lint:effect:fix` from the repository root. +Effect linting is part of the repo lint (`.oxlintrc.json` extends the Effect oxlint preset); run +`pnpm lint:check` or `pnpm lint:fix` from the repository root; `lint:fix` also applies suggestion +fixes, so review its diff. `ignorePatterns` keeps `.github/scripts`, the npm launcher shim, compute +stack templates and the generated API contracts and feedback types out of the lint. Instrument by default: when you add or change code that does I/O (HTTP, SQL, filesystem beyond a small read, child processes, containers, waits) or millisecond-scale work, give it a named span @@ -67,9 +69,9 @@ Keep the local feedback loop fast; full CI runs for ready PRs targeting `develop Use `pnpm test` only when its scope fits the change; the CLI's aggregate script includes full E2E, so use `pnpm run test:unit` and `pnpm run test:integration` with affected test files locally. When repo-wide validation is warranted, use root `pnpm check:all` or `pnpm fix:all`. These are the -repo-wide quality entrypoints: `check:all` runs generic and Effect lint, format, knip, type, and +repo-wide quality entrypoints: `check:all` runs lint, format, knip, type, and GitHub Actions workflow checks (jactionlint and shellcheck, installed by mise); `fix:all` applies -generic and Effect lint, format, and knip fixes. Do not use production `as` casts to silence type +lint, format, and knip fixes. Do not use production `as` casts to silence type errors. Use root Turbo entrypoints for live and auxiliary workflows: diff --git a/apps/cli-e2e/AGENTS.md b/apps/cli-e2e/AGENTS.md index 0b75112979..0ee5456bfc 100644 --- a/apps/cli-e2e/AGENTS.md +++ b/apps/cli-e2e/AGENTS.md @@ -80,7 +80,7 @@ The `workspace` fixture is what triggers scenario setup — if you destructure ` Always return `runE2e(...)` from the body: Vitest does not run a returned Effect, so the harness fails a test that requests `run` or `runNoProjectId` and passes without executing it. A custom runner added with `testBehaviour.extend` gets the same check by wrapping its CLI Effect in `runGuard.track(...)`, as `stackRun` does. Pass the test's `signal` so a Vitest timeout interrupts the body. -This workspace is under the Effect lint (`.oxlintrc.effect.json`, warnings are errors), which rejects `async` test bodies, global `fetch`, `process.env` and `JSON.parse` in Effect code. Decode JSON stdout with `decodeJsonOutput(schema)(result.stdout)`, which reports a stdout excerpt on failure. +This workspace is under the Effect lint (`.oxlintrc.json`, warnings are errors), which rejects `async` test bodies, global `fetch`, `process.env` and `JSON.parse` in Effect code. Decode JSON stdout with `decodeJsonOutput(schema)(result.stdout)`, which reports a stdout excerpt on failure. Available fixtures: diff --git a/apps/cli-e2e/fixtures/pg/toast-sizes.json b/apps/cli-e2e/fixtures/pg/toast-sizes.json new file mode 100644 index 0000000000..296e33254f --- /dev/null +++ b/apps/cli-e2e/fixtures/pg/toast-sizes.json @@ -0,0 +1,15 @@ +{ + "columns": [ + "name", + "total_size", + "heap_size", + "toast_size", + "toast_live_chunks", + "toast_dead_chunks", + "toast_dead_pct", + "last_autovacuum", + "last_vacuum" + ], + "typeOids": [25, 25, 25, 25, 20, 20, 1700, 25, 25], + "rows": [["public.documents", "4200 MB", "800 MB", "3400 MB", "250000", "18000", "6.7", "2026-04-23 12:00", ""]] +} diff --git a/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json new file mode 100644 index 0000000000..fe51488c70 --- /dev/null +++ b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-exits-non-zero-on-connection-refused/interactions.json @@ -0,0 +1 @@ +[] diff --git a/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json new file mode 100644 index 0000000000..fe51488c70 --- /dev/null +++ b/apps/cli-e2e/fixtures/scenarios/inspect-db-toast-sizes-renders-query-results-as-a-table/interactions.json @@ -0,0 +1 @@ +[] diff --git a/apps/cli-e2e/src/tests/inspect.e2e.test.ts b/apps/cli-e2e/src/tests/inspect.e2e.test.ts index 12f3e4d9d2..441b3888a3 100644 --- a/apps/cli-e2e/src/tests/inspect.e2e.test.ts +++ b/apps/cli-e2e/src/tests/inspect.e2e.test.ts @@ -29,6 +29,7 @@ const SUBCOMMANDS = [ { name: "vacuum-stats", fixtureKey: "vacuum-stats", assertValue: "public.events" }, { name: "table-stats", fixtureKey: "table-stats", assertValue: "public.orders" }, { name: "traffic-profile", fixtureKey: "traffic-profile", assertValue: "sessions" }, + { name: "toast-sizes", fixtureKey: "toast-sizes", assertValue: "public.documents" }, ] as const; describe("inspect:flags", () => { diff --git a/apps/cli/AGENTS.md b/apps/cli/AGENTS.md index 7f16660f9a..255aa73a61 100644 --- a/apps/cli/AGENTS.md +++ b/apps/cli/AGENTS.md @@ -271,6 +271,11 @@ Keep CLI tests hermetic: real filesystem layers that use `RuntimeInfo.homeDir` o or credentials layers also use `isolatedHomeLayer` so ambient `SUPABASE_HOME`, `SUPABASE_PROFILE`, and tokens cannot leak. +Cross-backend consumer behavior lives in `tests/contract/`: `cases.ts` holds supabase-js cases per +service that run unchanged against legacy, stack/container and stack/native, one +`contract..integration.test.ts` runner per setup. A parity fix adds the case it makes pass +there; a new service adds its case list next to the others. + Use `*.unit.test.ts` for pure logic and narrow edge cases, `*.integration.test.ts` for handlers and service behavior with realistic layered replacements, `*.e2e.test.ts` for local subprocess and Docker-stack behavior, and `*.live.test.ts` only when the command reaches the real platform or diff --git a/apps/cli/bench/workloads.full.json b/apps/cli/bench/workloads.full.json index 90fef53a28..0b17661da1 100644 --- a/apps/cli/bench/workloads.full.json +++ b/apps/cli/bench/workloads.full.json @@ -48,7 +48,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "db-query", @@ -61,7 +61,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "db-reset", @@ -90,7 +90,7 @@ "warmup": 1, "traceRuns": 1, "timeoutSeconds": 120, - "estimatedSeconds": 60 + "estimatedSeconds": 180 }, { "id": "gen-types-small", @@ -104,7 +104,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "gen-types-medium", @@ -118,7 +118,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "gen-types-large", @@ -132,7 +132,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 100 + "estimatedSeconds": 250 }, { "id": "db-diff-cold", @@ -183,7 +183,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 100 + "estimatedSeconds": 250 }, { "id": "db-lint", @@ -197,7 +197,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "migration-up", @@ -239,7 +239,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 90 + "estimatedSeconds": 220 }, { "id": "stack-start-docker", @@ -266,7 +266,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-stop-docker", @@ -294,7 +294,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-db-reset-docker", @@ -336,7 +336,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-stop-native", @@ -364,7 +364,7 @@ "runs": 10, "warmup": 1, "traceRuns": 2, - "estimatedSeconds": 60 + "estimatedSeconds": 150 }, { "id": "stack-db-reset-native", diff --git a/apps/cli/docs/analytics.md b/apps/cli/docs/analytics.md index fe808b4684..f56f6bc8c3 100644 --- a/apps/cli/docs/analytics.md +++ b/apps/cli/docs/analytics.md @@ -78,8 +78,12 @@ sanitized error classification: - `has_suggestion` - `suggestion_type` - `suggested_command` (only when the remediation is an allowlisted command) +- `config_key` (only on `invalid_config` failures from config resolution: the failing config key + path, such as `inbucket` or `auth.email.smtp.port`, comma-joined when several removed keys fail; + keys only, never values, file paths or user-chosen names, which collapse to their nearest known + parent key) -These values come exclusively from the closed taxonomy in +Apart from `config_key`, these values come exclusively from the closed taxonomy in `src/shared/telemetry/error-actionability.ts` — never from raw error text — and the KPI query semantics (strict recovery, repeat errors, internal/unknown bug rate) are documented there in `CliErrorActionabilityMetricDefinitions`. @@ -101,6 +105,15 @@ Local-runtime commands also carry properties that split error rates by runtime: - `orioledb` (boolean) once the command resolves local database config, `true` when the project selects an OrioleDB image (`db.orioledb_version` on a 15/17 project). It describes local config only; for `--remote` targets it says nothing about the remote database +- `serve_end_reason` on the Docker-runtime `functions serve`, set once its final outcome is known: + `signal` (the CLI received a shutdown signal), `container_stopped` (an external supervisor + stopped the Edge Runtime container, e.g. `supabase stop`), `container_exited` (the container + exited on its own with code 0), `container_removed` (the container disappeared), or + `container_crashed` (the container exited with a crash code, which also fails the command). A + restart records nothing; only the final end does +- `edge_runtime_exit_code` (number) alongside `serve_end_reason` when the container's exit code is + known: its stop code for `container_stopped`, `0` for `container_exited`, its crash code for + `container_crashed` Flag capture is intentionally conservative: `flags` records only flags changed on the command line, and redacts each value unless it is boolean, a choice flag, or listed in the command's diff --git a/apps/cli/docs/compute-commands.md b/apps/cli/docs/compute-commands.md index ae8b3a4395..d755a93c1a 100644 --- a/apps/cli/docs/compute-commands.md +++ b/apps/cli/docs/compute-commands.md @@ -39,6 +39,12 @@ When enabled, the CLI exposes: - `supabase compute logs` - `supabase compute delete` +`supabase compute logs` remains available until v4 with its existing `--tail`, +`--kind`, and `-f/--follow` options. It prints a deprecation warning on stderr. +For the new log browser, read Compute logs with `supabase logs --remote --service compute`. Use +`--resource ` to select an application and `--kind app|requests|builds` to +select its log stream. Press `L` in the interactive browser to load more logs. + Compute source directories live under `supabase/compute//`. `compute new` edits TOML configuration. It refuses projects whose authoritative diff --git a/apps/cli/docs/release-process.md b/apps/cli/docs/release-process.md index 55cc60a409..7ade1efbf7 100644 --- a/apps/cli/docs/release-process.md +++ b/apps/cli/docs/release-process.md @@ -213,7 +213,7 @@ Production releases live in a single `[.github/workflows/release.yml](../../../. | next | push: next (after `Test next` passes) | `next` | none | prerelease | `X.Y.Z-next.N` | | maintenance | manual dispatch on a `v.x` ref | `v.stable` | `supabase@` / `supabase-v-stable` | published, never latest | `N.Y.Z` | -`beta` auto-publishes on every merge to `develop` (CLI). `stable` auto-publishes after a develop→main fast-forward, which itself happens when the weekly `[deploy.yml](../../../.github/workflows/deploy.yml)` cron PR is approved (the FF push to `main` re-fires `release.yml` via the `push: branches: [main]` trigger). `next` auto-publishes on every push to the `next` branch, and `maintenance` is dispatched by hand (see [Maintenance lines](#maintenance-lines)). +`beta` auto-publishes on every merge to `develop` (CLI). `stable` auto-publishes after a develop→main fast-forward, which itself happens when the weekly `[deploy.yml](../../../.github/workflows/deploy.yml)` cron PR is approved (the FF push to `main` re-fires `release.yml` via the `push: branches: [main]` trigger). `next` auto-publishes on every push to the `next` branch that carries commits beyond `develop` (see [The `next` channel](#the-next-channel)), and `maintenance` is dispatched by hand (see [Maintenance lines](#maintenance-lines)). Versions are computed by `cycjimmy/semantic-release-action` from the first line of each squash commit (the PR title). Conventional commit titles use `feat:` → minor, `fix:`, `perf:`, or `revert:` → patch, and `!` after the type or scope (`feat!:` / `fix(scope)!:`) → major. Commit bodies, including `BREAKING CHANGE` notes, are ignored for version calculation. The `release` config lives in `apps/cli/package.json`: `develop` is a `beta` prerelease branch, `next` is a `next` prerelease branch, `main` emits plain `X.Y.Z`, and a wildcard entry `v+([0-9]).x` turns every `v.x` branch into a maintenance line on the `v.stable` channel (the entry is ignored while no such branch exists). @@ -245,7 +245,7 @@ flowchart LR flowchart TD pushDev[push: develop] --> plan pushMain[push: main] --> plan - pushNext[push: next] --> testNext["Test next
test.yml (reusable)"] + pushNext[push: next] -->|"only with commits beyond develop"| testNext["Test next
test.yml (reusable)"] testNext -->|passes| plan pr["approved PR
develop→main, next→develop, sync/*"] --> ff["fast-forward.yml
checks + git push via App token"] ff --> pushMain @@ -281,11 +281,14 @@ gh workflow run release.yml \ Auto-trigger paths leave `version` empty: semantic-release computes it from commits since the last tag. A dispatch with an empty `version` does the same for the dispatched ref, and fails if the channel does not match the ref or if semantic-release computes no release. +The `develop` and `next` plans wait, up to 120 minutes, until the `publish` job of any in-flight `Release` run on `main` has finished before reading tags: that job pushes the stable tag after the npm publish, so a prerelease planned earlier would miss it and compute a stale version. The rest of the `main` run (Homebrew, Scoop, smoke tests) does not delay the plan. If the wait times out, the plan job fails; re-run it once that job has finished. + The plan job refuses a dispatch when: - `channel=next` is not on the `next` ref, or `channel=maintenance` is not on a `v.x` ref; - `next` or a `v.x` ref is dispatched with `beta` or `stable` (those refs release only their own channel); - `channel=stable` is not a dry run and the ref is not `main` (any other ref, including `hotfix/*`, may only dry-run); +- `channel=next` is dispatched while `next` has no commits beyond `develop` (see [The `next` channel](#the-next-channel)); - a non-empty `version` is not semver (no `v` prefix, no build metadata) or does not match the channel: `next` needs `X.Y.Z-next.N`, `beta` needs `X.Y.Z-beta.N`, and `stable` and `maintenance` need plain `X.Y.Z`. ### Hotfix release flow @@ -315,9 +318,11 @@ Do not use `workflow_dispatch dry_run=false` as a hotfix path. A stable release `next` is one long-lived branch reused for every future major. Breaking PRs target `next` instead of `develop`, with a `type(scope)!:` title; they use the same merge queue, required checks, and review as `develop`. The `Lint Pull Request` check fails any `!` title whose base is not `next`, with one exception: the `next` → `develop` cut PR. -Every push to `next` first runs the full `test.yml` suite (the `Test next` job in `release.yml`). Only when it passes does the pipeline plan and publish `X.Y.Z-next.N` to the npm `next` dist-tag and a GitHub prerelease. `next` has no Homebrew, Scoop, or Linear publication. It does get the raw semantic-release changelog on its GitHub Release (`backfill-release-notes`); the LLM-proposed notes PR is stable-only, and `apply-release-notes` still accepts `-next` tags for a manual proposal. +Every push to `next` that carries commits beyond `develop` first runs the full `test.yml` suite (the `Test next` job in `release.yml`). Only when it passes does the pipeline plan and publish `X.Y.Z-next.N` to the npm `next` dist-tag and a GitHub prerelease. `next` has no Homebrew, Scoop, or Linear publication. It does get the raw semantic-release changelog on its GitHub Release (`backfill-release-notes`); the LLM-proposed notes PR is stable-only, and `apply-release-notes` still accepts `-next` tags for a manual proposal. + +`next` publishes use the `next` prerelease identifier so versions never collide with `beta` on npm after a cut. After a stable release, semantic-release on `next` computes past it on the following push with commits beyond `develop`. The npm `next` dist-tag is not moved by the stable release, because publishing is OIDC-only and `npm dist-tag add` needs a token the pipeline does not hold; `supabase@next` catches up on the next release from `next`, which needs commits beyond `develop`. A release can be re-cut by dispatching `release.yml` with `channel=next` on the `next` ref. -`next` publishes use the `next` prerelease identifier so versions never collide with `beta` on npm after a cut. After a stable release, semantic-release on `next` computes past it on the following push. The npm `next` dist-tag is not moved by the stable release, because publishing is OIDC-only and `npm dist-tag add` needs a token the pipeline does not hold; `supabase@next` catches up on the next push to `next`. A release can be re-cut by dispatching `release.yml` with `channel=next` on the `next` ref. +A push to `next` with no commits beyond `develop` skips `Test next` and the release. That is the usual state between majors, because the sync fast-forwards `next` onto `develop`. semantic-release reads channel notes per commit ([semantic-release#4073](https://github.com/semantic-release/semantic-release/issues/4073)), so a `next` tag on a commit `develop` already tagged misreads both tags. `supabase@next` stays on the last `next` release until a breaking change lands on `next`, and `supabase@beta` carries the newest code meanwhile. A `channel=next` dispatch is refused in that state. The nightly live e2e run covers `main`, `develop`, and `next`. @@ -529,7 +534,7 @@ Both updaters run automatically from `release-shared.yml`'s `publish-homebrew` a Once the channels are live, two reusable workflows run automatically (last in `release-shared.yml`, non-gating — by the time they run the artifacts are already published, so a failure surfaces as a red post-release signal rather than blocking distribution): -- `[setup-cli-smoke-test.yml](../../../.github/workflows/setup-cli-smoke-test.yml)` (`setup-cli-smoke` job) — installs the released version through `supabase/setup-cli` v1 and v2 (the GitHub Release download path) and v3 (the npm package) on Linux, macOS, Windows, and Alpine. +- `[setup-cli-smoke-test.yml](../../../.github/workflows/setup-cli-smoke-test.yml)` (`setup-cli-smoke` job) — installs the released version through `supabase/setup-cli` v1 and v2 (the GitHub Release download path) and v3 (the npm package) on Linux, macOS, Windows, and Alpine. v3 waits on each runner until npm serves the version (up to 15 minutes) and fails unless setup-cli installed from npm; one ubuntu job forces the GitHub release fallback. - `[verify-install-channels.yml](../../../.github/workflows/verify-install-channels.yml)` (`verify-install-channels` job) — runs a **real** `brew install` (macOS **and** Linux, so both the `on_macos` and `on_linux` stanzas of the formula are exercised), `scoop install`, and `curl|bash` install of the **published** install script (fetched from the release asset, not the repo checkout) against the just-published Homebrew tap, Scoop bucket, and GitHub Release. Each leg then asserts `supabase --version` matches and runs `supabase --help` to exercise the actual command tree. brew, scoop, and the install script each verify the published `sha256`/`hash` against the downloaded tarball, so this is the signal that would have caught CLI v2.107.0 (where the brew/scoop manifests shipped checksums that did not match the release tarballs and every `brew install` / `scoop install` failed). It only runs for `beta`/`stable`/`maintenance` (the channels that publish brew/scoop) and can be dispatched manually against any already-published version via the Actions tab. ### Code signing (macOS) diff --git a/apps/cli/docs/stack-commands.md b/apps/cli/docs/stack-commands.md index e26d1f5ed9..d2128923cc 100644 --- a/apps/cli/docs/stack-commands.md +++ b/apps/cli/docs/stack-commands.md @@ -226,20 +226,25 @@ commands retain their existing behavior. `functions serve` requires a running project stack. It attaches to the existing Functions member, waking it when needed. If Functions was excluded, it creates a temporary standalone Functions instance on the stack's API listener without changing the saved composition. Ctrl-C destroys that -temporary instance; an existing member remains available. `--no-verify-jwt` and `--env-file` apply -for the serving session and the previous configuration is restored on exit. Ordinary code edits +temporary instance; an existing member remains available. On a running stack, any difference +between the configuration serve resolves (env files, per-function `.env`, `[functions.]` +settings, `--env-file`, `--debug`, `--no-verify-jwt`) and the saved one restarts Functions for +the serving session, and the saved configuration is restored on exit. Edits made while serving +apply on the next `functions serve`, because the stack backend does not watch files. Ordinary code edits are picked up by the next invocation without a runtime restart. Import-map and inspector flags -and multiline environment values are currently unsupported on this backend. Normal exit owns cleanup; forcibly killing the CLI +are currently unsupported on this backend. Normal exit owns cleanup; forcibly killing the CLI cannot guarantee restoration or removal of its temporary instance. Concurrent override sessions for the same Functions instance are unsupported. Cleanup waits for submitted lifecycle operations; additional signals do not force it to abandon cleanup. Restoration restores configuration and keeps the service running; it does not return it to a sleeping state. -Stack mode reads the shared `supabase/functions/.env` file when creating Functions; `--env-file` -replaces those custom values for the session. A normal `start` applies changes to the shared file -and restores default JWT verification on the same Functions instance. Per-function -`.env` files are not loaded. Reserved -`SUPABASE_*` values in these files are ignored because the runtime supplies them. With +Stack mode resolves the Functions env like legacy `start` and `functions serve`: the shared +`supabase/functions/.env` file overrides `[edge_runtime.secrets]`, and `functions serve` also +reads each enabled function's own `.env` on every invocation; `--env-file` replaces those files +for the session. Multiline values are supported. `start` applies changes to the shared file and +restores default JWT verification only when it creates or replans the stack; a `start` against a +running stack keeps the current Functions configuration. Reserved `SUPABASE_*` +values in these files are skipped with a warning because the runtime supplies them. With `--output-format stream-json`, readiness emits a result containing the instance ID and URL, followed by live log events. @@ -275,6 +280,31 @@ lint transaction (always rolled back). It does not launch a client binary. ## Reading stack logs +`supabase logs` reads retained stdout/stderr from composition members and `gateway` +request lines from the shared API port. Interactive terminals open a browser; press `L` +to load more records. Pipes and machine output perform one finite read and exit. + +Select `--stack ` or `--stack-id `. Repeat `--service ` to select +services, use `--service postgres` for the database, or `--resource ` +for one instance. `--service gateway` reads only request lines. Reads work while +the stack is stopped and never start an owner or service. + +`--limit N` (default 100, maximum 1000) selects the initial newest lines. `--since` +accepts a duration (`10m`, `1h30m`), an ISO-8601 time, or `start` for each service's +latest launch. The default is the last hour. The requested lower bound stays fixed +when you load newer logs, and the header shows the captured end of each successful +load. Failed loads preserve the displayed rows and window. + +Use `--output-format json` for an array of retained records, or `stream-json` for +`log-entry` and `log-marker` events. Each event's `data` preserves the retained +record, including instance IDs, launch IDs, and positions. Missing stacks and +unmatched services fail with status 1. Browser interruption exits 130. + +The deprecated `supabase stack logs` keeps its existing `--tail` and `--follow` +behavior until removal in v4. + +### Legacy stack logs + `supabase stack logs` prints the retained stdout/stderr of composition members, plus the `gateway` request lines of the shared API port, and exits; `-f/--follow` then streams new lines until interrupted. Select `--stack ` or `--stack-id `; the @@ -296,6 +326,9 @@ prints the history as one array and does not accept `--follow`. Missing stacks, unmatched services, and `--follow` without an owner fail with status 1; interruption exits 130. +The command emits a deprecation warning on stderr and remains available until v4. +Stream JSON also includes a `data` object containing the original retained record. + ## Data and configuration The backends own separate state and databases. Enabling the flag does not import, copy, seed from, @@ -352,8 +385,8 @@ format, or comes from a newer CLI always fails the start. The lock's JSON Schema `https://supabase.com/schemas/supabase-lock/v1.json`. A database with initialized data keeps its build: any change other than a new `-rN` revision of the same build (a different upstream build, release line or major) is refused. The refusal offers to keep the data by pinning the recorded build with -`[local.services.database] version`, setting `[db] major_version` back when the major differs and removing any -`[db] orioledb_version` when an OrioleDB build is requested, or to run `supabase stack destroy` and recreate it. `supabase link` +`local.services.database.version` in your config file, setting `db.major_version` back when the major differs and removing any +`db.orioledb_version` when an OrioleDB build is requested, or to run `supabase stack destroy` and recreate it. `supabase link` and `supabase stack update` warn when the versions they lock would be refused by an existing stack's data. A database that never initialized adopts the change. See the [`start` side effects](../src/commands/experimental/stack/start/SIDE_EFFECTS.md#service-versions-and-the-lock) for @@ -441,7 +474,7 @@ unavailable fails with `StackStorageUnavailableError`, guiding the user to run ` status`/`supabase stack restart`, or `supabase start` when the stack has never been configured. Missing Storage HTTP endpoints fail with `StackStorageCapabilityError`, as do other Storage capability failures. A stack-gateway 502/503 encountered while Storage activates is reported as -`StackStorageCapabilityError` guiding the user to `supabase stack logs` then `supabase stack +`StackStorageCapabilityError` guiding the user to `supabase logs --service storage` then `supabase stack restart`, not as a raw status body — but only for a local target; a `--remote` failure passes through unchanged. All of these failures exit `1` for `storage`/`seed buckets`, and no HTTP diff --git a/apps/cli/docs/supabase/inspect/db-toast-sizes.md b/apps/cli/docs/supabase/inspect/db-toast-sizes.md new file mode 100644 index 0000000000..2a0debf6e9 --- /dev/null +++ b/apps/cli/docs/supabase/inspect/db-toast-sizes.md @@ -0,0 +1,13 @@ +# db-toast-sizes + +This command displays TOAST table sizes and dead chunk counts for every ordinary user table that has a TOAST relation (materialized views and partitioned parents are not listed). When a row exceeds ~2 kB, Postgres compresses and/or moves its largest variable-length values (TEXT, JSONB, bytea) out-of-line into a companion TOAST table. TOAST Size includes the TOAST index; Total Size also includes the table's own indexes. Autovacuum runs on the TOAST table independently from the main heap, so it can accumulate dead chunks even when the parent table looks healthy by its own dead-tuple count. + +A table that appears fine by `vacuum-stats` or `bloat` alone can still have significant TOAST bloat that wastes disk space and slows reads. High `TOAST Dead %` values indicate that autovacuum is not keeping up with the TOAST table and a manual `VACUUM` may be needed. + +``` + TABLE │ TOTAL SIZE │ HEAP SIZE │ TOAST SIZE │ TOAST LIVE CHUNKS │ TOAST DEAD CHUNKS │ TOAST DEAD % │ LAST AUTOVACUUM │ LAST VACUUM +────────────────────────┼────────────┼───────────┼────────────┼───────────────────┼───────────────────┼──────────────┼────────────────────┼───────────── + public.documents │ 4350 MB │ 800 MB │ 3400 MB │ 250000 │ 18000 │ 6.7 │ 2024-03-01 04:12 │ + public.media_assets │ 905 MB │ 120 MB │ 770 MB │ 80000 │ 200 │ 0.2 │ 2024-03-02 01:45 │ + public.messages │ 365 MB │ 280 MB │ 60 MB │ 95000 │ 0 │ 0.0 │ 2024-03-02 03:10 │ +``` diff --git a/apps/cli/docs/supabase/inspect/db-xid-age.md b/apps/cli/docs/supabase/inspect/db-xid-age.md new file mode 100644 index 0000000000..88a8479929 --- /dev/null +++ b/apps/cli/docs/supabase/inspect/db-xid-age.md @@ -0,0 +1,13 @@ +# db-xid-age + +This command lists user tables sorted by their transaction ID (XID) age, oldest first. PostgreSQL's transaction ID space wraps around at about 2.1 billion transactions. Once a table's XID age passes `autovacuum_freeze_max_age` (200 million by default), autovacuum forces a freeze on it; if the age keeps growing toward the wraparound limit, PostgreSQL stops assigning new transaction IDs to protect the data. `transactions_remaining` is the headroom to a 2 billion threshold, kept below the real limit as a safety margin. + +Tables with an XID age above 1.5 billion transactions (`transactions_remaining` below 500 million) should be treated as urgent: manual `VACUUM FREEZE` or a tuned autovacuum run is needed. The list covers ordinary user tables only: it leaves out materialized views, TOAST tables and Supabase-managed schemas, so it does not show the database-wide age. Check `SELECT age(datfrozenxid) FROM pg_database WHERE datname = current_database()` for that. + +``` + TABLE │ XID AGE │ TRANSACTIONS REMAINING +─────────────────────────┼───────────┼──────────────────────── + public.events │ 800000000 │ 1200000000 + public.users │ 500000000 │ 1500000000 + public.sessions │ 120000000 │ 1880000000 +``` diff --git a/apps/cli/docs/supabase/login.md b/apps/cli/docs/supabase/login.md index 5dbc7df448..66aef424d2 100644 --- a/apps/cli/docs/supabase/login.md +++ b/apps/cli/docs/supabase/login.md @@ -7,3 +7,5 @@ Your access token is stored securely in native credentials storage (the operatin > If this behavior is not desired, such as in a CI environment, you may skip login by specifying the `SUPABASE_ACCESS_TOKEN` environment variable in other commands. The Supabase CLI uses the stored token to access Management APIs for projects, functions, secrets, etc. + +> The Management API client retries retryable failures up to 5 times by default. Set `SUPABASE_API_MAX_RETRIES` to change that. It counts retries after the first attempt, so `0` sends each request once. diff --git a/apps/cli/docs/tracing-monitoring.md b/apps/cli/docs/tracing-monitoring.md index cdc4b92766..c42cee4b1e 100644 --- a/apps/cli/docs/tracing-monitoring.md +++ b/apps/cli/docs/tracing-monitoring.md @@ -178,9 +178,14 @@ the timing ranges, API calls per command, and the slowest phases. It reports: responses (the build did not follow the recorded scenario), non-zero or differing exit codes, missing traces, and too few runs for a timing verdict. Other count changes, such as 1 to 2 calls, appear in the API calls table with the changed endpoints but are not flagged. -- Wall time: base ("before") and branch ("after") medians and ranges. A verdict of slower or faster needs the branch - median outside the base min-max range, a change of at least 5 percent and 5 ms, and at least 5 - measured runs; otherwise it reads "within noise", or "not comparable" when a build failed. +- Wall time: base ("before") and branch ("after") medians and ranges. A verdict of slower or faster needs a + change of at least 5 percent and 5 ms that is also larger than 1.5 times the wider build's + interquartile range, a branch median beyond the base min-max range for run-scoped workloads, and + at least 5 measured runs; otherwise it reads "within noise", or "not + comparable" when a build failed. Block-scoped workloads record a block index per launch and, from + three paired blocks, judge block medians instead: the median per-block change must pass the same + tests and more than half of the blocks must move the same way past the floor, so one outlier + stack session can neither flag nor hide a change. Thresholds are named constants in `bench-report.ts`. - API calls per workload, base and branch. - Phase deltas: the three largest span deltas for each workload with a slower or faster verdict. @@ -297,7 +302,7 @@ described above: | Field | Meaning | | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | | `setup`, `teardown` | Lists of CLI arguments run untimed with the build under test. `teardown` always runs, even after a failure or timeout. | -| `setupScope` | `run` (default) repeats setup and teardown for every launch; `block` starts one stack per block of five launches. | +| `setupScope` | `run` (default) repeats setup and teardown for every launch; `block` starts one stack per block of two launches. | | `stack` | Sets `SUPABASE_EXPERIMENTAL_STACK=1` on every launch of the workload, the only variable a workload can set. | | `runtime` | `docker` or `native`. A stack workload must pass the same `--runtime` to every `stack start`. | | `fixture` | A directory under `apps/cli/bench/fixtures/` copied into the project's `supabase/` directory after `init`. | diff --git a/apps/cli/package.json b/apps/cli/package.json index b4693fbbec..6f2dda5d0e 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -60,7 +60,7 @@ "@supabase/pg-topo": "1.0.0-alpha.8", "@supabase/stack": "workspace:*", "@supabase/supabase-js": "catalog:", - "@supabase/typegen": "0.3.1", + "@supabase/typegen": "0.3.2", "@tsconfig/bun": "catalog:", "@types/bun": "catalog:", "@types/pg": "^8.23.1", diff --git a/apps/cli/scripts/backfill-release-notes.e2e.test.ts b/apps/cli/scripts/backfill-release-notes.e2e.test.ts new file mode 100644 index 0000000000..3323fcc37c --- /dev/null +++ b/apps/cli/scripts/backfill-release-notes.e2e.test.ts @@ -0,0 +1,180 @@ +import { BunFileSystem, BunPath } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Layer, Path, Schema } from "effect"; +import { fileURLToPath } from "node:url"; + +const scriptPath = fileURLToPath(new URL("./backfill-release-notes.ts", import.meta.url)); +const cliPackagePath = fileURLToPath(new URL("../package.json", import.meta.url)); +const analyzerPath = fileURLToPath(new URL("./analyze-commits-title.js", import.meta.url)); +const platformLayer = Layer.mergeAll(BunFileSystem.layer, BunPath.layer); + +const decodeCliPackage = Schema.decodeUnknownEffect( + Schema.fromJsonString(Schema.Struct({ release: Schema.Unknown })), +); +const encodePackage = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown)); + +const isolatedGit = { + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_CONFIG_NOSYSTEM: "1", + GIT_AUTHOR_NAME: "seed", + GIT_AUTHOR_EMAIL: "seed@example.com", + GIT_COMMITTER_NAME: "seed", + GIT_COMMITTER_EMAIL: "seed@example.com", +}; + +const spawn = (argv: ReadonlyArray, cwd: string, env: Record) => + Effect.acquireUseRelease( + Effect.sync(() => Bun.spawn([...argv], { cwd, env, stdout: "pipe", stderr: "pipe" })), + (child) => + Effect.gen(function* () { + const [exitCode, stdout, stderr] = yield* Effect.all( + [ + Effect.promise(() => child.exited), + Effect.promise(() => new Response(child.stdout).text()), + Effect.promise(() => new Response(child.stderr).text()), + ], + { concurrency: "unbounded" }, + ); + return { exitCode, stdout, stderr }; + }), + (child) => + child.exitCode === null && child.signalCode === null + ? Effect.sync(() => child.kill()).pipe(Effect.andThen(Effect.promise(() => child.exited))) + : Effect.void, + ); + +interface SeedHistory { + readonly git: (...args: ReadonlyArray) => Effect.Effect; + readonly commit: (message: string) => Effect.Effect; + readonly release: (tag: string, channels: string) => Effect.Effect; +} + +/** + * A working repository on `main` with an `origin` bare remote, `v1.0.0` on its first commit and + * the history `seed` adds on top. Every branch and tag is pushed, and every release also gets + * the per-tag channel note ref the release pipeline pushes. + */ +const repositoryWithHistory = (seed: (history: SeedHistory) => Effect.Effect) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "backfill-release-notes-" }); + const origin = path.join(root, "origin.git"); + const work = path.join(root, "work"); + const env = { ...process.env, ...isolatedGit }; + + const git = (cwd: string, ...args: ReadonlyArray) => + Effect.gen(function* () { + const result = yield* spawn(["git", ...args], cwd, env); + expect(result.exitCode, result.stderr).toBe(0); + return result.stdout; + }); + const commit = (message: string) => + git(work, "commit", "--quiet", "--allow-empty", "-m", message); + const release = (tag: string, channels: string) => + Effect.gen(function* () { + yield* git(work, "tag", "-a", tag, "-m", `Release ${tag}`); + yield* git( + work, + "notes", + `--ref=semantic-release-${tag}`, + "add", + "-m", + `{"channels":${channels}}`, + `${tag}^{commit}`, + ); + }); + + const cliPackage = yield* decodeCliPackage(yield* fs.readFileString(cliPackagePath)); + const packageJson = yield* encodePackage({ + name: "supabase", + version: "0.0.0-automated", + repository: { type: "git", url: "https://github.com/supabase/cli" }, + release: cliPackage.release, + }); + yield* fs.makeDirectory(path.join(work, "apps/cli/scripts"), { recursive: true }); + yield* fs.writeFileString(path.join(work, "apps/cli/package.json"), `${packageJson}\n`); + yield* fs.copyFile(analyzerPath, path.join(work, "apps/cli/scripts/analyze-commits-title.js")); + + yield* git(root, "init", "--quiet", "--bare", "--initial-branch=main", origin); + yield* git(work, "init", "--quiet", "--initial-branch=main"); + yield* git(work, "remote", "add", "origin", origin); + yield* git(work, "add", "-A"); + yield* commit("feat: initial"); + yield* release("v1.0.0", "[null]"); + + yield* seed({ git: (...args) => git(work, ...args), commit, release }); + + yield* git( + work, + "push", + "--quiet", + "origin", + "refs/heads/*:refs/heads/*", + "refs/tags/*:refs/tags/*", + "refs/notes/*:refs/notes/*", + ); + + const bunExecutable = Bun.which("bun"); + if (!bunExecutable) throw new Error("Bun executable not found"); + const { FORCE_COLOR: _forceColor, NO_COLOR: _noColor, ...inherited } = process.env; + + return { + backfill: (tag: string) => + spawn([bunExecutable, scriptPath, "--tag", tag], work, { ...inherited, ...isolatedGit }), + }; + }); + +/** + * `develop` and `next` share the commit that carries both `v1.1.0-beta.1` and `v1.1.0-next.1` + * (`next` is fast-forwarded onto `develop`), and `develop` moves on to `v1.1.0-beta.2`. + */ +const sharedBetaAndNextHistory = repositoryWithHistory(({ git, commit, release }) => + Effect.gen(function* () { + yield* git("switch", "--quiet", "--create", "develop"); + yield* commit("feat: add shared thing"); + yield* release("v1.1.0-beta.1", '["beta"]'); + yield* release("v1.1.0-next.1", '["next"]'); + yield* git("branch", "next"); + yield* commit("fix: patch beta"); + yield* release("v1.1.0-beta.2", '["beta"]'); + }), +); + +const forcedMaintenanceHistory = repositoryWithHistory(({ git, commit, release }) => + Effect.gen(function* () { + yield* commit("feat!: drop the legacy flag"); + yield* release("v2.0.0", "[null]"); + yield* git("switch", "--quiet", "--create", "v1.x", "v1.0.0"); + yield* commit("ci: pin the runner image"); + yield* release("v1.0.1", '["v1.stable"]'); + }), +); + +describe("backfill-release-notes.ts on a repository whose beta and next tags share a commit", () => { + it.live("backfills the next beta from the beta line instead of the next channel's tags", () => + Effect.gen(function* () { + const repository = yield* sharedBetaAndNextHistory; + + const { exitCode, stdout, stderr } = yield* repository.backfill("v1.1.0-beta.2"); + + expect(exitCode, stderr).toBe(0); + expect(stdout, stderr).toContain("compare/v1.1.0-beta.1...v1.1.0-beta.2"); + expect(stdout, stderr).toContain("patch beta"); + expect(stdout, stderr).not.toContain("add shared thing"); + }).pipe(Effect.provide(platformLayer)), + ); +}); + +describe("backfill-release-notes.ts on a maintenance line", () => { + it.live("backfills a forced maintenance release whose commits imply no release", () => + Effect.gen(function* () { + const repository = yield* forcedMaintenanceHistory; + + const { exitCode, stdout, stderr } = yield* repository.backfill("v1.0.1"); + + expect(exitCode, stderr).toBe(0); + expect(stdout, stderr).toContain("compare/v1.0.0...v1.0.1"); + }).pipe(Effect.provide(platformLayer)), + ); +}); diff --git a/apps/cli/scripts/backfill-release-notes.ts b/apps/cli/scripts/backfill-release-notes.ts index e5b81aed56..265b9b9874 100644 --- a/apps/cli/scripts/backfill-release-notes.ts +++ b/apps/cli/scripts/backfill-release-notes.ts @@ -20,11 +20,16 @@ // Without it, raw markdown notes are printed to stdout. import { BunFileSystem, BunPath } from "@effect/platform-bun"; import { $ } from "bun"; -import { Console, Data, Effect, FileSystem, Layer, Path, Schema } from "effect"; +import { Array as Arr, Console, Data, Effect, FileSystem, Layer, Path, Schema } from "effect"; import process from "node:process"; import { parseArgs } from "node:util"; -import semanticRelease from "semantic-release"; -import { channelForTag, releaseBranchForTag } from "./release-channels.ts"; +import semanticRelease, { type AnalyzeCommitsContext } from "semantic-release"; +import { + channelForTag, + isInChannelHistory, + releaseBranchForTag, + releaseTypeBetween, +} from "./release-channels.ts"; class JsonParseFailed extends Data.TaggedError("JsonParseFailed")<{ readonly cause: unknown }> {} @@ -151,19 +156,6 @@ const main = Effect.gen(function* () { $`git -C ${repoRoot} remote get-url origin`.text(), )).trim(); - // Notes refs aren't fetched by `git clone`. Pull from the source repo first - // (network-free), then from origin so even tags whose notes haven't been - // sync'd locally yet are available. - yield* shell(() => - $`git -C ${clone} fetch --no-tags --quiet ${repoRoot} +refs/notes/*:refs/notes/*` - .nothrow() - .quiet(), - ); - yield* shell(() => - $`git -C ${clone} fetch --no-tags --quiet ${originUrl} +refs/notes/*:refs/notes/*` - .nothrow() - .quiet(), - ); // One fetch per branch: git aborts a multi-ref fetch when any named ref is missing, and `next` // may not exist. The `v*.x` glob matches zero or more branches without failing. for (const branchSpec of ["main", "develop", "next", "v*.x"]) { @@ -207,6 +199,13 @@ const main = Effect.gen(function* () { for (const ct of coincidentTags) { yield* shell(() => $`git -C ${clone} tag -d ${ct}`.quiet().nothrow()); } + const targetChannel = channelForTag(tag); + const tagsOutsideChannel = (yield* shell(() => $`git -C ${clone} tag --list`.text())) + .split("\n") + .filter((t) => t && !isInChannelHistory(t, targetChannel)); + if (tagsOutsideChannel.length > 0) { + yield* shell(() => $`git -C ${clone} tag -d ${tagsOutsideChannel}`.quiet()); + } yield* shell(() => $`git -C ${clone} checkout -B ${branch} ${sha} --quiet`); // The clone only carries refs/heads/$BRANCH locally; seed the other @@ -224,23 +223,34 @@ const main = Effect.gen(function* () { } } - // semantic-release's notes reader returns channels=[null] for any tag - // missing an annotation, and the prerelease filter drops those tags - // entirely — walking lastRelease back far enough to drag unrelated commits - // into the changelog. Seed a channel note for every reachable tag that - // lacks one, inferring the channel from the tag name. + // semantic-release reads one channel note per commit (semantic-release/semantic-release#4073), + // so the clone keeps only this channel's prereleases and rebuilds one note per tagged + // commit from tag names. const mergedTagsOut = yield* shell(() => $`git -C ${clone} tag --merged HEAD --sort=v:refname`.text(), ); const mergedTags = mergedTagsOut.split("\n").filter((t) => t && t !== tag); - for (const prevTag of mergedTags) { - const noteCheck = yield* shell(() => - $`git -C ${clone} notes --ref semantic-release show ${prevTag}`.nothrow().quiet(), + const taggedCommits = (yield* shell(() => + $`git -C ${clone} rev-parse ${mergedTags.map((t) => `refs/tags/${t}^{commit}`)}`.text(), + )) + .split("\n") + .filter(Boolean); + if (taggedCommits.length !== mergedTags.length) { + yield* Console.error( + `Resolved ${taggedCommits.length} commits for ${mergedTags.length} tags merged into ${tag}`, ); - if (noteCheck.exitCode === 0) continue; - const payload = yield* encodeJson({ channels: [channelForTag(prevTag)] }); + return yield* Effect.sync(() => process.exit(1)); + } + const channelByCommit = new Map>(); + for (const [prevTag, commit] of Arr.zip(mergedTags, taggedCommits)) { + if (prevTag.includes("-") || !channelByCommit.has(commit)) { + channelByCommit.set(commit, channelForTag(prevTag)); + } + } + for (const [commit, channel] of channelByCommit) { + const payload = yield* encodeJson({ channels: [channel] }); yield* shell(() => - $`git -C ${clone} notes --ref semantic-release add -f -m ${payload} ${prevTag}^{commit}`.quiet(), + $`git -C ${clone} notes --ref semantic-release add -f -m ${payload} ${commit}`.quiet(), ); } @@ -280,10 +290,25 @@ const main = Effect.gen(function* () { delete childEnv[key]; } + const expected = tag.replace(/^v/, ""); + // A dispatch `version` can force a release its commits do not imply (e.g. only `ci:` commits). + const releaseFloor = { + analyzeCommits: (_config: unknown, context: AnalyzeCommitsContext) => + releaseTypeBetween(context.lastRelease.version, expected), + }; + const configuredPlugins = Array.isArray(rootPkg.release?.plugins) + ? rootPkg.release.plugins + : []; + const result = yield* Effect.tryPromise({ try: () => semanticRelease( - { dryRun: true, noCi: true, repositoryUrl: repoUrl }, + { + dryRun: true, + noCi: true, + repositoryUrl: repoUrl, + plugins: [...configuredPlugins, releaseFloor], + }, { cwd: path.join(clone, "apps/cli"), env: childEnv, @@ -299,7 +324,6 @@ const main = Effect.gen(function* () { return yield* Effect.sync(() => process.exit(1)); } - const expected = tag.replace(/^v/, ""); if (result.nextRelease.version !== expected) { yield* Console.error( `semantic-release computed v${result.nextRelease.version} but expected ${tag}; ` + diff --git a/apps/cli/scripts/bench-cli.integration.test.ts b/apps/cli/scripts/bench-cli.integration.test.ts index 6744dbe420..0254e056bd 100644 --- a/apps/cli/scripts/bench-cli.integration.test.ts +++ b/apps/cli/scripts/bench-cli.integration.test.ts @@ -204,13 +204,21 @@ describe("runWorkloadBench", () => { expect(report.base).toHaveLength(6); expect(report.branch).toHaveLength(6); const boots = calls.filter((call) => call.command === "boot"); - expect(boots.map((call) => call.build)).toEqual(["base", "branch", "branch", "base"]); - expect(calls.filter((call) => call.command === "halt")).toHaveLength(4); - // One warmup plus five measured launches share the first block's stack. + expect(boots.map((call) => call.build)).toEqual([ + "base", + "branch", + "branch", + "base", + "base", + "branch", + ]); + expect(calls.filter((call) => call.command === "halt")).toHaveLength(6); + expect(report.base.map((sample) => sample.block)).toEqual([0, 0, 1, 1, 2, 2]); + // One warmup plus two measured launches share the first block's stack. const firstBlock = calls.filter((call) => call.home === boots[0]?.home); expect(firstBlock.map((call) => call.command)).toEqual([ "boot", - ...Array.from({ length: 6 }, () => "work"), + ...Array.from({ length: 3 }, () => "work"), "halt", ]); }), diff --git a/apps/cli/scripts/bench-cli.ts b/apps/cli/scripts/bench-cli.ts index a048619fa6..5f8cc7c8ca 100644 --- a/apps/cli/scripts/bench-cli.ts +++ b/apps/cli/scripts/bench-cli.ts @@ -463,11 +463,13 @@ interface LaunchOutcome { /** Planned units of work for one build: a run-scoped unit holds one launch, a block-scoped one many. */ export interface PlannedUnit { readonly build: Build; + /** Block index for block-scoped units. */ + readonly block?: number; readonly launches: ReadonlyArray<"warmup" | "timed">; } /** Measured launches per block for `block`-scoped workloads. */ -export const BLOCK_SIZE = 5; +export const BLOCK_SIZE = 2; /** * Units in execution order. `run` scope reuses {@link buildRunPlan}, one launch per unit. @@ -495,7 +497,7 @@ export function buildUnitPlan( ...Array.from({ length: measured }, () => "timed" as const), ]; const order: ReadonlyArray = block % 2 === 0 ? ["base", "branch"] : ["branch", "base"]; - for (const build of order) units.push({ build, launches }); + for (const build of order) units.push({ build, block, launches }); } return units; } @@ -644,7 +646,7 @@ export const runWorkloadBench = ( const plan = buildUnitPlan(runs, warmup, scope); const started = yield* Clock.currentTimeNanos; - const timed: Record> = { + const timed: Record> = { base: [], branch: [], }; @@ -668,10 +670,14 @@ export const runWorkloadBench = ( (options.cpuProf && build === "branch" && !alreadyProfiled), ); - const record = (build: Build, outcomes: ReadonlyArray) => { + const record = (build: Build, outcomes: ReadonlyArray, block?: number) => { for (const outcome of outcomes) { if (outcome.kind === "timed") { - timed[build].push({ wallMs: outcome.wallMs, exitCode: outcome.exitCode }); + timed[build].push({ + wallMs: outcome.wallMs, + exitCode: outcome.exitCode, + ...(block === undefined ? {} : { block }), + }); } else if (outcome.kind === "traced") { traced[build].push({ exitCode: outcome.exitCode, trace: outcome.trace }); } @@ -728,6 +734,7 @@ export const runWorkloadBench = ( teardownFailures, onProfile, }), + planned.block, ); } diff --git a/apps/cli/scripts/bench-cli.unit.test.ts b/apps/cli/scripts/bench-cli.unit.test.ts index 15f8b80855..f577d8aa74 100644 --- a/apps/cli/scripts/bench-cli.unit.test.ts +++ b/apps/cli/scripts/bench-cli.unit.test.ts @@ -137,6 +137,21 @@ describe("buildUnitPlan", () => { }); }); +describe("buildUnitPlan block sizing", () => { + it("gives ten runs five blocks per build by default, each tagged with its block index", () => { + const plan = buildUnitPlan(10, 1, "block"); + + for (const build of ["base", "branch"] as const) { + expect(plan.filter((unit) => unit.build === build).map((unit) => unit.block)).toEqual([ + 0, 1, 2, 3, 4, + ]); + } + expect( + plan.every((unit) => unit.launches.filter((kind) => kind === "timed").length === 2), + ).toBe(true); + }); +}); + describe("parseBenchArgs for stack runs", () => { it("lets explicit --runs and --warmup override the manifest, and defaults otherwise", () => { expect(parseBenchArgs(required)).toMatchObject({ diff --git a/apps/cli/scripts/bench-report.ts b/apps/cli/scripts/bench-report.ts index a866d304d2..dede4dc6a0 100644 --- a/apps/cli/scripts/bench-report.ts +++ b/apps/cli/scripts/bench-report.ts @@ -29,8 +29,13 @@ export interface TimingSummary { readonly min: number; readonly max: number; readonly p90: number; + readonly iqr: number; } +/** Interquartile range by nearest rank, so one extreme value at either end cannot move it. */ +const iqr = (values: ReadonlyArray): number => + percentile(values, 75) - percentile(values, 25); + export function summarizeTimings(values: ReadonlyArray): TimingSummary { const sorted = [...values].sort((a, b) => a - b); return { @@ -38,6 +43,7 @@ export function summarizeTimings(values: ReadonlyArray): TimingSummary { min: round(sorted[0] ?? 0), max: round(sorted[sorted.length - 1] ?? 0), p90: round(percentile(sorted, 90)), + iqr: round(iqr(sorted)), }; } @@ -289,6 +295,8 @@ const TraceSample = Schema.Struct({ const RunSample = Schema.Struct({ wallMs: Schema.Finite, + /** Block index of a launch from a block-scoped workload; absent for run-scoped launches. */ + block: Schema.optionalKey(Schema.Finite), exitCode: Schema.Finite, /** Exit code of the traced launch, recorded even when no spans could be decoded. */ tracedExitCode: Schema.optionalKey(Schema.Finite), @@ -345,27 +353,88 @@ export const encodeBenchReport = Schema.encodeEffect(BenchReportJson); export const VERDICT_MIN_PCT = 5; export const VERDICT_MIN_MS = 5; export const VERDICT_MIN_SAMPLES = 5; +/** Paired blocks needed before the block, not the launch, is the unit of evidence for a verdict. */ +export const VERDICT_MIN_BLOCKS = 3; +export const VERDICT_IQR_FACTOR = 1.5; export const PHASE_ROWS = 3; export type TimingVerdict = "slower" | "faster" | "within noise" | "not comparable"; +/** Median wall time of one block for each build, paired by block index. */ +export interface BlockPair { + readonly base: number; + readonly branch: number; +} + +const exceedsFloor = (delta: Delta): boolean => + Math.abs(delta.pct) >= VERDICT_MIN_PCT && Math.abs(delta.ms) >= VERDICT_MIN_MS; + /** - * "slower" or "faster" only when the branch median is outside the base min-max range and the - * change is at least {@link VERDICT_MIN_PCT} percent and {@link VERDICT_MIN_MS} ms. + * "slower" or "faster" only when the median change passes the {@link VERDICT_MIN_PCT} percent and + * {@link VERDICT_MIN_MS} ms floor and is larger than {@link VERDICT_IQR_FACTOR} times the wider + * build's interquartile range, so one extreme sample cannot fake a change. Run-scoped timings + * also need the branch median beyond the base range; with at least {@link VERDICT_MIN_BLOCKS} + * paired blocks the spread test runs on block medians instead, and a strict majority of blocks + * must pass the floor in the same direction. */ export function timingVerdict( base: TimingSummary, branch: TimingSummary, - context: { readonly comparable: boolean; readonly samples: number }, + context: { + readonly comparable: boolean; + readonly samples: number; + readonly blocks?: ReadonlyArray; + }, ): TimingVerdict { if (!context.comparable || context.samples < VERDICT_MIN_SAMPLES) return "not comparable"; + const blocks = context.blocks; + if (blocks !== undefined && blocks.length >= VERDICT_MIN_BLOCKS) { + const baseMedians = blocks.map((block) => block.base); + const branchMedians = blocks.map((block) => block.branch); + const reference = median(baseMedians); + const ms = median(blocks.map((block) => block.branch - block.base)); + const pct = reference === 0 ? 0 : (ms / reference) * 100; + if (!exceedsFloor({ ms: round(ms), pct: round(pct) })) return "within noise"; + const agreeing = blocks.filter((block) => { + const blockDelta = deltaOf(block.base, block.branch); + return exceedsFloor(blockDelta) && blockDelta.ms > 0 === ms > 0; + }).length; + if (agreeing * 2 <= blocks.length) return "within noise"; + if (Math.abs(ms) <= VERDICT_IQR_FACTOR * Math.max(iqr(baseMedians), iqr(branchMedians))) { + return "within noise"; + } + return ms > 0 ? "slower" : "faster"; + } const delta = deltaOf(base.median, branch.median); - if (Math.abs(delta.pct) < VERDICT_MIN_PCT || Math.abs(delta.ms) < VERDICT_MIN_MS) { + if (!exceedsFloor(delta)) return "within noise"; + const outsideRange = delta.ms > 0 ? branch.median > base.max : branch.median < base.min; + if (!outsideRange) return "within noise"; + if (Math.abs(delta.ms) <= VERDICT_IQR_FACTOR * Math.max(base.iqr, branch.iqr)) { return "within noise"; } - if (branch.median > base.max) return "slower"; - if (branch.median < base.min) return "faster"; - return "within noise"; + return delta.ms > 0 ? "slower" : "faster"; +} + +/** Pairs the per-block medians of both builds; empty unless every sample carries a block index. */ +export function pairBlocks( + base: ReadonlyArray, + branch: ReadonlyArray, +): ReadonlyArray { + const group = (samples: ReadonlyArray): Map> | undefined => { + const blocks = new Map>(); + for (const sample of samples) { + if (sample.block === undefined) return undefined; + blocks.set(sample.block, [...(blocks.get(sample.block) ?? []), sample.wallMs]); + } + return blocks; + }; + const baseBlocks = group(base); + const branchBlocks = group(branch); + if (baseBlocks === undefined || branchBlocks === undefined) return []; + return [...baseBlocks.entries()].flatMap(([block, values]) => { + const other = branchBlocks.get(block); + return other === undefined ? [] : [{ base: median(values), branch: median(other) }]; + }); } export interface ApiSummary { @@ -559,6 +628,7 @@ export function analyzeWorkload(workload: WorkloadReport, traced: boolean): Work const verdict = timingVerdict(base.timing, branch.timing, { comparable: !differ && runsCleanly(base) && runsCleanly(branch), samples: Math.min(base.samples, branch.samples), + blocks: pairBlocks(workload.base, workload.branch), }); const spans = base.spans === undefined || branch.spans === undefined diff --git a/apps/cli/scripts/bench-report.unit.test.ts b/apps/cli/scripts/bench-report.unit.test.ts index 9f41a41d98..00743cebae 100644 --- a/apps/cli/scripts/bench-report.unit.test.ts +++ b/apps/cli/scripts/bench-report.unit.test.ts @@ -13,6 +13,7 @@ import { parseWorkloadManifest, summarizeSpanRun, summarizeTimings, + pairBlocks, timingVerdict, validateWorkloads, type BenchReport, @@ -51,6 +52,7 @@ describe("summarizeTimings", () => { min: 10, max: 50, p90: 50, + iqr: 20, }); }); }); @@ -180,34 +182,116 @@ describe("classifyHttpSpans", () => { }); }); +const timings = (values: ReadonlyArray) => summarizeTimings(values); +const repeat = (count: number, value: number) => Array.from({ length: count }, () => value); + describe("timingVerdict", () => { - const base = { median: 100, min: 95, max: 108, p90: 107 }; const context = { comparable: true, samples: 10 }; it.each([ - { branch: { median: 120, min: 115, max: 125, p90: 124 }, expected: "slower" }, - { branch: { median: 80, min: 78, max: 90, p90: 88 }, expected: "faster" }, - { branch: { median: 103, min: 99, max: 110, p90: 109 }, expected: "within noise" }, - { branch: { median: 107, min: 99, max: 118, p90: 117 }, expected: "within noise" }, - ])("calls a branch median of $branch.median $expected", ({ branch, expected }) => { - expect(timingVerdict(base, branch, context)).toBe(expected); + { branch: 120, expected: "slower" }, + { branch: 80, expected: "faster" }, + { branch: 103, expected: "within noise" }, + ])("calls a steady branch of $branch ms $expected", ({ branch, expected }) => { + expect(timingVerdict(timings(repeat(10, 100)), timings(repeat(10, branch)), context)).toBe( + expected, + ); }); it("ignores a large percentage change that is under the absolute floor", () => { - expect( - timingVerdict( - { median: 10, min: 9, max: 11, p90: 11 }, - { median: 14, min: 13, max: 15, p90: 15 }, - context, - ), - ).toBe("within noise"); + expect(timingVerdict(timings(repeat(10, 10)), timings(repeat(10, 14)), context)).toBe( + "within noise", + ); }); it("declines to compare when runs are not comparable or too few", () => { - const slow = { median: 200, min: 190, max: 210, p90: 209 }; + const base = timings(repeat(10, 100)); + const slow = timings(repeat(10, 200)); expect(timingVerdict(base, slow, { comparable: false, samples: 10 })).toBe("not comparable"); expect(timingVerdict(base, slow, { comparable: true, samples: 3 })).toBe("not comparable"); }); + + it("flags slower although one branch launch was fast", () => { + const branch = timings([...repeat(9, 125), 90]); + expect(timingVerdict(timings(repeat(10, 100)), branch, context)).toBe("slower"); + }); + + it("reads a branch median inside a noisy base range as noise on real five-run data", () => { + const base = timings([328, 234, 228, 229, 237]); + const branch = timings([303, 317, 273, 268, 313]); + expect(timingVerdict(base, branch, { comparable: true, samples: 5 })).toBe("within noise"); + }); + + it("reads a change past the floor but within 1.5 times the wider spread as noise", () => { + const base = timings([88, 92, 95, 100, 100, 100, 103, 107, 110, 112]); + const branch = timings([105, 110, 112, 115, 115, 115, 118, 120, 122, 125]); + expect(base.iqr).toBe(12); + expect(branch.iqr).toBe(8); + expect(branch.median).toBeGreaterThan(base.max); + expect(timingVerdict(base, branch, context)).toBe("within noise"); + }); +}); + +describe("timingVerdict with blocks", () => { + const context = { comparable: true, samples: 10 }; + const unused = timings(repeat(10, 100)); + const pairs = (...branchMedians: ReadonlyArray) => + branchMedians.map((branch) => ({ base: 100, branch })); + + it("flags slower when four of five blocks are clearly slower and one is fast", () => { + expect( + timingVerdict(unused, unused, { ...context, blocks: pairs(125, 126, 124, 125, 80) }), + ).toBe("slower"); + }); + + it("flags faster when most blocks are clearly faster", () => { + expect(timingVerdict(unused, unused, { ...context, blocks: pairs(75, 74, 76, 75, 120) })).toBe( + "faster", + ); + }); + + it("reads mixed-sign blocks as within noise", () => { + expect(timingVerdict(unused, unused, { ...context, blocks: pairs(180, 70, 130, 75, 85) })).toBe( + "within noise", + ); + }); + + it("reads block medians with wide spread as noise", () => { + const blocks = [ + [222, 276], + [224, 252], + [227, 225], + [233, 299], + [222, 291], + ].map(([base, branch]) => ({ base: base!, branch: branch! })); + expect(timingVerdict(unused, unused, { ...context, blocks })).toBe("within noise"); + }); + + it("falls back to the run path below three blocks", () => { + const branch = timings(repeat(10, 130)); + expect(timingVerdict(unused, branch, { ...context, blocks: pairs(60, 60) })).toBe("slower"); + }); +}); + +describe("pairBlocks", () => { + const at = (block: number, wallMs: number) => ({ + wallMs, + exitCode: 0, + block, + }); + + it("pairs per-block medians of both builds by block index", () => { + expect( + pairBlocks([at(0, 10), at(0, 20), at(1, 30)], [at(0, 40), at(0, 60), at(1, 50)]), + ).toEqual([ + { base: 15, branch: 50 }, + { base: 30, branch: 50 }, + ]); + }); + + it("returns no pairs when a sample has no block index", () => { + expect(pairBlocks([{ wallMs: 10, exitCode: 0 }], [at(0, 10)])).toEqual([]); + }); }); const sample = ( diff --git a/apps/cli/scripts/compile-options.ts b/apps/cli/scripts/compile-options.ts index 93797ca83c..41c9be4d24 100644 --- a/apps/cli/scripts/compile-options.ts +++ b/apps/cli/scripts/compile-options.ts @@ -1,6 +1,6 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { stackSourceDigest } from "@supabase/stack/internal/release"; -import { Effect } from "effect"; +import { Effect, Layer } from "effect"; /** * Shared compile options keep release, local, and E2E builds exercising the shipped binary @@ -21,6 +21,8 @@ export const compileOptions = { * started from the same stack sources, whether compiled or run from source. */ export const stackReleaseDefine = () => - Effect.runPromise(stackSourceDigest.pipe(Effect.provide(BunServices.layer))).then((digest) => ({ + Effect.runPromise( + stackSourceDigest.pipe(Effect.provide(Layer.merge(BunServices.layer, BunCrypto.layerDigest))), + ).then((digest) => ({ SUPABASE_STACK_BUILD_ID: JSON.stringify(digest), })); diff --git a/apps/cli/scripts/npm-registry.ts b/apps/cli/scripts/npm-registry.ts index 3759b8858c..2f4863be96 100644 --- a/apps/cli/scripts/npm-registry.ts +++ b/apps/cli/scripts/npm-registry.ts @@ -2,7 +2,7 @@ import { Data, Effect, Ref, Schedule, Schema } from "effect"; import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/http"; const POLL_INTERVAL = "10 seconds"; -const POLL_BUDGET = "15 minutes"; +const POLL_BUDGET = "30 minutes"; const PROBE_TIMEOUT = "30 seconds"; const RETRYABLE_CLIENT_STATUSES = new Set([404, 408, 429]); diff --git a/apps/cli/scripts/npm-registry.unit.test.ts b/apps/cli/scripts/npm-registry.unit.test.ts index f2639a2822..56923cd397 100644 --- a/apps/cli/scripts/npm-registry.unit.test.ts +++ b/apps/cli/scripts/npm-registry.unit.test.ts @@ -108,7 +108,7 @@ describe("waitUntilInstallable", () => { onInstallable: (pkg) => Effect.sync(() => void installable.push(pkg.name)), }).pipe(Effect.flip, Effect.forkChild({ startImmediately: true })); - yield* TestClock.adjust("14 minutes"); + yield* TestClock.adjust("29 minutes"); expect(fiber.pollUnsafe()).toBeUndefined(); yield* TestClock.adjust("2 minutes"); diff --git a/apps/cli/scripts/published-schemas.ts b/apps/cli/scripts/published-schemas.ts index 31f2ca2073..3f190b008f 100644 --- a/apps/cli/scripts/published-schemas.ts +++ b/apps/cli/scripts/published-schemas.ts @@ -1,3 +1,4 @@ +import { withoutSafeIntegerBounds } from "@supabase/config/internal"; import { HOSTED_LOCK_SCHEMA_URL, Lock, LOCK_SCHEMA_URL } from "@supabase/stack/effect"; import { JsonSchema, Predicate, Schema } from "effect"; @@ -60,8 +61,13 @@ const acceptHostedSchemaUrl = (schema: JsonSchema.JsonSchema): JsonSchema.JsonSc /** The draft-07 JSON Schema of `supabase/supabase.lock`, rendered from the stack's `Lock` schema. */ export const lockSchemaDocument = () => { + const generated = Schema.toJsonSchemaDocument(Lock); const { schema, definitions } = dedupeDefinitions( - JsonSchema.toDocumentDraft07(Schema.toJsonSchemaDocument(Lock)), + JsonSchema.toDocumentDraft07({ + ...generated, + schema: withoutSafeIntegerBounds(generated.schema), + definitions: withoutSafeIntegerBounds(generated.definitions), + }), ); return { $schema: JsonSchema.META_SCHEMA_URI_DRAFT_07, diff --git a/apps/cli/scripts/release-channels.ts b/apps/cli/scripts/release-channels.ts index bf3746c553..b3c31384e3 100644 --- a/apps/cli/scripts/release-channels.ts +++ b/apps/cli/scripts/release-channels.ts @@ -1,3 +1,5 @@ +import { semver } from "bun"; + const MAINTENANCE_TAG = /^v[1-9]\d*\.stable$/; const PUBLISHABLE_TAGS = new Set(["latest", "alpha", "beta", "next"]); @@ -38,3 +40,31 @@ export function channelForTag(tag: string): "beta" | "next" | "alpha" | "latest" if (tag.includes("-alpha.")) return "alpha"; return "latest"; } + +/** Whether `tag` is a CLI release tag in `channel`'s history: stable tags always, prereleases only on their own channel. */ +export function isInChannelHistory( + tag: string, + channel: ReturnType, +): boolean { + return /^v\d/.test(tag) && (!tag.includes("-") || channelForTag(tag) === channel); +} + +function versionBase(version: string): readonly [number, number, number] | undefined { + const match = /^(\d+)\.(\d+)\.(\d+)/.exec(version); + return match ? [Number(match[1]), Number(match[2]), Number(match[3])] : undefined; +} + +/** The smallest release type that takes `lastVersion` to `version`, or null unless `version` is later by SemVer precedence. */ +export function releaseTypeBetween( + lastVersion: string | undefined, + version: string, +): "major" | "minor" | "patch" | null { + if (lastVersion === undefined || semver.order(version, lastVersion) !== 1) return null; + const last = versionBase(lastVersion); + const next = versionBase(version); + if (!last || !next) return null; + + if (next[0] !== last[0]) return "major"; + if (next[1] !== last[1]) return "minor"; + return "patch"; +} diff --git a/apps/cli/scripts/release-channels.unit.test.ts b/apps/cli/scripts/release-channels.unit.test.ts index 5a157f5517..439c21c027 100644 --- a/apps/cli/scripts/release-channels.unit.test.ts +++ b/apps/cli/scripts/release-channels.unit.test.ts @@ -3,8 +3,10 @@ import { channelForTag, homebrewClassName, homebrewConflictsLine, + isInChannelHistory, isPublishableNpmTag, releaseBranchForTag, + releaseTypeBetween, } from "./release-channels.ts"; describe("isPublishableNpmTag", () => { @@ -72,3 +74,45 @@ describe("channelForTag", () => { expect(channelForTag(tag)).toBe(expected); }); }); + +describe("isInChannelHistory", () => { + test.each([ + ["v3.1.0-beta.1", "beta", true], + ["v3.0.0", "beta", true], + ["v3.1.0-next.1", "beta", false], + ["v2.99.0-alpha.1", "beta", false], + ["v3.1.0-next.1", "next", true], + ["v3.0.0", "next", true], + ["v3.1.0-beta.1", "next", false], + ["v2.99.0-alpha.1", "next", false], + ["v3.0.0", "latest", true], + ["v2.151.1", "latest", true], + ["v3.1.0-beta.1", "latest", false], + ["v3.1.0-next.1", "latest", false], + ["v2.99.0-alpha.1", "latest", false], + ["config-v1.2.0", "latest", false], + ["apps/cli-go/pkg/v1.0.0", "latest", false], + ["config-v1.2.0", "beta", false], + ] as const)("%s in the %s history -> %s", (tag, channel, expected) => { + expect(isInChannelHistory(tag, channel)).toBe(expected); + }); +}); + +describe("releaseTypeBetween", () => { + test.each([ + ["2.122.0", "2.122.1", "patch"], + ["2.122.1", "2.123.0", "minor"], + ["2.123.4", "3.0.0", "major"], + ["3.1.0-beta.4", "3.1.0-beta.5", "patch"], + ["3.0.0", "3.1.0-beta.1", "minor"], + ["2.151.0", "3.0.0-beta.1", "major"], + [undefined, "3.0.0", null], + ["2.122.1", "2.122.1", null], + ["2.122.1", "2.122.0", null], + ["3.1.0", "2.151.0", null], + ["3.1.0-beta.5", "3.1.0-beta.4", null], + ["3.1.0", "3.1.0-beta.1", null], + ] as const)("%s -> %s requires %s", (lastVersion, version, expected) => { + expect(releaseTypeBetween(lastVersion, version)).toBe(expected); + }); +}); diff --git a/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md b/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md index 1dec99b0f8..11b03f843e 100644 --- a/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md +++ b/apps/cli/src/SIDE_EFFECTS_TEMPLATE.md @@ -58,10 +58,11 @@ -| Variable | Purpose | Required? | -| ----------------------- | ---------------------------------------------------- | ------------------------------------------------------- | -| `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | -| `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| Variable | Purpose | Required? | +| -------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------- | +| `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | +| `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| `SUPABASE_API_MAX_RETRIES` | Management API client retries after the first attempt (`0` = send once), not command loops | no (falls back to 5) | ## Exit Codes diff --git a/apps/cli/src/auth/command-platform-api.layer.ts b/apps/cli/src/auth/command-platform-api.layer.ts index 112f224e19..fce96774e1 100644 --- a/apps/cli/src/auth/command-platform-api.layer.ts +++ b/apps/cli/src/auth/command-platform-api.layer.ts @@ -67,6 +67,7 @@ export const makeCommandPlatformApi = Effect.gen(function* () { }, { transformClient, + retry: { maxRetries: Option.getOrUndefined(cliSettings.apiMaxRetries) }, }, ); }).pipe(Effect.withSpan("CommandPlatformApi.build")); diff --git a/apps/cli/src/auth/command-platform-api.layer.unit.test.ts b/apps/cli/src/auth/command-platform-api.layer.unit.test.ts index ac98a9add2..944256b937 100644 --- a/apps/cli/src/auth/command-platform-api.layer.unit.test.ts +++ b/apps/cli/src/auth/command-platform-api.layer.unit.test.ts @@ -38,6 +38,7 @@ function mockCliSettings(opts: { userAgent?: string; profile?: string; projectHost?: string; + apiMaxRetries?: number; }) { return Layer.succeed(CommandSettings, { githubToken: Option.none(), @@ -55,6 +56,7 @@ function mockCliSettings(opts: { workdir: "/tmp", explicitWorkdir: false, userAgent: opts.userAgent ?? "SupabaseCLI/0.0.0-dev", + apiMaxRetries: Option.fromUndefinedOr(opts.apiMaxRetries), }); } @@ -184,7 +186,7 @@ function withBaseDeps( ); } -function captureRequests(responseHeaders: Record = {}) { +function captureRequests(responseHeaders: Record = {}, status = 200) { const requests: Array<{ url: string; headers: Readonly>; @@ -195,7 +197,7 @@ function captureRequests(responseHeaders: Record = {}) { HttpClientResponse.fromWeb( request, new Response(JSON.stringify([]), { - status: 200, + status, headers: { "content-type": "application/json", ...responseHeaders }, }), ), @@ -306,6 +308,39 @@ describe("commandPlatformApiLayer", () => { }).pipe(Effect.provide(layer)); }); + it.effect("sends a failing request once when SUPABASE_API_MAX_RETRIES is 0", () => { + // `Retry-After: 0` keeps any retry from waiting on the test clock. + const http = captureRequests({ "Retry-After": "0" }, 500); + const layer = commandPlatformApiLayer.pipe( + Layer.provide(mockCliSettings({ accessToken: VALID_TOKEN, apiMaxRetries: 0 })), + Layer.provide(mockCredentials(Option.none())), + Layer.provide(http.layer), + withBaseDeps(), + ); + return Effect.gen(function* () { + const api = yield* CommandPlatformApi; + const exit = yield* Effect.exit(api.v1.listAllProjects()); + expect(Exit.isFailure(exit)).toBe(true); + expect(http.requests).toHaveLength(1); + }).pipe(Effect.provide(layer)); + }); + + it.effect("keeps the client's default retries when SUPABASE_API_MAX_RETRIES is unset", () => { + const http = captureRequests({ "Retry-After": "0" }, 500); + const layer = commandPlatformApiLayer.pipe( + Layer.provide(mockCliSettings({ accessToken: VALID_TOKEN })), + Layer.provide(mockCredentials(Option.none())), + Layer.provide(http.layer), + withBaseDeps(), + ); + return Effect.gen(function* () { + const api = yield* CommandPlatformApi; + const exit = yield* Effect.exit(api.v1.listAllProjects()); + expect(Exit.isFailure(exit)).toBe(true); + expect(http.requests).toHaveLength(6); + }).pipe(Effect.provide(layer)); + }); + it.effect("targets the configured apiUrl rather than SUPABASE_API_URL env", () => { const http = captureRequests(); const layer = commandPlatformApiLayer.pipe( diff --git a/apps/cli/src/cli/complete.ts b/apps/cli/src/cli/complete.ts index 190c764622..ff8ba4b3b2 100644 --- a/apps/cli/src/cli/complete.ts +++ b/apps/cli/src/cli/complete.ts @@ -1,14 +1,12 @@ import { BunCrypto, BunServices } from "@effect/platform-bun"; -import { Cause, Clock, Crypto, DateTime, Effect, Layer, Option } from "effect"; +import { Cause, Clock, DateTime, Effect, Layer, Option } from "effect"; +import { SecureRandom } from "effect/crypto"; import { FetchHttpClient } from "effect/http"; import { GlobalFlag } from "effect/cli"; import type { Command, Param, Primitive } from "effect/cli"; import type { PlatformError } from "effect/PlatformError"; import process from "node:process"; -import { - QUERY_OUTPUT_FORMATS, - RESOURCE_OUTPUT_FORMATS, -} from "../command-internal/output-formats.ts"; +import { outputFormatsForCommand } from "../command-internal/output-formats.ts"; import { unwrapParam } from "../command-internal/param-introspection.ts"; import { REMOTE_FLAG_NAME } from "../command-internal/target/remote-vocabulary.ts"; import { isValidBase0Int64, parseUintBase0 } from "../shared/config/parse-uint.ts"; @@ -650,18 +648,6 @@ function isValidCsvFlagValue(value: string): boolean { } } -/** - * `--output` is modeled as one global flag whose `choiceKeys` is the union of two distinct - * enums (`env|pretty|json|toml|yaml` everywhere else, `json|table|csv` for `db query`), so - * `flag.choiceKeys` alone can't tell which applies at the resolved command. This restores - * per-command validation. - */ -function outputFlagChoiceKeys(matchedPath: ReadonlyArray): ReadonlyArray { - return matchedPath.length === 2 && matchedPath[0] === "db" && matchedPath[1] === "query" - ? QUERY_OUTPUT_FORMATS - : RESOURCE_OUTPUT_FORMATS; -} - function remoteTakesWord( remoteLeaf: RemoteArgvLeaf | undefined, trimmedArgs: ReadonlyArray, @@ -709,7 +695,7 @@ function isValidFlagValue( return parseBoolLiteral(value) !== undefined; case "Choice": if (flag.name === "output") { - return outputFlagChoiceKeys(matchedPath).includes(value); + return outputFormatsForCommand(matchedPath).includes(value); } return flag.choiceKeys !== undefined && flag.choiceKeys.includes(value); case "Int": @@ -1122,10 +1108,10 @@ export function formatCompletionResponse( export function captureCompleteTelemetryEffect( exitCode: number, durationMs: number, -): Effect.Effect { +): Effect.Effect { return Effect.gen(function* () { - const crypto = yield* Crypto.Crypto; - const commandRunId = yield* crypto.randomUUIDv4; + const crypto = yield* SecureRandom.SecureRandom; + const commandRunId = yield* crypto.uuidV4(); const analytics = yield* Analytics; yield* analytics .capture(EventCommandExecuted, { @@ -1162,7 +1148,7 @@ const completeAnalyticsLayer = (mode: ProjectStateMode) => Layer.provide(FetchHttpClient.layer), Layer.provide(cliConfigProviderLayer), Layer.provide(BunServices.layer), - Layer.merge(BunCrypto.layer), + Layer.provideMerge(BunCrypto.layerSecureRandom), ); /** diff --git a/apps/cli/src/cli/complete.unit.test.ts b/apps/cli/src/cli/complete.unit.test.ts index e8cc3f4505..aeb903f191 100644 --- a/apps/cli/src/cli/complete.unit.test.ts +++ b/apps/cli/src/cli/complete.unit.test.ts @@ -529,6 +529,25 @@ describe("respondToComplete", () => { }); describe("--output's choice values are validated per-command, not the widened global union (CLI-1965 review)", () => { + it.each([ + { path: ["logs"], format: "table" }, + { path: ["logs"], format: "csv" }, + { path: ["logs", "sources"], format: "table" }, + { path: ["logs", "sources"], format: "csv" }, + { path: ["logs"], format: "env" }, + ])("rejects $path -o $format with a valid output-format control", ({ path, format }) => { + const control = respondToComplete(rootCommand, [ + "__complete", + ...path, + "--output-format", + "json", + "--rem", + ]); + expect(control?.candidates.map((candidate) => candidate.name)).toContain("--remote="); + const result = respondToComplete(rootCommand, ["__complete", ...path, "-o", format, "--rem"]); + expect(result).toEqual({ candidates: [], directive: CompletionDirective.Default }); + }); + it("rejects db query's own local values (table/csv) everywhere else", () => { const result = respondToComplete(rootCommand, ["__complete", "--output", "table", ""]); expect(result).toEqual({ candidates: [], directive: CompletionDirective.Default }); diff --git a/apps/cli/src/cli/root.ts b/apps/cli/src/cli/root.ts index 93baf60356..9e41bc8ff5 100644 --- a/apps/cli/src/cli/root.ts +++ b/apps/cli/src/cli/root.ts @@ -31,6 +31,7 @@ import { initCommand } from "../commands/init/init.command.ts"; import { inspectCommand } from "../commands/inspect/inspect.command.ts"; import { issueCommand } from "../commands/issue/issue.command.ts"; import { linkCommand } from "../commands/link/link.command.ts"; +import { logsCommandFor } from "../commands/logs/logs.command.ts"; import { loginCommand } from "../commands/login/login.command.ts"; import { logoutCommand } from "../commands/logout/logout.command.ts"; import { migrationCommand } from "../commands/migration/migration.command.ts"; @@ -179,6 +180,7 @@ export const rootCommandForFeatures = (options: RootCommandFeatures = {}) => linkCommand, loginCommand, logoutCommand, + logsCommandFor(), migrationCommand, networkBansCommand, networkRestrictionsCommand, diff --git a/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts b/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts index 5cf56eaa7d..b26708d63f 100644 --- a/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts +++ b/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, FileSystem, Layer, Ref } from "effect"; @@ -23,7 +23,7 @@ const processControl = Layer.succeed(ProcessControl, { getExitCode: Effect.undefined, }); -const base = Layer.mergeAll(BunServices.layer, processControl); +const base = Layer.mergeAll(BunServices.layer, BunCrypto.layer, processControl); const docker = dockerRunLayer.pipe(Layer.provide(base)); const client = bundledPostgresClientLayer.pipe( Layer.provide(Layer.mergeAll(base, docker, mockOutput().layer)), diff --git a/apps/cli/src/command-internal/bundled-postgres-client.ts b/apps/cli/src/command-internal/bundled-postgres-client.ts index e30c5ae3f7..10215da982 100644 --- a/apps/cli/src/command-internal/bundled-postgres-client.ts +++ b/apps/cli/src/command-internal/bundled-postgres-client.ts @@ -1,7 +1,6 @@ import { Context, Config, - Crypto, Effect, FileSystem, Layer, @@ -10,6 +9,7 @@ import { PlatformError, Stream, } from "effect"; +import { Digest, SecureRandom } from "effect/crypto"; import { FetchHttpClient } from "effect/http"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; import { @@ -216,7 +216,8 @@ export const bundledPostgresClientLayer: Layer.Layer< never, | FileSystem.FileSystem | Path.Path - | Crypto.Crypto + | SecureRandom.SecureRandom + | Digest.Digest | ChildProcessSpawner.ChildProcessSpawner | DockerRun | RuntimeInfo @@ -228,7 +229,8 @@ export const bundledPostgresClientLayer: Layer.Layer< Effect.gen(function* () { const fileSystem = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const crypto = yield* Crypto.Crypto; + const crypto = yield* SecureRandom.SecureRandom; + const digest = yield* Digest.Digest; const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; const docker = yield* DockerRun; const runtimeInfo = yield* RuntimeInfo; @@ -258,7 +260,8 @@ export const bundledPostgresClientLayer: Layer.Layer< Effect.provide(FetchHttpClient.layer), Effect.provideService(FileSystem.FileSystem, fileSystem), Effect.provideService(Path.Path, path), - Effect.provideService(Crypto.Crypto, crypto), + Effect.provideService(SecureRandom.SecureRandom, crypto), + Effect.provideService(Digest.Digest, digest), Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), Effect.mapError(mapArtifactError), ); diff --git a/apps/cli/src/command-internal/colors.ts b/apps/cli/src/command-internal/colors.ts index 3eed599f46..f5c13f1c88 100644 --- a/apps/cli/src/command-internal/colors.ts +++ b/apps/cli/src/command-internal/colors.ts @@ -93,3 +93,7 @@ export function blue(text: string, stream: ColorStream = process.stderr): string export function dim(text: string, stream: ColorStream = process.stderr): string { return supportsColor(stream) ? styleText("dim", text, { validateStream: false }) : text; } + +export function inverse(text: string, stream: ColorStream = process.stderr): string { + return supportsColor(stream) ? styleText("inverse", text, { validateStream: false }) : text; +} diff --git a/apps/cli/src/command-internal/colors.unit.test.ts b/apps/cli/src/command-internal/colors.unit.test.ts index 44c7550096..502c0c744d 100644 --- a/apps/cli/src/command-internal/colors.unit.test.ts +++ b/apps/cli/src/command-internal/colors.unit.test.ts @@ -1,7 +1,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import type { ColorStream } from "./colors.ts"; -import { aqua, bold, green, red, yellow } from "./colors.ts"; +import { aqua, bold, green, red, yellow, inverse } from "./colors.ts"; // Bun's `util.styleText` ignores `validateStream`, so `colors.ts` implements the colour gate // itself. These tests pin that gate deterministically with fake streams and stubbed env vars. @@ -25,6 +25,7 @@ afterEach(() => { describe("colors TTY gating (termenv parity)", () => { it("styles on a colour-capable TTY", () => { expect(aqua("supabase", colorTty)).toBe("\u001b[36msupabase\u001b[39m"); + expect(inverse("selected", colorTty)).toBe("\u001b[7mselected\u001b[27m"); expect(bold("text", colorTty)).toBe("\u001b[1mtext\u001b[22m"); expect(yellow("warning", colorTty)).toBe("\u001b[33mwarning\u001b[39m"); expect(red("error", colorTty)).toBe("\u001b[31merror\u001b[39m"); @@ -33,6 +34,7 @@ describe("colors TTY gating (termenv parity)", () => { it("renders plain on a piped stream (no hasColors), like lipgloss's Ascii profile", () => { expect(aqua("supabase", piped)).toBe("supabase"); + expect(inverse("selected", piped)).toBe("selected"); expect(bold("text", piped)).toBe("text"); expect(yellow("warning", piped)).toBe("warning"); expect(red("error", piped)).toBe("error"); @@ -47,6 +49,7 @@ describe("colors TTY gating (termenv parity)", () => { vi.stubEnv("NO_COLOR", "1"); vi.stubEnv("CLICOLOR_FORCE", "1"); expect(aqua("supabase", colorTty)).toBe("supabase"); + expect(inverse("selected", colorTty)).toBe("selected"); }); it("CLICOLOR=0 disables colour on a capable TTY", () => { diff --git a/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts b/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts index 7cdd957a98..527cd603aa 100644 --- a/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts +++ b/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts @@ -73,20 +73,39 @@ describe("command permission mapping completeness", () => { for (const { name, tree } of trees) { describe(name, () => { - const leaves = [...tree].filter(([, node]) => node.isLeaf); + const executableCommands = [...tree].filter(([, node]) => node.isExecutable); - it("declares permissions on every leaf", () => { - for (const [path, node] of leaves) { + it("requires hosted log access only for the executable viewer, not the source catalog", () => { + const viewer = tree.get("logs"); + const sources = tree.get("logs sources"); + expect(viewer?.isLeaf).toBe(false); + expect(viewer?.isExecutable).toBe(true); + const viewerPermissions = viewer === undefined ? undefined : declarationOf(viewer); + const sourcePermissions = sources === undefined ? undefined : declarationOf(sources); + expect(viewerPermissions?.status).toBe("mapped"); + expect(sourcePermissions?.status).toBe("mapped"); + if (viewerPermissions?.status !== "mapped" || sourcePermissions?.status !== "mapped") + throw new Error("Log permission declarations are missing"); + expect(viewerPermissions.operations.map((entry) => entry.operationId)).toContain( + "v1-get-project-logs", + ); + expect(sourcePermissions.operations.map((entry) => entry.operationId)).not.toContain( + "v1-get-project-logs", + ); + }); + + it("declares permissions on every executable command", () => { + for (const [path, node] of executableCommands) { expect(declarationOf(node), `"${path}" declares no permissions`).toBeDefined(); } }); - it("declares permissions only on leaf commands", () => { + it("declares permissions only on executable commands", () => { for (const [path, node] of tree) { - if (node.isLeaf) continue; + if (node.isExecutable) continue; expect( declarationOf(node), - `"${path}" is a command group; declare permissions on its leaves instead`, + `"${path}" is a command group without a handler; declare permissions on its executable commands instead`, ).toBeUndefined(); } }); @@ -94,7 +113,7 @@ describe("command permission mapping completeness", () => { it("classifies every flag a mapped leaf accepts, and only references flags that exist", () => { const globalFlags = new Set(GLOBAL_NO_API_EFFECT_FLAGS); - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const permissions = declarationOf(node); if (permissions?.status !== "mapped") continue; @@ -195,7 +214,7 @@ describe("command permission mapping completeness", () => { }); it("references only operationIds that exist in the bundled spec and declare FGA permissions", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const permissions = declarationOf(node); if (permissions?.status !== "mapped") continue; for (const entry of permissions.operations) { @@ -213,7 +232,7 @@ describe("command permission mapping completeness", () => { }); it("composes branch resolution on every leaf that accepts --remote, except the branches that act on a parent project", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const permissions = declarationOf(node); if (permissions?.status !== "mapped") continue; if (!node.flags.has("remote") || path.startsWith("branches ")) continue; @@ -226,7 +245,7 @@ describe("command permission mapping completeness", () => { }); it("only marks a leaf unmapped when it's on the known go-delegation allowlist", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const permissions = declarationOf(node); if (permissions?.status !== "unmapped") continue; expect( @@ -242,7 +261,7 @@ describe("command permission mapping completeness", () => { for (const path of GO_DELEGATED_PATHS) { const declared = trees .flatMap(({ tree }) => [tree.get(path)]) - .filter((node) => node?.isLeaf === true) + .filter((node) => node?.isExecutable === true) .map((node) => (node === undefined ? undefined : declarationOf(node))); expect( declared.length, diff --git a/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts b/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts index a506950c1f..a8b801f817 100644 --- a/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts +++ b/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts @@ -39,7 +39,7 @@ describe("permission help section across the command tree", () => { describe(name, () => { const tree = commandTreeFor(variant); const rootFlags = permissionFlagsOf(userGlobalFlagParams(rootCommandForFeatures(variant))); - const leaves = [...tree].filter(([, node]) => node.isLeaf); + const executableCommands = [...tree].filter(([, node]) => node.isExecutable); // Help lists only visible flags of a parent group, but the command's own flags come with // their hidden ones from what `withPermissions` recorded. @@ -71,7 +71,7 @@ describe("permission help section across the command tree", () => { it.effect("gives a flag a default only when parsing no arguments yields that value", () => Effect.gen(function* () { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared?.status !== "mapped") continue; const params = paramsFor(path); @@ -106,7 +106,7 @@ describe("permission help section across the command tree", () => { ); it("never tests the presence of a text flag that has a default", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared?.status !== "mapped") continue; const catalog = catalogFor(path); @@ -124,7 +124,7 @@ describe("permission help section across the command tree", () => { it.effect("never turns on by default a switch a condition tests", () => Effect.gen(function* () { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared?.status !== "mapped") continue; const params = paramsFor(path); @@ -148,7 +148,7 @@ describe("permission help section across the command tree", () => { ); it("records every flag a command accepts when it declares its permissions", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const recorded = Context.getOption(node.command.annotations, CommandFlagsAnnotation); const expected = permissionFlagsOf(ownParams(node.command)).map((flag) => flag.name); expect(Option.isSome(recorded), `"${path}" records no flags`).toBe(true); @@ -164,7 +164,7 @@ describe("permission help section across the command tree", () => { // Help sees a parent group's flags only as visible names, so a condition cannot depend on // a parent's hidden flag or on its choice values. it("only conditions on flags a parent group shares when help can describe them", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared?.status !== "mapped") continue; const parts = path.split(" "); @@ -193,7 +193,7 @@ describe("permission help section across the command tree", () => { }); it("renders a section for every command that calls an API operation", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared === undefined) continue; const section = resolvePermissionSection(declared, catalogFor(path)); @@ -216,7 +216,7 @@ describe("permission help section across the command tree", () => { }); it("names only flags the command shows in its help", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared === undefined) continue; const catalog = catalogFor(path); @@ -233,7 +233,7 @@ describe("permission help section across the command tree", () => { }); it("prints each heading once, and never lists a row as both required and optional", () => { - for (const [path, node] of leaves) { + for (const [path, node] of executableCommands) { const declared = Option.getOrUndefined(readPermissions(node.command)); if (declared === undefined) continue; const section = resolvePermissionSection(declared, catalogFor(path)); diff --git a/apps/cli/src/command-internal/config-advice.ts b/apps/cli/src/command-internal/config-advice.ts new file mode 100644 index 0000000000..c512c1ac5b --- /dev/null +++ b/apps/cli/src/command-internal/config-advice.ts @@ -0,0 +1,63 @@ +import type { ConfigFormat } from "@supabase/config/effect"; +import { cliConfigFormatForPath } from "@supabase/config/internal"; +import type { Path } from "effect"; + +/** The project config file a message points at: its format and workdir-relative path. */ +export interface ConfigFileRef { + readonly format: ConfigFormat; + readonly path: string; +} + +/** Where `supabase init` writes the config, so the file a project without one will get. */ +export const DEFAULT_CONFIG_FILE: ConfigFileRef = { + format: "toml", + path: "supabase/config.toml", +}; + +/** A config file's workdir-relative path as messages show it, with `/` separators on every platform. */ +export const configDisplayPath = (path: Path.Path, workdir: string, filePath: string): string => + path.relative(workdir, filePath).replaceAll("\\", "/"); + +/** The config file at an absolute `configPath`; `undefined` reads as no file. */ +export const configFileRefFromPath = ( + path: Path.Path, + workdir: string, + configPath: string | undefined, +): ConfigFileRef => + configPath === undefined + ? DEFAULT_CONFIG_FILE + : { + format: cliConfigFormatForPath(configPath), + path: configDisplayPath(path, workdir, configPath), + }; + +export const configFileRefOf = ( + path: Path.Path, + workdir: string, + resolvedConfig: { + readonly hasConfigFile: boolean; + readonly loaded: { readonly path: string }; + }, +): ConfigFileRef => + configFileRefFromPath( + path, + workdir, + resolvedConfig.hasConfigFile ? resolvedConfig.loaded.path : undefined, + ); + +/** A setting as advice reads it: `[section] key = value` in TOML, `` `section.key` to `value` `` in JSON. */ +export const configSetting = ( + format: ConfigFormat, + section: string, + key: string, + value: string, +): string => + format === "json" ? `\`${section}.${key}\` to \`${value}\`` : `[${section}] ${key} = ${value}`; + +/** A setting's key as advice names it: `[section].key` in TOML, `section.key` in JSON. */ +export const configKey = (format: ConfigFormat, section: string, key: string): string => + format === "json" ? `${section}.${key}` : `[${section}].${key}`; + +/** The config file name without its directory, as "not declared" notes name it. */ +export const configFileName = (configPath: string): string => + configPath.split(/[\\/]/).pop() ?? configPath; diff --git a/apps/cli/src/command-internal/config-advice.unit.test.ts b/apps/cli/src/command-internal/config-advice.unit.test.ts new file mode 100644 index 0000000000..da673736d0 --- /dev/null +++ b/apps/cli/src/command-internal/config-advice.unit.test.ts @@ -0,0 +1,57 @@ +import { Path } from "effect"; +import { BunPath } from "@effect/platform-bun"; +import { describe, expect, it } from "vitest"; +import { Effect } from "effect"; +import { + configFileName, + configFileRefFromPath, + configKey, + configSetting, + DEFAULT_CONFIG_FILE, +} from "./config-advice.ts"; +import { schemaPathsTransitionWarning } from "./diff-engine.ts"; +import { enableLocalWebhooksSuggestion } from "./pg-net-guidance.ts"; + +const json = { format: "json", path: "supabase/config.json" } as const; + +describe("config advice", () => { + it("renders a setting as TOML section syntax or a JSON dotted key", () => { + expect(configSetting("toml", "storage", "enabled", "true")).toBe("[storage] enabled = true"); + expect(configSetting("json", "storage", "enabled", "true")).toBe("`storage.enabled` to `true`"); + expect(configKey("toml", "db", "password")).toBe("[db].password"); + expect(configKey("json", "db", "password")).toBe("db.password"); + }); + + it("maps a config path to its format and workdir-relative display path", () => + Effect.gen(function* () { + const path = yield* Path.Path; + expect(configFileRefFromPath(path, "/work", "/work/supabase/config.jsonc")).toEqual({ + format: "json", + path: "supabase/config.jsonc", + }); + expect(configFileRefFromPath(path, "/work", undefined)).toEqual(DEFAULT_CONFIG_FILE); + expect(configFileName("supabase/config.json")).toBe("config.json"); + }).pipe(Effect.provide(BunPath.layer), Effect.runSync)); + + it("names the config path with forward slashes on Windows", () => + Effect.gen(function* () { + const path = yield* Path.Path; + expect(configFileRefFromPath(path, "C:\\work", "C:\\work\\supabase\\config.toml")).toEqual({ + format: "toml", + path: "supabase/config.toml", + }); + }).pipe(Effect.provide(BunPath.layerWin32), Effect.runSync)); + + it("keeps TOML guidance text and gives JSON projects a JSON snippet", () => { + expect(enableLocalWebhooksSuggestion()).toBe( + "Add the following to supabase/config.toml and retry:\n\n[experimental.webhooks]\nenabled = true", + ); + expect(enableLocalWebhooksSuggestion(json)).toContain( + '{\n "experimental": {\n "webhooks": {\n "enabled": true', + ); + expect(schemaPathsTransitionWarning()).toContain("[db.migrations].schema_paths"); + expect(schemaPathsTransitionWarning(json)).toContain( + "`experimental.pgdelta.enabled` to `false`", + ); + }); +}); diff --git a/apps/cli/src/command-internal/config-file-ref.ts b/apps/cli/src/command-internal/config-file-ref.ts new file mode 100644 index 0000000000..8401ea95be --- /dev/null +++ b/apps/cli/src/command-internal/config-file-ref.ts @@ -0,0 +1,23 @@ +import { findCliProjectPaths } from "@supabase/config/effect"; +import { Effect, Option, Path } from "effect"; +import { CliConfigValues } from "../config/cli-config-values.service.ts"; +import { configFileRefFromPath, configFileRefOf, DEFAULT_CONFIG_FILE } from "./config-advice.ts"; + +/** The workdir's config file, from the memoised resolved config; a failed load reads as no file. */ +export const loadConfigFileRef = Effect.fnUntraced(function* (workdir: string) { + const path = yield* Path.Path; + const values = yield* CliConfigValues; + return yield* values.load({ workdir, projectRef: Option.none() }).pipe( + Effect.map((resolvedConfig) => configFileRefOf(path, workdir, resolvedConfig)), + Effect.orElseSucceed(() => DEFAULT_CONFIG_FILE), + ); +}); + +/** The workdir's config file found on disk (no resolved-config load); no file reads as the default. */ +export const findConfigFileRef = Effect.fnUntraced(function* (workdir: string) { + const path = yield* Path.Path; + const found = yield* findCliProjectPaths(workdir, { search: false }).pipe( + Effect.orElseSucceed(() => null), + ); + return configFileRefFromPath(path, workdir, found?.configPath); +}); diff --git a/apps/cli/src/command-internal/config-pull-run.ts b/apps/cli/src/command-internal/config-pull-run.ts index 6588290991..b5c934d23e 100644 --- a/apps/cli/src/command-internal/config-pull-run.ts +++ b/apps/cli/src/command-internal/config-pull-run.ts @@ -35,6 +35,7 @@ import { configIsRecord, configPathKey, } from "../commands/config/config.paths.ts"; +import { configFileName } from "./config-advice.ts"; import { loadDeclaredFileConfig, relativeConfigPath } from "../commands/config/config.load.ts"; import { CONFIG_CLASS_LABELS, @@ -833,10 +834,10 @@ function writtenCount(plan: ConfigPullPlan, outcome: ConfigPullOutcome): number return outcome.dryRun || outcome.declined ? 0 : plan.writes.length; } -function renderLocal(local: unknown, declared: boolean): string { +function renderLocal(local: unknown, declared: boolean, configPath: string): string { const value = configRenderValue(local, "(unset)"); return local !== undefined && !declared - ? `${value} (schema default — not declared in config.toml)` + ? `${value} (schema default — not declared in ${configFileName(configPath)})` : value; } @@ -942,7 +943,7 @@ export function renderConfigPullText( change.envVariables === undefined ? "" : ` (from env ${sanitizeInlineName(change.envVariables.join(", "))})`; - lines.push(` local: ${renderLocal(change.local, change.declared)}${env}`); + lines.push(` local: ${renderLocal(change.local, change.declared, configPath)}${env}`); lines.push(` remote: ${configRenderValue(change.remote, "(not returned)")}`); lines.push(""); } diff --git a/apps/cli/src/command-internal/config-validate.parity.unit.test.ts b/apps/cli/src/command-internal/config-validate.parity.unit.test.ts index 662b9e3d9b..0a8ea60460 100644 --- a/apps/cli/src/command-internal/config-validate.parity.unit.test.ts +++ b/apps/cli/src/command-internal/config-validate.parity.unit.test.ts @@ -1,5 +1,5 @@ import { unusedStackServices } from "../../tests/helpers/unused-stack.ts"; -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { CliConfigSchema, type CliConfig } from "@supabase/config"; import { stringify as stringifyToml } from "smol-toml"; @@ -7,7 +7,7 @@ import { Effect, Exit, FileSystem, Layer, Path, Schema } from "effect"; import { withEnvVar, mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; -import { readDbToml } from "./db-config.toml-read.ts"; +import { readDbConfig } from "./db-config.read.ts"; import { resolveStorageCredentials } from "./storage-credentials.ts"; import { resolveLocalConfigValues } from "./local-config-values.ts"; import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; @@ -17,7 +17,7 @@ const stringifyJson = Schema.encodeSync(Schema.fromJsonString(Schema.Unknown)); /** * Cross-caller parity coverage: for a table of shared misconfigurations, drives both real - * pipelines — D (`readDbToml`, Effect/raw-TOML) and L (`resolveLocalConfigValues`, + * pipelines — D (`readDbConfig`, Effect/raw-TOML) and L (`resolveLocalConfigValues`, * `@supabase/config`-decoded) — and asserts they fail with the same error-message substring, * since both route through the single `validateResolvedConfig`. A third caller, the * storage-credentials resolver (S), shares the `api.port`/`api.tls` presence branches and the @@ -25,7 +25,7 @@ const stringifyJson = Schema.encodeSync(Schema.fromJsonString(Schema.Unknown)); * that pipeline. * * D's harness replicates the `withConfig`/`read`/`failsWith` pattern from - * `db-config.toml-read.unit.test.ts` (file-local there, not exported). L's harness replicates + * `db-config.read.unit.test.ts` (file-local there, not exported). L's harness replicates * the `baseConfig`/`WORKDIR` pattern from `local-config-values.unit.test.ts`. */ @@ -37,14 +37,16 @@ const withConfig = (content: string) => yield* fs.makeDirectory(path.join(dir, "supabase"), { recursive: true }); yield* fs.writeFileString(path.join(dir, "supabase", "config.toml"), content); return dir; - }).pipe(Effect.provide(BunServices.layer)); + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])); const readD = (workdir: string) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - return yield* readDbToml(fs, path, workdir); - }).pipe(Effect.provide(Layer.merge(BunServices.layer, cliConfigValuesTestLayer))); + return yield* readDbConfig(fs, path, workdir); + }).pipe( + Effect.provide(Layer.mergeAll(BunServices.layer, BunCrypto.layer, cliConfigValuesTestLayer)), + ); /** Drives D's real pipeline and asserts the failure message contains `message`. */ function failsWithD(tomlLines: ReadonlyArray, message: string) { @@ -77,7 +79,7 @@ function failsWithL( resolveLocalConfigValues(config, "127.0.0.1", WORKDIR, document), ); expect(error.message).toContain(message); - }).pipe(Effect.provide(BunServices.layer)); + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])); } interface ParityScenario { @@ -344,6 +346,7 @@ describe("shared api + auth validation branches, cross-caller parity (S vs L)", unusedStackServices, cliConfigValuesTestLayer, BunServices.layer, + BunCrypto.layer, runtimeInfoLayer, mockCommandSettings({ workdir: dir }), // Local-path parity only — the remote branch (and with it the diff --git a/apps/cli/src/command-internal/config-validate.ts b/apps/cli/src/command-internal/config-validate.ts index dbec5fff8b..a525f82a81 100644 --- a/apps/cli/src/command-internal/config-validate.ts +++ b/apps/cli/src/command-internal/config-validate.ts @@ -15,7 +15,7 @@ import { parseUrl } from "./storage-url.ts"; /** * Single home for config validation, shared by the two config readers: * - * - **D** = `db-config.toml-read.ts`: raw TOML document, Effect-based, fails with + * - **D** = `db-config.read.ts`: raw TOML document, Effect-based, fails with * `DbConfigLoadError`. * - **L** = `local-config-values.ts`: decoded `@supabase/config` `CliConfig`, Effect-based, * fails with an `Error`. @@ -191,6 +191,8 @@ export interface ExperimentalInput { * `studio`/`localSmtp` entirely; both D and L omit `auth` when auth is disabled). */ export interface ConfigValidationInput { + /** The config file's name for messages that name it; defaults to `config.toml`. */ + readonly configFileName?: string; /** L only — D doesn't validate `project_id` here. */ readonly projectId?: string; /** L only — D has no `[api]` section. */ @@ -545,7 +547,7 @@ export function validateResolvedConfig(input: ConfigValidationInput): void { } if (input.analytics.gcpJwtPath.length === 0) { throw new ConfigValidateError( - "Path to GCP Service Account Key must be provided in config, relative to config.toml: analytics.gcp_jwt_path", + `Path to GCP Service Account Key must be provided in config, relative to ${input.configFileName ?? "config.toml"}: analytics.gcp_jwt_path`, ); } } @@ -609,7 +611,15 @@ export function signingKeysDecodeErrorMessage(cause: unknown): string { * project root all fail. This applies unconditionally to every caller (config validation, * `config push` content loading, `start`'s pre-Docker containment pass) — there is no opt-out. */ -export const resolveEmailTemplateContentPath = Effect.fnUntraced(function* (args: { +export const resolveEmailTemplateContentPath = ( + args: Parameters[0], +) => resolveEmailTemplatePaths(args).pipe(Effect.map((paths) => paths?.canonical)); + +/** + * {@link resolveEmailTemplateContentPath} with the configured candidate it validated, resolved + * once, which the stack reads per fetch so an in-root link may retarget. + */ +export const resolveEmailTemplatePaths = Effect.fnUntraced(function* (args: { readonly section: "template" | "notification"; readonly name: string; /** Post-env-expand; `""` = absent. */ @@ -627,12 +637,7 @@ export const resolveEmailTemplateContentPath = Effect.fnUntraced(function* (args return undefined; } const path = yield* Path.Path; - const candidate = - args.section === "notification" - ? yield* resolveNotificationContentPath(args.base, args.contentPath) - : path.isAbsolute(args.contentPath) - ? args.contentPath - : path.join(args.base, args.contentPath); + const candidate = yield* emailTemplateCandidatePath(args.section, args.base, args.contentPath); const resolvedCanonical = yield* canonicalPathForContainment(candidate); const canonicalBase = yield* canonicalPathForContainment(args.base); if (!isPathContainedInRoot(path, canonicalBase, resolvedCanonical)) { @@ -645,7 +650,19 @@ export const resolveEmailTemplateContentPath = Effect.fnUntraced(function* (args `move the file inside the project, or use a relative path that stays inside it.`, ); } - return resolvedCanonical; + return { candidate, canonical: resolvedCanonical }; +}); + +/* A non-empty `content_path` as configured, joined to `base` but not dereferenced, with the + notification fallback below. */ +const emailTemplateCandidatePath = Effect.fnUntraced(function* ( + section: "template" | "notification", + base: string, + contentPath: string, +) { + const path = yield* Path.Path; + if (section === "notification") return yield* resolveNotificationContentPath(base, contentPath); + return path.isAbsolute(contentPath) ? contentPath : path.join(base, contentPath); }); /** diff --git a/apps/cli/src/command-internal/config-validate.unit.test.ts b/apps/cli/src/command-internal/config-validate.unit.test.ts index daf5921a8f..405a13d0e3 100644 --- a/apps/cli/src/command-internal/config-validate.unit.test.ts +++ b/apps/cli/src/command-internal/config-validate.unit.test.ts @@ -97,7 +97,7 @@ describe("CLERK_DOMAIN_PATTERN", () => { }); // Direct coverage for the containment check shared by every caller (config push's -// loadAuthEmailContent, db-config.toml-read.ts, local-config-values.ts, start.handler.ts's +// loadAuthEmailContent, db-config.read.ts, local-config-values.ts, start.handler.ts's // pre-Docker pass), cheaper to pin here than to re-derive through every caller's own fixtures. layer(BunServices.layer)("resolveEmailTemplateContentPath", (it) => { const tempDir = (prefix: string) => diff --git a/apps/cli/src/command-internal/connect-errors.ts b/apps/cli/src/command-internal/connect-errors.ts index 7fd6054e1f..69a6260429 100644 --- a/apps/cli/src/command-internal/connect-errors.ts +++ b/apps/cli/src/command-internal/connect-errors.ts @@ -55,6 +55,13 @@ export function isIPv6ConnectivityError(message: string): boolean { export const SUGGEST_ENV_VAR = "Connect to your database by setting the env var correctly: SUPABASE_DB_PASSWORD"; +/** + * Hint shown when the server certificate does not verify against the CA the connection trusts, + * which on a hosted project is the bundled Supabase CA unless the user chose a trust setting. + */ +export const SUGGEST_CA_VERIFICATION = + "The server certificate did not verify against the trusted CA (the bundled Supabase CA on a hosted project). Behind a TLS-inspecting proxy, set PGSSLROOTCERT to a CA bundle that trusts it, or set PGSSLMODE=require to skip certificate verification."; + /** * Shown instead of the remote-only "Network Restrictions" hint when `ctx.isLocal` is true — a * refused `--local` connection (Docker/Postgres not running) needs a different fix than an @@ -70,13 +77,16 @@ export function localStackSuggestion(linkedCheckout: boolean | undefined): strin : SUGGEST_LOCAL_STACK; } +export const SUGGEST_NETWORK_RESTRICTIONS = + "Make sure your local IP is allowed in Network Restrictions and Network Bans."; + /** * Remote-only "Network Restrictions" hint, shown for a connection refused or blocked by IP * allow-listing. Shared by both the always-remote `Address not in tenant allow_list` branch * and the non-local `ECONNREFUSED`/`connection refused` branch in `connectSuggestion`. */ -function suggestNetworkRestrictions(dashboardUrl: string): string { - return `Make sure your local IP is allowed in Network Restrictions and Network Bans.\n${dashboardUrl}/project/_/database/settings`; +export function suggestNetworkRestrictions(dashboardUrl: string): string { + return `${SUGGEST_NETWORK_RESTRICTIONS}\n${dashboardUrl}/project/_/database/settings`; } /** Context the connect-suggestion needs but cannot derive from the error alone. */ @@ -221,6 +231,31 @@ const TLS_ERROR_CODES = new Set([ "CERT_REJECTED", "HOSTNAME_MISMATCH", ]); + +const CERT_VERIFICATION_CODES: ReadonlySet = new Set([ + ...[...TLS_ERROR_CODES].filter((code) => code !== "OUT_OF_MEM"), + // Node and Bun's `checkServerIdentity` failure, which `verify-full` raises on a hostname mismatch. + "ERR_TLS_CERT_ALTNAME_INVALID", +]); + +/** + * Whether the remote pooler rejected the caller's IP for project network restrictions. Only + * the allow-list changing can fix it, so it sets `DbConnectError.networkRestricted`, which the + * temp-role probe keys off to stop retrying. + */ +export function isNetworkRestrictionFailure(error: unknown): boolean { + return collectConnectErrorText(error).includes("Address not in tenant allow_list"); +} + +/** + * Whether a connect failure is an X509 certificate-verification rejection. It is deterministic: + * the same server certificate fails the same way on every attempt. Sets + * `DbConnectError.certificateRejected`, which the temp-role probe keys off to stop retrying. + */ +export function isCertificateVerificationFailure(error: unknown): boolean { + const cause = deepestConnectCause(error); + return hasStringCode(cause) && CERT_VERIFICATION_CODES.has(cause.code); +} // node-postgres' own message when the server answers `N` to SSLRequest // (`pg/lib/connection.js`); rendered as `tls error (server refused TLS connection)`. const SERVER_REFUSED_SSL = "The server does not support SSL connections"; @@ -233,6 +268,35 @@ const SERVER_REFUSED_SSL = "The server does not support SSL connections"; // post-handshake `read ECONNRESET` is not phase-specific and stays verbatim. const TLS_DISCONNECT_MESSAGE = "Client network socket disconnected before secure TLS connection was established"; +// node-postgres' message when the server answers the SSLRequest with anything but `S`/`N`. +const SSL_NEGOTIATION_ERROR = "There was an error establishing an SSL connection"; +// Handshake failures that carry no TLS-specific code: Node's `EPROTO` for a malformed TLS record, +// and Bun's `ERR_SOCKET_CLOSED` when the server hangs up after accepting the SSLRequest. +const TLS_HANDSHAKE_ERRNOS = new Set(["EPROTO", "ERR_SOCKET_CLOSED"]); + +const isTlsStageCause = (text: unknown, code: unknown): boolean => + text === SERVER_REFUSED_SSL || + text === TLS_DISCONNECT_MESSAGE || + (typeof code === "string" && + (TLS_ERROR_CODES.has(code) || code.startsWith("ERR_TLS") || code.startsWith("ERR_SSL"))); + +/** + * Whether a connect failure came from negotiating TLS (the server declined or botched the + * SSLRequest, or the handshake failed) rather than from dialing, authenticating, or a server + * error. The `sslmode=prefer` plaintext fallback retries on these. + */ +export function isTlsNegotiationFailure(error: unknown): boolean { + const cause = deepestConnectCause(error); + if (typeof cause !== "object" || cause === null) return false; + const message = Reflect.get(cause, "message"); + const code = Reflect.get(cause, "code"); + return ( + isTlsStageCause(message, code) || + message === SSL_NEGOTIATION_ERROR || + (typeof code === "string" && TLS_HANDSHAKE_ERRNOS.has(code)) + ); +} + // A Postgres SQLSTATE is exactly five uppercase alphanumerics. Combined with the // `severity` field this identifies a node-postgres `DatabaseError` (a server // ErrorResponse), never a node system error (whose codes are longer `E…` names). @@ -281,12 +345,7 @@ function connectCauseDetail(cause: unknown): string { if (syscall === "connect" || (typeof code === "string" && DIAL_ERROR_CODES.has(code))) { return `dial error (${text})`; } - if ( - text === SERVER_REFUSED_SSL || - text === TLS_DISCONNECT_MESSAGE || - (typeof code === "string" && - (TLS_ERROR_CODES.has(code) || code.startsWith("ERR_TLS") || code.startsWith("ERR_SSL"))) - ) { + if (isTlsStageCause(text, code)) { return `tls error (${text})`; } return text; @@ -354,7 +413,7 @@ export function connectSuggestion( const text = collectConnectErrorText(error); // "Address not in tenant allow_list" only ever comes from the remote pooler rejecting the // caller's IP, so it always means network restrictions. - if (text.includes("Address not in tenant allow_list")) { + if (isNetworkRestrictionFailure(error)) { return suggestNetworkRestrictions(ctx.dashboardUrl); } // Don't send the user to the dashboard's Network Restrictions page for a --local connection. @@ -363,6 +422,7 @@ export function connectSuggestion( ? localStackSuggestion(ctx.linkedCheckout) : suggestNetworkRestrictions(ctx.dashboardUrl); } + if (isCertificateVerificationFailure(error)) return SUGGEST_CA_VERIFICATION; if ( text.includes("SCRAM exchange: Wrong password") || text.includes("failed SASL auth") || @@ -370,8 +430,8 @@ export function connectSuggestion( ) { return SUGGEST_ENV_VAR; } - // An unset `sslmode` negotiates TLS and fails rather than downgrading, so a server without - // TLS needs the caller to opt into plaintext explicitly. + // Reached when plaintext is not an allowed fallback (`sslmode=require` or stricter), so the + // caller must opt into plaintext explicitly. if (text.includes(SERVER_REFUSED_SSL) || text.includes("server refused TLS connection")) { return "This server does not accept TLS. Set `sslmode=disable` on the connection string to connect in plaintext."; } diff --git a/apps/cli/src/command-internal/connect-errors.unit.test.ts b/apps/cli/src/command-internal/connect-errors.unit.test.ts index 3b31e9d043..33f6c0f953 100644 --- a/apps/cli/src/command-internal/connect-errors.unit.test.ts +++ b/apps/cli/src/command-internal/connect-errors.unit.test.ts @@ -3,14 +3,18 @@ import * as Pg from "pg"; import { describe, expect, it } from "vitest"; import { + SUGGEST_CA_VERIFICATION, SUGGEST_ENV_VAR, SUGGEST_LOCAL_STACK, connectFailureMessage, connectSuggestion, ipv6Suggestion, + isCertificateVerificationFailure, + isNetworkRestrictionFailure, isDialFailure, isIPv6ConnectivityError, isIPv6ConnectivityErrorCause, + isTlsNegotiationFailure, } from "./connect-errors.ts"; // The real `@effect/sql` wrapper produced by the connect probe @@ -269,6 +273,13 @@ describe("connectSuggestion", () => { ); }); + it("maps a certificate-verification failure to the CA hint", () => { + const err = sqlError( + systemError("self-signed certificate in certificate chain", "SELF_SIGNED_CERT_IN_CHAIN"), + ); + expect(connectSuggestion(err, ctx)).toBe(SUGGEST_CA_VERIFICATION); + }); + it("maps a refused local connection to the local stack hint", () => { const err = sqlError(systemError("connect ECONNREFUSED 127.0.0.1:54322", "ECONNREFUSED")); expect(connectSuggestion(err, { ...ctx, isLocal: true })).toBe(SUGGEST_LOCAL_STACK); @@ -421,6 +432,34 @@ describe("connectSuggestion", () => { }); }); +describe("isCertificateVerificationFailure", () => { + it("matches the X509 verification codes but not other TLS or dial failures", () => { + const withCode = (code: string) => + realSqlConnectError(Object.assign(new Error("failure"), { code })); + expect(isCertificateVerificationFailure(withCode("SELF_SIGNED_CERT_IN_CHAIN"))).toBe(true); + expect(isCertificateVerificationFailure(withCode("CERT_HAS_EXPIRED"))).toBe(true); + expect(isCertificateVerificationFailure(withCode("ERR_TLS_CERT_ALTNAME_INVALID"))).toBe(true); + expect(isCertificateVerificationFailure(withCode("OUT_OF_MEM"))).toBe(false); + expect(isCertificateVerificationFailure(withCode("ECONNREFUSED"))).toBe(false); + expect(isCertificateVerificationFailure(withCode("ECONNRESET"))).toBe(false); + expect(isCertificateVerificationFailure(authFailedError())).toBe(false); + }); +}); + +describe("isNetworkRestrictionFailure", () => { + it("matches the pooler's allow-list rejection but not other failures", () => { + expect( + isNetworkRestrictionFailure( + realSqlConnectError(new Error("Address not in tenant allow_list: 1.2.3.4")), + ), + ).toBe(true); + expect(isNetworkRestrictionFailure(authFailedError())).toBe(false); + expect(isNetworkRestrictionFailure(realSqlConnectError(new Error("some other failure")))).toBe( + false, + ); + }); +}); + describe("isDialFailure", () => { it("classifies dial errno failures", () => { expect(isDialFailure(realSqlConnectError(dialError("ECONNREFUSED", "127.0.0.1", 54322)))).toBe( @@ -530,3 +569,30 @@ describe("isIPv6ConnectivityErrorCause", () => { ).toBe(true); }); }); + +describe("isTlsNegotiationFailure (sslmode=prefer plaintext fallback gate)", () => { + it("is true for a declined SSLRequest, a botched reply, and TLS handshake failures", () => { + for (const cause of [ + new Error("The server does not support SSL connections"), + new Error("There was an error establishing an SSL connection"), + new Error("Client network socket disconnected before secure TLS connection was established"), + Object.assign(new Error("wrong version number"), { code: "ERR_SSL_WRONG_VERSION_NUMBER" }), + Object.assign(new Error("write EPROTO wrong version number"), { code: "EPROTO" }), + Object.assign(new Error("Socket is closed"), { code: "ERR_SOCKET_CLOSED" }), + ]) { + expect(isTlsNegotiationFailure(realSqlConnectError(cause))).toBe(true); + } + }); + + it("is false for dial, timeout, authentication and unclassified failures", () => { + for (const cause of [ + dialError("ECONNREFUSED", "127.0.0.1", 5432), + new Error("Connection terminated unexpectedly"), + new Error("timeout expired"), + Object.assign(new Error("password authentication failed"), { code: "28P01" }), + ]) { + expect(isTlsNegotiationFailure(realSqlConnectError(cause))).toBe(false); + } + expect(isTlsNegotiationFailure("boom")).toBe(false); + }); +}); diff --git a/apps/cli/src/command-internal/container-cli.ts b/apps/cli/src/command-internal/container-cli.ts index c6f165ea50..5371f4ba02 100644 --- a/apps/cli/src/command-internal/container-cli.ts +++ b/apps/cli/src/command-internal/container-cli.ts @@ -307,18 +307,35 @@ function isDockerApiVersionAtLeast(version: string, minVersion: string): boolean * fall back to Podman, whose `volume prune` has no `--all` flag to gate. */ export const dockerSupportsVolumePruneAllFlag = (spawner: Spawner) => + dockerApiVersionsSatisfy(spawner, (version) => isDockerApiVersionAtLeast(version, "1.42")); + +/** + * `docker create --health-start-interval` needs API >= 1.44 on both ends and is a hard failure + * below that. The Podman CLI rejects the flag, and a `docker` shim for it fails this format or + * prints Podman's own `6.x`, so a major other than 1 is not Docker's API. The real `docker` client + * on a Podman socket reporting 1.44 passes: Podman keeps the start period and drops the start + * interval. + */ +export const dockerSupportsHealthStartInterval = (spawner: Spawner) => + dockerApiVersionsSatisfy( + spawner, + (version) => version.startsWith("1.") && isDockerApiVersionAtLeast(version, "1.44"), + ); + +/* Reads the client's own and effective API versions and the daemon's, `false` on any failure. + `DOCKER_API_VERSION` can lift the effective one past what the client binary supports. */ +const dockerApiVersionsSatisfy = (spawner: Spawner, supported: (version: string) => boolean) => Effect.scoped( Effect.gen(function* () { - const child = yield* spawner.spawn( - ChildProcess.make( - "docker", - ["version", "--format", "{{.Client.APIVersion}} {{.Server.APIVersion}}"], - { - stdin: "ignore", - stdout: "pipe", - stderr: "ignore", - }, - ), + const child = yield* spawnRuntime( + spawner, + "docker", + [ + "version", + "--format", + "{{.Client.DefaultAPIVersion}} {{.Client.APIVersion}} {{.Server.APIVersion}}", + ], + { stdin: "ignore", stdout: "pipe", stderr: "ignore" }, ); const [exitCode, stdout] = yield* Effect.all( [child.exitCode.pipe(Effect.map(Number)), collectText(child.stdout)], @@ -326,9 +343,6 @@ export const dockerSupportsVolumePruneAllFlag = (spawner: Spawner) => ); if (exitCode !== 0) return false; const versions = stdout.trim().split(/\s+/); - return ( - versions.length === 2 && - versions.every((version) => isDockerApiVersionAtLeast(version, "1.42")) - ); + return versions.length === 3 && versions.every(supported); }), ).pipe(Effect.orElseSucceed(() => false)); diff --git a/apps/cli/src/command-internal/container-cli.unit.test.ts b/apps/cli/src/command-internal/container-cli.unit.test.ts index d65317e74c..01fbd0fdf9 100644 --- a/apps/cli/src/command-internal/container-cli.unit.test.ts +++ b/apps/cli/src/command-internal/container-cli.unit.test.ts @@ -6,6 +6,7 @@ import { containerCliExitCode, containerRuntimeNotFoundMessage, describeContainerCliFailure, + dockerSupportsHealthStartInterval, dockerSupportsVolumePruneAllFlag, isContainerNotFoundMessage, spawnContainerCli, @@ -17,7 +18,11 @@ function mockSpawner( readonly bothMissing?: boolean; readonly exitCode?: number; readonly stdout?: string; - readonly apiVersions?: { readonly client: string; readonly server: string }; + readonly apiVersions?: { + readonly clientDefault?: string; + readonly client: string; + readonly server: string; + }; } = {}, ) { const spawned: Array<{ readonly command: string; readonly args: ReadonlyArray }> = []; @@ -25,7 +30,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { const cmd = command._tag === "StandardCommand" ? command.command : ""; - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push({ command: cmd, args }); if ((opts.dockerMissing && cmd === "docker") || opts.bothMissing === true) { @@ -40,6 +45,10 @@ function mockSpawner( const stdout = opts.apiVersions !== undefined && args[0] === "version" ? (args[args.indexOf("--format") + 1] ?? "") + .replaceAll( + "{{.Client.DefaultAPIVersion}}", + opts.apiVersions.clientDefault ?? opts.apiVersions.client, + ) .replaceAll("{{.Client.APIVersion}}", opts.apiVersions.client) .replaceAll("{{.Server.APIVersion}}", opts.apiVersions.server) : opts.stdout; @@ -229,6 +238,57 @@ describe("container CLI process spans", () => { ); }); +describe("dockerSupportsHealthStartInterval", () => { + it.live.each([ + { name: "both ends at the 1.44 minimum", client: "1.44", server: "1.44" }, + { name: "a new client against a newer daemon", client: "1.47", server: "1.56" }, + { name: "a three-part API version", client: "1.44.0", server: "1.44.0" }, + ])("returns true for $name", ({ client, server }) => { + const mock = mockSpawner({ apiVersions: { client, server } }); + return dockerSupportsHealthStartInterval(mock.spawner).pipe( + Effect.map((supported) => { + expect(supported).toBe(true); + }), + ); + }); + + it.live.each([ + { + name: "a client below API 1.44", + options: { apiVersions: { client: "1.43", server: "1.56" } }, + }, + { + name: "an old client with DOCKER_API_VERSION forced up", + options: { apiVersions: { clientDefault: "1.43", client: "1.44", server: "1.56" } }, + }, + { + name: "a daemon below API 1.44", + options: { apiVersions: { client: "1.56", server: "1.43" } }, + }, + { + name: "a docker shim printing Podman's own version", + options: { apiVersions: { client: "6.1.3", server: "6.1.3" } }, + }, + { + name: "a shim whose major starts with a 1", + options: { apiVersions: { client: "10.1.0", server: "10.1.0" } }, + }, + { name: "fewer than three versions", options: { stdout: "1.56 1.56" } }, + { + name: "the version command exiting non-zero", + options: { exitCode: 1, stdout: "1.56 1.56 1.56" }, + }, + { name: "docker not being installed", options: { dockerMissing: true } }, + ])("returns false for $name", ({ options }) => { + const mock = mockSpawner(options); + return dockerSupportsHealthStartInterval(mock.spawner).pipe( + Effect.map((supported) => { + expect(supported).toBe(false); + }), + ); + }); +}); + describe("dockerSupportsVolumePruneAllFlag", () => { it.live( "returns true when the client and daemon both report an API version at or above 1.42", @@ -240,7 +300,11 @@ describe("dockerSupportsVolumePruneAllFlag", () => { expect(mock.spawned).toEqual([ { command: "docker", - args: ["version", "--format", "{{.Client.APIVersion}} {{.Server.APIVersion}}"], + args: [ + "version", + "--format", + "{{.Client.DefaultAPIVersion}} {{.Client.APIVersion}} {{.Server.APIVersion}}", + ], }, ]); }), @@ -266,8 +330,8 @@ describe("dockerSupportsVolumePruneAllFlag", () => { name: "versions that only compare above 1.42 lexicographically", options: { apiVersions: { client: "1.9", server: "1.9" } }, }, - { name: "a single version instead of a client and daemon pair", options: { stdout: "1.51" } }, - { name: "more than two versions", options: { stdout: "1.51 1.51 1.51" } }, + { name: "a single version", options: { stdout: "1.51" } }, + { name: "more than three versions", options: { stdout: "1.51 1.51 1.51 1.51" } }, { name: "empty output", options: { stdout: "" } }, ])("returns false for $name", ({ options }) => { const mock = mockSpawner(options); diff --git a/apps/cli/src/command-internal/db-bootstrap/await-storage-ready.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/await-storage-ready.unit.test.ts index cb35de6488..52a45f9678 100644 --- a/apps/cli/src/command-internal/db-bootstrap/await-storage-ready.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/await-storage-ready.unit.test.ts @@ -19,7 +19,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push(args); const result = handler(args); diff --git a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts index 85457f9137..e5246a7cc1 100644 --- a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts +++ b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.ts @@ -34,6 +34,7 @@ import { import { buildStartContainerCreateArgs, applyBitbucketStartContainerFilter, + applyHealthStartInterval, isDockerClientEnvKey, type StartContainerSpec, } from "./docker-create-args.ts"; @@ -124,6 +125,11 @@ export interface ContainerOpts { readonly projectId: string; /** Passed in (rather than read via env here) so this module stays a pure effect orchestrator. */ readonly isBitbucketPipeline: boolean; + /** + * Whether the engine takes `--health-start-interval`, probed once per run by each command that + * reads Docker's health status. Shadow databases leave it unset: nothing reads theirs. + */ + readonly healthStartInterval?: boolean; /** * Stamped onto every created container as {@link CLI_WORKDIR_LABEL} so a later `stop` or * {@link rollbackStart} can find this directory from the container's own label alone, without @@ -183,7 +189,7 @@ function portConflictSuggestion(hostPort: string, serviceLabel: string): string return ( `\nTry stopping the project or container already using ${hostPort} ` + "(`docker ps` lists what's bound to it, or `supabase stop` for another local Supabase project), " + - `or configure a different ${serviceLabel} port in supabase/config.toml.` + `or configure a different ${serviceLabel} port in your config file.` ); } @@ -716,7 +722,10 @@ export function createContainer( } } - const finalSpec = applyBitbucketStartContainerFilter(labeledSpec, opts.isBitbucketPipeline); + const finalSpec = applyHealthStartInterval( + applyBitbucketStartContainerFilter(labeledSpec, opts.isBitbucketPipeline), + opts.healthStartInterval === true, + ); const createArgs = buildStartContainerCreateArgs(finalSpec); // `isDockerClientEnvKey` keys are already emitted inline as `-e KEY=value` above; see diff --git a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts index 648474c306..0cf21ec579 100644 --- a/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/container-lifecycle.unit.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "@effect/vitest"; -import { Deferred, Effect, PlatformError, Sink, Stream } from "effect"; +import { Deferred, Effect, PlatformError, type Redacted, Sink, Stream } from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; import { @@ -31,14 +31,14 @@ function mockSpawner( const spawned: Array> = []; const spawnedOptions: Array<{ readonly args: ReadonlyArray; - readonly env: Record | undefined; + readonly env: Record | undefined> | undefined; readonly extendEnv: boolean | undefined; readonly stdin: ChildProcess.CommandInput | ChildProcess.StdinConfig | undefined; }> = []; const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push(args); if (command._tag === "StandardCommand") { spawnedOptions.push({ @@ -297,6 +297,65 @@ describe("createContainer", () => { ); }); + describe("Docker's first health check", () => { + const steady = { test: ["CMD", "true"], intervalSeconds: 10, timeoutSeconds: 2, retries: 3 }; + const steadyFlags = { + "--health-cmd": "true", + "--health-interval": "10s", + "--health-timeout": "2s", + "--health-retries": "3", + }; + const startFlags = { "--health-start-period": "10s", "--health-start-interval": "1s" }; + + it.live.each([ + { + name: "adds start flags where the engine takes them", + healthcheck: steady, + supported: true, + expected: { ...steadyFlags, ...startFlags }, + }, + { + name: "keeps a service's own start period", + healthcheck: { ...steady, startPeriodSeconds: 20 }, + supported: true, + expected: { ...steadyFlags, ...startFlags, "--health-start-period": "20s" }, + }, + { + name: "adds none where the engine rejects them", + healthcheck: steady, + supported: false, + expected: steadyFlags, + }, + { + name: "adds none for a caller that never asked", + healthcheck: steady, + expected: steadyFlags, + }, + { name: "adds none without a healthcheck", supported: true, expected: {} }, + ])("$name", ({ healthcheck, supported, expected }) => { + const mock = alwaysSucceed(); + return createContainer( + mock.spawner, + { ...baseSpec, binds: [], ...(healthcheck === undefined ? {} : { healthcheck }) }, + { + projectId: "proj", + isBitbucketPipeline: false, + workdir: workdir.current, + extraHosts: [], + ...(supported === undefined ? {} : { healthStartInterval: supported }), + }, + ).pipe( + Effect.map(() => { + const args = mock.spawned.find((spawned) => spawned[0] === "create") ?? []; + const flags = args.flatMap((arg, index) => + arg.startsWith("--health-") ? [[arg, args[index + 1]]] : [], + ); + expect(Object.fromEntries(flags)).toEqual(expected); + }), + ); + }); + }); + it.live("fails with ContainerStartError, unmodified, on a plain start failure", () => { const mock = mockSpawner((args) => { if (args[0] === "create") return { exitCode: 0, stdout: "abc\n" }; @@ -346,7 +405,7 @@ describe("createContainer", () => { expect(error).toBeInstanceOf(ContainerStartError); expect(error.message).toContain('failed to start docker container "supabase_db_proj"'); expect(error.message).toContain("0.0.0.0:5432"); - expect(error.message).toContain("db port in supabase/config.toml"); + expect(error.message).toContain("db port in your config file"); }), ); }, diff --git a/apps/cli/src/command-internal/db-bootstrap/db-setup.ts b/apps/cli/src/command-internal/db-bootstrap/db-setup.ts index d823034fb0..b3f5fccbe1 100644 --- a/apps/cli/src/command-internal/db-bootstrap/db-setup.ts +++ b/apps/cli/src/command-internal/db-bootstrap/db-setup.ts @@ -30,7 +30,7 @@ import type { } from "../../config/cli-config.errors.ts"; import type { ProjectRefReadError } from "../../shared/config/legacy-project-state.service.ts"; import { DbConfigLoadError } from "../db-config.errors.ts"; -import { checkDbToml } from "../db-config.toml-read.ts"; +import { checkDbConfig } from "../db-config.read.ts"; import { CLI_PROJECT_LABEL, localDbContainerId } from "../docker-ids.ts"; import { DockerRun, type DockerRunOpts } from "../docker-run.service.ts"; import { migrateAndSeed } from "../migrate-and-seed.ts"; @@ -851,10 +851,10 @@ export const startSetupLocalDatabase = ( Effect.gen(function* () { const { session, fs, path, workdir } = input; - // `warnOnUnresolvedEnv: false`: the caller's own handler already ran an earlier `checkDbToml` + // `warnOnUnresolvedEnv: false`: the caller's own handler already ran an earlier `checkDbConfig` // pass before reaching this fresh-volume setup, which already printed any unresolved-env // warning. Without this, this module's own duplicate config-load pass would print it twice. - const toml = yield* checkDbToml(fs, path, workdir, undefined, { + const toml = yield* checkDbConfig(fs, path, workdir, undefined, { warnOnUnresolvedEnv: false, }); @@ -874,6 +874,7 @@ export const startSetupLocalDatabase = ( pgDeltaEnabled: toml.pgDelta.enabled, schemaPaths: toml.schemaPaths, localDatabaseWebhooksEnabled: toml.webhooksEnabled, + configFile: toml.configFile, }); }); diff --git a/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts b/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts index d379bc7a39..1c38d17ae1 100644 --- a/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts +++ b/apps/cli/src/command-internal/db-bootstrap/docker-create-args.ts @@ -23,6 +23,8 @@ interface StartHealthcheckSpec { readonly retries?: number; /** Emitted as `--health-start-period s`; omitted when unset. */ readonly startPeriodSeconds?: number; + /** Emitted as `--health-start-interval s`; see {@link applyHealthStartInterval}. */ + readonly startIntervalSeconds?: number; } /** One `-p :[/]` entry. */ @@ -203,6 +205,12 @@ function buildHealthcheckArgs(healthcheck: StartHealthcheckSpec): ReadonlyArray< if (healthcheck.startPeriodSeconds !== undefined) { args.push("--health-start-period", formatDockerDurationSeconds(healthcheck.startPeriodSeconds)); } + if (healthcheck.startIntervalSeconds !== undefined) { + args.push( + "--health-start-interval", + formatDockerDurationSeconds(healthcheck.startIntervalSeconds), + ); + } return args; } @@ -288,3 +296,26 @@ export function applyBitbucketStartContainerFilter( securityOpt: [], }; } + +/* Docker returns to `--health-interval` after the first pass or the end of the start period. */ +const HEALTH_START_INTERVAL_SECONDS = 1; +/* The start interval only runs inside a start period, whose failures do not count. One steady + interval keeps `unhealthy` where it was, or one interval later if a slow probe straddles it. */ +const HEALTH_START_PERIOD_SECONDS = 10; + +/* Runs a container's first health checks right after it starts instead of a full interval later. + `supported` comes from `dockerSupportsHealthStartInterval`; without it the spec is unchanged. */ +export function applyHealthStartInterval( + spec: StartContainerSpec, + supported: boolean, +): StartContainerSpec { + if (!supported || spec.healthcheck === undefined) return spec; + return { + ...spec, + healthcheck: { + ...spec.healthcheck, + startPeriodSeconds: spec.healthcheck.startPeriodSeconds ?? HEALTH_START_PERIOD_SECONDS, + startIntervalSeconds: HEALTH_START_INTERVAL_SECONDS, + }, + }; +} diff --git a/apps/cli/src/command-internal/db-bootstrap/health-check.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/health-check.unit.test.ts index 81aba24804..a92c938686 100644 --- a/apps/cli/src/command-internal/db-bootstrap/health-check.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/health-check.unit.test.ts @@ -79,7 +79,7 @@ function mockHealthSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; const binary = command._tag === "StandardCommand" ? command.command : ""; // Mirrors a host where only one runtime is installed: `spawnContainerCli` // tries `docker` first and falls back to `podman` when the spawn fails. diff --git a/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts index 2e4835d52e..53bc69f843 100644 --- a/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts @@ -15,7 +15,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push(args); const result = handler(args); @@ -114,7 +114,7 @@ describe("ensureImagesCached", () => { const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; if (args[0] === "image" && args[1] === "inspect") { const count = yield* Ref.updateAndGet(inFlight, (n) => n + 1); yield* Ref.update(peak, (n) => Math.max(n, count)); diff --git a/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts b/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts index 3cb27f822b..e772c960fc 100644 --- a/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.precedence.integration.test.ts @@ -1,9 +1,10 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, it } from "@effect/vitest"; import { ConfigProvider, Effect, FileSystem, Layer, Option, Path } from "effect"; import { ChildProcessSpawner } from "effect/process"; import * as HttpClient from "effect/http/HttpClient"; import * as HttpClientResponse from "effect/http/HttpClientResponse"; +import { catalogPins } from "@supabase/stack/internal/artifacts"; import { cliConfigValuesTestLayer } from "../../../tests/helpers/config-values-layer.ts"; import { mockRuntimeInfo } from "../../../tests/helpers/mocks.ts"; @@ -16,6 +17,9 @@ import { buildLocalDbContainerInputs } from "./local-container-inputs.ts"; const TARGET_REF = "abcdefghijklmnopqrst"; const GOLDEN_DIR = "./testdata/config-precedence/local-container-inputs"; +const DEFAULT_POSTGRES_IMAGE = catalogPins().find( + (entry) => entry.service === "database" && entry.isDefault, +)!.pin.upstreamImage; const BASE_CONFIG = `project_id = "golden-shadow" @@ -142,6 +146,7 @@ describe("local container inputs precedence goldens", () => { const layer = Layer.mergeAll( BunServices.layer, + BunCrypto.layer, cliConfigValuesTestLayer, Layer.succeed(ConfigEnvPins, fixture.shellEnv ?? {}), mockRuntimeInfo({ platform: "linux" }), @@ -192,7 +197,10 @@ describe("local container inputs precedence goldens", () => { projectEnvValues: configEnvOnly(setup.projectEnvValues ?? {}), }, }; - const goldenText = yield* goldenJson(golden, { [workdir]: "" }); + const goldenText = (yield* goldenJson(golden, { [workdir]: "" })).replaceAll( + `"${DEFAULT_POSTGRES_IMAGE}"`, + `""`, + ); yield* Effect.promise(() => ctx.expect(goldenText).toMatchFileSnapshot(`${GOLDEN_DIR}/${fixture.golden}.json`), ); diff --git a/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.ts b/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.ts index 6986dff839..d0766f0f17 100644 --- a/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.ts +++ b/apps/cli/src/command-internal/db-bootstrap/local-container-inputs.ts @@ -8,7 +8,8 @@ * separately-resolved `--experimental` gate (needed earlier, before `cfg.isLocal` is known). */ -import { Crypto, Effect, FileSystem, Option, Path } from "effect"; +import { Effect, FileSystem, Option, Path } from "effect"; +import { Digest } from "effect/crypto"; import { HttpClient } from "effect/http"; import type { ChildProcessSpawner } from "effect/process/ChildProcessSpawner"; import type { GlobalFlag } from "effect/cli"; @@ -93,7 +94,7 @@ export const buildLocalDbContainerInputs = ( | FileSystem.FileSystem | Path.Path | RuntimeInfo - | Crypto.Crypto + | Digest.Digest | GlobalFlag.Setting.Identifier<"experimental"> | CliArgs | HttpClient.HttpClient @@ -111,9 +112,13 @@ export const buildLocalDbContainerInputs = ( const document = resolvedConfig.loaded.document ?? {}; const experimental = yield* resolveExperimentalWithProjectEnv(projectEnvValues); - const values = yield* resolveLocalConfigValues(config, hostname, workdir, document).pipe( - Effect.mapError((cause) => mapError(cause.message)), - ); + const values = yield* resolveLocalConfigValues( + config, + hostname, + workdir, + document, + resolvedConfig.hasConfigFile ? resolvedConfig.loaded.path : undefined, + ).pipe(Effect.mapError((cause) => mapError(cause.message))); const bootstrapConfig = yield* resolveDbBootstrapConfig( fs, diff --git a/apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts b/apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts index c04f53fada..6e91a82c81 100644 --- a/apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts @@ -1,4 +1,4 @@ -import { BunHttpServer, BunServices, BunSocketServer } from "@effect/platform-bun"; +import { BunCrypto, BunHttpServer, BunServices, BunSocketServer } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { ConfigProvider, Effect, FileSystem, Layer, Option, Path, Schedule, Stream } from "effect"; import { HttpServerRequest, HttpServerResponse } from "effect/http"; @@ -121,7 +121,7 @@ const missingSocketEndpoint = Effect.gen(function* () { }); const localDockerEngineTestLayer = localDockerEngineLayer.pipe( - Layer.provide(BunServices.layer), + Layer.provide([BunServices.layer, BunCrypto.layer]), Layer.provide(runtimeInfoLayer), ); @@ -151,7 +151,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { expect(answer).toEqual(Option.some({ exists: true, running: true, status: "running" })); expect(engine.requestedUrls).toEqual(["/containers/supabase_db_engine-probe/json"]); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live.each([ @@ -182,7 +182,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.some({ exists: true, running, status })); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live("answers absent for an Engine 404", () => @@ -197,7 +197,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.some({ exists: false })); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live("gives no answer for a responder that does not identify as a Docker engine", () => @@ -238,7 +238,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { expect(absent).toEqual(Option.none()); expect(viaServerHeader).toEqual(Option.some({ exists: false })); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live("gives no answer for an empty or malformed Engine 200 body", () => @@ -263,7 +263,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { expect(empty).toEqual(Option.none()); expect(malformed).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live( @@ -280,7 +280,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live( @@ -295,7 +295,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 15_000, ); @@ -311,7 +311,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live("gives no answer when the peer resets after the headers", () => @@ -327,7 +327,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live("traces the request, the decline, and the fallback through DebugLogger", () => { @@ -379,7 +379,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { lines.some((l) => l.startsWith("debug:") && l.includes("no definitive Engine answer")), ).toBe(true); }), - ).pipe(Effect.provide(BunServices.layer)); + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.live( @@ -406,7 +406,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 15_000, ); }); @@ -420,7 +420,7 @@ describe("LocalDockerEngine (direct Engine-API transport)", () => { ); expect(answer).toEqual(Option.none()); }), - ).pipe(Effect.provide(BunServices.layer)), + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.live("gives no answer for an endpoint the transport cannot address (ssh)", () => @@ -592,7 +592,7 @@ describe("isLocalDbRunning", () => { ["container", "inspect", "supabase_db_engine-probe"], ]); }), - ).pipe(Effect.provide(BunServices.layer)); + ).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])); }, ); }); diff --git a/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts b/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts index df89a0d462..80098d71bd 100644 --- a/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts +++ b/apps/cli/src/command-internal/db-bootstrap/local-db-running.ts @@ -1,6 +1,7 @@ import http from "node:http"; -import { Context, Crypto, Data, Effect, FileSystem, Layer, Option, Path } from "effect"; +import { Context, Data, Effect, FileSystem, Layer, Option, Path } from "effect"; +import { Digest } from "effect/crypto"; import type { ChildProcessSpawner } from "effect/process/ChildProcessSpawner"; import { @@ -271,19 +272,19 @@ const inspectContainerOverSocket = ( export const localDockerEngineLayer: Layer.Layer< LocalDockerEngine, never, - Crypto.Crypto | FileSystem.FileSystem | Path.Path | RuntimeInfo + Digest.Digest | FileSystem.FileSystem | Path.Path | RuntimeInfo > = Layer.effect( LocalDockerEngine, Effect.gen(function* () { const runtime = yield* RuntimeInfo; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const crypto = yield* Crypto.Crypto; + const crypto = yield* Digest.Digest; const resolveEndpoint = resolveDockerDaemonEndpoint().pipe( Effect.provideService(RuntimeInfo, runtime), Effect.provideService(FileSystem.FileSystem, fs), Effect.provideService(Path.Path, path), - Effect.provideService(Crypto.Crypto, crypto), + Effect.provideService(Digest.Digest, crypto), Effect.orElseSucceed(() => Option.none()), ); const debugLogger = yield* Effect.serviceOption(DebugLogger); @@ -390,16 +391,14 @@ export function localDbState( if (!isContainerNotFoundMessage(stderr)) { // Surface the Docker install hint instead of raw stderr when the daemon is unreachable. const daemonDown = isDockerDaemonUnreachable(stderr); - return yield* Effect.fail( - new LocalDbRunningError({ - message: - stderr.length > 0 - ? `failed to inspect service: ${stderr}` - : "failed to inspect service", - daemonDown, - ...(daemonDown ? { suggestion: SUGGEST_DOCKER_INSTALL } : {}), - }), - ); + return yield* new LocalDbRunningError({ + message: + stderr.length > 0 + ? `failed to inspect service: ${stderr}` + : "failed to inspect service", + daemonDown, + ...(daemonDown ? { suggestion: SUGGEST_DOCKER_INSTALL } : {}), + }); } return { exists: false } satisfies LocalDbState; }), diff --git a/apps/cli/src/command-internal/db-bootstrap/postgres.service.ts b/apps/cli/src/command-internal/db-bootstrap/postgres.service.ts index ac8db9b61b..9bd3aaf722 100644 --- a/apps/cli/src/command-internal/db-bootstrap/postgres.service.ts +++ b/apps/cli/src/command-internal/db-bootstrap/postgres.service.ts @@ -10,7 +10,6 @@ import type { CliConfig } from "@supabase/config"; import { localDbContainerId } from "../docker-ids.ts"; -import { toDockerMountPath } from "../docker-path.ts"; import { encodeToml } from "../output.encoders.ts"; import { POSTGRES_DEFAULT_ROOT_KEY } from "../local-config-values.ts"; import type { StartContainerSpec } from "./docker-create-args.ts"; @@ -314,9 +313,7 @@ export function buildPostgresStartContainerSpec( binds: [ `${containerName}:${POSTGRES_DATA_DIR}`, // The backup file bind is appended only on the `fromBackup` branch. - ...(input.fromBackup === undefined - ? [] - : [`${toDockerMountPath(input.fromBackup)}:/etc/backup.sql:ro`]), + ...(input.fromBackup === undefined ? [] : [`${input.fromBackup}:/etc/backup.sql:ro`]), ], // Tmpfs is keyed on `isPg14OrEarlier` alone, independent of `isRestore`. ...(isPg14OrEarlier ? { tmpfs: { "/docker-entrypoint-initdb.d": "" } } : {}), diff --git a/apps/cli/src/command-internal/db-bootstrap/postgres.service.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/postgres.service.unit.test.ts index 47f7bf755c..df486ec49b 100644 --- a/apps/cli/src/command-internal/db-bootstrap/postgres.service.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/postgres.service.unit.test.ts @@ -295,11 +295,11 @@ describe("buildPostgresStartContainerSpec", () => { ]); }); - test("--from-backup: converts a Windows-style host path through toDockerMountPath for the backup bind", () => { + test("--from-backup: keeps the Windows drive in the backup bind source", () => { const spec = buildPostgresStartContainerSpec( baseInput({ fromBackup: "C:\\Users\\me\\backup.sql" }), ); - expect(spec.binds).toContain("/Users/me/backup.sql:/etc/backup.sql:ro"); + expect(spec.binds).toContain("C:\\Users\\me\\backup.sql:/etc/backup.sql:ro"); }); test("no --from-backup: binds only the data volume, matching the pre-existing behavior", () => { diff --git a/apps/cli/src/command-internal/db-bootstrap/realtime-env.ts b/apps/cli/src/command-internal/db-bootstrap/realtime-env.ts index 9bb7c2bc98..19983243a0 100644 --- a/apps/cli/src/command-internal/db-bootstrap/realtime-env.ts +++ b/apps/cli/src/command-internal/db-bootstrap/realtime-env.ts @@ -6,6 +6,7 @@ import type { CliConfig } from "@supabase/config"; import { + DEFAULT_ERL_MAX_PORTS, DEFAULT_LOCAL_SERVICE_SECRET_KEY_BASE, DEFAULT_REALTIME_DB_ENCRYPTION_KEY, } from "@supabase/stack/defaults"; @@ -56,6 +57,7 @@ export function buildRealtimeEnv(input: RealtimeEnvInput): Record { DB_ENC_KEY: "supabaserealtime", APP_NAME: "realtime", SECRET_KEY_BASE: "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG", + ERL_MAX_PORTS: "65536", DNS_NODES: "''", RLIMIT_NOFILE: "", SEED_SELF_HOST: "true", diff --git a/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts b/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts index ce496f87c0..6d18a459c3 100644 --- a/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/recreate-local-database.ts @@ -23,8 +23,8 @@ import { ErrorActionabilityId, } from "../../shared/telemetry/error-actionability.ts"; import { isSqlState } from "../connect-errors.ts"; -import { isContainerNotFoundMessage } from "../container-cli.ts"; -import { checkDbToml } from "../db-config.toml-read.ts"; +import { dockerSupportsHealthStartInterval, isContainerNotFoundMessage } from "../container-cli.ts"; +import { checkDbConfig } from "../db-config.read.ts"; import { DbConnection, type DbSession } from "../db-connection.service.ts"; import { DbExecError, type DbConnectError } from "../db-connection.errors.ts"; import { CLI_PROJECT_LABEL } from "../docker-ids.ts"; @@ -292,7 +292,10 @@ const recreateLocalDatabase15 = ( ...input.postgresSpec, image: resolvedPostgresImage, }); - yield* createContainer(spawner, postgresSpec, input.containerOpts); + yield* createContainer(spawner, postgresSpec, { + ...input.containerOpts, + healthStartInterval: yield* dockerSupportsHealthStartInterval(spawner), + }); // Never swallowed — reset has no `--from-backup`-equivalent gate. yield* waitForPostgresReady( @@ -373,7 +376,7 @@ const recreateLocalDatabase14 = ( Effect.gen(function* () { const { setup, fs, path, workdir } = input; const dbPassword = startInternalDbPassword(setup.dbUrl); - const toml = yield* checkDbToml(fs, path, workdir); + const toml = yield* checkDbConfig(fs, path, workdir); const dbConnection = yield* DbConnection; const output = yield* Output; @@ -447,6 +450,7 @@ const recreateLocalDatabase14 = ( pgDeltaEnabled: toml.pgDelta.enabled, schemaPaths: toml.schemaPaths, localDatabaseWebhooksEnabled: toml.webhooksEnabled, + configFile: toml.configFile, }); }), ); diff --git a/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts b/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts index b41987edda..843b7e3bf0 100644 --- a/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts @@ -30,7 +30,7 @@ import { } from "../../shared/telemetry/error-actionability.ts"; import { aqua, yellow } from "../colors.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; -import { checkDbToml, loadProjectEnvValues, readDbToml } from "../db-config.toml-read.ts"; +import { checkDbConfig, loadProjectEnvValues, readDbConfig } from "../db-config.read.ts"; import { loadLocalProjectContext } from "../local-project-context.ts"; import { hasConfiguredBuckets, seedBucketsRun } from "../seed-buckets.ts"; import { awaitStorageReady } from "./await-storage-ready.ts"; @@ -42,10 +42,12 @@ import { stackOpenReadyProject } from "../stack-local-database.ts"; import { classifyStorageCapability, describeStorageCapability, + enableStorageAdvice, StackStorageCapabilityError, StackStorageUnavailableError, stackStorageEndpointFor, } from "../stack-storage.ts"; +import { configFileRefOf } from "../config-advice.ts"; import { loadStackConfig } from "../stack-config.ts"; import { readSelectionOrBundled } from "../stack-lock.ts"; import { catalogDatabaseServices, StackCatalogSetup } from "../stack-catalog-setup.ts"; @@ -116,7 +118,7 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu const experimental = yield* resolveExperimentalWithProjectEnv(projectEnv); // Abort on a bad config before wiping the local database. - yield* checkDbToml(fs, path, workdir); + yield* checkDbConfig(fs, path, workdir); yield* Effect.annotateCurrentSpan("stack.backend", backend.kind); if (backend.kind === "stack") { @@ -128,7 +130,7 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu const config = yield* loadStackConfig(workdir, { selection: readSelectionOrBundled }).pipe( Effect.mapError((cause) => resetFailed(cause.message, cause)), ); - const toml = yield* readDbToml(fs, path, workdir); + const toml = yield* readDbConfig(fs, path, workdir); const databaseStatus = yield* opened.value.database.status.pipe( Effect.mapError((cause) => resetFailed(`failed to inspect stack: ${cause.message}`)), ); @@ -235,18 +237,19 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu yield* Effect.gen(function* () { const context = yield* loadLocalProjectContext(workdir, (message) => resetFailed(message)); if (!hasConfiguredBuckets(context.config)) return; + const configFile = configFileRefOf(path, workdir, context.resolvedConfig); const capability = storage === undefined ? undefined : yield* storage.status; const decision = classifyStorageCapability(capability); if (decision !== "proceed") { yield* skipSeeding( describeStorageCapability(capability), decision === "disabled" - ? "Set [storage] enabled = true in supabase/config.toml, run supabase stack stop followed by supabase stack start without -x storage, then supabase seed buckets." + ? `${enableStorageAdvice(configFile)}, run supabase stack stop followed by supabase stack start without -x storage, then supabase seed buckets.` : undefined, ); return; } - const credentials = yield* stackStorageEndpointFor(opened.value.stack); + const credentials = yield* stackStorageEndpointFor(opened.value.stack, configFile); yield* seedBucketsRun({ projectRef: "", emitSummary: false, @@ -259,6 +262,7 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu }, projectEnvValues: projectEnv, workdir, + configFile, }); }).pipe( Effect.catch((error) => skipSeeding(error.message, suggestionOf(error))), @@ -354,6 +358,7 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu resolvedConfig: { config, document: resolvedConfig.loaded.document }, // The same nested-dotenv walk already resolved for `yes`/`experimental` above. projectEnvValues: projectEnv, + configFile: configFileRefOf(path, workdir, resolvedConfig), }).pipe( // An invalid bucket entry (bad name, unparseable `file_size_limit`, …) can't abort here — // recreate already dropped/rebuilt the DB — so warn and skip buckets instead. diff --git a/apps/cli/src/command-internal/db-bootstrap/restart-services.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/restart-services.unit.test.ts index 3d861c826f..66f824418b 100644 --- a/apps/cli/src/command-internal/db-bootstrap/restart-services.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/restart-services.unit.test.ts @@ -17,7 +17,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push(args); const result = handler(args); @@ -122,7 +122,7 @@ describe("restartServicesAndReloadKong", () => { const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; if (args[0] === "restart") { restarted.push(args[1] ?? ""); inFlight++; diff --git a/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts index ff072f9066..35f8c3d5c3 100644 --- a/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts @@ -33,7 +33,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push(args); const result = opts.respond?.(args) ?? opts; diff --git a/apps/cli/src/command-internal/db-bootstrap/shadow-database.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/shadow-database.unit.test.ts index 0467e64888..3c0a6575c8 100644 --- a/apps/cli/src/command-internal/db-bootstrap/shadow-database.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/shadow-database.unit.test.ts @@ -110,7 +110,7 @@ function mockSpawner() { const encoder = new TextEncoder(); const spawner = ChildProcessSpawner.make((command) => Effect.sync(() => { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push(args); // `ensureNetwork` probes with `network inspect` first; report it as missing so // `createShadowDatabase` actually reaches `network create` instead of short-circuiting. diff --git a/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts b/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts index 37ac986ec0..082fdb4ea6 100644 --- a/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/start-local-database.ts @@ -15,7 +15,7 @@ import { Output } from "../../shared/output/output.service.ts"; import { RuntimeInfo } from "../../shared/runtime/runtime-info.service.ts"; import { NetworkIdFlag, resolveDebugWithProjectEnv } from "../global-flags.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; -import { checkDbToml } from "../db-config.toml-read.ts"; +import { checkDbConfig } from "../db-config.read.ts"; import { DbConfigLoadError } from "../db-config.errors.ts"; import { resolveAuthExternalProviders, @@ -28,6 +28,7 @@ import { parseUrl } from "../storage-url.ts"; import { loadLocalProjectContext } from "../local-project-context.ts"; import { cliProjectFilterValue, localDbContainerId } from "../docker-ids.ts"; import { isBitbucketPipeline } from "../bitbucket-pipeline.ts"; +import { dockerSupportsHealthStartInterval } from "../container-cli.ts"; import { holdProjectLifecycleLock } from "../project-lifecycle-lock.ts"; import { dbNotRunningError } from "../status-errors.ts"; import { buildLocalDbContainerInputs } from "./local-container-inputs.ts"; @@ -75,9 +76,9 @@ const startLocalDatabaseInScope = Effect.fn("DbBootstrap.startLocalDatabase")(fu // Config is loaded first thing: a missing config is tolerated (defaults), but a present config // that is malformed, references an undecryptable `encrypted:` secret, or fails validation - // aborts before any container work. `checkDbToml` does that load+validate, not + // aborts before any container work. `checkDbConfig` does that load+validate, not // `isLocalDbRunning`'s best-effort read, which swallows config errors. - const dbTomlValues = yield* checkDbToml(fs, path, cliSettings.workdir); + const dbTomlValues = yield* checkDbConfig(fs, path, cliSettings.workdir); // Threaded into `rollbackDatabaseStart`'s own teardown (gates its `Pruned …:` stderr reports) and into // `buildLocalDbContainerInputs`'s own `setup.debug`, so a failed fresh-volume migrate job tees @@ -172,6 +173,7 @@ const startLocalDatabaseInScope = Effect.fn("DbBootstrap.startLocalDatabase")(fu hostnameForValidation, cliSettings.workdir, document, + resolvedConfig.hasConfigFile ? resolvedConfig.loaded.path : undefined, ).pipe(Effect.mapError((cause) => new DbConfigLoadError({ message: cause.message }))); // Held until this call returns, after any rollback has run. @@ -267,7 +269,10 @@ const startLocalDatabaseInScope = Effect.fn("DbBootstrap.startLocalDatabase")(fu hostname, dbContainerId, dbPort: values.dbPort, - containerOpts, + containerOpts: { + ...containerOpts, + healthStartInterval: yield* dockerSupportsHealthStartInterval(spawner), + }, // `db reset` has no `fromBackup` concept at all, so `postgresSpecBase` omits it. postgresSpec: { ...postgresSpecBase, fromBackup }, // Only the `db` container's own image, resolved lazily, right where it would be diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json index dce71ff5c6..cbb5b4258a 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-project-id-env.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": true, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_shell-project-id", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json index 4272d72177..020b32c15a 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote-with-shell-env.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": false, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_abcdefghijklmnopqrst", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json index 980df2596f..83e8204e77 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/matched-remote.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": true, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_abcdefghijklmnopqrst", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json index 2f86ba9c79..e7cfb518fa 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/no-overrides.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": true, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_golden-shadow", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json index 3d85d0db5a..95d14e1655 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/project-dotenv.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": false, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_golden-shadow", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, @@ -145,6 +145,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json index a49d200a86..913d7a795f 100644 --- a/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json +++ b/apps/cli/src/command-internal/db-bootstrap/testdata/config-precedence/local-container-inputs/shell-env.json @@ -5,8 +5,8 @@ "dbHealthTimeoutSeconds": 120, "majorVersion": 17, "orioledbVersion": null, - "postgresConfigImage": "supabase/postgres:17.11.0.011", - "postgresImage": "supabase/postgres:17.11.0.011", + "postgresConfigImage": "", + "postgresImage": "", "realtimeEnabledForSetup": false, "realtimeIpVersion": "IPv4", "realtimeMaxHeaderLength": 4096, @@ -32,7 +32,7 @@ "experimental": false, "networkId": "supabase_network_golden-shadow", "postgresSpecBase": { - "configImage": "supabase/postgres:17.11.0.011", + "configImage": "", "db": { "health_timeout": "2m", "major_version": 17, @@ -138,6 +138,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/db-config.errors.ts b/apps/cli/src/command-internal/db-config.errors.ts index 530baed442..9772a5fc57 100644 --- a/apps/cli/src/command-internal/db-config.errors.ts +++ b/apps/cli/src/command-internal/db-config.errors.ts @@ -1,6 +1,8 @@ import { Data } from "effect"; import { actionability, + CliErrorCategory, + CliErrorKind, type CliErrorActionabilityDeclaration, CliSuggestionType, ErrorActionabilityId, @@ -74,6 +76,23 @@ export class DbConfigLoginRoleStatusError extends Data.TaggedError("DbConfigLogi } } +/** + * The Management API could not mint a login role because the project database lacks + * `supabase_privileged_role` (Postgres `42501`); the dedicated tag isolates these projects in telemetry. + */ +export class DbConfigLoginRoleMissingPrivilegedRoleError extends Data.TaggedError( + "DbConfigLoginRoleMissingPrivilegedRoleError", +)<{ readonly message: string; readonly suggestion: string; readonly projectRef: string }> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return { + error_kind: CliErrorKind.ExternalService, + error_category: CliErrorCategory.ApiStatus, + has_suggestion: true, + suggestion_type: CliSuggestionType.SetEnvVar, + }; + } +} + /** Transport failure listing network bans (`V1ListAllNetworkBans`). */ export class DbConfigListBansNetworkError extends Data.TaggedError("DbConfigListBansNetworkError")<{ readonly message: string; diff --git a/apps/cli/src/command-internal/db-config.integration.test.ts b/apps/cli/src/command-internal/db-config.integration.test.ts index 192a7440c6..a972f6e8f2 100644 --- a/apps/cli/src/command-internal/db-config.integration.test.ts +++ b/apps/cli/src/command-internal/db-config.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Cause, @@ -85,8 +85,15 @@ import type { DbConnectionInput } from "./db-config.types.ts"; import { hostedMutationPreapproval } from "./hosted-consent/hosted-consent.ts"; import type { HostedMutationConsent } from "./hosted-consent/hosted-consent.token.ts"; import type { ResolvedTarget } from "./target/resolve-target.ts"; +import { + SUGGEST_CA_VERIFICATION, + SUGGEST_ENV_VAR, + SUGGEST_NETWORK_RESTRICTIONS, +} from "./connect-errors.ts"; import { DbConnectError } from "./db-connection.errors.ts"; +import { toConnectError } from "./db-connection.sql-pg.layer.ts"; import { DbConnection, type DbSession, type PgConnInput } from "./db-connection.service.ts"; +import { rootCaBundle } from "./supabase-tls.ts"; import { StackApi } from "./stack-api.ts"; import { stackBackendLayer } from "./stack-backend.ts"; import { selection } from "../../tests/helpers/pins.ts"; @@ -279,7 +286,7 @@ describe("dbConfigResolver (local + db-url)", () => { const dir = yield* withWorkdir('[api]\nschemas = ["app", "public"]'); const result = yield* resolveConnection(dir, inputFor(target)); expect(result.exposedSchemas).toEqual(["app", "public"]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("local mode: uses 127.0.0.1 with config.toml db.port/password and is local", () => @@ -308,7 +315,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -326,7 +333,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -344,7 +351,7 @@ describe("dbConfigResolver (local + db-url)", () => { ); const r = yield* resolveConnection(dir, inputFor(localTarget)); expect(r.conn.suggestionContext).toMatchObject({ linkedCheckout: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect( @@ -356,7 +363,7 @@ describe("dbConfigResolver (local + db-url)", () => { projectId: "abcdefghijklmnopqrst", }); expect(r.conn.suggestionContext).toMatchObject({ linkedCheckout: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("local mode on an unlinked checkout adds no --linked pointer", () => @@ -364,7 +371,7 @@ describe("dbConfigResolver (local + db-url)", () => { const dir = yield* withWorkdir(); const r = yield* resolveConnection(dir, inputFor(localTarget)); expect(r.conn.suggestionContext?.linkedCheckout).toBeUndefined(); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("local mode: falls back to default port/password without a config.toml", () => @@ -380,7 +387,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -409,7 +416,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -427,7 +434,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -445,7 +452,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -465,7 +472,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -485,7 +492,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -506,7 +513,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -527,7 +534,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -554,7 +561,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -573,7 +580,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }); @@ -595,7 +602,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ), ); @@ -625,7 +632,7 @@ describe("dbConfigResolver (local + db-url)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -741,14 +748,14 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { stackApi: projectStackApi(dir, "running"), }).pipe(Effect.provide(stackBackendLayer("stack"))); expect(resolved.exposedSchemas).toEqual(["app", "public"]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("treats the url printed by `stack status --env` as the local database", () => Effect.gen(function* () { const resolved = yield* resolveOnStack(yield* withWorkdir(), stackUrl(STACK_SQL_PORT)); expect(resolved.isLocal).toBe(true); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("fills a passwordless stack url from the stack's credentials, not [db].password", () => @@ -760,14 +767,14 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { ); expect(resolved.isLocal).toBe(true); expect(resolved.conn.password).toBe("stack-password"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); - it.effect("keeps a loopback url on another port remote so it still requires TLS", () => + it.effect("keeps a loopback url on another port remote so it still negotiates TLS", () => Effect.gen(function* () { const resolved = yield* resolveOnStack(yield* withWorkdir(), stackUrl(STACK_SQL_PORT + 1)); expect(resolved.isLocal).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("fills from the stack's credentials when the stack port is also [db].port", () => @@ -781,7 +788,7 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { ); expect(resolved.isLocal).toBe(true); expect(resolved.conn.password).toBe("stack-password"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); for (const state of ["unregistered", "stopped"] as const) { @@ -794,7 +801,7 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { projectStackApi(dir, state), ); expect(resolved.isLocal).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); } @@ -806,7 +813,7 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { ); expect(resolved.conn.fallbacks).toEqual([{ host: "db.example.com", port: 5432 }]); expect(resolved.isLocal).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("does not consult the stack for a non-loopback host", () => @@ -824,7 +831,7 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { untouchedStackApi, ); expect(resolved.isLocal).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); }); @@ -861,7 +868,7 @@ describe("dbConfigResolver (linked config ordering)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }); @@ -984,6 +991,8 @@ describe("dbConfigResolver (linked config ordering)", () => { user: `cli_login_role.${adHocRef}`, password: "temporary-role-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), suggestionContext: { dashboardUrl: "https://supabase.com/dashboard", profileName: "supabase", @@ -1008,6 +1017,8 @@ describe("dbConfigResolver (linked config ordering)", () => { user: `cli_login_role.${adHocRef}`, password: "temporary-role-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), }, isLocal: false, dnsResolver: "native", @@ -1021,7 +1032,7 @@ describe("dbConfigResolver (linked config ordering)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }); @@ -1149,6 +1160,8 @@ describe("dbConfigResolver (linked config ordering)", () => { user: `cli_login_role.${adHocRef}`, password: "temporary-role-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), suggestionContext: { dashboardUrl: "https://supabase.com/dashboard", profileName: "supabase", @@ -1173,6 +1186,8 @@ describe("dbConfigResolver (linked config ordering)", () => { user: `cli_login_role.${adHocRef}`, password: "temporary-role-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), }, isLocal: false, dnsResolver: "native", @@ -1186,7 +1201,7 @@ describe("dbConfigResolver (linked config ordering)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -1293,6 +1308,8 @@ describe("dbConfigResolver (linked config ordering)", () => { user: `postgres.${linkedRef}`, password: "linked-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), suggestionContext: { dashboardUrl: "https://supabase.com/dashboard", profileName: "supabase", @@ -1314,7 +1331,7 @@ describe("dbConfigResolver (linked config ordering)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }); }); @@ -1403,6 +1420,8 @@ describe("dbConfigResolver (an explicit --project-ref fetches the pooler config) user: `postgres.${ref}`, password: "ambient-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), suggestionContext: { dashboardUrl: "https://supabase.com/dashboard", profileName: "supabase", @@ -1421,7 +1440,7 @@ describe("dbConfigResolver (an explicit --project-ref fetches the pooler config) }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -1521,6 +1540,8 @@ describe("dbConfigResolver (an explicit --project-ref fetches the pooler config) user: `postgres.${targetRef}`, password: "flag-password", database: "postgres", + sslmode: "require", + sslrootcertInline: rootCaBundle(), suggestionContext: { dashboardUrl: "https://supabase.com/dashboard", profileName: "supabase", @@ -1539,7 +1560,7 @@ describe("dbConfigResolver (an explicit --project-ref fetches the pooler config) }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -1589,7 +1610,7 @@ describe("dbConfigResolver (an explicit --project-ref fetches the pooler config) }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -1640,7 +1661,7 @@ describe("dbConfigResolver (an explicit --project-ref fetches the pooler config) }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -1673,7 +1694,7 @@ describe("dbConfigResolver (Management API calls)", () => { const r = yield* resolveConnection(dir, inputFor(localTarget)); expect(r.isLocal).toBe(true); noApiCalls(requests, { local: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("--db-url makes no Management API request and the block declares none", () => { @@ -1687,7 +1708,7 @@ describe("dbConfigResolver (Management API calls)", () => { ); expect(r.isLocal).toBe(false); noApiCalls(requests, { "db-url": true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect( @@ -1729,7 +1750,7 @@ describe("dbConfigResolver (Management API calls)", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -1781,7 +1802,7 @@ describe("dbConfigResolver (Management API calls)", () => { `/v1/projects/${ref}/cli/login-role`, ); assertDbConfigDrift(requests, { linked: true, "project-ref": ref, password: "" }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); for (const forbidden of ["retrieve", "delete"] as const) { @@ -1862,7 +1883,7 @@ describe("dbConfigResolver (Management API calls)", () => { : { method: "DELETE", path: `/v1/projects/${ref}/network-bans` }, ); assertDbConfigDrift(requests, { linked: true, "project-ref": ref }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }, ); } @@ -1919,6 +1940,8 @@ describe("dbConfigResolver (container pooler fallback)", () => { host: "aws-0-us-east-1.pooler.supabase.com", user: `cli_login_role.${ref}`, password: "temporary-role-password", + sslmode: "require", + sslrootcertInline: rootCaBundle(), }); } expect(requests).toEqual([ @@ -1926,7 +1949,57 @@ describe("dbConfigResolver (container pooler fallback)", () => { { method: "POST", path: loginRolePath }, ]); assertFallbackDrift(requests, { linked: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }); + + it.effect( + "reports a missing supabase_privileged_role when the login-role mint fails with 42501", + () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, (method, path) => { + if (method === "GET" && path === poolerPath) + return Response.json(primaryPoolerConfigs(ref)); + if (method === "POST" && path === loginRolePath) { + return Response.json( + { message: "Failed to create login role: permission denied to alter role" }, + { status: 400 }, + ); + } + return undefined; + }); + const dir = yield* withWorkdir(); + const error = yield* resolvePoolerFallback(dir, inputFor(explicitRefTarget(ref)), { + dbConnection: connects, + configEnv: { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }, + }).pipe(Effect.flip); + expect(error._tag).toBe("DbConfigLoginRoleMissingPrivilegedRoleError"); + expect(error).toMatchObject({ suggestion: expect.stringContaining("--password") }); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }, + ); + + it.effect("keeps the generic status error for an unrelated login-role 400", () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, (method, path) => { + if (method === "GET" && path === poolerPath) + return Response.json(primaryPoolerConfigs(ref)); + if (method === "POST" && path === loginRolePath) { + return Response.json( + { message: "42501: permission denied for table x" }, + { status: 400 }, + ); + } + return undefined; + }); + const dir = yield* withWorkdir(); + const error = yield* resolvePoolerFallback(dir, inputFor(explicitRefTarget(ref)), { + dbConnection: connects, + configEnv: { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }, + }).pipe(Effect.flip); + expect(error._tag).toBe("DbConfigLoginRoleStatusError"); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("fetches the pooler config but mints no login role when a password is set", () => { @@ -1948,7 +2021,7 @@ describe("dbConfigResolver (container pooler fallback)", () => { ); expect(requests).toEqual([{ method: "GET", path: poolerPath }]); assertFallbackDrift(requests, { linked: true, password: "secret" }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("returns no pooler when the pooler config is forbidden", () => { @@ -1967,7 +2040,7 @@ describe("dbConfigResolver (container pooler fallback)", () => { expect(Option.isNone(pooler)).toBe(true); expect(requests).toEqual([{ method: "GET", path: poolerPath }]); assertFallbackDrift(requests, { linked: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("--local returns no pooler and makes no Management API request", () => { @@ -1979,7 +2052,7 @@ describe("dbConfigResolver (container pooler fallback)", () => { expect(Option.isNone(pooler)).toBe(true); expect(requests).toEqual([]); assertFallbackDrift(requests, { local: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); }); @@ -2023,6 +2096,8 @@ describe("resolveLinkedConn libpq TLS", () => { readonly reachable?: boolean; readonly password?: Option.Option; readonly poolerUrl?: string; + readonly profile?: string; + readonly projectHost?: string; } = {}, ) => withConfigEnv( @@ -2032,11 +2107,16 @@ describe("resolveLinkedConn libpq TLS", () => { return yield* resolveLinkedConn( ref, opts.workdir ?? (yield* projectWorkdir([])), - "invalid", + opts.profile ?? "supabase", + opts.projectHost ?? "invalid", "supabase.com", "native", opts.password ?? Option.some("secret"), - { removeNetworkBans: false, poolerUrl: Option.fromNullishOr(opts.poolerUrl) }, + { + removeNetworkBans: false, + poolerUrl: Option.fromNullishOr(opts.poolerUrl), + dashboardUrl: "https://supabase.com/dashboard", + }, ); }), ).pipe(Effect.scoped); @@ -2118,6 +2198,163 @@ describe("resolveLinkedConn libpq TLS", () => { }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), ); + const pinned = { sslmode: "require", sslrootcertInline: rootCaBundle() }; + const serviceFileWith = (content: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const dir = yield* fs.makeTempDirectoryScoped({ prefix: "cli-linked-service-" }); + const serviceFile = path.join(dir, "pg_service.conf"); + yield* fs.writeFileString(serviceFile, content); + return { PGSERVICE: "svc", PGSERVICEFILE: serviceFile }; + }); + const hostedPoolerUrl = `postgres://postgres.${ref}:[YOUR-PASSWORD]@${poolerHost}:6543/postgres`; + + describe("Supabase CA pin", () => { + for (const [profile, projectHost] of [ + ["supabase", "supabase.co"], + ["supabase-staging", "supabase.red"], + ["snap", "snapcloud.dev"], + ] as const) { + it.effect(`pins the bundled CA on the ${profile} direct host when TLS is unset`, () => + Effect.gen(function* () { + const conn = yield* linkedConn({}, { profile, projectHost }); + expect(conn).toMatchObject({ host: `db.${ref}.${projectHost}`, ...pinned }); + }).pipe(Effect.provide(linkedConnLayer)), + ); + } + + it.effect("pins the bundled CA on the direct host before minting a login role", () => + Effect.gen(function* () { + const conn = yield* linkedConn( + {}, + { projectHost: "supabase.co", password: Option.some("") }, + ); + expect(conn).toMatchObject({ user: "cli_login_x", password: "temp", ...pinned }); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("keeps verify-full with the user's own root cert", () => + Effect.gen(function* () { + const conn = yield* linkedConn( + { PGSSLMODE: "verify-full", PGSSLROOTCERT: "/ca.pem" }, + { projectHost: "supabase.co" }, + ); + expect(conn).toMatchObject({ sslmode: "verify-full", sslrootcert: "/ca.pem" }); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("keeps PGSSLMODE=disable", () => + Effect.gen(function* () { + const conn = yield* linkedConn({ PGSSLMODE: "disable" }, { projectHost: "supabase.co" }); + expect(conn.sslmode).toBe("disable"); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("verifies against a PGSSLROOTCERT with no sslmode instead of the bundled CA", () => + Effect.gen(function* () { + const conn = yield* linkedConn( + { PGSSLROOTCERT: "/ca.pem" }, + { projectHost: "supabase.co" }, + ); + expect(conn).toMatchObject({ sslmode: "require", sslrootcert: "/ca.pem" }); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("keeps a service-file sslmode", () => + Effect.gen(function* () { + const env = yield* serviceFileWith("[svc]\nsslmode=require\n"); + const conn = yield* linkedConn(env, { projectHost: "supabase.co" }); + expect(conn.sslmode).toBe("require"); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("pins the bundled CA on the pooler when TLS is unset", () => + Effect.gen(function* () { + const conn = yield* linkedConn( + {}, + { reachable: false, poolerUrl: hostedPoolerUrl, password: Option.some("secret") }, + ); + expect(conn).toMatchObject({ host: poolerHost, ...pinned }); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("keeps an explicit sslmode in the saved pooler URL", () => + Effect.gen(function* () { + const conn = yield* linkedConn( + {}, + { + reachable: false, + poolerUrl: `${hostedPoolerUrl}?sslmode=disable`, + password: Option.some("secret"), + }, + ); + expect(conn.sslmode).toBe("disable"); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("pins the temp-role probe connection through the pooler", () => { + const probed: Array = []; + const probeSession: DbSession = { + exec: () => Effect.void, + execBatch: () => Effect.void, + query: () => Effect.succeed([]), + extensionExists: () => Effect.succeed(false), + copyToCsv: () => Effect.succeed(new Uint8Array()), + queryRaw: () => Effect.succeed({ fields: [], rows: [], commandTag: "SELECT 1" }), + }; + return Effect.gen(function* () { + const conn = yield* linkedConn( + {}, + { reachable: false, poolerUrl: hostedPoolerUrl, password: Option.some("") }, + ); + expect(probed).toHaveLength(1); + expect(probed[0]).toMatchObject({ + host: poolerHost, + user: `cli_login_x.${ref}`, + ...pinned, + }); + expect(conn).toMatchObject(pinned); + }).pipe( + Effect.scoped, + Effect.provideService(DbConnection, { + connect: (conn) => { + probed.push(conn); + return Effect.succeed(probeSession); + }, + }), + Effect.provide(linkedConnLayer), + ); + }); + + for (const [profile, projectHost] of [ + ["supabase-local", "supabase.red"], + ["my-profile", "supabase.co"], + ] as const) { + it.effect(`does not pin the ${profile} profile`, () => + Effect.gen(function* () { + const conn = yield* linkedConn({}, { profile, projectHost }); + expect(conn.host).toBe(`db.${ref}.${projectHost}`); + expect(conn.sslmode).toBeUndefined(); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.provide(linkedConnLayer)), + ); + } + + it.effect("does not pin a host outside the profile's domains", () => + Effect.gen(function* () { + const conn = yield* linkedConn({}, { profile: "supabase", projectHost: "supabase.red" }); + expect(conn.sslmode).toBeUndefined(); + expect(conn.sslrootcertInline).toBeUndefined(); + }).pipe(Effect.provide(linkedConnLayer)), + ); + }); + it.effect("names the libpq variables when PGSSLMODE is invalid on a reachable direct host", () => Effect.gen(function* () { const error = yield* linkedConn({ PGSSLMODE: "verifyfull" }).pipe(Effect.flip); @@ -2230,7 +2467,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(r.isLocal).toBe(true); expect(r.target).toEqual({ kind: "local" }); expect(output.stderrText).toBe("Target: local stack (127.0.0.1:55555)\n"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("db-url target announces the host and never the credentials", () => @@ -2242,7 +2479,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(r.conn).toMatchObject({ host: "db.example.com", port: 6543, user: "alice" }); expect(r.isLocal).toBe(false); expect(output.stderrText).toBe("Target: custom Postgres (db.example.com:6543)\n"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("a db-url that fails to parse echoes the url with the password redacted", () => @@ -2252,7 +2489,7 @@ describe("dbConfigResolver.resolveConnection", () => { const exit = yield* Effect.exit(resolveConnection(dir, inputFor({ kind: "db-url", url }))); expect(failureText(exit)).toContain("DbConfigParseUrlError"); expect(failureText(exit)).not.toContain("secret"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); const directTargets: ReadonlyArray = [ @@ -2272,14 +2509,14 @@ describe("dbConfigResolver.resolveConnection", () => { ); expect(failureText(exit)).toContain("TargetPasswordConflictError"); expect(output.stderrText).toBe(""); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect(`accepts an empty --password for a ${label} target`, () => Effect.gen(function* () { const dir = yield* withWorkdir(); yield* resolveConnection(dir, inputFor(target, { password: Option.some("") })); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); } @@ -2309,7 +2546,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(r.target).toEqual(target); expect(requests).toEqual([]); expect(output.stderrText).toBe(""); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect( @@ -2338,7 +2575,7 @@ describe("dbConfigResolver.resolveConnection", () => { { method: "GET", path: `/v1/projects/${ref}/config/database/pooler` }, ]); assertDbConfigDrift(requests, { remote: true }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }, ); @@ -2376,7 +2613,7 @@ describe("dbConfigResolver.resolveConnection", () => { }), ), Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), ); }, ); @@ -2415,7 +2652,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(requests).toEqual([ { method: "GET", path: `/v1/projects/${ref}/config/database/pooler` }, ]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); describe("network bans on the temporary login role", () => { @@ -2447,19 +2684,23 @@ describe("dbConfigResolver.resolveConnection", () => { } return undefined; }); - const failingConnect = (failures: number) => { + const failingConnect = ( + failures: number, + failure: (cfg: PgConnInput) => DbConnectError = () => + new DbConnectError({ message: "password authentication failed" }), + ) => { const attempts = { count: 0 }; const layer = Layer.succeed(DbConnection, { - connect: () => + connect: (cfg) => Effect.suspend(() => { attempts.count += 1; - return attempts.count <= failures - ? Effect.fail(new DbConnectError({ message: "password authentication failed" })) - : Effect.succeed(session); + return attempts.count <= failures ? Effect.fail(failure(cfg)) : Effect.succeed(session); }), }); return { attempts, layer }; }; + const driverFailure = (code: string, message: string) => (cfg: PgConnInput) => + toConnectError(cfg, false, Object.assign(new Error(message), { code })); const forkResolve = ( input: DbConnectionInput, connection: Layer.Layer, @@ -2515,10 +2756,130 @@ describe("dbConfigResolver.resolveConnection", () => { ); expect(connection.attempts.count).toBe(9); expect(requests.some((request) => request.method === "DELETE")).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }, ); + it.effect( + "keeps looking for bans when the dial fails, as a ban surfaces as a dial failure", + () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + const connection = failingConnect( + Number.POSITIVE_INFINITY, + driverFailure("ETIMEDOUT", "connect ETIMEDOUT 203.0.113.9:6543"), + ); + return Effect.gen(function* () { + yield* bannedRequests(requests, ["203.0.113.7"]); + const { fiber, advance } = yield* forkResolve( + inputFor(linkedTarget(ref, { source: "flag", matchesLinkedRef: false })), + connection.layer, + ); + yield* advance(8); + const exit = yield* Fiber.await(fiber); + expect(failureText(exit)).toContain("LinkedDbNetworkBannedError"); + expect(connection.attempts.count).toBe(9); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }, + ); + + it.effect("fails at once on a rejected server certificate and names the CA setting", () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + const connection = failingConnect( + Number.POSITIVE_INFINITY, + driverFailure("SELF_SIGNED_CERT_IN_CHAIN", "self-signed certificate in certificate chain"), + ); + return Effect.gen(function* () { + yield* bannedRequests(requests, ["203.0.113.7"]); + const { fiber } = yield* forkResolve( + inputFor(linkedTarget(ref, { source: "flag", matchesLinkedRef: false })), + connection.layer, + ); + const exit = yield* Fiber.await(fiber); + const error = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(error).toEqual( + Option.some( + expect.objectContaining({ + _tag: "DbConfigConnectTempRoleError", + message: expect.stringContaining("tls error (self-signed certificate"), + suggestion: SUGGEST_CA_VERIFICATION, + }), + ), + ); + expect(SUGGEST_CA_VERIFICATION).not.toBe(SUGGEST_ENV_VAR); + expect(connection.attempts.count).toBe(1); + expect(requests.some((request) => request.path.includes("network-bans"))).toBe(false); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }); + + it.effect("stops retrying authentication failures once the certificate is rejected", () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + const connection = failingConnect(Number.POSITIVE_INFINITY, (cfg) => + connection.attempts.count <= 2 + ? new DbConnectError({ message: "password authentication failed" }) + : driverFailure("CERT_HAS_EXPIRED", "certificate has expired")(cfg), + ); + return Effect.gen(function* () { + yield* bannedRequests(requests, []); + const { fiber, advance } = yield* forkResolve( + inputFor(linkedTarget(ref, { source: "flag", matchesLinkedRef: false })), + connection.layer, + ); + yield* advance(2); + const exit = yield* Fiber.await(fiber); + expect(failureText(exit)).toContain("DbConfigConnectTempRoleError"); + expect(connection.attempts.count).toBe(3); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }); + + it.effect("fails at once on an IP rejected by network restrictions and lists no bans", () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + const connection = failingConnect( + Number.POSITIVE_INFINITY, + () => + new DbConnectError({ + message: "failed to connect to postgres: Address not in tenant allow_list", + networkRestricted: true, + }), + ); + return Effect.gen(function* () { + yield* bannedRequests(requests, ["203.0.113.7"]); + const { fiber } = yield* forkResolve( + inputFor(linkedTarget(ref, { source: "flag", matchesLinkedRef: false })), + connection.layer, + ); + const exit = yield* Fiber.await(fiber); + const error = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(error).toEqual( + Option.some( + expect.objectContaining({ + _tag: "DbConfigConnectTempRoleError", + message: expect.stringContaining("Address not in tenant allow_list"), + suggestion: `${SUGGEST_NETWORK_RESTRICTIONS}\nhttps://supabase.com/dashboard/project/_/database/settings`, + }), + ), + ); + expect(connection.attempts.count).toBe(1); + expect(requests.some((request) => request.path.includes("network-bans"))).toBe(false); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }); + + it.effect("retries a plain authentication failure to the end and lists no ban", () => { + const requests: Array<{ readonly method: string; readonly path: string }> = []; + const connection = failingConnect(Number.POSITIVE_INFINITY); + return Effect.gen(function* () { + yield* bannedRequests(requests, []); + const { fiber, advance } = yield* forkResolve( + inputFor(linkedTarget(ref, { source: "flag", matchesLinkedRef: false })), + connection.layer, + ); + yield* advance(8); + const exit = yield* Fiber.await(fiber); + expect(failureText(exit)).toContain("DbConfigConnectTempRoleError"); + expect(connection.attempts.count).toBe(9); + expect(requests.some((request) => request.method === "DELETE")).toBe(false); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); + }); + it.effect("connects without removing bans that are not blocking the caller", () => { const requests: Array<{ readonly method: string; readonly path: string }> = []; const connection = failingConnect(3); @@ -2533,7 +2894,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(r.conn).toMatchObject({ user: `cli_login_role.${ref}` }); expect(connection.attempts.count).toBe(4); expect(requests.some((request) => request.method === "DELETE")).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("keeps retrying without removing anything when no ban is listed", () => { @@ -2550,7 +2911,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(r.conn).toMatchObject({ user: `cli_login_role.${ref}` }); expect(connection.attempts.count).toBe(4); expect(requests.some((request) => request.method === "DELETE")).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("removes the bans and says so once the caller holds a --yes approval", () => { @@ -2576,7 +2937,7 @@ describe("dbConfigResolver.resolveConnection", () => { }); expect(output.stderrText).toContain("Removing network bans (203.0.113.7)...\n"); assertDbConfigDrift(requests, { linked: true, "project-ref": ref }); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("does not remove bans for an approval obtained by prompting", () => { @@ -2599,7 +2960,7 @@ describe("dbConfigResolver.resolveConnection", () => { const exit = yield* Fiber.await(fiber); expect(failureText(exit)).toContain("LinkedDbNetworkBannedError"); expect(requests.some((request) => request.method === "DELETE")).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect( @@ -2629,7 +2990,7 @@ describe("dbConfigResolver.resolveConnection", () => { ); expect(failureText(exit)).not.toContain("--yes"); expect(connection.attempts.count).toBe(9); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }, ); @@ -2649,7 +3010,7 @@ describe("dbConfigResolver.resolveConnection", () => { expect(failureText(exit)).toContain("DbConfigConnectTempRoleError"); expect(failureText(exit)).not.toContain("--yes"); expect(requests.some((request) => request.method === "DELETE")).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); it.effect("keeps a failed removal when a later listing fails under --yes", () => { @@ -2688,7 +3049,7 @@ describe("dbConfigResolver.resolveConnection", () => { const exit = yield* Fiber.await(fiber); expect(failureText(exit)).toContain("DbConfigUnbanStatusError"); expect(listCalls).toBeGreaterThan(1); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); }); }); }); diff --git a/apps/cli/src/command-internal/db-config.layer.ts b/apps/cli/src/command-internal/db-config.layer.ts index f42092c3fe..99f4ee6c1c 100644 --- a/apps/cli/src/command-internal/db-config.layer.ts +++ b/apps/cli/src/command-internal/db-config.layer.ts @@ -1,7 +1,6 @@ import * as net from "node:net"; -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { - Crypto, Duration, Effect, FileSystem, @@ -12,6 +11,7 @@ import { Redacted, Struct, } from "effect"; +import { Digest } from "effect/crypto"; import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; import { CliArgs } from "../shared/cli/cli-args.service.ts"; @@ -33,7 +33,12 @@ import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { Tty } from "../shared/runtime/tty.service.ts"; import { Analytics } from "../shared/telemetry/analytics.service.ts"; import { TelemetryRuntime } from "../shared/telemetry/runtime.service.ts"; -import { type ConnectSuggestionContext, SUGGEST_ENV_VAR } from "./connect-errors.ts"; +import { + type ConnectSuggestionContext, + SUGGEST_CA_VERIFICATION, + SUGGEST_ENV_VAR, + suggestNetworkRestrictions, +} from "./connect-errors.ts"; import { DbConnection, type PgConnInput } from "./db-connection.service.ts"; import { IdentityStitch } from "./identity-stitch.ts"; import { @@ -49,7 +54,7 @@ import { redactConnectionString, } from "./db-config.parse.ts"; import { DbConfigResolver, type DbConfigError } from "./db-config.service.ts"; -import { loadProjectEnvValues, readDbToml } from "./db-config.toml-read.ts"; +import { loadProjectEnvValues, readDbConfig } from "./db-config.read.ts"; import type { DbConnectionInput } from "./db-config.types.ts"; import { DebugLogger } from "../shared/output/debug-logger.service.ts"; import { getHostname } from "./hostname.ts"; @@ -57,6 +62,7 @@ import { mapHttpError } from "./http-errors.ts"; import { currentStackBackend } from "./stack-backend.ts"; import { StackApi, stackApiLayer } from "./stack-api.ts"; import { stackLocalDatabaseConn } from "./stack-local-database.ts"; +import { pinSupabaseTls } from "./supabase-tls.ts"; import { block, type OperationEntry } from "./command-permissions/model.ts"; import { renderDbUrlTarget, renderLocalTarget } from "./target/target-identity.ts"; import { isCheckoutLinked } from "../shared/config/link-snapshot.ts"; @@ -198,6 +204,19 @@ const loginRoleErrorMapper = mapHttpError({ statusMessage: (status, body) => `unexpected login role status ${status}: ${body}`, }); +// The Management API surfaces Postgres 42501 from `ALTER ROLE`, which on these projects means +// `supabase_privileged_role` was never granted to the login-role creator. +const isMissingPrivilegedRole = (error: Errors.DbConfigLoginRoleStatusError): boolean => + error.status === 400 && /permission denied to alter role/i.test(error.body); + +const missingPrivilegedRoleError = (ref: string, error: Errors.DbConfigLoginRoleStatusError) => + new Errors.DbConfigLoginRoleMissingPrivilegedRoleError({ + projectRef: ref, + message: `failed to create a temporary login role, likely because the project database is missing supabase_privileged_role: ${error.body}`, + suggestion: + "Connect with your database password instead, which skips the temporary login role: --password where the command supports it, SUPABASE_DB_PASSWORD for the linked project, or a Postgres URL that includes the password (--db-url, or --from/--to for db diff). Contact support at https://supabase.com/dashboard/support/new to repair the project's roles.", + }); + const listBansErrorMapper = mapHttpError({ networkError: Errors.DbConfigListBansNetworkError, statusError: Errors.DbConfigListBansStatusError, @@ -264,9 +283,12 @@ const initLoginRole = Effect.fn("DbConfig.initLoginRole")(function* ( const api = yield* (yield* CommandPlatformApiFactory).make; // Written to stderr unconditionally (not gated on --debug). yield* output.raw("Initialising login role...\n", "stderr"); - const role = yield* api.v1 - .createLoginRole({ ref, read_only: false }) - .pipe(Effect.catch(loginRoleErrorMapper)); + const role = yield* api.v1.createLoginRole({ ref, read_only: false }).pipe( + Effect.catch(loginRoleErrorMapper), + Effect.catchTag("DbConfigLoginRoleStatusError", (error) => + Effect.fail(isMissingPrivilegedRole(error) ? missingPrivilegedRoleError(ref, error) : error), + ), + ); return { ...conn, user: role.role, password: role.password }; }); @@ -300,12 +322,17 @@ const NO_BANS: BanState = { listed: [], failure: Option.none() }; // Verify-connect with backoff while the pooler refreshes the temp password. From attempt 3 on, look // for network bans on the project: with approval they are removed and a failed removal is what the -// run reports if the retries run out; without it the last listing is reported instead. +// run reports if the retries run out; without it the last listing is reported instead. A rejected +// server certificate or an IP rejected by the project's network restrictions ends the loop at +// once: neither changes between attempts, and bans do not cause them. Every other failure keeps +// retrying, dial failures included, because a ban surfaces as a dial failure and only the retry +// path lists bans. const waitForTempRole = Effect.fn("DbConfig.waitForTempRole")(function* ( ref: string, conn: PgConnInput, dnsResolver: "native" | "https", removeNetworkBans: boolean, + dashboardUrl: string, ) { const dbConn = yield* DbConnection; const debug = yield* DebugLogger; @@ -318,6 +345,22 @@ const waitForTempRole = Effect.fn("DbConfig.waitForTempRole")(function* ( Effect.scoped(dbConn.connect(conn, { isLocal: false, dnsResolver }).pipe(Effect.asVoid)).pipe( Effect.ensuring(Effect.annotateCurrentSpan("retry.attempt_count", n)), Effect.catch((cause) => { + if (cause.certificateRejected === true) { + return Effect.fail( + new Errors.DbConfigConnectTempRoleError({ + message: `failed to connect as temp role: ${cause.message}`, + suggestion: SUGGEST_CA_VERIFICATION, + }), + ); + } + if (cause.networkRestricted === true) { + return Effect.fail( + new Errors.DbConfigConnectTempRoleError({ + message: `failed to connect as temp role: ${cause.message}`, + suggestion: suggestNetworkRestrictions(dashboardUrl), + }), + ); + } // 8 retries after the initial attempt allows 9 total attempts. `n` is 1-based, so give // up only after attempt 9 (`n > MAX_RETRIES`), not at attempt 8. if (n > MAX_RETRIES) { @@ -496,6 +539,7 @@ const fetchApiPoolerUrl = Effect.fn("DbConfig.fetchApiPoolerUrl")( const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( ref: string, workdir: string, + profile: string, poolerHost: string, dnsResolver: "native" | "https", password: string, @@ -508,6 +552,7 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( ignoreSavedUrl = false, resolveVaultSecrets = true, removeNetworkBans: boolean, + dashboardUrl: string, ) { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; @@ -515,7 +560,7 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( // on the main linked branch rather than validating base config. For an ad-hoc `--project-ref`, // skip the saved workdir pooler URL because it belongs to the linked project, not necessarily // the explicit ref. - yield* readDbToml(fs, path, workdir, ref, { + yield* readDbConfig(fs, path, workdir, ref, { resolveVaultSecrets, }); let connectionString = ignoreSavedUrl ? undefined : Option.getOrUndefined(savedPoolerUrl); @@ -535,14 +580,14 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( } } if (Option.isNone(pooler)) return Option.none(); - const poolerConn = pooler.value; + const poolerConn = pinSupabaseTls(pooler.value, profile); if (password.length > 0) return Option.some({ ...poolerConn, password }); // Mint a temp role; preserve Supavisor's `.` tenant suffix. const originalUser = poolerConn.user; const withRole = yield* initLoginRole(ref, poolerConn); const finalUser = originalUser.endsWith(`.${ref}`) ? `${withRole.user}.${ref}` : withRole.user; const tempConn = { ...withRole, user: finalUser }; - yield* waitForTempRole(ref, tempConn, dnsResolver, removeNetworkBans); + yield* waitForTempRole(ref, tempConn, dnsResolver, removeNetworkBans, dashboardUrl); return Option.some(tempConn); }); @@ -562,6 +607,7 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(function* ( ref: string, workdir: string, + profile: string, projectHost: string, poolerHost: string, dnsResolver: "native" | "https", @@ -569,6 +615,8 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio options: { /** Whether the temp-role retry may clear listed bans; only a `--yes` approval sets it. */ readonly removeNetworkBans: boolean; + /** The active profile's dashboard, linked from the network restrictions hint. */ + readonly dashboardUrl: string; readonly adHocProjectRef?: boolean; readonly resolveVaultSecrets?: boolean; /** The pooler URL the link recorded for `ref`, used when the direct host is unreachable. */ @@ -589,6 +637,7 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio fetchPoolerFromApi = false, hasPasswordFlag = false, removeNetworkBans, + dashboardUrl, poolerUrl = Option.none(), } = options; const dbPassword = yield* resolveLinkedPassword(ref, workdir, explicitPassword, hasPasswordFlag); @@ -605,24 +654,28 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio yield* Effect.annotateCurrentSpan("db.direct_reachable", reachable); if (reachable) { /* Same libpq TLS and timeout precedence as the pooler URL; host, port, role and database stay - pinned. The empty `passfile=` skips a `.pgpass` read for a password that is discarded. */ + pinned. The empty `passfile=` skips a `.pgpass` read for a password that is discarded. Absent an + explicit `sslmode`/`sslrootcert`, the Supabase CA is pinned, as on the pooler path. */ const parsed = yield* parseConnectionString( `postgresql://postgres@${host}:${DIRECT_PORT}/postgres?passfile=`, ); if (parsed === undefined) { return yield* new Errors.DbConfigLibpqEnvError({ message: LIBPQ_ENV_MESSAGE }); } - const direct = { - ...base, - ...Struct.pick(parsed, [ - "sslmode", - "sslrootcert", - "sslcert", - "sslkey", - "sslpassword", - "connectTimeoutSeconds", - ]), - }; + const direct = pinSupabaseTls( + { + ...base, + ...Struct.pick(parsed, [ + "sslmode", + "sslrootcert", + "sslcert", + "sslkey", + "sslpassword", + "connectTimeoutSeconds", + ]), + }, + profile, + ); if (direct.password.length > 0) return direct; return yield* initLoginRole(ref, direct); } @@ -633,6 +686,7 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio const poolerConn = yield* resolvePoolerConn( ref, workdir, + profile, poolerHost, dnsResolver, base.password, @@ -641,6 +695,7 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio adHocProjectRef, resolveVaultSecrets, removeNetworkBans, + dashboardUrl, ); if (Option.isNone(poolerConn)) { return yield* new Errors.DbConfigIpv6Error({ @@ -663,7 +718,7 @@ export const dbConfigResolverLayer = Layer.effect( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const runtimeInfo = yield* RuntimeInfo; - const crypto = yield* Crypto.Crypto; + const crypto = yield* Digest.Digest; const debug = yield* DebugLogger; const output = yield* Output; const dbConn = yield* DbConnection; @@ -726,6 +781,7 @@ export const dbConfigResolverLayer = Layer.effect( onSome: (value) => Layer.succeed(CliArgs, value), }), BunServices.layer, + BunCrypto.layerSecureRandom, ); // Compile-time guard: if `linkedDbResolverRuntimeLayer`'s requirements ever // grow a service not captured above, this assignment fails to type-check (the @@ -759,7 +815,7 @@ export const dbConfigResolverLayer = Layer.effect( Effect.provideService(RuntimeInfo, runtimeInfo), Effect.provideService(FileSystem.FileSystem, fs), Effect.provideService(Path.Path, path), - Effect.provideService(Crypto.Crypto, crypto), + Effect.provideService(Digest.Digest, crypto), Effect.catchTag("CliConfigLoadError", (error) => Effect.fail(new Errors.DbConfigLoadError({ message: error.message })), ), @@ -768,7 +824,7 @@ export const dbConfigResolverLayer = Layer.effect( const resolveDbUrl = (dbUrl: string, localHost: string, resolveVaultSecrets: boolean) => Effect.gen(function* () { - const tomlValues = yield* readDbToml(fs, path, cliSettings.workdir, undefined, { + const tomlValues = yield* readDbConfig(fs, path, cliSettings.workdir, undefined, { resolveVaultSecrets, }); // The project `.env*` files populate the environment that the libpq `PG*` fallbacks @@ -814,7 +870,7 @@ export const dbConfigResolverLayer = Layer.effect( const resolveLocalStack = (localHost: string, resolveVaultSecrets: boolean) => Effect.gen(function* () { - const tomlValues = yield* readDbToml(fs, path, cliSettings.workdir, undefined, { + const tomlValues = yield* readDbConfig(fs, path, cliSettings.workdir, undefined, { resolveVaultSecrets, }); const backend = yield* currentStackBackend; @@ -854,12 +910,13 @@ export const dbConfigResolverLayer = Layer.effect( }, ) => Effect.gen(function* () { - yield* readDbToml(fs, path, cliSettings.workdir, ref, { + yield* readDbConfig(fs, path, cliSettings.workdir, ref, { resolveVaultSecrets: options.resolveVaultSecrets, }); return yield* resolveLinkedConn( ref, cliSettings.workdir, + cliSettings.profile, cliSettings.projectHost, cliSettings.poolerHost, options.dnsResolver, @@ -871,6 +928,7 @@ export const dbConfigResolverLayer = Layer.effect( hasPasswordFlag: options.hasPasswordFlag, removeNetworkBans: options.removeNetworkBans, poolerUrl: options.poolerUrl, + dashboardUrl: cliSettings.dashboardUrl, }, ); }); @@ -894,7 +952,15 @@ export const dbConfigResolverLayer = Layer.effect( target.kind === "local" && (yield* isCheckoutLinked(fs, path, cliSettings.workdir, cliSettings.projectId)); if (target.kind !== "linked") { - yield* rejectTargetPassword(target.kind, input.password, linkedCheckout); + yield* rejectTargetPassword( + target.kind, + input.password, + linkedCheckout, + cliSettings.workdir, + ).pipe( + Effect.provideService(FileSystem.FileSystem, fs), + Effect.provideService(Path.Path, path), + ); } const resolveVaultSecrets = input.resolveVaultSecrets ?? true; if (target.kind === "linked") { @@ -952,6 +1018,7 @@ export const dbConfigResolverLayer = Layer.effect( return yield* resolvePoolerConn( target.ref, cliSettings.workdir, + cliSettings.profile, cliSettings.poolerHost, input.dnsResolver, password, @@ -960,6 +1027,7 @@ export const dbConfigResolverLayer = Layer.effect( !target.matchesLinkedRef, input.resolveVaultSecrets ?? true, canRemoveNetworkBans(input), + cliSettings.dashboardUrl, ); }).pipe( Effect.provide( diff --git a/apps/cli/src/command-internal/db-config.toml-read.precedence.unit.test.ts b/apps/cli/src/command-internal/db-config.read.precedence.unit.test.ts similarity index 98% rename from apps/cli/src/command-internal/db-config.toml-read.precedence.unit.test.ts rename to apps/cli/src/command-internal/db-config.read.precedence.unit.test.ts index d75c0cd9ae..beb851d057 100644 --- a/apps/cli/src/command-internal/db-config.toml-read.precedence.unit.test.ts +++ b/apps/cli/src/command-internal/db-config.read.precedence.unit.test.ts @@ -6,7 +6,7 @@ import { ConfigEnvPins } from "../../tests/helpers/config-env-pins.ts"; import { goldenJson, useShellEnvPin } from "../../tests/helpers/config-goldens.ts"; import { useTempWorkdir } from "../../tests/helpers/command-mocks.ts"; import { cliConfigValuesTestLayer } from "../../tests/helpers/config-values-layer.ts"; -import { checkDbToml } from "./db-config.toml-read.ts"; +import { checkDbConfig } from "./db-config.read.ts"; const TARGET_REF = "abcdefghijklmnopqrst"; const OTHER_REF = "zyxwvutsrqponmlkjihg"; @@ -213,7 +213,7 @@ describe("db toml reader precedence goldens", () => { } pinShellEnv(shell); - const outcome = yield* checkDbToml(fs, path, workdir, fixture.ref).pipe( + const outcome = yield* checkDbConfig(fs, path, workdir, fixture.ref).pipe( Effect.provide(cliConfigValuesTestLayer), Effect.provideService(ConfigEnvPins, shell), Effect.provideService( diff --git a/apps/cli/src/command-internal/db-config.toml-read.ts b/apps/cli/src/command-internal/db-config.read.ts similarity index 93% rename from apps/cli/src/command-internal/db-config.toml-read.ts rename to apps/cli/src/command-internal/db-config.read.ts index 56f30ef128..a1957a02ac 100644 --- a/apps/cli/src/command-internal/db-config.toml-read.ts +++ b/apps/cli/src/command-internal/db-config.read.ts @@ -27,6 +27,7 @@ import { validateAuthSmsProviders, validateResolvedConfig, } from "./config-validate.ts"; +import { type ConfigFileRef, configFileRefOf } from "./config-advice.ts"; import { DbConfigLoadError, isPassthroughLoadFailure } from "./db-config.errors.ts"; import { recordOrioleDbTelemetry, selectsOrioleDb } from "./db-image.ts"; import { resolveAuthExternalProviders } from "./local-config-values.ts"; @@ -42,7 +43,7 @@ type EnvLookup = (name: string) => string | undefined; * defaults; a malformed one aborts the command instead of running against the default local * database. */ -export interface DbTomlValues { +export interface DbConfigValues { readonly projectEnv: Readonly>; readonly apiSchemas: ReadonlyArray; /** Schemas the Data API serves; empty when `[api].enabled` is false. */ @@ -71,7 +72,7 @@ export interface DbTomlValues { * `[experimental.pgdelta]` config, consumed by the declarative-schema commands * (`db schema declarative generate` / `sync`). */ - readonly pgDelta: PgDeltaTomlConfig; + readonly pgDelta: PgDeltaConfig; /** Effective `[experimental.webhooks].enabled`; false when the section is absent. */ readonly webhooksEnabled: boolean; /** @@ -79,13 +80,13 @@ export interface DbTomlValues { * therefore the declarative catalog-cache key (`setupInputsToken`). Drift in * any of these must self-invalidate cached catalogs. */ - readonly baseline: BaselineTomlConfig; + readonly baseline: BaselineConfig; /** `[db.migrations] enabled` (default true) — gates `up`/`down` migration apply. */ readonly migrationsEnabled: boolean; /** * `[db.migrations] schema_paths`, default `[]` — resolved (supabase-prefixed * when relative) and overridable via `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS`, - * same as {@link DbSeedTomlConfig.sqlPaths}. Resolved unconditionally, not + * same as {@link DbSeedConfig.sqlPaths}. Resolved unconditionally, not * gated on `migrationsEnabled`. */ readonly schemaPaths: ReadonlyArray; @@ -98,18 +99,20 @@ export interface DbTomlValues { */ readonly schemaPathPatterns: ReadonlyArray; /** `[db.seed]` enabled + supabase-prefixed `sql_paths` globs — used by `down`. */ - readonly seed: DbSeedTomlConfig; + readonly seed: DbSeedConfig; /** `[db.vault]` secrets (name → resolved value) — upserted by `up`/`down`. */ - readonly vault: ReadonlyArray; + readonly vault: ReadonlyArray; /** * The matched `[remotes.]` block name when a linked ref merged its override * (`Loading config override: [remotes.]` line), else `undefined`. */ readonly appliedRemote: string | undefined; + /** The config file the values came from, for messages that point at it. */ + readonly configFile: ConfigFileRef; } /** `[db.seed]` config surfaced for `migration down`'s seed step. */ -interface DbSeedTomlConfig { +interface DbSeedConfig { readonly enabled: boolean; /** Glob patterns, each supabase-prefixed when relative. */ readonly sqlPaths: ReadonlyArray; @@ -120,14 +123,14 @@ interface DbSeedTomlConfig { * decrypted if it was a dotenvx `encrypted:` ciphertext). `resolved` is true * once the value is a non-empty, non-`env(...)` string. */ -interface DbVaultSecretToml { +interface DbVaultSecret { readonly name: string; readonly value: string; readonly resolved: boolean; } /** Cache-key inputs from `[auth]`/`[storage]`/`[realtime]`/`[api]`/`[db.vault]`. */ -interface BaselineTomlConfig { +interface BaselineConfig { /** `[auth] enabled`, default true. Gates `initSchema`'s auth service migration. */ readonly authEnabled: boolean; /** `[storage] enabled`, default true. */ @@ -145,7 +148,7 @@ interface BaselineTomlConfig { } /** The `[experimental.pgdelta]` subtree. */ -export interface PgDeltaTomlConfig { +export interface PgDeltaConfig { /** `[experimental.pgdelta] enabled`, default true. */ readonly enabled: boolean; /** @@ -343,7 +346,7 @@ const toDbConfigLoadError = (error: ResolvedConfigLoadError) => { } }; -const loadDbTomlResolvedConfig = ( +const loadDbResolvedConfig = ( workdir: string, ref: string | undefined, ignoreConfigFile: boolean, @@ -357,14 +360,14 @@ const loadDbTomlResolvedConfig = ( ).pipe(Effect.mapError(toDbConfigLoadError)); /** - * Projects the `CliConfigValues` resolved config of `/supabase/config.{toml,json}` (flags aside) - * onto {@link DbTomlValues}. `fs`/`path` are passed in so the resolver can capture them once and + * Projects the `CliConfigValues` resolved config of `/supabase/config.{toml,json,jsonc}` (flags aside) + * onto {@link DbConfigValues}. `fs`/`path` are passed in so the resolver can capture them once and * keep its own `R` at `never`. * * Fails with `DbConfigLoadError` when the config is present but unreadable, undecodable or invalid; * an absent file is not an error. */ -const readDbTomlCore = Effect.fnUntraced(function* ( +const readDbConfigCore = Effect.fnUntraced(function* ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, @@ -377,7 +380,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( warnOnUnresolvedEnv = true, resolveVaultSecrets = true, ) { - const resolvedConfig = yield* loadDbTomlResolvedConfig(workdir, ref, ignoreConfigFile); + const resolvedConfig = yield* loadDbResolvedConfig(workdir, ref, ignoreConfigFile); const projectEnv = resolvedConfig.projectEnvValues; const { config } = resolvedConfig.materialized; const fail = (message: string) => Effect.fail(new DbConfigLoadError({ message })); @@ -640,7 +643,9 @@ const readDbTomlCore = Effect.fnUntraced(function* ( webhooksEnabled, pgdeltaFormatOptions: pgDeltaConfig?.format_options ?? "", }; + const configFile = configFileRefOf(path, workdir, resolvedConfig); const validationInput: ConfigValidationInput = { + configFileName: path.basename(configFile.path), db: { port, majorVersion }, storageBucketNames: Object.keys(buckets), functionSlugs: Object.keys(config.functions ?? {}), @@ -668,7 +673,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const vaultConfig = config.db.vault ?? {}; const vaultNames = Object.keys(vaultConfig).sort(); - const vault: Array = []; + const vault: Array = []; if (resolveVaultSecrets) { for (const name of vaultNames) { const value = vaultConfig[name] ?? ""; @@ -695,7 +700,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( ? [...new Set(apiSchemas.map((schema) => schema.trim()).filter((schema) => schema.length > 0))] : []; - const values: DbTomlValues = { + const values: DbConfigValues = { projectEnv, apiSchemas, exposedSchemas, @@ -725,6 +730,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( seed: { enabled: seedEnabled, sqlPaths: seedSqlPaths }, vault, appliedRemote: Option.getOrUndefined(resolvedConfig.appliedRemote), + configFile, }; return values; }); @@ -735,12 +741,12 @@ const readDbTomlCore = Effect.fnUntraced(function* ( * validation aborts with a matching error. Call this before asserting the stack is * running, prompting, or any destructive work. */ -export const checkDbToml = ( +export const checkDbConfig = ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, ref?: string, - // See `readDbTomlCore`'s doc comment. Pass `false` only when an earlier, + // See `readDbConfigCore`'s doc comment. Pass `false` only when an earlier, // same-invocation call already printed the OrioleDB S3 WARN once. opts?: { readonly warnOnUnresolvedEnv?: boolean; @@ -748,7 +754,7 @@ export const checkDbToml = ( readonly resolveVaultSecrets?: boolean; }, ) => - readDbTomlCore( + readDbConfigCore( fs, path, workdir, @@ -760,12 +766,12 @@ export const checkDbToml = ( /** * Reads `config.toml`. Defaults to the same validating behavior as - * {@link checkDbToml}; pass `{ validate: false }` for a best-effort read that never + * {@link checkDbConfig}; pass `{ validate: false }` for a best-effort read that never * throws — a config-load failure falls back to pure defaults (env overrides still * applied), for callers that only need `projectId` and don't require a fully * validated config. */ -export const readDbToml = ( +export const readDbConfig = ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, @@ -779,17 +785,41 @@ export const readDbToml = ( const warnOnUnresolvedEnv = opts?.warnOnUnresolvedEnv ?? true; const resolveVaultSecrets = opts?.resolveVaultSecrets ?? true; return opts?.validate === false - ? readDbTomlCore(fs, path, workdir, ref, false, warnOnUnresolvedEnv, resolveVaultSecrets).pipe( + ? readDbConfigCore( + fs, + path, + workdir, + ref, + false, + warnOnUnresolvedEnv, + resolveVaultSecrets, + ).pipe( // Fall back to the ignore-file defaults path (never re-reads the broken config) // so a best-effort caller gets a well-formed defaults result instead of a throw. Effect.catchTags({ DbConfigLoadError: () => - readDbTomlCore(fs, path, workdir, ref, true, warnOnUnresolvedEnv, resolveVaultSecrets), + readDbConfigCore( + fs, + path, + workdir, + ref, + true, + warnOnUnresolvedEnv, + resolveVaultSecrets, + ), CliConfigValueError: () => - readDbTomlCore(fs, path, workdir, ref, true, warnOnUnresolvedEnv, resolveVaultSecrets), + readDbConfigCore( + fs, + path, + workdir, + ref, + true, + warnOnUnresolvedEnv, + resolveVaultSecrets, + ), }), ) - : readDbTomlCore(fs, path, workdir, ref, false, warnOnUnresolvedEnv, resolveVaultSecrets); + : readDbConfigCore(fs, path, workdir, ref, false, warnOnUnresolvedEnv, resolveVaultSecrets); }; /** @@ -798,7 +828,7 @@ export const readDbToml = ( * default `supabase/schemas`. `path` joins the segments so the separator matches the * host platform. */ -export function resolveDeclarativeDir(path: Path.Path, pgDelta: PgDeltaTomlConfig): string { +export function resolveDeclarativeDir(path: Path.Path, pgDelta: PgDeltaConfig): string { return Option.getOrElse(pgDelta.declarativeSchemaPath, () => path.join(...DEFAULT_DECLARATIVE_DIR_SEGMENTS), ); diff --git a/apps/cli/src/command-internal/db-config.toml-read.unit.test.ts b/apps/cli/src/command-internal/db-config.read.unit.test.ts similarity index 97% rename from apps/cli/src/command-internal/db-config.toml-read.unit.test.ts rename to apps/cli/src/command-internal/db-config.read.unit.test.ts index 59dbec46a3..00da986f48 100644 --- a/apps/cli/src/command-internal/db-config.toml-read.unit.test.ts +++ b/apps/cli/src/command-internal/db-config.read.unit.test.ts @@ -17,12 +17,12 @@ import type { PlatformError } from "effect/PlatformError"; import { withConfigEnv } from "../../tests/helpers/command-mocks.ts"; import { cliConfigValuesTestLayer } from "../../tests/helpers/config-values-layer.ts"; import { - checkDbToml, + checkDbConfig, loadProjectEnvValues, - readDbToml, + readDbConfig, resolveDeclarativeDir, - type DbTomlValues, -} from "./db-config.toml-read.ts"; + type DbConfigValues, +} from "./db-config.read.ts"; import { resolveSeedSqlPath } from "../shared/config/seed-path.ts"; import { sanitizeProjectId } from "../shared/config/project-id.ts"; import type { CliConfigValues } from "../config/cli-config-values.service.ts"; @@ -69,14 +69,14 @@ const read = (workdir: string) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - return yield* readDbToml(fs, path, workdir); + return yield* readDbConfig(fs, path, workdir); }).pipe(Effect.provide(servicesLive)); const readRef = (workdir: string, ref: string) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - return yield* readDbToml(fs, path, workdir, ref); + return yield* readDbConfig(fs, path, workdir, ref); }).pipe(Effect.provide(servicesLive)); const loadEnv = (workdir: string) => @@ -97,9 +97,9 @@ describe("read (lenient) vs check (throws) split", () => { return yield* run(fs, path); }).pipe(Effect.provide(servicesLive)); - it.effect("checkDbToml throws on an undecryptable secret", () => { + it.effect("checkDbConfig throws on an undecryptable secret", () => { return withConfig('[db]\nroot_key = "encrypted:anything"\n').pipe( - Effect.flatMap((dir) => withServices(dir, (fs, path) => checkDbToml(fs, path, dir))), + Effect.flatMap((dir) => withServices(dir, (fs, path) => checkDbConfig(fs, path, dir))), Effect.exit, Effect.tap((exit) => Effect.sync(() => { @@ -114,10 +114,10 @@ describe("read (lenient) vs check (throws) split", () => { ); }); - it.effect("checkDbToml surfaces an invalid env value as CliConfigValueError", () => { + it.effect("checkDbConfig surfaces an invalid env value as CliConfigValueError", () => { return withConfig("[db]\nport = 5432\n").pipe( Effect.tap((dir) => writeFile(dir, ["supabase", ".env"], "SUPABASE_DB_PORT=notaport\n")), - Effect.flatMap((dir) => withServices(dir, (fs, path) => checkDbToml(fs, path, dir))), + Effect.flatMap((dir) => withServices(dir, (fs, path) => checkDbConfig(fs, path, dir))), Effect.flip, Effect.tap((error) => Effect.sync(() => { @@ -142,7 +142,7 @@ describe("read (lenient) vs check (throws) split", () => { ).pipe( Effect.flatMap((dir) => withServices(dir, (fs, path) => - checkDbToml(fs, path, dir, undefined, { resolveVaultSecrets: false }), + checkDbConfig(fs, path, dir, undefined, { resolveVaultSecrets: false }), ), ), Effect.tap((values) => @@ -154,24 +154,31 @@ describe("read (lenient) vs check (throws) split", () => { ); }); - it.effect("readDbToml({ validate: false }) tolerates the same secret, returning defaults", () => { - return withConfig('[db]\nroot_key = "encrypted:anything"\n').pipe( - Effect.flatMap((dir) => - withServices(dir, (fs, path) => readDbToml(fs, path, dir, undefined, { validate: false })), - ), - Effect.tap((v) => - Effect.sync(() => { - expect(v.vault).toEqual([]); - expect(v.port).toBeGreaterThan(0); - }), - ), - ); - }); + it.effect( + "readDbConfig({ validate: false }) tolerates the same secret, returning defaults", + () => { + return withConfig('[db]\nroot_key = "encrypted:anything"\n').pipe( + Effect.flatMap((dir) => + withServices(dir, (fs, path) => + readDbConfig(fs, path, dir, undefined, { validate: false }), + ), + ), + Effect.tap((v) => + Effect.sync(() => { + expect(v.vault).toEqual([]); + expect(v.port).toBeGreaterThan(0); + }), + ), + ); + }, + ); - it.effect("readDbToml({ validate: false }) still returns a valid config's values", () => { + it.effect("readDbConfig({ validate: false }) still returns a valid config's values", () => { return withConfig('project_id = "lenientproj"\n').pipe( Effect.flatMap((dir) => - withServices(dir, (fs, path) => readDbToml(fs, path, dir, undefined, { validate: false })), + withServices(dir, (fs, path) => + readDbConfig(fs, path, dir, undefined, { validate: false }), + ), ), Effect.tap((v) => Effect.sync(() => { @@ -182,7 +189,7 @@ describe("read (lenient) vs check (throws) split", () => { }); }); -describe("readDbToml", () => { +describe("readDbConfig", () => { it.effect("returns defaults when config.toml is absent", () => { return Effect.gen(function* () { const path = yield* Path.Path; @@ -341,7 +348,7 @@ describe("readDbToml", () => { // A bare leading `/` has no Windows volume name, so it resolves as relative and joins to // `supabase/`, unlike Node's `path.win32.isAbsolute`, which treats a leading separator as // rooted at the current drive. Tests `resolveSeedSqlPath` directly with - // `BunPath.layerWin32` instead of the full `readDbToml` pipeline, since that pipeline + // `BunPath.layerWin32` instead of the full `readDbConfig` pipeline, since that pipeline // needs the real (POSIX-pathed) `Path.Path` service to open the temp config file on disk. const originalPlatform = process.platform; Object.defineProperty(process, "platform", { value: "win32" }); @@ -533,7 +540,7 @@ describe("readDbToml", () => { table: "db.seed", field: "enabled", value: "0", - read: (v: DbTomlValues) => v.seed.enabled, + read: (v: DbConfigValues) => v.seed.enabled, expected: false, }, { @@ -541,7 +548,7 @@ describe("readDbToml", () => { table: "db.migrations", field: "enabled", value: "0", - read: (v: DbTomlValues) => v.migrationsEnabled, + read: (v: DbConfigValues) => v.migrationsEnabled, expected: false, }, { @@ -549,7 +556,7 @@ describe("readDbToml", () => { table: "experimental.pgdelta", field: "enabled", value: "1", - read: (v: DbTomlValues) => v.pgDelta.enabled, + read: (v: DbConfigValues) => v.pgDelta.enabled, expected: true, }, { @@ -557,7 +564,7 @@ describe("readDbToml", () => { table: "auth", field: "enabled", value: '"0"', - read: (v: DbTomlValues) => v.baseline.authEnabled, + read: (v: DbConfigValues) => v.baseline.authEnabled, expected: false, }, { @@ -565,7 +572,7 @@ describe("readDbToml", () => { table: "api", field: "auto_expose_new_tables", value: '"TRUE"', - read: (v: DbTomlValues) => Option.getOrNull(v.baseline.apiAutoExposeNewTables), + read: (v: DbConfigValues) => Option.getOrNull(v.baseline.apiAutoExposeNewTables), expected: true, }, ])("decodes $name as a bool", ({ table, field, value, read: pick, expected }) => { @@ -584,25 +591,25 @@ describe("readDbToml", () => { name: "db.seed.enabled", table: "db.seed", field: "enabled", - read: (v: DbTomlValues) => v.seed.enabled, + read: (v: DbConfigValues) => v.seed.enabled, }, { name: "db.migrations.enabled", table: "db.migrations", field: "enabled", - read: (v: DbTomlValues) => v.migrationsEnabled, + read: (v: DbConfigValues) => v.migrationsEnabled, }, { name: "experimental.pgdelta.enabled", table: "experimental.pgdelta", field: "enabled", - read: (v: DbTomlValues) => v.pgDelta.enabled, + read: (v: DbConfigValues) => v.pgDelta.enabled, }, { name: "api.auto_expose_new_tables", table: "api", field: "auto_expose_new_tables", - read: (v: DbTomlValues) => Option.getOrNull(v.baseline.apiAutoExposeNewTables), + read: (v: DbConfigValues) => Option.getOrNull(v.baseline.apiAutoExposeNewTables), }, ])( "expands an env() reference in $name through the shell and project .env", @@ -1789,7 +1796,7 @@ describe("readDbToml", () => { Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - return yield* readDbToml(fs, path, dir, undefined, { warnOnUnresolvedEnv: false }); + return yield* readDbConfig(fs, path, dir, undefined, { warnOnUnresolvedEnv: false }); }), ), Effect.provide(servicesLive), @@ -2206,7 +2213,7 @@ describe("readDbToml", () => { ); }); -describe("readDbToml [experimental.pgdelta]", () => { +describe("readDbConfig [experimental.pgdelta]", () => { it.effect("defaults pg-delta to enabled with no config", () => { return withConfig(undefined).pipe( Effect.flatMap(read), @@ -2293,7 +2300,7 @@ describe("resolveDeclarativeDir", () => { ); }); -describe("readDbToml auth.Enabled validation", () => { +describe("readDbConfig auth.Enabled validation", () => { // Fails the config load with `message` contained in the surfaced error. const failsWith = ( lines: ReadonlyArray, @@ -2565,7 +2572,7 @@ describe("readDbToml auth.Enabled validation", () => { ); }); -describe("readDbToml encrypted secret decryption", () => { +describe("readDbConfig encrypted secret decryption", () => { // An undecryptable `encrypted:` value anywhere in config.toml aborts the load with // `failed to parse config: `. const expectFails = (lines: ReadonlyArray, message: string) => @@ -2647,7 +2654,7 @@ describe("readDbToml encrypted secret decryption", () => { ); }); -describe("readDbToml non-scalar config booleans", () => { +describe("readDbConfig non-scalar config booleans", () => { // A present non-scalar boolean must fail the config load rather than falling through to the // schema default, which would let `db reset` prompt and drop schemas. const failsInvalid = (lines: ReadonlyArray, field: string) => @@ -2667,7 +2674,7 @@ describe("readDbToml non-scalar config booleans", () => { ); }); -describe("readDbToml project env scoping", () => { +describe("readDbConfig project env scoping", () => { const LINKED = "linkedlinkedlinkedlin"; const OTHER = "otherotherotherother"; @@ -2721,7 +2728,7 @@ describe("readDbToml project env scoping", () => { }); }); -describe("readDbToml empty project_id", () => { +describe("readDbConfig empty project_id", () => { it.effect("rejects a present-but-empty top-level project_id", () => { return withConfig('project_id = ""\n').pipe( Effect.flatMap(read), @@ -2751,7 +2758,7 @@ describe("readDbToml empty project_id", () => { }); }); -describe("readDbToml [analytics] validation", () => { +describe("readDbConfig [analytics] validation", () => { const failsWith = (lines: ReadonlyArray, message: string) => Effect.gen(function* () { const dir = yield* withConfig(lines.join("\n")); @@ -2823,7 +2830,7 @@ describe("readDbToml [analytics] validation", () => { }); }); -describe("readDbToml SUPABASE_PROJECT_ID override", () => { +describe("readDbConfig SUPABASE_PROJECT_ID override", () => { it.effect("overrides the TOML project_id with SUPABASE_PROJECT_ID", () => { return withConfig(['project_id = "toml-project"', ""].join("\n")).pipe( Effect.flatMap(read), @@ -2891,7 +2898,7 @@ describe("readDbToml SUPABASE_PROJECT_ID override", () => { }); }); -describe("readDbToml OrioleDB telemetry", () => { +describe("readDbConfig OrioleDB telemetry", () => { const recordedAfterRead = (workdir: string) => Effect.gen(function* () { const recorded = yield* Ref.make({}); @@ -2921,7 +2928,7 @@ describe("readDbToml OrioleDB telemetry", () => { }); }); -describe("readDbToml Data API exposure", () => { +describe("readDbConfig Data API exposure", () => { it.effect.each([ { name: "disabled in config", diff --git a/apps/cli/src/command-internal/db-config.service.ts b/apps/cli/src/command-internal/db-config.service.ts index d2d91992fc..0831ccd838 100644 --- a/apps/cli/src/command-internal/db-config.service.ts +++ b/apps/cli/src/command-internal/db-config.service.ts @@ -8,6 +8,7 @@ import type { ProjectRefNotLinkedError, } from "../config/project-ref.errors.ts"; import type { ProfileLoadError } from "./profile-load.ts"; +import type { ApiMaxRetriesError } from "../config/command-settings.layer.ts"; import type { LinkedProjectRefReadError } from "../shared/config/project-link.ts"; import type { DbConnectError } from "./db-connection.errors.ts"; import type { LocalDbRunningError } from "./db-bootstrap/local-db-running.ts"; @@ -18,6 +19,7 @@ import type { DbConfigListBansNetworkError, DbConfigListBansStatusError, DbConfigLoadError, + DbConfigLoginRoleMissingPrivilegedRoleError, DbConfigLoginRoleNetworkError, DbConfigLoginRoleStatusError, DbConfigParseUrlError, @@ -47,6 +49,7 @@ export type DbConfigError = // A hard linked-ref load surfaces a real link-state read error instead of masking it // as not-linked. | LinkedProjectRefReadError + | DbConfigLoginRoleMissingPrivilegedRoleError | DbConfigLoginRoleNetworkError | DbConfigLoginRoleStatusError | DbConfigListBansNetworkError @@ -68,8 +71,9 @@ export type DbConfigError = // the pooler-config fallback. | CommandPlatformApiFactoryError // The lazy linked runtime rebuilds `commandSettingsLayer`, whose strict - // profile resolution can fail inside the resolver effect the same way. - | ProfileLoadError; + // profile and `SUPABASE_API_MAX_RETRIES` checks can fail inside the resolver effect the same way. + | ProfileLoadError + | ApiMaxRetriesError; // The `--linked` path builds a lazy Management API runtime (so `--local`/`--db-url` never need an // access token) from the resolver's captured context, so `resolve`'s R stays `never` and the diff --git a/apps/cli/src/command-internal/db-connection.errors.ts b/apps/cli/src/command-internal/db-connection.errors.ts index fc4391d2ac..ca9b241fff 100644 --- a/apps/cli/src/command-internal/db-connection.errors.ts +++ b/apps/cli/src/command-internal/db-connection.errors.ts @@ -17,6 +17,16 @@ export class DbConnectError extends Data.TaggedError("DbConnectError")<{ * fresh-db bootstrap's connect retry keys off this field. */ readonly retryable?: boolean; + /** + * True when the server certificate failed verification; the temp-role probe keys off this + * field to fail fast, since no retry can change the certificate. + */ + readonly certificateRejected?: boolean; + /** + * True when the pooler rejected the caller's IP for project network restrictions; the + * temp-role probe keys off this field to fail fast, since no retry can change the allow list. + */ + readonly networkRestricted?: boolean; /** * True when the failure is an IPv6 dial failure that an IPv4 pooler retry can recover; * `gen types`' pooler fallback keys off this field. diff --git a/apps/cli/src/command-internal/db-connection.pool-client.ts b/apps/cli/src/command-internal/db-connection.pool-client.ts index 6cffda0e5b..8285e8a22a 100644 --- a/apps/cli/src/command-internal/db-connection.pool-client.ts +++ b/apps/cli/src/command-internal/db-connection.pool-client.ts @@ -30,6 +30,7 @@ export const makePoolSqlClient = ( ): Effect.Effect => SqlClient.make({ acquirer: Effect.succeed(new PoolConnection(pool)), + driver: "pg", compiler: PgClient.makeCompiler(), spanAttributes: [ ["db.system.name", "postgresql"], diff --git a/apps/cli/src/command-internal/db-connection.service.ts b/apps/cli/src/command-internal/db-connection.service.ts index e743401536..64712d7ada 100644 --- a/apps/cli/src/command-internal/db-connection.service.ts +++ b/apps/cli/src/command-internal/db-connection.service.ts @@ -33,8 +33,8 @@ export interface PgConnInput { readonly runtimeParams?: Readonly>; /** * libpq `sslmode`, parsed from a `--db-url` query string. Controls whether the driver layer - * negotiates TLS and verifies the server certificate. Absent → TLS without certificate - * verification (libpq's `prefer`/`require` default). + * negotiates TLS and verifies the server certificate. Absent → TLS first, then plaintext if + * that fails (libpq's `prefer` default). */ readonly sslmode?: string; /** diff --git a/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts b/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts index 4f4d5264cf..236f621087 100644 --- a/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts +++ b/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts @@ -5,10 +5,16 @@ * structure, and the connect suggestion must classify real node-postgres error shapes. */ import * as net from "node:net"; +import * as tls from "node:tls"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Deferred, Duration, Effect, FileSystem, Layer, Path } from "effect"; +import { + KONG_LOCAL_TLS_CERT, + KONG_LOCAL_TLS_KEY, +} from "../commands/start/templates/kong-local-tls.ts"; +import { mockOutput } from "../../tests/helpers/mocks.ts"; import { SUGGEST_ENV_VAR, SUGGEST_LOCAL_STACK } from "./connect-errors.ts"; import type { DbConnectError, DbExecError } from "./db-connection.errors.ts"; import { type DbSession, type PgConnInput, DbConnection } from "./db-connection.service.ts"; @@ -146,16 +152,27 @@ const EMPTY_QUERY = wireMessage("I", Buffer.alloc(0)); * A fake Postgres server that completes an auth-less startup handshake, answers every * simple-protocol query with `SELECT 1`'s result (satisfying `acquireProbedPool`'s own probe), * and records whether the client sent an SSLRequest first — so a test can prove whether TLS was - * attempted independent of how the attempt is resolved. + * attempted independent of how the attempt is resolved. `sslReply` picks how an SSLRequest is + * answered: `N` (default, plaintext only), `S` then hang up (a failed handshake), an + * immediate hang-up (a failure that is not about TLS negotiation), or `S` with a real handshake + * followed by a 28000 rejection (`pg_hba` with only `hostnossl` entries), or `S` with a real + * handshake followed by a successful startup (a server that accepts TLS). `rejectPlaintext` + * answers a plaintext startup with 28000 too (`pg_hba` with no entry for the client at all). */ -const fakeStartupServer = (): Effect.Effect<{ +const fakeStartupServer = ( + sslReply: "N" | "S-then-hangup" | "hangup" | "S-then-28000" | "S-then-ok" = "N", + { rejectPlaintext = false }: { readonly rejectPlaintext?: boolean } = {}, +): Effect.Effect<{ readonly port: number; readonly close: () => void; readonly sawSslRequest: () => boolean; + readonly connectionCount: () => number; }> => Effect.callback((resume) => { let sawSslRequest = false; + let connectionCount = 0; const server = net.createServer((socket) => { + connectionCount += 1; let sawStartup = false; let pending = Buffer.alloc(0); socket.on("data", (data: Buffer) => { @@ -167,7 +184,52 @@ const fakeStartupServer = (): Effect.Effect<{ if (pending.length < length) return; if (pending.readInt32BE(4) === 80877103) { sawSslRequest = true; - socket.write("N"); + if (sslReply === "N") socket.write("N"); + else if (sslReply === "S-then-hangup") socket.end("S"); + else if (sslReply === "S-then-28000") { + socket.write("S"); + const secure = new tls.TLSSocket(socket, { + isServer: true, + key: KONG_LOCAL_TLS_KEY, + cert: KONG_LOCAL_TLS_CERT, + }); + secure.once("data", () => + secure.end( + errorResponse({ + S: "FATAL", + V: "FATAL", + C: "28000", + M: 'no pg_hba.conf entry for host "127.0.0.1", user "postgres", database "postgres", SSL encryption', + }), + ), + ); + secure.on("error", () => {}); + } else if (sslReply === "S-then-ok") { + socket.write("S"); + const secure = new tls.TLSSocket(socket, { + isServer: true, + key: KONG_LOCAL_TLS_KEY, + cert: KONG_LOCAL_TLS_CERT, + }); + secure.once("data", () => { + secure.write(Buffer.concat([AUTHENTICATION_OK, READY_FOR_QUERY])); + secure.on("data", () => + secure.write(Buffer.concat([commandComplete("SELECT 1"), READY_FOR_QUERY])), + ); + }); + secure.on("error", () => {}); + } else socket.destroy(); + return; + } else if (rejectPlaintext) { + socket.end( + errorResponse({ + S: "FATAL", + V: "FATAL", + C: "28000", + M: 'no pg_hba.conf entry for host "127.0.0.1", user "postgres", database "postgres", no encryption', + }), + ); + return; } else { sawStartup = true; socket.write(Buffer.concat([AUTHENTICATION_OK, READY_FOR_QUERY])); @@ -197,6 +259,7 @@ const fakeStartupServer = (): Effect.Effect<{ port: address.port, close: () => server.close(), sawSslRequest: () => sawSslRequest, + connectionCount: () => connectionCount, }), ); }); @@ -925,7 +988,7 @@ describe("a local target's explicit TLS request (CLI-2366: honor --db-url's own it.live( "stays plaintext for a local target with sslmode=prefer (CLI-2366 regression guard: " + - "libpq's TLS-then-plaintext fallback is not implemented by sslConfigsFor)", + "the loopback stack never gets a wasted TLS attempt)", () => Effect.gen(function* () { const server = yield* fakeStartupServer(); @@ -1063,3 +1126,190 @@ describe("a local target's explicit TLS request (CLI-2366: honor --db-url's own }), ); }); + +describe("a non-local plaintext-only target follows libpq sslmode semantics", () => { + const TLS_FALLBACK_WARNING = + "Connected to 127.0.0.1 without TLS: the TLS connection failed or was rejected. Set PGSSLMODE=require (or verify-full) to refuse unencrypted connections."; + + const connectAndQuery = ( + server: { readonly port: number }, + sslmode: string | undefined, + fallbacks?: PgConnInput["fallbacks"], + ) => { + const out = mockOutput(); + return Effect.gen(function* () { + const pool = yield* acquirePgPool( + { + host: "127.0.0.1", + port: server.port, + user: "postgres", + password: "postgres", + database: "postgres", + ...(sslmode !== undefined ? { sslmode } : {}), + ...(fallbacks !== undefined ? { fallbacks } : {}), + }, + { isLocal: false, dnsResolver: "native" }, + ); + yield* Effect.tryPromise(() => pool.query("select 1")); + return out.messages.filter((message) => message.type === "warn").map((m) => m.message); + }).pipe(Effect.scoped, Effect.provide(Layer.merge(BunServices.layer, out.layer))); + }; + + it.live("tries TLS, then connects in plaintext with a warning when sslmode is unset", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer(); + const warnings = yield* connectAndQuery(server, undefined).pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.sawSslRequest()).toBe(true); + expect(warnings).toEqual([TLS_FALLBACK_WARNING]); + }), + ); + + it.live("connects over TLS without a warning when the server accepts TLS", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer("S-then-ok"); + const warnings = yield* connectAndQuery(server, "prefer").pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.connectionCount()).toBe(1); + expect(warnings).toEqual([]); + }), + ); + + it.live("connects in plaintext without a warning when sslmode=allow", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer(); + const warnings = yield* connectAndQuery(server, "allow").pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.sawSslRequest()).toBe(false); + expect(warnings).toEqual([]); + }), + ); + + it.live("tries TLS, then connects in plaintext when sslmode=prefer", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer(); + const warnings = yield* connectAndQuery(server, "prefer").pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.sawSslRequest()).toBe(true); + expect(warnings).toEqual([TLS_FALLBACK_WARNING]); + }), + ); + + it.live("falls back to plaintext when the TLS handshake fails under sslmode=prefer", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer("S-then-hangup"); + yield* connectAndQuery(server, "prefer").pipe(Effect.ensuring(Effect.sync(server.close))); + expect(server.sawSslRequest()).toBe(true); + }), + ); + + it.live("does not retry in plaintext when the TLS attempt fails for a non-TLS reason", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer("hangup"); + const error = yield* connectFailure({ port: server.port, sslmode: "prefer" }, false).pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.connectionCount()).toBe(1); + expect(error.message).not.toContain("SSL"); + }), + ); + + it.live("skips the plaintext retry on a target that failed for a non-TLS reason", () => + Effect.gen(function* () { + const hangup = yield* fakeStartupServer("hangup"); + const plaintextOnly = yield* fakeStartupServer(); + yield* connectAndQuery(hangup, "prefer", [ + { host: "127.0.0.1", port: plaintextOnly.port }, + ]).pipe( + Effect.ensuring( + Effect.sync(() => { + hangup.close(); + plaintextOnly.close(); + }), + ), + ); + expect(hangup.connectionCount()).toBe(1); + expect(plaintextOnly.sawSslRequest()).toBe(true); + expect(plaintextOnly.connectionCount()).toBe(2); + }), + ); + + it.live("retries in plaintext when pg_hba rejects the TLS attempt with 28000 under prefer", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer("S-then-28000"); + const warnings = yield* connectAndQuery(server, "prefer").pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.sawSslRequest()).toBe(true); + expect(server.connectionCount()).toBe(2); + expect(warnings).toEqual([TLS_FALLBACK_WARNING]); + }), + ); + + it.live("stops at a target that rejects both the TLS and the plaintext attempt with 28000", () => + Effect.gen(function* () { + const rejecting = yield* fakeStartupServer("S-then-28000", { rejectPlaintext: true }); + const next = yield* fakeStartupServer(); + const error = yield* connectFailure( + { + port: rejecting.port, + sslmode: "prefer", + fallbacks: [{ host: "127.0.0.1", port: next.port }], + }, + false, + ).pipe( + Effect.ensuring( + Effect.sync(() => { + rejecting.close(); + next.close(); + }), + ), + ); + expect(rejecting.connectionCount()).toBe(2); + expect(next.connectionCount()).toBe(0); + expect(error.message).toContain("SQLSTATE 28000"); + expect(error.message).toContain("SSL encryption"); + expect(error.message).toContain("no encryption"); + }), + ); + + it.live("does not retry a 28000 rejection in plaintext when sslmode=require", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer("S-then-28000"); + const error = yield* connectFailure({ port: server.port, sslmode: "require" }, false).pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.connectionCount()).toBe(1); + expect(error.message).toContain("SQLSTATE 28000"); + }), + ); + + it.live("fails with the plaintext hint when sslmode=require", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer(); + const error = yield* connectFailure({ port: server.port, sslmode: "require" }, false).pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.connectionCount()).toBe(1); + expect(error.message).toContain("tls error (The server does not support SSL connections)"); + expect(error.suggestion).toBe( + "This server does not accept TLS. Set `sslmode=disable` on the connection string to connect in plaintext.", + ); + }), + ); + + it.live("connects in plaintext without an SSLRequest when sslmode=disable", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer(); + const warnings = yield* connectAndQuery(server, "disable").pipe( + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.sawSslRequest()).toBe(false); + expect(warnings).toEqual([]); + }), + ); +}); diff --git a/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts b/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts index 7f761f07c5..07664e9461 100644 --- a/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts +++ b/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts @@ -1,6 +1,6 @@ import * as net from "node:net"; import type { ConnectionOptions } from "node:tls"; -import { Cause, Duration, Effect, Exit, FileSystem, Layer, Scope } from "effect"; +import { Cause, Duration, Effect, Exit, FileSystem, Layer, Option, Scope } from "effect"; import * as Reactivity from "effect/reactivity/Reactivity"; import { ConnectionError, SqlError } from "effect/sql/SqlError"; // `pg` drives the pool behind `makePoolSqlClient` and the raw COPY and extended-protocol batch @@ -10,9 +10,12 @@ import { to as pgCopyTo } from "pg-copy-streams"; import { connectFailureMessage, connectSuggestion, + isCertificateVerificationFailure, + isNetworkRestrictionFailure, isDialFailure, isIPv6ConnectivityErrorCause, isSqlState, + isTlsNegotiationFailure, localStackSuggestion, } from "./connect-errors.ts"; import { DbConnectError, DbCopyError, DbExecError } from "./db-connection.errors.ts"; @@ -25,6 +28,7 @@ import { type DbSession, type PgConnInput, } from "./db-connection.service.ts"; +import { Output } from "../shared/output/output.service.ts"; import { resolveHostsOverHttps } from "./db-dns.ts"; // `@types/pg` doesn't declare `queryMode` on `QueryConfig`, `verify` on `PoolConfig` (run for @@ -86,7 +90,7 @@ function needsRoleStepDown(user: string): boolean { // These SQLSTATEs terminate the multi-host fallback chain instead of trying the next host, // since they indicate the server rejected the attempt rather than being unreachable; `28000` -// only terminates when the failed attempt used TLS. +// only terminates once the target has been tried over TLS. const TERMINAL_SQLSTATES = new Set(["28P01", "3D000", "42501"]); const TLS_GATED_SQLSTATE = "28000"; @@ -97,13 +101,16 @@ const TLS_GATED_SQLSTATE = "28000"; * Postgres SQLSTATE as a `code` property on the server error, carried through `@effect/sql`'s * `SqlError.cause`, so this walks the `cause` chain looking for one. */ -export function isTerminalConnectError(error: unknown, usedTls: boolean): boolean { +export function isTerminalConnectError(error: unknown, triedTls: boolean): boolean { const code = extractSqlState(error); if (code === undefined) return false; if (TERMINAL_SQLSTATES.has(code)) return true; - return code === TLS_GATED_SQLSTATE && usedTls; + return code === TLS_GATED_SQLSTATE && triedTls; } +const permitsPlaintextDowngrade = (error: unknown): boolean => + isTlsNegotiationFailure(error) || extractSqlState(error) === TLS_GATED_SQLSTATE; + /** * Extracts the Postgres SQLSTATE from a driver error. The `pg` driver attaches the * code as a `code` property on the server error, carried through `@effect/sql`'s @@ -396,9 +403,9 @@ export interface ClientCert { } /** - * `sslmode` values that demand TLS. `prefer` and `allow` describe a fallback libpq would perform - * and {@link sslConfigsFor} does not, so neither counts as a demand; treating them as one would - * turn a plaintext-capable target into a handshake failure. + * `sslmode` values that demand TLS. `prefer` and `allow` fall back across plaintext and TLS in + * {@link sslConfigsFor}, so neither counts as a demand; treating them as one would turn a + * plaintext-capable target into a handshake failure. */ const TLS_DEMANDING_SSLMODES = new Set(["require", "verify-ca", "verify-full"]); @@ -459,11 +466,12 @@ export function sslOptionFor( } /** - * The ordered list of `ssl` configs to try for a connection: `disable` → plaintext only; - * `allow` → plaintext then TLS; `prefer`/unset/`require`/`verify-ca`/`verify-full` → TLS only, - * so a failed handshake on the default `prefer` mode fails loudly rather than silently - * downgrading to plaintext. `servername` targets the original hostname per dial host when a - * DoH-resolved IP was substituted; `caCert` promotes `require` to `verify-ca` when set. + * The ordered list of `ssl` configs to try for a connection, following libpq: `disable` → + * plaintext only; `allow` → plaintext then TLS; `prefer`/unset → TLS then plaintext (the + * plaintext retry runs only after a TLS negotiation failure or a `28000` rejection, see + * {@link permitsPlaintextDowngrade}); `require`/`verify-ca`/`verify-full` → TLS only. + * `servername` targets the original hostname per dial host when a DoH-resolved IP was + * substituted; `caCert` promotes `require` to `verify-ca` when set. * `isLocal` is the caller's TLS-exemption decision, not the raw target classification: a local * target that explicitly set `sslmode`/`sslrootcert` (see {@link tlsExplicitlyRequested}) is not * exempt, so the caller passes `false` for it in that case. `allow`'s fallback bypasses that @@ -495,11 +503,12 @@ export function sslConfigsFor( ) { return [sslOptionFor(effectiveMode, false, servername, caCert, clientCert)]; } - // prefer (and the unset default) is TLS-only: a failed handshake must error, never downgrade - // to plaintext. - return [sslOptionFor(sslmode, false, servername, caCert, clientCert)]; + return [sslOptionFor(sslmode, false, servername, caCert, clientCert), false]; } +const isTlsConfig = (ssl: boolean | ConnectionOptions | undefined): boolean => + ssl !== undefined && ssl !== false; + /** * The raw `pg.ClientConfig` for a dial target: the connection-string form when a libpq * `options`/`runtimeParams` payload must reach the server (see {@link buildConnectionUrl}), @@ -641,15 +650,20 @@ export const toConnectError = ( cfg: PgConnInput, isLocal: boolean, error: unknown, + tlsError?: unknown, ): DbConnectError => { const suggestion = cfg.suggestionContext === undefined ? undefined : connectSuggestion(error, { ...cfg.suggestionContext, isLocal }); return new DbConnectError({ - message: `failed to connect to postgres: ${connectFailureMessage(cfg, error)}`, + message: `failed to connect to postgres: ${ + tlsError === undefined ? "" : `${connectFailureMessage(cfg, tlsError)}; then without TLS: ` + }${connectFailureMessage(cfg, error)}`, ...(suggestion === undefined ? {} : { suggestion }), ...(isDialFailure(error) ? { retryable: true } : {}), + ...(isCertificateVerificationFailure(error) ? { certificateRejected: true } : {}), + ...(isNetworkRestrictionFailure(error) ? { networkRestricted: true } : {}), ...(isIPv6ConnectivityErrorCause(error) ? { ipv6Unreachable: true } : {}), }); }; @@ -776,9 +790,10 @@ const acquirePgPoolConnection = (cfg: PgConnInput, { isLocal, dnsResolver }: DbC } : undefined; - // Builds the ordered attempt list: each TLS config from {@link sslConfigsFor} tried against + // Builds the ordered attempt list: each `ssl` config from {@link sslConfigsFor} tried against // each dial target (host × resolved IPs), with `servername` per target set to the original - // hostname when dialing a DoH-resolved IP. + // hostname when dialing a DoH-resolved IP. `isTlsDowngrade` marks the plaintext retry that + // follows a TLS attempt on the same target. const attempts = dialTargets.flatMap(({ dialHost, port, servername }) => sslConfigsFor( cfg.sslmode, @@ -787,11 +802,14 @@ const acquirePgPoolConnection = (cfg: PgConnInput, { isLocal, dnsResolver }: DbC caCert, dialHost, clientCert, - ).map((ssl) => ({ + ).map((ssl, index, configs) => ({ + host: servername ?? dialHost, pool: makePool(dialHost, port, ssl), - // The fallback chain only short-circuits on an auth error when the failed attempt used - // TLS; a TLS config is any non-plaintext `ssl` value. - usedTls: ssl !== undefined && ssl !== false, + // A 28000 rejection is final once the target has been tried over TLS: `allow` still + // follows a plaintext rejection with its TLS attempt, while a `prefer` plaintext retry + // that is rejected too ends the chain instead of moving on to the next host. + triedTls: configs.slice(0, index + 1).some(isTlsConfig), + isTlsDowngrade: index > 0 && !isTlsConfig(ssl) && isTlsConfig(configs[index - 1]), rawConfig: buildRawPgConfig(cfg, dialHost, port, ssl, connectTimeoutSeconds), })), ); @@ -812,7 +830,7 @@ const acquirePgPoolConnection = (cfg: PgConnInput, { isLocal, dnsResolver }: DbC const sessionScope = yield* Scope.Scope; const attemptScope = yield* Scope.fork(sessionScope); return yield* attempt.pool.pipe( - Effect.map((pool) => ({ pool, rawConfig: attempt.rawConfig })), + Effect.map((pool) => ({ pool, attempt })), Scope.provide(attemptScope), Effect.onExit((exit) => Exit.isSuccess(exit) ? Effect.void : Scope.close(attemptScope, exit), @@ -820,18 +838,55 @@ const acquirePgPoolConnection = (cfg: PgConnInput, { isLocal, dnsResolver }: DbC ); }); const lastIndex = attempts.length - 1; - const { pool, rawConfig: winningRawConfig } = yield* attempts - .slice(0, lastIndex) - .reduceRight( - (next, attempt) => - probe(attempt).pipe( - Effect.catch((error) => - isTerminalConnectError(error, attempt.usedTls) ? Effect.fail(error) : next, - ), - ), - probe(attempts[lastIndex]!), - ) - .pipe(Effect.mapError((error) => toConnectError(cfg, isLocal, error))); + // A failed attempt falls through to the next one unless its error is terminal. The plaintext + // retry after a TLS attempt (libpq's `prefer`) runs only when the server could not do TLS: + // negotiation failed or `pg_hba` has no TLS entry for the client (28000). Otherwise it is + // skipped, so an unreachable host or a wrong password is not retried in plaintext. + const run = ( + index: number, + ): Effect.Effect< + Effect.Success>, + { + readonly error: Effect.Error>; + readonly index: number; + readonly tlsError?: unknown; + }, + Effect.Services> + > => + probe(attempts[index]!).pipe( + Effect.catch((error) => { + if (index === lastIndex) return Effect.fail({ error, index }); + const next = attempts[index + 1]!; + if (next.isTlsDowngrade && permitsPlaintextDowngrade(error)) { + return run(index + 1).pipe( + Effect.mapError((failure) => + failure.index === index + 1 && extractSqlState(error) === TLS_GATED_SQLSTATE + ? { ...failure, tlsError: error } + : failure, + ), + ); + } + if (isTerminalConnectError(error, attempts[index]!.triedTls)) { + return Effect.fail({ error, index }); + } + const nextIndex = next.isTlsDowngrade ? index + 2 : index + 1; + return nextIndex > lastIndex ? Effect.fail({ error, index }) : run(nextIndex); + }), + ); + const { pool, attempt: winner } = yield* run(0).pipe( + Effect.mapError(({ error, tlsError }) => toConnectError(cfg, isLocal, error, tlsError)), + ); + const winningRawConfig = winner.rawConfig; + // libpq downgrades silently. Surfacing it tells a user who expected TLS (every hosted project + // offers it) that this connection is unencrypted, so a refused TLS attempt does not go unseen. + if (winner.isTlsDowngrade) { + const output = yield* Effect.serviceOption(Output); + if (Option.isSome(output)) { + yield* output.value.warn( + `Connected to ${winner.host} without TLS: the TLS connection failed or was rejected. Set PGSSLMODE=require (or verify-full) to refuse unencrypted connections.`, + ); + } + } // Steps down from the temp/privileged login role before any further SQL, remote // connections only; a local `--db-url` using `supabase_admin`/`cli_login_*` must not run it. diff --git a/apps/cli/src/command-internal/db-connection.sql-pg.unit.test.ts b/apps/cli/src/command-internal/db-connection.sql-pg.unit.test.ts index 500139e65e..e07fb25f94 100644 --- a/apps/cli/src/command-internal/db-connection.sql-pg.unit.test.ts +++ b/apps/cli/src/command-internal/db-connection.sql-pg.unit.test.ts @@ -226,9 +226,15 @@ describe("sslConfigsFor (fallback list)", () => { expect(sslConfigsFor("allow", true, undefined)).toEqual([false, { rejectUnauthorized: false }]); }); - it("prefer and unset are TLS only (ConnectByUrl strips the plaintext fallback)", () => { - expect(sslConfigsFor("prefer", false, undefined)).toEqual([{ rejectUnauthorized: false }]); - expect(sslConfigsFor(undefined, false, undefined)).toEqual([{ rejectUnauthorized: false }]); + it("prefer and unset try TLS first, then plaintext (libpq)", () => { + expect(sslConfigsFor("prefer", false, undefined)).toEqual([ + { rejectUnauthorized: false }, + false, + ]); + expect(sslConfigsFor(undefined, false, undefined)).toEqual([ + { rejectUnauthorized: false }, + false, + ]); }); it("require / verify-* are TLS only (no fallback)", () => { @@ -254,7 +260,10 @@ describe("sslConfigsFor (fallback list)", () => { it("does not attach a CA to non-verifying modes", () => { const ca = "ca-bundle"; - expect(sslConfigsFor("prefer", false, undefined, ca)).toEqual([{ rejectUnauthorized: false }]); + expect(sslConfigsFor("prefer", false, undefined, ca)).toEqual([ + { rejectUnauthorized: false }, + false, + ]); }); it("forces a single plaintext attempt for a unix-socket host regardless of sslmode", () => { @@ -556,7 +565,7 @@ describe("isTerminalConnectError (fallback termination)", () => { expect(isTerminalConnectError({ code: "42501" }, false)).toBe(true); }); - it("gates 28000 on the attempt having used TLS (a TLS attempt is required)", () => { + it("gates 28000 on the target having been tried over TLS", () => { expect(isTerminalConnectError({ code: "28000" }, true)).toBe(true); expect(isTerminalConnectError({ code: "28000" }, false)).toBe(false); }); diff --git a/apps/cli/src/command-internal/db-password-rejection.integration.test.ts b/apps/cli/src/command-internal/db-password-rejection.integration.test.ts index dc052493e1..9d65c83070 100644 --- a/apps/cli/src/command-internal/db-password-rejection.integration.test.ts +++ b/apps/cli/src/command-internal/db-password-rejection.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Cause, Effect, Exit, FileSystem, Layer, Path } from "effect"; import { CliOutput, Command } from "effect/cli"; @@ -99,7 +99,8 @@ const runCommand = ( Layer.succeed(CliArgs, { args }), mockOutput({ format: "text" }).layer, BunServices.layer, - legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), + BunCrypto.layer, + legacyProjectStateLayer.pipe(Layer.provide([BunServices.layer, BunCrypto.layer])), mockRuntimeInfo(), mockAnalytics().layer, mockTelemetryRuntime(), @@ -111,7 +112,7 @@ const runCommand = ( Effect.exit, ); return Exit.isFailure(exit) ? Cause.squash(exit.cause) : undefined; - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); describe("--password with a direct database target", () => { it("is discovered on commands that accept a direct target", () => { diff --git a/apps/cli/src/command-internal/db-pull-run.ts b/apps/cli/src/command-internal/db-pull-run.ts index 09ed990e67..3082c1f9fa 100644 --- a/apps/cli/src/command-internal/db-pull-run.ts +++ b/apps/cli/src/command-internal/db-pull-run.ts @@ -19,7 +19,7 @@ import type { ResolvedTarget } from "./target/resolve-target.ts"; import { flagsForTarget } from "./target/target-flags.ts"; import { resolveDbImage } from "./db-image.ts"; import { DbConnection, type PgConnInput } from "./db-connection.service.ts"; -import { loadProjectEnvValues, readDbToml, resolveDeclarativeDir } from "./db-config.toml-read.ts"; +import { loadProjectEnvValues, readDbConfig, resolveDeclarativeDir } from "./db-config.read.ts"; import { makeDir } from "./make-dir.ts"; import { toPostgresURL } from "./postgres-url.ts"; import { buildLocalDbContainerInputs } from "./db-bootstrap/local-container-inputs.ts"; @@ -156,7 +156,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( // Read the config for the matching `[remotes.]` block before connecting, so a // connection failure still reports which override matched. - const toml = yield* readDbToml(fs, path, cliSettings.workdir, linkedRef); + const toml = yield* readDbConfig(fs, path, cliSettings.workdir, linkedRef); if (toml.appliedRemote !== undefined) { yield* output.raw(`Loading config override: [remotes.${toml.appliedRemote}]\n`, "stderr"); } @@ -314,7 +314,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( // pg-delta ignores schema_paths in favor of the migrations baseline. if (usePgDeltaDiff && toml.schemaPaths !== undefined && toml.schemaPaths.length > 0) { - yield* output.raw(schemaPathsTransitionWarning, "stderr"); + yield* output.raw(schemaPathsTransitionWarning(toml.configFile), "stderr"); } // Migration-file path. @@ -492,7 +492,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( targetLocal: resolved.isLocal, migrationMode, // `toml.schemaPathPatterns` is used here, not the raw `@supabase/config` field on - // `localInputs`, since only `toml` (via `readDbToml`) resolves the + // `localInputs`, since only `toml` (via `readDbConfig`) resolves the // `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` env override. schemaPaths: toml.schemaPathPatterns, pgDelta: toml.pgDelta, diff --git a/apps/cli/src/command-internal/db-push-core.ts b/apps/cli/src/command-internal/db-push-core.ts index b42c8104fe..f72919df75 100644 --- a/apps/cli/src/command-internal/db-push-core.ts +++ b/apps/cli/src/command-internal/db-push-core.ts @@ -18,7 +18,8 @@ import { DbPushRolesError, } from "../commands/db/push/push.errors.ts"; import { aqua, bold } from "./colors.ts"; -import type { DbTomlValues } from "./db-config.toml-read.ts"; +import { configFileName } from "./config-advice.ts"; +import type { DbConfigValues } from "./db-config.read.ts"; import { DbConnection, type PgConnInput } from "./db-connection.service.ts"; import { applyMigrations, seedGlobals } from "./migration-apply.ts"; import { listRemoteMigrations, suggestRevertHistory } from "./migration-history.ts"; @@ -75,8 +76,8 @@ export interface DbPushCoreInput { readonly seed: DbSeedPlan; readonly includeVault: boolean; readonly dnsResolver: "native" | "https"; - /** Already loaded + validated `config.toml`, e.g. via `checkDbToml`. */ - readonly toml: DbTomlValues; + /** Already loaded + validated `config.toml`, e.g. via `checkDbConfig`. */ + readonly toml: DbConfigValues; /** Consent the caller already obtained; `None` asks once after the plan (never for local). */ readonly preapproved: Option.Option; /** @@ -127,7 +128,7 @@ export const dbPushCore = Effect.fn("DbPush.run")(function* (input: DbPushCoreIn let pending: ReadonlyArray = []; if (!toml.migrationsEnabled) { yield* output.raw( - `Skipping migrations because it is disabled in config.toml for project: ${projectRef}\n`, + `Skipping migrations because it is disabled in ${configFileName(toml.configFile.path)} for project: ${projectRef}\n`, "stderr", ); } else { @@ -160,7 +161,7 @@ export const dbPushCore = Effect.fn("DbPush.run")(function* (input: DbPushCoreIn if (includeSeed) { if (!seed.input.enabled) { yield* output.raw( - `Skipping seed because it is disabled in config.toml for project: ${projectRef}\n`, + `Skipping seed because it is disabled in ${configFileName(toml.configFile.path)} for project: ${projectRef}\n`, "stderr", ); } else { @@ -234,7 +235,7 @@ export const dbPushCore = Effect.fn("DbPush.run")(function* (input: DbPushCoreIn ? [ `Apply these migrations:\n${confirmPushAll(pending.map((p) => path.basename(p))).trimEnd()}`, ...(includeVault && vaultSecrets.some((secret) => secret.resolved) - ? ["Update vault secrets from config.toml"] + ? [`Update vault secrets from ${configFileName(toml.configFile.path)}`] : []), ] : []), diff --git a/apps/cli/src/command-internal/diff-engine.ts b/apps/cli/src/command-internal/diff-engine.ts index b6ff5b8430..3822eacc0c 100644 --- a/apps/cli/src/command-internal/diff-engine.ts +++ b/apps/cli/src/command-internal/diff-engine.ts @@ -1,8 +1,17 @@ // Pure diff-engine resolution shared by `db diff` and `db pull`. No Effect / service // dependencies — unit-tested directly. -export const schemaPathsTransitionWarning = - "WARNING: [db.migrations].schema_paths no longer changes the migrations baseline used by db diff or migration-style db pull. These commands always compare local migrations with the selected database. `supabase db schema declarative sync` reads declarative_schema_path (default supabase/schemas), not schema_paths: move those files there or point declarative_schema_path at them. Only migra (--use-migra, --diff-engine migra, or [experimental.pgdelta] enabled = false) still reads schema_paths, and only for local targets.\n"; +import { + type ConfigFileRef, + configKey, + configSetting, + DEFAULT_CONFIG_FILE, +} from "./config-advice.ts"; + +export const schemaPathsTransitionWarning = ( + configFile: ConfigFileRef = DEFAULT_CONFIG_FILE, +): string => + `WARNING: ${configKey(configFile.format, "db.migrations", "schema_paths")} no longer changes the migrations baseline used by db diff or migration-style db pull. These commands always compare local migrations with the selected database. \`supabase db schema declarative sync\` reads declarative_schema_path (default supabase/schemas), not schema_paths: move those files there or point declarative_schema_path at them. Only migra (--use-migra, --diff-engine migra, or ${configSetting(configFile.format, "experimental.pgdelta", "enabled", "false")}) still reads schema_paths, and only for local targets.\n`; /** * Reports whether `db diff` should run in pg-delta mode. A true `--use-migra` or `--use-pgadmin` diff --git a/apps/cli/src/command-internal/docker-image-resolve.unit.test.ts b/apps/cli/src/command-internal/docker-image-resolve.unit.test.ts index 3615514ab6..a2bd44af0f 100644 --- a/apps/cli/src/command-internal/docker-image-resolve.unit.test.ts +++ b/apps/cli/src/command-internal/docker-image-resolve.unit.test.ts @@ -26,7 +26,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; if (args[0] === "image" && args[1] === "inspect") { if (command._tag === "StandardCommand") imageInspectOptions.push(command.options); diff --git a/apps/cli/src/command-internal/docker-lifecycle.unit.test.ts b/apps/cli/src/command-internal/docker-lifecycle.unit.test.ts index f6491f3792..85c977a6de 100644 --- a/apps/cli/src/command-internal/docker-lifecycle.unit.test.ts +++ b/apps/cli/src/command-internal/docker-lifecycle.unit.test.ts @@ -33,7 +33,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { const cmd = command._tag === "StandardCommand" ? command.command : ""; - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; spawned.push({ command: cmd, args }); const exitDeferred = yield* Deferred.make(); @@ -343,6 +343,22 @@ describe("inspectContainerState", () => { ); }); + it.live.each([ + "failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory", + "error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.51/containers/supabase_db_my-app/json: open //./pipe/docker_engine: The system cannot find the file specified.", + "Is the docker daemon running?", + "permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.51/containers/supabase_db_my-app/json: dial unix /var/run/docker.sock: connect: permission denied", + ])("flags the daemon as down for the newer Docker message: %s", (stderr) => { + const mock = mockSpawner({ exitCode: 1, stderr: `${stderr}\n` }); + return inspectContainerState(mock.spawner, "supabase_db_my-app").pipe( + Effect.flip, + Effect.map((error) => { + expect(error.daemonDown).toBe(true); + expect(classifyCliErrorActionability(error).error_category).toBe("docker_not_running"); + }), + ); + }); + it.live( "fails with DockerLifecycleInspectError with a generic message when stderr is empty", () => { diff --git a/apps/cli/src/command-internal/docker-suggest.ts b/apps/cli/src/command-internal/docker-suggest.ts index 0054fb9d16..483ae36735 100644 --- a/apps/cli/src/command-internal/docker-suggest.ts +++ b/apps/cli/src/command-internal/docker-suggest.ts @@ -12,17 +12,20 @@ export const SUGGEST_DOCKER_START = export const SUGGEST_CONTAINER_MEMORY_LIMIT = "The container ran out of memory and was killed. Raise the memory allocated to your container runtime (Docker Desktop: Settings > Resources), or free up memory on the host, and try again."; +/** Docker CLI wordings for a daemon that cannot be reached, shared by every daemon-down matcher. */ +export const DOCKER_DAEMON_CONNECTION_FAILURE_PATTERN = + /failed to connect to the docker api|error during connect|is the docker daemon running|permission denied while trying to connect/iu; + /** * Whether a container-CLI stderr indicates the daemon is unreachable. Matches the docker/podman - * "cannot connect"/"is the docker daemon running" messages, a socket permission-denied message, - * the generic "error during connect" wrapper (which also covers Windows npipe daemon-down text), - * and {@link containerRuntimeNotFoundMessage} — a missing CLI binary gets the same install hint + * "cannot connect" messages, {@link DOCKER_DAEMON_CONNECTION_FAILURE_PATTERN}, and + * {@link containerRuntimeNotFoundMessage} — a missing CLI binary gets the same install hint * as a missing daemon. */ export function isDockerDaemonUnreachable(stderr: string): boolean { return ( - /cannot connect to the docker daemon|cannot connect to podman|is the docker daemon running|permission denied while trying to connect|error during connect/iu.test( - stderr, - ) || stderr.includes(containerRuntimeNotFoundMessage) + /cannot connect to the docker daemon|cannot connect to podman/iu.test(stderr) || + DOCKER_DAEMON_CONNECTION_FAILURE_PATTERN.test(stderr) || + stderr.includes(containerRuntimeNotFoundMessage) ); } diff --git a/apps/cli/src/command-internal/docker-suggest.unit.test.ts b/apps/cli/src/command-internal/docker-suggest.unit.test.ts index 52791dcf4d..df045863d3 100644 --- a/apps/cli/src/command-internal/docker-suggest.unit.test.ts +++ b/apps/cli/src/command-internal/docker-suggest.unit.test.ts @@ -28,6 +28,11 @@ describe("isDockerDaemonUnreachable", () => { "error during connect: in the default daemon configuration on Windows, the docker client must be run with elevated privileges to connect: open //./pipe/docker_engine: Access is denied.", ), ).toBe(true); + expect( + isDockerDaemonUnreachable( + "failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory", + ), + ).toBe(true); }); it("does not flag an unrelated inspect failure", () => { diff --git a/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts b/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts index 535e628a85..322ed5c109 100644 --- a/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts +++ b/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "@effect/vitest"; -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; import { vi } from "vitest"; @@ -58,6 +58,7 @@ function makeCliSettings(workdir = "/nonexistent-workdir") { workdir, explicitWorkdir: false, userAgent: "test", + apiMaxRetries: Option.none(), }); } @@ -224,7 +225,7 @@ describe("edgeRuntimeScriptLayer sentinel handling", () => { yield* runScript().pipe(Effect.provide(Layer.merge(layer, projectStateModeLayer("stack")))); expect(docker.lastOpts?.image).toContain("edge-runtime:v1.99.0"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect( @@ -271,7 +272,7 @@ describe("edgeRuntimeScriptLayer sentinel handling", () => { expect(docker.lastOpts?.image).toBe(lockedImage); }).pipe( Effect.scoped, - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), Effect.ensuring(Effect.sync(() => vi.unstubAllEnvs())), ), ); diff --git a/apps/cli/src/command-internal/edge-runtime-script.layer.ts b/apps/cli/src/command-internal/edge-runtime-script.layer.ts index 9762108961..5986d723a5 100644 --- a/apps/cli/src/command-internal/edge-runtime-script.layer.ts +++ b/apps/cli/src/command-internal/edge-runtime-script.layer.ts @@ -5,7 +5,7 @@ import { DebugFlag, NetworkIdFlag } from "./global-flags.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; import { CliConfigValues } from "../config/cli-config-values.service.ts"; -import { readDbToml } from "./db-config.toml-read.ts"; +import { readDbConfig } from "./db-config.read.ts"; import { getRegistryImageUrl } from "./docker-registry.ts"; import { DockerRun } from "./docker-run.service.ts"; import { resolveEdgeRuntimeImage, resolveLockedEdgeRuntimeImage } from "./edge-runtime-image.ts"; @@ -81,7 +81,7 @@ export const edgeRuntimeScriptLayer = Layer.effect( const workdir = opts.workdir ?? cliSettings.workdir; const denoVersion = opts.denoVersion ?? - (yield* readDbToml(fs, path, workdir).pipe( + (yield* readDbConfig(fs, path, workdir).pipe( Effect.provideService(CliConfigValues, configValues), Effect.mapError((error) => new EdgeRuntimeScriptError({ message: error.message })), )).denoVersion; diff --git a/apps/cli/src/command-internal/functions-local-config.ts b/apps/cli/src/command-internal/functions-local-config.ts index 8ded3eed41..cf4f29dae6 100644 --- a/apps/cli/src/command-internal/functions-local-config.ts +++ b/apps/cli/src/command-internal/functions-local-config.ts @@ -26,6 +26,7 @@ export const functionsLocalConfigLoader: FunctionsLocalConfigLoader = { context.hostname, projectRoot, context.resolvedConfig.loaded.document, + context.resolvedConfig.hasConfigFile ? context.resolvedConfig.loaded.path : undefined, ).pipe(Effect.mapError(toError)); return { loaded: { config: context.config, document: context.resolvedConfig.loaded.document }, diff --git a/apps/cli/src/command-internal/git-status.unit.test.ts b/apps/cli/src/command-internal/git-status.unit.test.ts index 21a1ee4cea..fd708f228d 100644 --- a/apps/cli/src/command-internal/git-status.unit.test.ts +++ b/apps/cli/src/command-internal/git-status.unit.test.ts @@ -21,7 +21,7 @@ function mockSpawner( const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { const cmd = command._tag === "StandardCommand" ? command.command : ""; - const args = command._tag === "StandardCommand" ? command.args : []; + const args = command._tag === "StandardCommand" ? command.args.map(String) : []; const cwd = command._tag === "StandardCommand" ? command.options.cwd : undefined; spawned.push({ command: cmd, args, cwd }); diff --git a/apps/cli/src/command-internal/glob.unit.test.ts b/apps/cli/src/command-internal/glob.unit.test.ts index a424671c2e..a7b4a1bc47 100644 --- a/apps/cli/src/command-internal/glob.unit.test.ts +++ b/apps/cli/src/command-internal/glob.unit.test.ts @@ -81,7 +81,7 @@ function fakeFileInfo(type: FileSystem.File.Type): FileSystem.File.Info { mtime: Option.none(), atime: Option.none(), birthtime: Option.none(), - dev: 0, + dev: 0n, ino: Option.none(), mode: 0, nlink: Option.none(), diff --git a/apps/cli/src/command-internal/hostname.ts b/apps/cli/src/command-internal/hostname.ts index 857353d9b1..ab32548820 100644 --- a/apps/cli/src/command-internal/hostname.ts +++ b/apps/cli/src/command-internal/hostname.ts @@ -1,4 +1,5 @@ -import { Config, Crypto, Effect, FileSystem, Option, Path, Schema } from "effect"; +import { Config, Effect, FileSystem, Option, Path, Schema } from "effect"; +import { Digest } from "effect/crypto"; import type { ResolvedCliConfig } from "../config/cli-config-values.service.ts"; import type { CliConfigLoadError } from "../shared/config/cli-config-load.errors.ts"; @@ -73,15 +74,10 @@ const dockerContextEndpointHost = (contextName: string) => { return Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const crypto = yield* Crypto.Crypto; + const crypto = yield* Digest.Digest; const configDir = yield* dockerConfigDir(); - const digest = yield* crypto - .digest("SHA-256", new TextEncoder().encode(contextName)) - .pipe(Effect.option); - if (Option.isNone(digest)) return Option.none(); - const contextId = Array.from(digest.value, (byte) => byte.toString(16).padStart(2, "0")).join( - "", - ); + const digest = yield* crypto.digest(Digest.sha256, new TextEncoder().encode(contextName)); + const contextId = Array.from(digest, (byte) => byte.toString(16).padStart(2, "0")).join(""); const metaPath = path.join(configDir, "contexts", "meta", contextId, "meta.json"); const content = yield* fs.readFileString(metaPath).pipe(Effect.option); if (Option.isNone(content)) return Option.none(); @@ -120,7 +116,7 @@ export function platformDefaultDockerHost(platform: NodeJS.Platform): string { export const resolveDockerDaemonEndpoint = Effect.fnUntraced(function* (): Effect.fn.Return< Option.Option, Config.ConfigError, - RuntimeInfo | FileSystem.FileSystem | Path.Path | Crypto.Crypto + RuntimeInfo | FileSystem.FileSystem | Path.Path | Digest.Digest > { const runtime = yield* RuntimeInfo; const dockerHost = yield* shellOption("DOCKER_HOST"); @@ -138,7 +134,7 @@ export const getHostname = ( ): Effect.Effect< string, Config.ConfigError | CliConfigLoadError, - RuntimeInfo | FileSystem.FileSystem | Path.Path | Crypto.Crypto + RuntimeInfo | FileSystem.FileSystem | Path.Path | Digest.Digest > => Effect.gen(function* () { const { SUPABASE_SERVICES_HOSTNAME: override } = yield* resolvedConfig.envValues([ diff --git a/apps/cli/src/command-internal/hostname.unit.test.ts b/apps/cli/src/command-internal/hostname.unit.test.ts index 72d6f8d273..471a51b39f 100644 --- a/apps/cli/src/command-internal/hostname.unit.test.ts +++ b/apps/cli/src/command-internal/hostname.unit.test.ts @@ -1,18 +1,9 @@ import { createHash } from "node:crypto"; -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { - Config, - ConfigProvider, - Crypto, - Effect, - FileSystem, - Layer, - Option, - Path, - Schema, -} from "effect"; +import { Config, ConfigProvider, Effect, FileSystem, Layer, Option, Path, Schema } from "effect"; +import { Digest } from "effect/crypto"; import type { CliConfigLoadError } from "../shared/config/cli-config-load.errors.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { @@ -36,7 +27,7 @@ const runtimeLayer = Layer.succeed( }), ); -type HostnameServices = RuntimeInfo | FileSystem.FileSystem | Path.Path | Crypto.Crypto; +type HostnameServices = RuntimeInfo | FileSystem.FileSystem | Path.Path | Digest.Digest; /** A resolved config whose env-tier lookup yields `env`; the real tier order is covered by the config service tests. */ const resolvedConfigWith = (env: Readonly> = {}) => ({ @@ -55,6 +46,7 @@ const noConfig = resolvedConfigWith(); function configLayer(env: Readonly>) { return Layer.mergeAll( BunServices.layer, + BunCrypto.layer, runtimeLayer, ConfigProvider.layer(ConfigProvider.fromEnvRecord(env, { preserveEmptyStrings: true })), ); @@ -93,7 +85,7 @@ function withDockerConfig( run: () => Effect.Effect, ) { return writeDockerConfigDir(options).pipe( - Effect.provide(BunServices.layer), + Effect.provide([BunServices.layer, BunCrypto.layer]), Effect.flatMap((configDir) => run().pipe(Effect.provide(configLayer({ ...env, DOCKER_CONFIG: configDir }))), ), @@ -241,6 +233,7 @@ describe("getHostname", () => { Effect.provide( Layer.mergeAll( BunServices.layer, + BunCrypto.layer, runtimeLayer, ConfigProvider.layer( ConfigProvider.fromEnvRecord({ DOCKER_HOST: "unix:///docker.sock" }), diff --git a/apps/cli/src/command-internal/link-services-core.ts b/apps/cli/src/command-internal/link-services-core.ts index 7cbd5e28b6..37bf717237 100644 --- a/apps/cli/src/command-internal/link-services-core.ts +++ b/apps/cli/src/command-internal/link-services-core.ts @@ -44,7 +44,9 @@ export interface LinkServicesInput { * storage-version}`. Each probe is independently best-effort — an * unreachable service never fails the caller. Does not write `project-ref`, * the linked-project cache, or fire `cli_project_linked`; the `link` command - * owns those, and `bootstrap` calls this directly to skip them. + * owns those, and `bootstrap` calls this directly to skip them. Unless the saved + * `project-ref` is `ref`, the files above except `pooler-url` are removed first, + * since their readers do no ref check. */ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( input: LinkServicesInput, @@ -58,6 +60,19 @@ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( const state = yield* LegacyProjectState; const { workdir } = input; + const refChanged = yield* state.readProjectRef(workdir).pipe( + Effect.map((linked) => !Option.contains(linked, input.ref)), + Effect.orElseSucceed(() => true), + ); + yield* Effect.annotateCurrentSpan("link.ref_changed", refChanged); + if (refChanged) { + yield* Effect.forEach( + ["storageMigration", "restVersion", "gotrueVersion", "storageVersion"] as const, + (file) => state.remove(workdir, file).pipe(Effect.ignore), + { discard: true }, + ); + } + yield* linkStorageMigration(api, input.ref, state, workdir); yield* linkPooler({ api, ref: input.ref, skipPooler: input.skipPooler, state, workdir }); if (input.serviceKey.length === 0) return; diff --git a/apps/cli/src/command-internal/local-config-values.ts b/apps/cli/src/command-internal/local-config-values.ts index 878e44541b..740e870a03 100644 --- a/apps/cli/src/command-internal/local-config-values.ts +++ b/apps/cli/src/command-internal/local-config-values.ts @@ -28,6 +28,7 @@ import { ErrorActionabilityId, } from "../shared/telemetry/error-actionability.ts"; import { resolveApiExternalUrl } from "./api-url.ts"; +import { configFileName } from "./config-advice.ts"; import { sanitizeProjectId } from "../shared/config/project-id.ts"; import { apiTlsCertReadErrorMessage, @@ -773,6 +774,8 @@ export const resolveLocalConfigValues = Effect.fnUntraced(function* ( * (not the always-defaulted decoded value). `undefined` callers skip those checks. */ document?: Readonly>, + /** The loaded config file's path, so messages name it; `undefined` reads as `config.toml`. */ + configPath?: string, ) { const resolvedProjectId = config.project_id ?? ""; @@ -954,6 +957,7 @@ export const resolveLocalConfigValues = Effect.fnUntraced(function* ( edgeRuntimeDenoVersion: denoVersion, analytics: analyticsInput, experimental: experimentalInput, + configFileName: configPath === undefined ? undefined : configFileName(configPath), }; yield* attempt(() => validateResolvedConfig(input)); if (authEnabled) { diff --git a/apps/cli/src/command-internal/local-project-context.ts b/apps/cli/src/command-internal/local-project-context.ts index 57ac81dba7..d024d1bbde 100644 --- a/apps/cli/src/command-internal/local-project-context.ts +++ b/apps/cli/src/command-internal/local-project-context.ts @@ -1,5 +1,6 @@ import type { CliConfig } from "@supabase/config"; -import { Crypto, Effect, FileSystem, Option, Path } from "effect"; +import { Effect, FileSystem, Option, Path } from "effect"; +import { Digest } from "effect/crypto"; import { CliConfigValues, type ResolvedCliConfig } from "../config/cli-config-values.service.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; @@ -30,7 +31,7 @@ export const loadLocalProjectContext = ( ): Effect.Effect< LocalProjectContext, E, - FileSystem.FileSystem | Path.Path | RuntimeInfo | Crypto.Crypto | CliConfigValues + FileSystem.FileSystem | Path.Path | RuntimeInfo | Digest.Digest | CliConfigValues > => Effect.gen(function* () { const { resolvedConfig, config, projectEnvValues, hostname, projectId } = diff --git a/apps/cli/src/command-internal/local-project-context.unit.test.ts b/apps/cli/src/command-internal/local-project-context.unit.test.ts index 1b121b4c5e..a62d989d0c 100644 --- a/apps/cli/src/command-internal/local-project-context.unit.test.ts +++ b/apps/cli/src/command-internal/local-project-context.unit.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Config, ConfigProvider, Data, Effect, FileSystem, Layer, Option, Path } from "effect"; @@ -51,11 +51,13 @@ const layerWithShellEnv = (env: Readonly> = {}) => Layer.fresh( Layer.mergeAll( BunServices.layer, + BunCrypto.layer, runtimeInfoLayer, cliConfigValuesLayer.pipe( Layer.provide( Layer.mergeAll( BunServices.layer, + BunCrypto.layer, mockOutput().layer, Layer.succeed(CliConfigFlagInputs, makeCliConfigFlagInputs()), ), diff --git a/apps/cli/src/command-internal/login-api.layer.integration.test.ts b/apps/cli/src/command-internal/login-api.layer.integration.test.ts index 6e7da75165..4e3175b4da 100644 --- a/apps/cli/src/command-internal/login-api.layer.integration.test.ts +++ b/apps/cli/src/command-internal/login-api.layer.integration.test.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from "@effect/vitest"; import { Effect, Layer, Option } from "effect"; +import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { LoginApi } from "../commands/login/login-api.service.ts"; import { @@ -13,11 +14,14 @@ import { loginApiLayer, loginProfile } from "./login-api.layer.ts"; const PROFILE = { gotrue_id: "gotrue-user-123", primary_email: "a@b.co", username: "ab" }; -function setup(opts: Parameters[0]) { +function setup( + opts: Parameters[0], + apiMaxRetries = Option.none(), +) { const api = mockCommandPlatformApi(opts); const layer = loginApiLayer.pipe( Layer.provide(api.httpClientLayer), - Layer.provide(mockCommandSettings({ workdir: "/unused-workdir" })), + Layer.provide(mockCommandSettings({ workdir: "/unused-workdir", apiMaxRetries })), ); return { api, layer }; } @@ -59,4 +63,26 @@ describe("loginApiLayer", () => { expect(api.requests).toHaveLength(1); }).pipe(Effect.provide(layer)); }); + + it.effect("looks the profile up once when SUPABASE_API_MAX_RETRIES is 0", () => { + const { api, layer } = setup( + { + // `Retry-After: 0` keeps any retry from waiting on the test clock. + handler: (request) => + Effect.succeed( + HttpClientResponse.fromWeb( + request, + new Response("{}", { status: 500, headers: { "Retry-After": "0" } }), + ), + ), + }, + Option.some(0), + ); + return Effect.gen(function* () { + const loginApi = yield* LoginApi; + const gotrueId = yield* loginApi.fetchGotrueId(DEFAULT_API_URL, VALID_TOKEN); + expect(gotrueId).toEqual(Option.none()); + expect(api.requests).toHaveLength(1); + }).pipe(Effect.provide(layer)); + }); }); diff --git a/apps/cli/src/command-internal/login-api.layer.ts b/apps/cli/src/command-internal/login-api.layer.ts index 14c78dbe13..f94ae30aee 100644 --- a/apps/cli/src/command-internal/login-api.layer.ts +++ b/apps/cli/src/command-internal/login-api.layer.ts @@ -88,11 +88,10 @@ export const loginApiLayer = Layer.effect( // host and token. fetchGotrueId: (apiHost: string, token: string) => Effect.gen(function* () { - const api = yield* makeApiClient({ - baseUrl: apiHost, - accessToken: token, - userAgent: cliSettings.userAgent, - }); + const api = yield* makeApiClient( + { baseUrl: apiHost, accessToken: token, userAgent: cliSettings.userAgent }, + { retry: { maxRetries: Option.getOrUndefined(cliSettings.apiMaxRetries) } }, + ); const profile = yield* api.v1.getProfile(); return profile.gotrue_id.length > 0 ? Option.some(profile.gotrue_id) diff --git a/apps/cli/src/command-internal/logs/local-history.ts b/apps/cli/src/command-internal/logs/local-history.ts new file mode 100644 index 0000000000..b3d843c02a --- /dev/null +++ b/apps/cli/src/command-internal/logs/local-history.ts @@ -0,0 +1,130 @@ +import type { LogPosition, LogRecord, StackLogRecord } from "@supabase/stack/effect"; + +const isLine = (record: LogRecord) => record.kind === "stdout" || record.kind === "stderr"; + +export interface LogWindow { + readonly records: ReadonlyArray; + readonly shown: number; + readonly total: number; +} + +const tailWindow = (records: ReadonlyArray, tail: number): LogWindow => { + const total = records.filter(isLine).length; + if (total <= tail) return { records, shown: total, total }; + let seen = 0; + let start = records.length; + while (seen < tail && start > 0) { + start -= 1; + const record = records[start]; + if (record !== undefined && isLine(record)) seen += 1; + } + return { records: seen === 0 ? [] : records.slice(start), shown: seen, total }; +}; + +const comparePositions = (left: LogPosition, right: LogPosition) => + left.generation - right.generation || left.byteOffset - right.byteOffset; + +const origin: LogPosition = { generation: 0, byteOffset: 0 }; +const compareText = (left: string, right: string) => (left < right ? -1 : left > right ? 1 : 0); +const compareRecords = (left: StackLogRecord, right: StackLogRecord) => + compareText(left.timestamp, right.timestamp) || + compareText(left.service, right.service) || + compareText(left.instanceId, right.instanceId) || + comparePositions( + left.position ?? left.resumeAt ?? origin, + right.position ?? right.resumeAt ?? origin, + ); + +interface Launch { + readonly id: number; + readonly timestamp: string; +} + +interface InstanceTail { + records: Array; + head: number; + lines: number; + total: number; + launch: Launch | undefined; +} + +export const isFromLaunch = (record: LogRecord, launchId: number | undefined) => + launchId === undefined || record.launchId === undefined || record.launchId >= launchId; + +const fromLaunch = (record: StackLogRecord, launch: Launch) => + record.launchId === undefined + ? record.timestamp >= launch.timestamp + : isFromLaunch(record, launch.id); + +export const makeHistoryCollector = (tail: number, fromLatestLaunch: boolean) => { + const instances = new Map(); + const positions = new Map(); + const launches = new Map(); + const push = (record: StackLogRecord) => { + const position = record.position; + const last = positions.get(record.instanceId); + if (position !== undefined && (last === undefined || comparePositions(position, last) > 0)) + positions.set(record.instanceId, position); + let state = instances.get(record.instanceId); + if (state === undefined) { + state = { records: [], head: 0, lines: 0, total: 0, launch: undefined }; + instances.set(record.instanceId, state); + } + if (fromLatestLaunch) { + const latest = state.launch; + if ( + record.kind === "launch" && + record.launchId !== undefined && + (latest === undefined || record.launchId > latest.id) + ) { + const launch = { id: record.launchId, timestamp: record.timestamp }; + state.records = state.records.slice(state.head).filter((kept) => fromLaunch(kept, launch)); + state.head = 0; + state.lines = state.records.filter(isLine).length; + state.total = state.lines; + state.launch = launch; + launches.set(record.instanceId, launch.id); + } else if (latest !== undefined && !fromLaunch(record, latest)) return; + } + const line = isLine(record); + if (line) state.total += 1; + const oldest = state.records[state.head]; + if (state.lines >= tail && oldest !== undefined && compareRecords(record, oldest) < 0) return; + let low = state.head; + let high = state.records.length; + while (low < high) { + const middle = (low + high) >>> 1; + const kept = state.records[middle]; + if (kept !== undefined && compareRecords(kept, record) <= 0) low = middle + 1; + else high = middle; + } + if (low === state.records.length) state.records.push(record); + else state.records.splice(low, 0, record); + if (line) state.lines += 1; + while (state.lines > tail && state.head < state.records.length) { + const dropped = state.records[state.head]; + state.head += 1; + if (dropped !== undefined && isLine(dropped)) state.lines -= 1; + } + if (state.head > 1024 && state.head * 2 > state.records.length) { + state.records = state.records.slice(state.head); + state.head = 0; + } + }; + const finish = (): { + readonly window: LogWindow; + readonly positions: typeof positions; + readonly launches: ReadonlyMap; + } => { + const kept = [...instances.values()].flatMap((state) => state.records.slice(state.head)); + const total = [...instances.values()].reduce((sum, state) => sum + state.total, 0); + const window = tailWindow(kept.toSorted(compareRecords), tail); + return { window: { ...window, total }, positions, launches }; + }; + return { push, finish }; +}; + +export const isAfter = (record: LogRecord, printed: LogPosition | undefined) => + printed === undefined || + record.position === undefined || + comparePositions(record.position, printed) > 0; diff --git a/apps/cli/src/command-internal/logs/local-sources.ts b/apps/cli/src/command-internal/logs/local-sources.ts new file mode 100644 index 0000000000..28f6d08b69 --- /dev/null +++ b/apps/cli/src/command-internal/logs/local-sources.ts @@ -0,0 +1,8 @@ +import { sharesApiEndpoint, type SavedStack } from "@supabase/stack/effect"; + +export const hasGatewayLogs = (definition: SavedStack): boolean => { + const members = new Set(definition.composition.members.map(({ id }) => id)); + return definition.instances.some( + ({ id, creation }) => members.has(id) && sharesApiEndpoint(creation), + ); +}; diff --git a/apps/cli/src/command-internal/logs/since.ts b/apps/cli/src/command-internal/logs/since.ts new file mode 100644 index 0000000000..5f204863d4 --- /dev/null +++ b/apps/cli/src/command-internal/logs/since.ts @@ -0,0 +1,34 @@ +import { DateTime, Option } from "effect"; + +export type SinceBound = + | { readonly kind: "time"; readonly iso: string } + | { readonly kind: "start" }; + +const durationShape = /^(?:\d+(?:\.\d+)?(?:ms|s|m|h|d))+$/u; +const durationPart = /(\d+(?:\.\d+)?)(ms|s|m|h|d)/gu; +const unitMillis: Readonly> = { + ms: 1, + s: 1_000, + m: 60_000, + h: 3_600_000, + d: 86_400_000, +}; +const isoDate = /^\d{4}-\d{2}-\d{2}/u; + +export const parseSince = (value: string, nowMillis: number): SinceBound | undefined => { + if (value === "start") return { kind: "start" }; + if (durationShape.test(value)) { + let millis = 0; + for (const [, amount, unit] of value.matchAll(durationPart)) + millis += Number(amount) * (unitMillis[unit ?? ""] ?? 0); + return Option.match(DateTime.make(nowMillis - millis), { + onNone: () => undefined, + onSome: (time) => ({ kind: "time", iso: DateTime.formatIso(time) }), + }); + } + if (!isoDate.test(value)) return undefined; + return Option.match(DateTime.make(value), { + onNone: () => undefined, + onSome: (time) => ({ kind: "time", iso: DateTime.formatIso(time) }), + }); +}; diff --git a/apps/cli/src/command-internal/migrate-and-seed.ts b/apps/cli/src/command-internal/migrate-and-seed.ts index 1bf8470264..adf396e587 100644 --- a/apps/cli/src/command-internal/migrate-and-seed.ts +++ b/apps/cli/src/command-internal/migrate-and-seed.ts @@ -5,7 +5,13 @@ import type { DbSession } from "./db-connection.service.ts"; import type { DbExecError } from "./db-connection.errors.ts"; import { MigrationApplyError, applyMigrationFile, applySchemaFiles } from "./migration-apply.ts"; import { loadPartialMigrations } from "./migration-history.ts"; -import { ENABLE_LOCAL_WEBHOOKS_SUGGESTION, isPgNetUnavailableError } from "./pg-net-guidance.ts"; +import { enableLocalWebhooksSuggestion, isPgNetUnavailableError } from "./pg-net-guidance.ts"; +import { + type ConfigFileRef, + configKey, + configSetting, + DEFAULT_CONFIG_FILE, +} from "./config-advice.ts"; import { applySeedFiles, type SeedConfig } from "./seed.ts"; /** Config consumed by `migrateAndSeed`. */ @@ -27,12 +33,15 @@ export interface MigrateAndSeedConfig { * this is not a local start/reset replay and disables local-only remediation. */ readonly localDatabaseWebhooksEnabled?: boolean; + /** The project config file the guidance points at; absent means `config.toml`. */ + readonly configFile?: ConfigFileRef | undefined; } const migrationApplyError = ( message: string, dbError: DbExecError | undefined, localDatabaseWebhooksEnabled: boolean | undefined, + configFile: ConfigFileRef | undefined, ): MigrationApplyError => { const pgNetUnavailable = localDatabaseWebhooksEnabled === false && @@ -40,14 +49,16 @@ const migrationApplyError = ( isPgNetUnavailableError(dbError); return new MigrationApplyError({ message, - suggestion: pgNetUnavailable ? ENABLE_LOCAL_WEBHOOKS_SUGGESTION : undefined, + suggestion: pgNetUnavailable ? enableLocalWebhooksSuggestion(configFile) : undefined, reason: pgNetUnavailable ? "local_pg_net_unavailable" : undefined, }); }; /** Warns a versionless `--experimental` rebuild that `schema_paths` is ignored while pg-delta is on. */ -export const experimentalSchemaPathsIgnoredWarning = - "WARNING: [db.migrations].schema_paths is not applied while pg-delta is enabled (the default). Set [experimental.pgdelta] enabled = false to apply schema_paths files.\n"; +export const experimentalSchemaPathsIgnoredWarning = ( + configFile: ConfigFileRef = DEFAULT_CONFIG_FILE, +): string => + `WARNING: ${configKey(configFile.format, "db.migrations", "schema_paths")} is not applied while pg-delta is enabled (the default). Set ${configSetting(configFile.format, "experimental.pgdelta", "enabled", "false")} to apply schema_paths files.\n`; /** Whether {@link experimentalSchemaPathsIgnoredWarning} applies to this rebuild. */ export const ignoresExperimentalSchemaPaths = (inputs: { @@ -81,7 +92,7 @@ export const migrateAndSeed = ( Effect.gen(function* () { const output = yield* Output; if (ignoresExperimentalSchemaPaths({ ...config, version })) { - yield* output.raw(experimentalSchemaPathsIgnoredWarning, "stderr"); + yield* output.raw(experimentalSchemaPathsIgnoredWarning(config.configFile), "stderr"); } if (config.experimental && version.length === 0 && !config.pgDeltaEnabled) { yield* applySchemaFiles( @@ -100,7 +111,12 @@ export const migrateAndSeed = ( for (const migrationPath of pending) { yield* output.raw(`Applying migration ${path.basename(migrationPath)}...\n`, "stderr"); yield* applyMigrationFile(session, fs, path, migrationPath, (message, dbError) => - migrationApplyError(message, dbError, config.localDatabaseWebhooksEnabled), + migrationApplyError( + message, + dbError, + config.localDatabaseWebhooksEnabled, + config.configFile, + ), ); } } diff --git a/apps/cli/src/command-internal/migration-apply.ts b/apps/cli/src/command-internal/migration-apply.ts index 40eb63121c..ce101df88b 100644 --- a/apps/cli/src/command-internal/migration-apply.ts +++ b/apps/cli/src/command-internal/migration-apply.ts @@ -311,7 +311,7 @@ const TYPE_NAME_PATTERN = /type "([^"]+)" does not exist/; /** * Renders a failed statement's error context: the `^` caret under the server-reported error - * position, the `Detail` line when present, the SQLSTATE-42704 extension hint, then + * position, the `Detail` line when present, the SQLSTATE-42704 or 25P01 hint, then * `At statement: ` and the caret-marked statement text. The structured `detail`/`position` * fields are only set by the driver for real server error responses. * @@ -334,6 +334,12 @@ export const formatExecBatchError = (e: DbExecError, index: number, stat: string msg.push(` CREATE TABLE example (col extensions.${typeName});`); msg.push(" Learn more: supabase migration new --help"); } + if (e.code === "25P01") { + msg.push(""); + msg.push("Hint: This statement must run inside an explicit BEGIN; ... COMMIT; block;"); + msg.push(" the CLI adds none. Review any existing transaction boundaries."); + msg.push(" Move each non-transactional statement (e.g. VACUUM) to its own file."); + } msg.push(`At statement: ${index}`, marked); return formattedExecBatchFailure(`${errorMessage(e)}\n${msg.join("\n")}`, e); }; diff --git a/apps/cli/src/command-internal/migration-apply.unit.test.ts b/apps/cli/src/command-internal/migration-apply.unit.test.ts index 224e8f2618..a18f432ec4 100644 --- a/apps/cli/src/command-internal/migration-apply.unit.test.ts +++ b/apps/cli/src/command-internal/migration-apply.unit.test.ts @@ -712,7 +712,7 @@ describe("hasTransactionControl", () => { }); layer(BunServices.layer)("migration failure rendering", (it) => { - // Error message layout: `\n[Detail]\n[42704 hint]\n` then + // Error message layout: `\n[Detail]\n[42704 or 25P01 hint]\n` then // `At statement: \n`. const failing = ( sql: string, @@ -765,7 +765,7 @@ layer(BunServices.layer)("migration failure rendering", (it) => { }), ); - it.effect("skips the hint when the SQLSTATE is not 42704", () => + it.effect("skips the 42704 hint when the SQLSTATE is not 42704", () => Effect.gen(function* () { const stat = "CREATE TABLE test (path ltree NOT NULL)"; const message = yield* failing(stat, { @@ -795,6 +795,25 @@ layer(BunServices.layer)("migration failure rendering", (it) => { }), ); + it.effect("recommends an authored BEGIN/COMMIT when LOCK TABLE needs a transaction block", () => + Effect.gen(function* () { + const stat = "LOCK TABLE public.t IN ACCESS EXCLUSIVE MODE"; + const message = yield* failing(stat, { + message: "ERROR: LOCK TABLE can only be used in transaction blocks (SQLSTATE 25P01)", + code: "25P01", + }); + expect(message).toBe( + "ERROR: LOCK TABLE can only be used in transaction blocks (SQLSTATE 25P01)\n" + + "\n" + + "Hint: This statement must run inside an explicit BEGIN; ... COMMIT; block;\n" + + " the CLI adds none. Review any existing transaction boundaries.\n" + + " Move each non-transactional statement (e.g. VACUUM) to its own file.\n" + + "At statement: 0\n" + + "LOCK TABLE public.t IN ACCESS EXCLUSIVE MODE", + ); + }), + ); + it.effect("keeps the plain layout for errors without position or detail", () => Effect.gen(function* () { const stat = "ALTER TABLE a ADD COLUMN b int"; diff --git a/apps/cli/src/command-internal/migration-fetch-run.ts b/apps/cli/src/command-internal/migration-fetch-run.ts index 45bf6ec894..bf66853271 100644 --- a/apps/cli/src/command-internal/migration-fetch-run.ts +++ b/apps/cli/src/command-internal/migration-fetch-run.ts @@ -7,7 +7,7 @@ import { CommandSettings } from "../config/command-settings.service.ts"; import { bold } from "./colors.ts"; import { DbConfigResolver } from "./db-config.service.ts"; import { DbConnection } from "./db-connection.service.ts"; -import { loadProjectEnvValues } from "./db-config.toml-read.ts"; +import { loadProjectEnvValues } from "./db-config.read.ts"; import { announcedTarget, hostedMutationPreapproval } from "./hosted-consent/hosted-consent.ts"; import { readMigrationTable } from "./migration-history.ts"; import { promptYesNo } from "./prompt-yes-no.ts"; diff --git a/apps/cli/src/command-internal/output-formats.ts b/apps/cli/src/command-internal/output-formats.ts index 52d4e85d8b..3cffaa89f3 100644 --- a/apps/cli/src/command-internal/output-formats.ts +++ b/apps/cli/src/command-internal/output-formats.ts @@ -18,6 +18,15 @@ export const RESOURCE_OUTPUT_FORMATS = ["env", "pretty", "json", "toml", "yaml"] /** Accepted `-o`/`--output` values for `db query`. */ export const QUERY_OUTPUT_FORMATS = ["json", "table", "csv"] as const; +const commandOutputFormats: Readonly>> = { + "db query": QUERY_OUTPUT_FORMATS, + logs: [], + "logs sources": [], +}; + +export const outputFormatsForCommand = (path: ReadonlyArray): ReadonlyArray => + commandOutputFormats[path.join(" ")] ?? RESOURCE_OUTPUT_FORMATS; + /** * Raised when `-o`/`--output` carries a value the active command doesn't accept. The message * has the form `invalid argument "" for "" flag: `. diff --git a/apps/cli/src/command-internal/pg-dump.native.integration.test.ts b/apps/cli/src/command-internal/pg-dump.native.integration.test.ts index 09d9b28825..a63ba731cb 100644 --- a/apps/cli/src/command-internal/pg-dump.native.integration.test.ts +++ b/apps/cli/src/command-internal/pg-dump.native.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { FetchHttpClient } from "effect/http"; import { Effect, FileSystem, Layer, Option, Redacted, Ref, Stream } from "effect"; @@ -16,7 +16,7 @@ import { destroyTestStack } from "../../tests/helpers/stack-cleanup.ts"; import { selection } from "../../tests/helpers/pins.ts"; const runtimes = ["native", "docker"] as const; -const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); +const liveStackApi = stackApiLayer.pipe(Layer.provide([BunServices.layer, BunCrypto.layer])); describe("managed pg_dump against a live stack", { timeout: 180_000 }, () => { for (const runtime of runtimes) { diff --git a/apps/cli/src/command-internal/pg-net-guidance.ts b/apps/cli/src/command-internal/pg-net-guidance.ts index 600dfa1714..419926b750 100644 --- a/apps/cli/src/command-internal/pg-net-guidance.ts +++ b/apps/cli/src/command-internal/pg-net-guidance.ts @@ -1,10 +1,16 @@ +import { type ConfigFileRef, DEFAULT_CONFIG_FILE } from "./config-advice.ts"; import type { DbExecError } from "./db-connection.errors.ts"; /** Canonical remediation when local migrations need pg_net but webhooks are disabled. */ -export const ENABLE_LOCAL_WEBHOOKS_SUGGESTION = - "Add the following to supabase/config.toml and retry:\n\n" + - "[experimental.webhooks]\n" + - "enabled = true"; +export const enableLocalWebhooksSuggestion = ( + configFile: ConfigFileRef = DEFAULT_CONFIG_FILE, +): string => + `Add the following to ${configFile.path} and retry:\n\n` + + (configFile.format === "json" + ? '{\n "experimental": {\n "webhooks": {\n "enabled": true\n }\n }\n}' + : "[experimental.webhooks]\nenabled = true"); + +export const ENABLE_LOCAL_WEBHOOKS_SUGGESTION = enableLocalWebhooksSuggestion(); const MISSING_NET_SCHEMA_PATTERN = /schema "net" does not exist/iu; const MISSING_PG_NET_FUNCTION_PATTERN = /function net\.http_[a-z0-9_]*\([^)]*\) does not exist/iu; diff --git a/apps/cli/src/command-internal/pgdelta-ssl.ts b/apps/cli/src/command-internal/pgdelta-ssl.ts deleted file mode 100644 index 9d981842c0..0000000000 --- a/apps/cli/src/command-internal/pgdelta-ssl.ts +++ /dev/null @@ -1,7 +0,0 @@ -/** - * The Supabase CA bundle for TLS-requiring remote Postgres endpoints (staging + production + - * snapshot roots, concatenated). The migra engine passes it to its edge-runtime script as - * `SSL_CA` when the SSL probe reports the server requires TLS. - */ -export const PG_DELTA_CA_BUNDLE = - "-----BEGIN CERTIFICATE-----\nMIID1DCCArygAwIBAgIUbYRdq/8/uNq8G9stMCdOFSBgA2MwDQYJKoZIhvcNAQEL\nBQAwczELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0RlbHdhcmUxEzARBgNVBAcMCk5l\ndyBDYXN0bGUxFTATBgNVBAoMDFN1cGFiYXNlIEluYzEmMCQGA1UEAwwdU3VwYWJh\nc2UgU3RhZ2luZyBSb290IDIwMjEgQ0EwHhcNMjEwNDI4MTAzNjEzWhcNMzEwNDI2\nMTAzNjEzWjBzMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRGVsd2FyZTETMBEGA1UE\nBwwKTmV3IENhc3RsZTEVMBMGA1UECgwMU3VwYWJhc2UgSW5jMSYwJAYDVQQDDB1T\ndXBhYmFzZSBTdGFnaW5nIFJvb3QgMjAyMSBDQTCCASIwDQYJKoZIhvcNAQEBBQAD\nggEPADCCAQoCggEBAN0AKRE8a56O8LaZxiOAcHFUFnwiKUvPoXPq26Ifw+Nv+7zg\nN2V5WnMZbbw24q61Os60ZUn0XmbVtuIeJ+stPHsO7qxxuL+bmPR+qU5tkDrIOyEe\nYD/2u8/q6ssVv42k4XcXbhM6RVz7CkCDY0TiBm1bMtRZso3xB6E9wAjxDf43XfV5\nPAGs3JI+Zo/vyqCDlN0hHOrB/aBl01JXqQWI84Gia5ooucq4SjA1CyawBcQ2IAvG\nrXuy1BouY+xM3zRuNvtfFP6rb5Mta+jCYEMh1AZ8yP8sYUWAyhxX6k9EbOb009wQ\naZljbUCh/UglGWuBxdzePavx+zPjzWXB1NyVkpkCAwEAAaNgMF4wCwYDVR0PBAQD\nAgEGMB0GA1UdDgQWBBQFx+PHLf27iIo/PMfIfGqXF7Zb+DAfBgNVHSMEGDAWgBQF\nx+PHLf27iIo/PMfIfGqXF7Zb+DAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEB\nCwUAA4IBAQB/xIiz5dDqzGXjqYqXZYx4iSfSxsVayeOPDMfmaiCfSMJEUG4cUiwG\nOvMPGztaUEYeip5SCvSKuAAjVkXyP7ahKR7t7lZ9mErVXyxSZoVLbOd578CuYiZk\nOgT17UjPv66WMzEKEr8wGpomTYWWfEkuqt8ENdiM1Z4LNFahdKj36+jm6/a+9R8K\n25VIL68DTaQpBxFWG6ixC1HRMHJ12lDhKsshIi099BVpkGibESlxPrQOdKKqBB/J\nvIX+/Hb+mS4H5zYMeK2wX0onp+GBcD6X9L1UJuXMVd+BRan8RFidXL5s3++xXjQq\nNzbc6lnA69urKffvcT07YwMsY/OmHzVa\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIDxDCCAqygAwIBAgIUbLxMod62P2ktCiAkxnKJwtE9VPYwDQYJKoZIhvcNAQEL\nBQAwazELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0RlbHdhcmUxEzARBgNVBAcMCk5l\ndyBDYXN0bGUxFTATBgNVBAoMDFN1cGFiYXNlIEluYzEeMBwGA1UEAwwVU3VwYWJh\nc2UgUm9vdCAyMDIxIENBMB4XDTIxMDQyODEwNTY1M1oXDTMxMDQyNjEwNTY1M1ow\nazELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0RlbHdhcmUxEzARBgNVBAcMCk5ldyBD\nYXN0bGUxFTATBgNVBAoMDFN1cGFiYXNlIEluYzEeMBwGA1UEAwwVU3VwYWJhc2Ug\nUm9vdCAyMDIxIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqQXW\nQyHOB+qR2GJobCq/CBmQ40G0oDmCC3mzVnn8sv4XNeWtE5XcEL0uVih7Jo4Dkx1Q\nDmGHBH1zDfgs2qXiLb6xpw/CKQPypZW1JssOTMIfQppNQ87K75Ya0p25Y3ePS2t2\nGtvHxNjUV6kjOZjEn2yWEcBdpOVCUYBVFBNMB4YBHkNRDa/+S4uywAoaTWnCJLUi\ncvTlHmMw6xSQQn1UfRQHk50DMCEJ7Cy1RxrZJrkXXRP3LqQL2ijJ6F4yMfh+Gyb4\nO4XajoVj/+R4GwywKYrrS8PrSNtwxr5StlQO8zIQUSMiq26wM8mgELFlS/32Uclt\nNaQ1xBRizkzpZct9DwIDAQABo2AwXjALBgNVHQ8EBAMCAQYwHQYDVR0OBBYEFKjX\nuXY32CztkhImng4yJNUtaUYsMB8GA1UdIwQYMBaAFKjXuXY32CztkhImng4yJNUt\naUYsMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAB8spzNn+4VU\ntVxbdMaX+39Z50sc7uATmus16jmmHjhIHz+l/9GlJ5KqAMOx26mPZgfzG7oneL2b\nVW+WgYUkTT3XEPFWnTp2RJwQao8/tYPXWEJDc0WVQHrpmnWOFKU/d3MqBgBm5y+6\njB81TU/RG2rVerPDWP+1MMcNNy0491CTL5XQZ7JfDJJ9CCmXSdtTl4uUQnSuv/Qx\nCea13BX2ZgJc7Au30vihLhub52De4P/4gonKsNHYdbWjg7OWKwNv/zitGDVDB9Y2\nCMTyZKG3XEu5Ghl1LEnI3QmEKsqaCLv12BnVjbkSeZsMnevJPs1Ye6TjjJwdik5P\no/bKiIz+Fq8=\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIDxzCCAq+gAwIBAgIUeX+gpfmsRW9asFkRvjyXjHxbfgcwDQYJKoZIhvcNAQEL\nBQAwazELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0RlbHdhcmUxEzARBgNVBAcMCk5l\ndyBDYXN0bGUxFTATBgNVBAoMDFN1cGFiYXNlIEluYzEeMBwGA1UEAwwVU3VwYWJh\nc2UgUm9vdCAyMDIxIENBMB4XDTI1MDkwMzA4MDEyNVoXDTM1MDkwMTA4MDEyNVow\nazELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0RlbHdhcmUxEzARBgNVBAcMCk5ldyBD\nYXN0bGUxFTATBgNVBAoMDFN1cGFiYXNlIEluYzEeMBwGA1UEAwwVU3VwYWJhc2Ug\nUm9vdCAyMDIxIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA5Ve7\ni9UAmc7luUilELPtqzEk8nGHxg7nY0aCStr625M7+K4OPO6RUllTsHh47k1jWyzm\nLXLlyYwCsYCjQp+3vn06H+F/HRUxBt6CK2B7bNng230exTunk0xFvfkX6YgHR7B3\n1B7L25Rq3PhuRFPV4hnGYRam2XBZC4UNPqoAgrhV0HOYzXXAVoTr2yaBTMnB331Z\nRwOmINh7eqTCk/JRZbb6vfZOhZRAVAe9AoRLoG8aKwmeoLGwlu0UuFx6z3E+6bmA\nfSNa8Lx02GEoCdPLw9IRKUFq/SgBpQUKm44H1fDwTjH2CMM0N4p0mL/6wXnNeHvt\nC40MmKZ0RcVmHE5wBwIDAQABo2MwYTAdBgNVHQ4EFgQUjvEE541toZcwtXQlZlcB\nYOBRTnowHwYDVR0jBBgwFoAUjvEE541toZcwtXQlZlcBYOBRTnowDwYDVR0TAQH/\nBAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQELBQADggEBACD5IcGP\nXKvS9qg0CgEQPFqYavt5c7P+0xxFgiZe+xoG8fUw58yNeK2APtgGPRpxEOGfAlNx\nz9HDt4gcyHEE00B3qAVDm49pqNxioFWzNqU2LGfM/HL1QmN6urR7hCOkVCJddvOc\nFhFX4nZDuRfaBboDvS5HlK3Pzxddp9hvrJi2bemr8HLqYc3HzmVckgPGSLML6t+h\n4LRCXSlQsDgQ1LZ4KHsl4cq7K51N6FOXQBLB5q4lMKhs0VUhCT8Pdsj12+84laCV\nc22q6p2mdT9SaernCSRnWazXWisgpjv3H7Ex4S1DCYjJIwn3PUToGFv1r8YRN2/S\nO19yVSxxCIf64Sg=\n-----END CERTIFICATE-----\n"; diff --git a/apps/cli/src/command-internal/pgdelta-ssl.unit.test.ts b/apps/cli/src/command-internal/pgdelta-ssl.unit.test.ts deleted file mode 100644 index 68844c665d..0000000000 --- a/apps/cli/src/command-internal/pgdelta-ssl.unit.test.ts +++ /dev/null @@ -1,9 +0,0 @@ -import { describe, expect, it } from "vitest"; - -import { PG_DELTA_CA_BUNDLE } from "./pgdelta-ssl.ts"; - -describe("PG_DELTA_CA_BUNDLE", () => { - it("concatenates the three Supabase CA certificates", () => { - expect(PG_DELTA_CA_BUNDLE.match(/BEGIN CERTIFICATE/g)).toHaveLength(3); - }); -}); diff --git a/apps/cli/src/command-internal/pgdelta.ts b/apps/cli/src/command-internal/pgdelta.ts index 2e8e35090a..3ee8f4488f 100644 --- a/apps/cli/src/command-internal/pgdelta.ts +++ b/apps/cli/src/command-internal/pgdelta.ts @@ -16,7 +16,7 @@ export interface PgDeltaContext { * forwarded to the edge-runtime container so migra runs under the configured Deno image. */ readonly denoVersion: number; - /** The project's parsed `supabase/.env` (`readDbToml`'s `projectEnv`). */ + /** The project's parsed `supabase/.env` (`readDbConfig`'s `projectEnv`). */ readonly projectEnv: Readonly>; } diff --git a/apps/cli/src/command-internal/project-lifecycle-lock.integration.test.ts b/apps/cli/src/command-internal/project-lifecycle-lock.integration.test.ts index e38174ca5c..11676f8740 100644 --- a/apps/cli/src/command-internal/project-lifecycle-lock.integration.test.ts +++ b/apps/cli/src/command-internal/project-lifecycle-lock.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Deferred, Effect, Exit, Fiber, FileSystem, Layer, Option, Path } from "effect"; import * as HttpClient from "effect/http/HttpClient"; @@ -197,7 +197,7 @@ describe("project lifecycle lock", () => { yield* Fiber.join(stopping); expect(docker.containerNames()).toEqual([]); expect(docker.networkNames()).toEqual([]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 10_000, ); @@ -229,7 +229,7 @@ describe("project lifecycle lock", () => { yield* Fiber.join(stopping); expect(docker.containerNames()).toEqual([]); expect(docker.networkNames()).toEqual([]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 10_000, ); @@ -263,7 +263,7 @@ describe("project lifecycle lock", () => { yield* Fiber.join(stopping); yield* Fiber.join(starting); expect(docker.networkNames()).toEqual([NETWORK]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 10_000, ); @@ -296,7 +296,7 @@ describe("project lifecycle lock", () => { yield* Fiber.join(startingBeta); expect(docker.containerNames()).toContain("supabase_db_alpha"); expect(docker.networkNames()).toEqual(["supabase_network_alpha", "supabase_network_beta"]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 10_000, ); @@ -337,7 +337,7 @@ describe("project lifecycle lock", () => { yield* Fiber.join(secondStarting); expect(docker.networkNames()).toEqual([NETWORK]); expect(docker.containerNames()).toContain("supabase_auth_demo"); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 10_000, ); @@ -370,7 +370,7 @@ describe("project lifecycle lock", () => { yield* Fiber.join(yield* runStop(stopper)); expect(docker.containerNames()).toEqual([]); expect(docker.networkNames()).toEqual([]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), 10_000, ); @@ -402,7 +402,7 @@ describe("project lifecycle lock", () => { expect(yield* Fiber.join(blocked)).toBeInstanceOf(ProjectLifecycleLockBusyError); expect(docker.networkNames()).toEqual([NETWORK]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); for (const format of ["json", "stream-json"] as const) { @@ -428,7 +428,7 @@ describe("project lifecycle lock", () => { expect(yield* Fiber.join(blocked)).toBeInstanceOf(ProjectLifecycleLockBusyError); expect(waiter.out.rawChunks).toEqual([]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); } @@ -451,7 +451,7 @@ describe("project lifecycle lock", () => { expect(docker.containerNames()).toContain("supabase_db_demo"); expect(cli.out.stderrText).not.toContain("Waiting for another Supabase CLI command"); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); it.effect("stops the stack without the lock", () => @@ -468,7 +468,7 @@ describe("project lifecycle lock", () => { expect(docker.containerNames()).toEqual([]); expect(docker.networkNames()).toEqual([]); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), ); }); }); diff --git a/apps/cli/src/command-internal/project-lifecycle-lock.ts b/apps/cli/src/command-internal/project-lifecycle-lock.ts index 609f176ca7..34fb543d8d 100644 --- a/apps/cli/src/command-internal/project-lifecycle-lock.ts +++ b/apps/cli/src/command-internal/project-lifecycle-lock.ts @@ -5,17 +5,8 @@ import { isLockBusy, type NamespaceError, } from "@supabase/stack/effect"; -import { - Clock, - Crypto, - Data, - Duration, - Effect, - FileSystem, - Option, - Path, - type Scope, -} from "effect"; +import { Clock, Data, Duration, Effect, FileSystem, Option, Path, type Scope } from "effect"; +import { Digest } from "effect/crypto"; import { CommandSettings } from "../config/command-settings.service.ts"; import { Output } from "../shared/output/output.service.ts"; @@ -65,7 +56,7 @@ export const holdProjectLifecycleLock = Effect.fn("ProjectLifecycleLock.hold")(f const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const output = yield* Output; - const crypto = yield* Crypto.Crypto; + const crypto = yield* Digest.Digest; const debugLogger = yield* Effect.serviceOption(DebugLogger); const allProjects = projectId.length === 0; @@ -115,7 +106,7 @@ export const holdProjectLifecycleLock = Effect.fn("ProjectLifecycleLock.hold")(f } const digest = yield* crypto.digest( - "SHA-256", + Digest.sha256, new TextEncoder().encode(cliProjectFilterValue(projectId)), ); const key = Array.from(digest, (byte) => byte.toString(16).padStart(2, "0")).join(""); diff --git a/apps/cli/src/command-internal/resolve-token.unit.test.ts b/apps/cli/src/command-internal/resolve-token.unit.test.ts index e52ce3b99c..32830931bd 100644 --- a/apps/cli/src/command-internal/resolve-token.unit.test.ts +++ b/apps/cli/src/command-internal/resolve-token.unit.test.ts @@ -22,6 +22,7 @@ const settings = (accessToken: Option.Option>) => workdirEnvValue: Option.none(), githubToken: Option.none(), userAgent: "SupabaseCLI/test", + apiMaxRetries: Option.none(), }); const credentials = ( diff --git a/apps/cli/src/command-internal/resolved-config-context.ts b/apps/cli/src/command-internal/resolved-config-context.ts index dfd48e1982..bffb5a5ba5 100644 --- a/apps/cli/src/command-internal/resolved-config-context.ts +++ b/apps/cli/src/command-internal/resolved-config-context.ts @@ -79,7 +79,9 @@ export const describeConfigLoadFailure = (cause: unknown): string => cause._tag === "CliConfigFlagConflictError" || cause._tag === "CliConfigLoadError" || cause._tag === "CliConfigRemovedKeyError" || - cause._tag === "CliConfigRemovedEnvError") && + cause._tag === "CliConfigRemovedEnvError" || + cause._tag === "CliConfigFileConflictError" || + cause._tag === "CliConfigSchemaVersionError") && "message" in cause && typeof cause.message === "string" ? cause.message diff --git a/apps/cli/src/command-internal/resolved-config-context.unit.test.ts b/apps/cli/src/command-internal/resolved-config-context.unit.test.ts index 893c5dc73d..e9fa44ffe8 100644 --- a/apps/cli/src/command-internal/resolved-config-context.unit.test.ts +++ b/apps/cli/src/command-internal/resolved-config-context.unit.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, FileSystem, Layer, Path } from "effect"; @@ -18,18 +18,20 @@ const writeProjectEnv = (workdir: string, contents: string) => const supabaseDir = path.join(workdir, "supabase"); yield* fs.makeDirectory(supabaseDir, { recursive: true }); yield* fs.writeFileString(path.join(supabaseDir, ".env"), contents); - }).pipe(Effect.provide(BunServices.layer)); + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])); /** Runs against a shell environment of exactly `env`, with the real config service and an empty Docker config dir. */ const layerWithShellEnv = (workdir: string, env: Readonly> = {}) => Layer.fresh( Layer.mergeAll( BunServices.layer, + BunCrypto.layer, runtimeInfoLayer, cliConfigValuesLayer.pipe( Layer.provide( Layer.mergeAll( BunServices.layer, + BunCrypto.layer, mockOutput().layer, Layer.succeed(CliConfigFlagInputs, makeCliConfigFlagInputs()), ), diff --git a/apps/cli/src/command-internal/scratch-paths.integration.test.ts b/apps/cli/src/command-internal/scratch-paths.integration.test.ts index ae63a8112e..f7c2f46feb 100644 --- a/apps/cli/src/command-internal/scratch-paths.integration.test.ts +++ b/apps/cli/src/command-internal/scratch-paths.integration.test.ts @@ -1,6 +1,6 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect } from "effect"; +import { Effect, Layer } from "effect"; import { mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; import { pgdeltaScratchDir, scratchDir } from "./scratch-paths.ts"; @@ -12,7 +12,7 @@ const resolve = (kind: "legacy" | "stack", workdir: string) => pgdelta: pgdeltaScratchDir(workdir), }).pipe( Effect.provide([ - BunServices.layer, + Layer.merge(BunServices.layer, BunCrypto.layer), mockCommandSettings({ workdir, supabaseHome: "/home/.supabase" }), stackBackendLayer(kind), ]), diff --git a/apps/cli/src/command-internal/scratch-paths.ts b/apps/cli/src/command-internal/scratch-paths.ts index db9d3a1697..1f21b5d1d4 100644 --- a/apps/cli/src/command-internal/scratch-paths.ts +++ b/apps/cli/src/command-internal/scratch-paths.ts @@ -1,4 +1,5 @@ -import { Crypto, Effect, Path } from "effect"; +import { Effect, Path } from "effect"; +import { Digest } from "effect/crypto"; import { CommandSettings } from "../config/command-settings.service.ts"; import { legacyProjectStatePaths } from "../shared/config/legacy-project-state.paths.ts"; @@ -19,8 +20,8 @@ export const pgdeltaScratchDir = Effect.fnUntraced(function* (projectRoot: strin const path = yield* Path.Path; const root = path.join(yield* scratchDir(projectRoot), "pgdelta"); if (!(yield* usesStackState)) return root; - const digest = yield* (yield* Crypto.Crypto).digest( - "SHA-256", + const digest = yield* (yield* Digest.Digest).digest( + Digest.sha256, new TextEncoder().encode(path.resolve(projectRoot)), ); return path.join( diff --git a/apps/cli/src/command-internal/seed-buckets.ts b/apps/cli/src/command-internal/seed-buckets.ts index 8e0ced02b7..f7a8efd92d 100644 --- a/apps/cli/src/command-internal/seed-buckets.ts +++ b/apps/cli/src/command-internal/seed-buckets.ts @@ -18,6 +18,7 @@ import { type StorageCredentials, } from "./storage-credentials.ts"; import { currentStackBackend } from "./stack-backend.ts"; +import { type ConfigFileRef, configFileRefOf, DEFAULT_CONFIG_FILE } from "./config-advice.ts"; import { withStackStorageGuidance } from "./stack-storage.ts"; import { parseFileSizeLimit, resolveBucketProps } from "./storage-bucket-config.ts"; import { @@ -39,7 +40,6 @@ import { bucketObjectKey } from "../commands/seed/buckets/buckets.upload.ts"; // same tag `seedBucketsRun` itself fails with, without importing across command boundaries. export { SeedConfigLoadError }; -const CONFIG_PATH = "supabase/config.toml"; const UPLOAD_CONCURRENCY = 5; /** Whether `[storage.buckets]` or `[storage.vector.buckets]` declares anything to seed. */ @@ -150,6 +150,8 @@ export const seedBucketsRun = Effect.fnUntraced(function* < readonly credentials?: StorageCredentials; /** Explicit project root for config/env loading and `objects_path` resolution. */ readonly workdir?: string; + /** The project config file named in prune messages; defaults to the loaded config, else `supabase/config.toml`. */ + readonly configFile?: ConfigFileRef; /** * When true, overwrite/prune decisions resolve from consent alone and no prompt runs, so the * caller cannot consume a line of its parent's stdin. `start` sets this: it has already @@ -208,6 +210,11 @@ export const seedBucketsRun = Effect.fnUntraced(function* < )); // `--yes` OR `SUPABASE_YES`. const yes = opts.yes ?? (yield* resolveYesWithProjectEnv(projectEnvValues)); + const configPath = + opts.configFile?.path ?? + (context === undefined + ? DEFAULT_CONFIG_FILE.path + : configFileRefOf(path, workdir, context.resolvedConfig).path); const config = opts.resolvedConfig?.config ?? context?.config ?? decodeDefaultCliConfig({}); const document = opts.resolvedConfig?.document ?? context?.document; @@ -281,7 +288,13 @@ export const seedBucketsRun = Effect.fnUntraced(function* < // A hosted run never prompts per item: one consent covers every overwrite and prune. const confirmHostedWrites = projectRef === "" ? undefined : opts.confirmHostedWrites; if (confirmHostedWrites !== undefined) { - const details = yield* planHostedWrites(gateway, config, bucketPropsByName, existingBuckets); + const details = yield* planHostedWrites( + gateway, + config, + bucketPropsByName, + existingBuckets, + configPath, + ); if (details.length > 0) yield* confirmHostedWrites(details); } const approved = confirmHostedWrites !== undefined || yes; @@ -309,6 +322,7 @@ export const seedBucketsRun = Effect.fnUntraced(function* < gateway, Object.keys(config.storage.analytics.buckets), summary, + configPath, ); } @@ -323,6 +337,7 @@ export const seedBucketsRun = Effect.fnUntraced(function* < gateway, vectorBucketNames, summary, + configPath, ).pipe(Effect.catch((error) => handleVectorError(output, error, summary))); } @@ -426,6 +441,7 @@ const upsertVectorBuckets = Effect.fnUntraced(function* ( gateway: StorageGateway, configuredNames: ReadonlyArray, summary: SeedSummary, + configPath: string, ) { const existing = yield* gateway.listVectorBuckets; const existingSet = new Set(existing); @@ -448,14 +464,14 @@ const upsertVectorBuckets = Effect.fnUntraced(function* ( : yield* promptYesNo( output, yes, - `Bucket ${bold(name)} not found in ${bold(CONFIG_PATH)}. Do you want to prune it?`, + `Bucket ${bold(name)} not found in ${bold(configPath)}. Do you want to prune it?`, false, interactive, ); if (!prune) { if (promptless) { yield* output.raw( - `Keeping vector bucket ${bold(name)}: not declared in ${bold(CONFIG_PATH)}. Run ${bold("supabase seed buckets")} to prune.\n`, + `Keeping vector bucket ${bold(name)}: not declared in ${bold(configPath)}. Run ${bold("supabase seed buckets")} to prune.\n`, "stderr", ); } @@ -475,6 +491,7 @@ const upsertAnalyticsBuckets = Effect.fnUntraced(function* ( gateway: StorageGateway, configuredNames: ReadonlyArray, summary: SeedSummary, + configPath: string, ) { const existing = yield* gateway.listAnalyticsBuckets; const existingSet = new Set(existing); @@ -497,14 +514,14 @@ const upsertAnalyticsBuckets = Effect.fnUntraced(function* ( : yield* promptYesNo( output, yes, - `Bucket ${bold(name)} not found in ${bold(CONFIG_PATH)}. Do you want to prune it?`, + `Bucket ${bold(name)} not found in ${bold(configPath)}. Do you want to prune it?`, false, interactive, ); if (!prune) { if (promptless) { yield* output.raw( - `Keeping analytics bucket ${bold(name)}: not declared in ${bold(CONFIG_PATH)}. Run ${bold("supabase seed buckets")} to prune.\n`, + `Keeping analytics bucket ${bold(name)}: not declared in ${bold(configPath)}. Run ${bold("supabase seed buckets")} to prune.\n`, "stderr", ); } @@ -527,6 +544,7 @@ const planHostedWrites = Effect.fnUntraced(function* ( config: CliConfig, propsByName: ReadonlyMap, existingBuckets: ReadonlyArray, + configPath: string, ) { const details: Array = []; const existing = new Set(existingBuckets.map((bucket) => bucket.name)); @@ -553,7 +571,7 @@ const planHostedWrites = Effect.fnUntraced(function* ( const pruned = [...present].filter((name) => !declared.includes(name)); if (created.length > 0) details.push(`Create analytics buckets: ${joinNames(created)}`); if (pruned.length > 0) { - details.push(`Delete analytics buckets not declared in ${CONFIG_PATH}: ${joinNames(pruned)}`); + details.push(`Delete analytics buckets not declared in ${configPath}: ${joinNames(pruned)}`); } } @@ -574,7 +592,7 @@ const planHostedWrites = Effect.fnUntraced(function* ( const pruned = listed.value.filter((name) => !declaredVector.includes(name)); if (created.length > 0) details.push(`Create vector buckets: ${joinNames(created)}`); if (pruned.length > 0) { - details.push(`Delete vector buckets not declared in ${CONFIG_PATH}: ${joinNames(pruned)}`); + details.push(`Delete vector buckets not declared in ${configPath}: ${joinNames(pruned)}`); } } } diff --git a/apps/cli/src/command-internal/stack-api.ts b/apps/cli/src/command-internal/stack-api.ts index a69db5c5ad..38c191188b 100644 --- a/apps/cli/src/command-internal/stack-api.ts +++ b/apps/cli/src/command-internal/stack-api.ts @@ -1,4 +1,5 @@ -import { Context, Crypto, Effect, FileSystem, Layer, Option, Path, Scope } from "effect"; +import { Context, Effect, FileSystem, Layer, Option, Path, Scope } from "effect"; +import { Digest, SecureRandom } from "effect/crypto"; import { FetchHttpClient, HttpClient } from "effect/http"; import { ChildProcessSpawner } from "effect/process"; import { @@ -39,14 +40,16 @@ export const stackApiLayer = Layer.effect( Effect.gen(function* () { const fileSystem = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const crypto = yield* Crypto.Crypto; + const crypto = yield* SecureRandom.SecureRandom; + const digest = yield* Digest.Digest; const childProcess = yield* ChildProcessSpawner.ChildProcessSpawner; const http = yield* HttpClient.HttpClient; const provideServices = (effect: Effect.Effect) => effect.pipe( Effect.provideService(FileSystem.FileSystem, fileSystem), Effect.provideService(Path.Path, path), - Effect.provideService(Crypto.Crypto, crypto), + Effect.provideService(SecureRandom.SecureRandom, crypto), + Effect.provideService(Digest.Digest, digest), Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, childProcess), Effect.provideService(HttpClient.HttpClient, http), ); diff --git a/apps/cli/src/command-internal/stack-auth-config.ts b/apps/cli/src/command-internal/stack-auth-config.ts index 524c0644b8..dc594fc690 100644 --- a/apps/cli/src/command-internal/stack-auth-config.ts +++ b/apps/cli/src/command-internal/stack-auth-config.ts @@ -74,6 +74,7 @@ export const resolveAuthConfig = Effect.fn("StackAuthConfig.resolve")( double_confirm_changes: auth.email.double_confirm_changes, enable_confirmations: auth.email.enable_confirmations, secure_password_change: auth.email.secure_password_change, + require_current_password: auth.email.require_current_password, max_frequency: auth.email.max_frequency, otp_length: auth.email.otp_length, otp_expiry: auth.email.otp_expiry, diff --git a/apps/cli/src/command-internal/stack-bootstrap.ts b/apps/cli/src/command-internal/stack-bootstrap.ts index 6c7724111d..efb82990b2 100644 --- a/apps/cli/src/command-internal/stack-bootstrap.ts +++ b/apps/cli/src/command-internal/stack-bootstrap.ts @@ -4,7 +4,7 @@ import { Output } from "../shared/output/output.service.ts"; import { applyDatabaseWebhooks, ensureStackWebhookSchema } from "./db-bootstrap/db-setup.ts"; import { parseConnectionString } from "./db-config.parse.ts"; import { DbConnection, type DbSession } from "./db-connection.service.ts"; -import type { DbTomlValues } from "./db-config.toml-read.ts"; +import type { DbConfigValues } from "./db-config.read.ts"; import { migrateAndSeed } from "./migrate-and-seed.ts"; import { StackCatalogSetup, type StackCatalogSetupInput } from "./stack-catalog-setup.ts"; @@ -86,8 +86,8 @@ export const applyStackWebhooksOnly = ( interface StackMigrations { readonly workdir: string; readonly toml: Pick< - DbTomlValues, - "migrationsEnabled" | "seed" | "pgDelta" | "schemaPaths" | "webhooksEnabled" + DbConfigValues, + "migrationsEnabled" | "seed" | "pgDelta" | "schemaPaths" | "webhooksEnabled" | "configFile" >; readonly experimental: boolean; /** Applies migrations up to this version; omitted applies all of them. */ @@ -116,6 +116,7 @@ const applyStackMigrateAndSeed = ( pgDeltaEnabled: toml.pgDelta.enabled, schemaPaths: toml.schemaPaths, localDatabaseWebhooksEnabled: toml.webhooksEnabled, + configFile: toml.configFile, }); }), "postgres", @@ -123,7 +124,7 @@ const applyStackMigrateAndSeed = ( /** The catalog overlay declared by a project's `config.toml`. */ export const projectCatalogOverlay = ( - toml: Pick, + toml: Pick, workdir: string, ): StackCatalogSetupInput["overlay"] => ({ webhooks: "config", diff --git a/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts b/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts index dac1a0cd9e..a750f7389b 100644 --- a/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts +++ b/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Deferred, @@ -357,7 +357,7 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { buildOutput.layer, stackCatalogSetupLayer, dbConnectionLayer, - ).pipe(Layer.provideMerge(BunServices.layer)), + ).pipe(Layer.provideMerge(Layer.merge(BunServices.layer, BunCrypto.layer))), ), ); }, diff --git a/apps/cli/src/command-internal/stack-config.precedence.integration.test.ts b/apps/cli/src/command-internal/stack-config.precedence.integration.test.ts index bdaf084706..a7f8d75890 100644 --- a/apps/cli/src/command-internal/stack-config.precedence.integration.test.ts +++ b/apps/cli/src/command-internal/stack-config.precedence.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, it } from "@effect/vitest"; import { ConfigProvider, Effect, FileSystem, Layer, Path } from "effect"; @@ -153,6 +153,7 @@ describe("stack config precedence goldens", () => { const layer = Layer.mergeAll( BunServices.layer, + BunCrypto.layer, runtimeInfoLayer, mockCommandSettings({ workdir, supabaseHome: workdir }), cliConfigValuesTestLayer, diff --git a/apps/cli/src/command-internal/stack-config.ts b/apps/cli/src/command-internal/stack-config.ts index 2f89f78f36..23c8f839e7 100644 --- a/apps/cli/src/command-internal/stack-config.ts +++ b/apps/cli/src/command-internal/stack-config.ts @@ -8,7 +8,8 @@ import { } from "@supabase/stack/defaults"; import { type ServiceCreationInput as ServiceCreationType } from "@supabase/stack/effect"; import { type ArtifactSelection } from "@supabase/stack/internal/artifacts"; -import { Crypto, Effect, Data, FileSystem, Option, Path, Redacted, Schema } from "effect"; +import { Effect, Data, FileSystem, Option, Path, Redacted, Schema } from "effect"; +import { SecureRandom } from "effect/crypto"; import { FetchHttpClient } from "effect/http"; import { CliConfigKeys, type AnyCliConfigKey } from "../config/cli-config-keys.ts"; @@ -20,6 +21,7 @@ import type { import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { CLI_VERSION } from "../shared/cli/version.ts"; import { decryptSecretMap } from "../shared/config/vault-decrypt.ts"; +import { toEdgeRuntimeSecretsEnv } from "../shared/functions/functions-config.ts"; import { describeConfigLoadFailure, loadResolvedConfigContext, @@ -42,6 +44,7 @@ import { resolveAuthExternalProviders, } from "./local-config-values.ts"; import { generateAsymmetricLocalJwt } from "./local-jwt.ts"; +import { resolveEmailTemplatePaths } from "./config-validate.ts"; import { recordOrioleDbTelemetry } from "./db-image.ts"; import { resolveRemoteJwks, @@ -96,7 +99,7 @@ type StackConfigEffect = Effect.Effect< | FileSystem.FileSystem | Path.Path | RuntimeInfo - | Crypto.Crypto + | SecureRandom.SecureRandom | CommandSettings | CliConfigValues | R @@ -118,7 +121,7 @@ interface StackConfigOptions { ) => Effect.Effect< ArtifactSelection, StackConfigError | LockResolutionError | E, - FileSystem.FileSystem | Path.Path | Crypto.Crypto | R + FileSystem.FileSystem | Path.Path | SecureRandom.SecureRandom | R >; /** Fails with `frozen-missing` when there is no `supabase/config.toml` to lock. */ readonly frozen?: boolean; @@ -231,14 +234,6 @@ const firstDifference = (left: unknown, right: unknown, path: string): string | const configValidationError = (config: CliConfig): string | undefined => { const defaults = getDefaultCliConfig(); - if (config.auth.enabled) { - for (const [name, template] of Object.entries(config.auth.email.template)) - if (template.content_path !== "") - return `auth.email.template.${name}.content_path requires template serving, which is not supported by the experimental stack`; - for (const [name, notification] of Object.entries(config.auth.email.notification)) - if (notification.enabled && notification.content_path !== "") - return `auth.email.notification.${name}.content_path requires template serving, which is not supported by the experimental stack`; - } for (const { path, active } of unsupportedConfigPaths) { if (!active(config)) continue; const value = pathValue(config, path); @@ -467,13 +462,13 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ), catch: (cause) => new StackConfigError({ message: String(cause) }), }); + const resolvedSecrets = yield* resolveConfigSubtree( + resolvedConfig, + validatedConfig.edge_runtime.secrets ?? {}, + "edge_runtime.secrets", + ).pipe(Effect.mapError(toStackConfigError)); const functionsEnvResult = decryptSecretMap( - Object.fromEntries( - Object.entries(validatedConfig.edge_runtime.secrets ?? {}).map(([key, value]) => [ - key, - value ?? "", - ]), - ), + toEdgeRuntimeSecretsEnv(resolvedSecrets), resolvedConfig.dotenvPrivateKeys, ); if (!functionsEnvResult.ok) @@ -544,10 +539,48 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( config: {}, endpoints: { http: endpoint(undefined) }, }; + const { template, notification } = validatedConfig.auth.email; + const emailTemplates = Effect.forEach( + !validatedConfig.auth.enabled + ? [] + : [ + ...Object.entries(template).map(([name, entry]) => ({ + section: "template" as const, + name, + id: name, + contentPath: entry.content_path, + })), + ...Object.entries(notification) + .filter(([, entry]) => entry.enabled) + .map(([name, entry]) => ({ + section: "notification" as const, + name, + id: `${name}_notification`, + contentPath: entry.content_path, + })), + ], + ({ section, name, id, contentPath }) => + resolveEmailTemplatePaths({ + section, + name, + contentPath, + contentPresent: false, + base: projectRoot, + }).pipe( + Effect.map((paths) => + paths === undefined ? [] : [{ id, path: paths.candidate, root: projectRoot }], + ), + ), + ).pipe( + Effect.map((templates) => templates.flat()), + Effect.provideService(FileSystem.FileSystem, yield* FileSystem.FileSystem), + Effect.provideService(Path.Path, path), + Effect.mapError(toStackConfigError), + ); const createCreations = ( stackId: string, ): Effect.Effect, StackConfigError> => - Effect.sync(() => { + Effect.map(emailTemplates, (templates) => { return [ { service: "database", @@ -617,6 +650,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( config: { ...authConfig, ...(jwtSecret === undefined ? {} : { jwtSecret: Redacted.value(jwtSecret) }), + ...(templates.length === 0 ? {} : { templates }), }, endpoints: { http: endpoint(apiPort) }, } satisfies ServiceCreationType, diff --git a/apps/cli/src/command-internal/stack-functions-env.ts b/apps/cli/src/command-internal/stack-functions-env.ts index 4d7c33290e..09924f91a3 100644 --- a/apps/cli/src/command-internal/stack-functions-env.ts +++ b/apps/cli/src/command-internal/stack-functions-env.ts @@ -1,73 +1,27 @@ -import { Data, Effect, FileSystem, Predicate } from "effect"; +import { Effect, Option } from "effect"; import type { ServiceCreationInput } from "@supabase/stack/effect"; -import { parseDotEnv } from "../shared/config/dotenv.ts"; -import { - actionability, - type CliErrorActionabilityDeclaration, - ErrorActionabilityId, -} from "../shared/telemetry/error-actionability.ts"; +import { resolveFunctionsEnv } from "../shared/functions/functions-config.ts"; -export class StackFunctionsEnvError extends Data.TaggedError("StackFunctionsEnvError")<{ - readonly message: string; - readonly path: string; - readonly cause?: unknown; -}> { - get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { - return actionability.invalidConfig; - } -} - -/** Reads a Functions dotenv file and removes reserved Stack-provided variables. */ -export const readStackFunctionsEnv = Effect.fn("StackFunctionsEnv.read")(function* ( - filePath: string, - optional: boolean, -) { - const fs = yield* FileSystem.FileSystem; - const contents = yield* fs.readFileString(filePath).pipe( - Effect.catchTag("PlatformError", (error) => - optional && Predicate.isTagged(error.reason, "NotFound") - ? Effect.succeed("") - : Effect.fail( - new StackFunctionsEnvError({ - path: filePath, - message: `Unable to load environment file ${filePath}: ${error.message}`, - cause: error, - }), - ), - ), - ); - const parsed = yield* Effect.try({ - try: () => parseDotEnv(contents), - catch: (cause) => - new StackFunctionsEnvError({ - path: filePath, - message: `Unable to parse environment file ${filePath}: ${cause instanceof Error ? cause.message : String(cause)}`, - cause, - }), - }); - const env = Object.fromEntries( - Object.entries(parsed).filter(([key]) => !key.startsWith("SUPABASE_")), - ); - if (Object.keys(env).some((key) => !/^[A-Za-z_][A-Za-z0-9_]*$/u.test(key))) - return yield* new StackFunctionsEnvError({ - path: filePath, - message: `Environment names in ${filePath} must start with a letter or underscore and contain only letters, digits, and underscores.`, - }); - if (Object.values(env).some((value) => /[\0\r\n]/u.test(value))) - return yield* new StackFunctionsEnvError({ - path: filePath, - message: `Multiline environment values in ${filePath} are not supported by the stack backend.`, - }); - return env; -}); - -/** Adds the project Functions dotenv values to a Functions creation; configured values win. */ -export const withProjectFunctionsEnv = (creation: ServiceCreationInput) => +/** Adds the project Functions env, plus `debug` when set, to a Functions creation. */ +export const withProjectFunctionsConfig = ( + creation: ServiceCreationInput, + options: { readonly warn: boolean; readonly debug: boolean }, +) => creation.service === "functions" - ? readStackFunctionsEnv(`${creation.config.functionsRoot}/.env`, true).pipe( - Effect.map((env): ServiceCreationInput => ({ + ? resolveFunctionsEnv({ + secrets: creation.config.env ?? {}, + functionsDir: creation.config.functionsRoot, + envFile: Option.none(), + functionSlugs: [], + warn: options.warn, + }).pipe( + Effect.map(({ shared }): ServiceCreationInput => ({ ...creation, - config: { ...creation.config, env: { ...env, ...creation.config.env } }, + config: { + ...creation.config, + env: shared, + ...(options.debug ? { debug: true } : {}), + }, })), ) : Effect.succeed(creation); diff --git a/apps/cli/src/command-internal/stack-local-database.ts b/apps/cli/src/command-internal/stack-local-database.ts index ed610c193c..969b7ac379 100644 --- a/apps/cli/src/command-internal/stack-local-database.ts +++ b/apps/cli/src/command-internal/stack-local-database.ts @@ -14,7 +14,7 @@ import { currentStackBackend } from "./stack-backend.ts"; import { StackApi } from "./stack-api.ts"; import { loadStackConfig } from "./stack-config.ts"; import { lockedSelection } from "./stack-lock.ts"; -import { readDbToml } from "./db-config.toml-read.ts"; +import { readDbConfig } from "./db-config.read.ts"; import { catalogDatabaseServices } from "./stack-catalog-setup.ts"; import { resolveExperimentalWithProjectEnv } from "./global-flags.ts"; import { @@ -230,7 +230,7 @@ export const stackEnsurePostgresOnlyStarted = Effect.fn( const config = yield* loadStackConfig(settings.workdir, { selection: lockedSelection({ frozen: false, output }), }).pipe(Effect.mapError(startFailed)); - const toml = yield* readDbToml(fs, path, settings.workdir).pipe(Effect.mapError(startFailed)); + const toml = yield* readDbConfig(fs, path, settings.workdir).pipe(Effect.mapError(startFailed)); const experimental = yield* resolveExperimentalWithProjectEnv({ ...toml.projectEnv }); const existing = yield* stackForProject(api, settings, path).pipe(Effect.mapError(startFailed)); const keys = diff --git a/apps/cli/src/command-internal/stack-lock.integration.test.ts b/apps/cli/src/command-internal/stack-lock.integration.test.ts index d5f5aef027..adfedc24ed 100644 --- a/apps/cli/src/command-internal/stack-lock.integration.test.ts +++ b/apps/cli/src/command-internal/stack-lock.integration.test.ts @@ -1,8 +1,9 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { LOCK_SCHEMA_URL, writeLock, type Lock } from "@supabase/stack/effect"; import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; -import { Crypto, Effect, FileSystem, Path } from "effect"; +import { Effect, FileSystem, Path } from "effect"; +import { Digest } from "effect/crypto"; import { database15, @@ -22,12 +23,12 @@ import { selectionForRequest } from "./stack-project-selection.ts"; const inProject = ( body: ( projectRoot: string, - ) => Effect.Effect, + ) => Effect.Effect, ) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; return yield* body(yield* fs.makeTempDirectoryScoped({ prefix: "stack-lock-int-" })); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); const requestFor = ( projectRoot: string, diff --git a/apps/cli/src/command-internal/stack-lock.ts b/apps/cli/src/command-internal/stack-lock.ts index 2cd70b06bc..17eb781389 100644 --- a/apps/cli/src/command-internal/stack-lock.ts +++ b/apps/cli/src/command-internal/stack-lock.ts @@ -29,7 +29,6 @@ import { import { Clock, Config, - Crypto, DateTime, Effect, FileSystem, @@ -39,6 +38,7 @@ import { Result, Schema, } from "effect"; +import { Digest } from "effect/crypto"; import { FetchHttpClient } from "effect/http"; import { ensureGitignore, readProjectLink } from "../shared/config/project-link.ts"; @@ -747,7 +747,7 @@ export const withProjectLease = (projectRoot: string, effect: Effect.Ef holdLease(projectRoot).pipe(Effect.andThen(effect), Effect.scoped); /** - * Fails a project without `supabase/config.toml` that is frozen, explicitly or by `CI` with a + * Fails a project without a config file that is frozen, explicitly or by `CI` with a * lock present: with no config the lock cannot be checked, so the bundled versions must not * silently stand in for it. An explicitly frozen project with no lock fails on the missing lock. */ @@ -758,7 +758,7 @@ export const rejectFrozenWithoutConfig = Effect.fn("StackLock.rejectFrozenWithou if (Option.isSome(lock)) return yield* new LockResolutionError({ reason: "frozen-missing", - message: `supabase/config.toml not found, so ${LOCK_FILE} cannot be checked; commit supabase/config.toml next to it`, + message: `supabase/config.toml, config.json or config.jsonc not found, so ${LOCK_FILE} cannot be checked; commit your config file next to it`, }); if (frozen) return yield* new LockResolutionError({ @@ -1000,7 +1000,7 @@ export const lockedResolution = (options: { ): Effect.fn.Return< LockedResolution, StackConfigError | LockResolutionError, - FileSystem.FileSystem | Path.Path | Crypto.Crypto + FileSystem.FileSystem | Path.Path | Digest.Digest > { const requests = yield* requestsOf(request); const ci = yield* continuousIntegration; @@ -1111,7 +1111,7 @@ export const updatedLock = (options: { ): Effect.fn.Return< LockUpdate, StackConfigError | LockResolutionError | E, - FileSystem.FileSystem | Path.Path | Crypto.Crypto | CommandSettings | R + FileSystem.FileSystem | Path.Path | Digest.Digest | CommandSettings | R > { const requests = yield* requestsOf(request); const { update, published } = yield* Effect.gen(function* () { @@ -1266,7 +1266,7 @@ export const linkedLock = (options: { ): Effect.fn.Return< LinkedLock, StackConfigError | LockResolutionError | E | E2, - FileSystem.FileSystem | Path.Path | Crypto.Crypto | CommandSettings | R | R2 + FileSystem.FileSystem | Path.Path | Digest.Digest | CommandSettings | R | R2 > { const requests = yield* requestsOf(request); const ci = yield* continuousIntegration; @@ -1390,7 +1390,7 @@ export const outdatedServices = Effect.fn("StackLock.outdated")(function* (reque const newest: Effect.Effect< ResolvedPin | undefined, StackConfigError | LockResolutionError, - Crypto.Crypto + Digest.Digest > = reason !== undefined || isExactRelease(requested) ? Effect.undefined diff --git a/apps/cli/src/command-internal/stack-log-events.ts b/apps/cli/src/command-internal/stack-log-events.ts index f8ef296b09..e8c483a79c 100644 --- a/apps/cli/src/command-internal/stack-log-events.ts +++ b/apps/cli/src/command-internal/stack-log-events.ts @@ -5,7 +5,10 @@ import type { StreamEvent } from "../shared/output/types.ts"; export type LogSource = "history" | "live"; /** The `log-entry` or `log-marker` event of a record. */ -export const logEvent = (record: StackLogRecord, source: LogSource): StreamEvent => { +export const logEvent = ( + record: StackLogRecord, + source: LogSource, +): Extract => { const subject = { service: record.service, instance_id: record.instanceId }; if (record.kind === "stdout" || record.kind === "stderr") return { diff --git a/apps/cli/src/command-internal/stack-shadow.integration.test.ts b/apps/cli/src/command-internal/stack-shadow.integration.test.ts index d16481ab95..c90e162bf9 100644 --- a/apps/cli/src/command-internal/stack-shadow.integration.test.ts +++ b/apps/cli/src/command-internal/stack-shadow.integration.test.ts @@ -5,7 +5,7 @@ import { type Stack, } from "@supabase/stack/effect"; import { CliConfigSchema } from "@supabase/config"; -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Deferred, Effect, FileSystem, Layer, Option, Path, Ref, Schema } from "effect"; import { mockOutput } from "../../tests/helpers/mocks.ts"; @@ -327,7 +327,7 @@ describe("stack shadow databases", () => { const api = yield* StackApi; expect(yield* api.discover({ stateRoot: path.join(root, "stacks") })).toEqual([]); }).pipe(Effect.provide(layers)); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), 420_000, ); @@ -370,7 +370,7 @@ describe("stack shadow databases", () => { }).pipe(Effect.provide(layers)); for (const shadowRoot of roots) expect(yield* fs.exists(shadowRoot)).toBe(false); expect(output.stderrText).not.toContain("Failed to destroy"); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), 180_000, ); @@ -430,7 +430,7 @@ describe("stack shadow databases", () => { }), ); }).pipe(Effect.provide(layers)); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), 180_000, ); @@ -467,7 +467,7 @@ describe("stack shadow databases", () => { expect(output.stderrText).not.toContain("Failed to destroy"); expect(roots).toHaveLength(1); for (const root of roots) expect(yield* fs.exists(root)).toBe(false); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), 120_000, ); @@ -506,7 +506,7 @@ describe("stack shadow databases", () => { ), }); }), - ).pipe(Layer.provide(stackApiLayer), Layer.provide(BunServices.layer)); + ).pipe(Layer.provide(stackApiLayer), Layer.provide([BunServices.layer, BunCrypto.layer])); yield* stackWithShadowDatabase(input(fs, path, root), () => Effect.void, { runtime: "native", @@ -616,7 +616,7 @@ describe("stack shadow databases", () => { ); expect(pins).toEqual([target.pin]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); } @@ -731,7 +731,7 @@ describe("stack shadow databases", () => { ); expect(pins).toEqual([remote.pin]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); } @@ -767,6 +767,6 @@ describe("stack shadow databases", () => { "db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = 17.0.0.000 is not an OrioleDB build this CLI ships", ); expect(roots).toEqual([]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])), ); }); diff --git a/apps/cli/src/command-internal/stack-storage.native.integration.test.ts b/apps/cli/src/command-internal/stack-storage.native.integration.test.ts index 77f0170d10..8dbc746934 100644 --- a/apps/cli/src/command-internal/stack-storage.native.integration.test.ts +++ b/apps/cli/src/command-internal/stack-storage.native.integration.test.ts @@ -1,6 +1,7 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Crypto, Effect, FileSystem, Layer, Option, Path, Stream } from "effect"; +import { Effect, FileSystem, Layer, Option, Path, Stream } from "effect"; +import { SecureRandom } from "effect/crypto"; import { FetchHttpClient } from "effect/http"; import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; @@ -12,11 +13,7 @@ import { stackStorageEndpointFor, } from "./stack-storage.ts"; import { makeStorageGateway } from "./storage-gateway.ts"; -import { - StackApi, - stackApiLayer, - stackTargetResolverLayer, -} from "../commands/experimental/stack/stack.shared.ts"; +import { StackApi, stackApiLayer, stackTargetResolverLayer } from "./stack-target.ts"; import { mockCommandSettings, mockTelemetryStateTracked, @@ -25,7 +22,7 @@ import { mockOutput, mockTty } from "../../tests/helpers/mocks.ts"; import { CliArgs } from "../shared/cli/cli-args.service.ts"; import { stdinLayer } from "../shared/runtime/stdin.layer.ts"; import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; -import { ExperimentalFlag, YesFlag } from "./global-flags.ts"; +import { DebugFlag, ExperimentalFlag, YesFlag } from "./global-flags.ts"; import { stackStart } from "../commands/experimental/stack/start/start.handler.ts"; import { destroyTestStacks } from "../../tests/helpers/stack-cleanup.ts"; import { cliConfigValuesTestLayer } from "../../tests/helpers/config-values-layer.ts"; @@ -92,6 +89,7 @@ const makeLayers = (root: string) => { stackCatalogSetupLayer, dbConnectionLayer, Layer.succeed(ExperimentalFlag, false), + Layer.succeed(DebugFlag, false), Layer.succeed(YesFlag, false), Layer.succeed(CliArgs, { args: ["stack", "start"] }), Layer.succeed(CommandPlatformApiFactory, { make: Effect.die("unused") }), @@ -134,8 +132,8 @@ describe("native stack Storage gateway", () => { apiKey: credentials.apiKey, userAgent: "stack-storage-native-integration", }); - const crypto = yield* Crypto.Crypto; - const bucket = `native-${(yield* crypto.randomUUIDv4).slice(0, 8)}`; + const crypto = yield* SecureRandom.SecureRandom; + const bucket = `native-${(yield* crypto.uuidV4()).slice(0, 8)}`; const local = path.join(root, "payload.txt"); yield* fs.writeFileString(local, "native storage payload"); @@ -181,7 +179,7 @@ describe("native stack Storage gateway", () => { yield* destroyTestStacks(api, path.join(root, "stacks"), path.join(root, "cache")); }), ).pipe(Effect.provide(layers)); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide([BunServices.layer, BunCrypto.layer])), { timeout: 180_000 }, ); }); diff --git a/apps/cli/src/command-internal/stack-storage.ts b/apps/cli/src/command-internal/stack-storage.ts index 45b8fc9d84..3b0cec84c5 100644 --- a/apps/cli/src/command-internal/stack-storage.ts +++ b/apps/cli/src/command-internal/stack-storage.ts @@ -3,7 +3,7 @@ * endpoint; lazy Storage members may be woken by the first gateway request. */ -import { Data, Effect, Match, Option, Path } from "effect"; +import { Data, Effect, FileSystem, Match, Option, Path } from "effect"; import type { Observation, ServiceInstance, Stack } from "@supabase/stack/effect"; import { actionability, @@ -17,10 +17,18 @@ import { stackOpenProjectBy } from "./stack-local-database.ts"; import { sanitizeInlineName } from "./http-errors.ts"; import { StorageGatewayStatusError } from "./storage-gateway.errors.ts"; import type { StorageCredentials } from "./storage-credentials.ts"; +import { type ConfigFileRef, configSetting } from "./config-advice.ts"; +import { findConfigFileRef } from "./config-file-ref.ts"; const INSPECT_OR_RESTART_SUGGESTION = "Run supabase stack status to inspect the stack, or supabase stack restart."; +/** The instruction to enable Storage; without a known config file it stays format-neutral. */ +export const enableStorageAdvice = (configFile: ConfigFileRef | undefined): string => + configFile === undefined + ? "Set `storage.enabled` to `true` in your config file" + : `Set ${configSetting(configFile.format, "storage", "enabled", "true")} in ${configFile.path}`; + /** The stack cannot currently be reached to serve Storage requests. */ export class StackStorageUnavailableError extends Data.TaggedError("StackStorageUnavailableError")<{ readonly message: string; @@ -139,6 +147,7 @@ const storageCredentialsFrom = ( /** Resolves the composition Storage endpoint without launching the owner or any member. */ export const stackStorageEndpointFor = ( stack: Stack, + configFile?: ConfigFileRef, ): Effect.Effect => Effect.gen(function* () { const composition = yield* stack.composition.describe.pipe( @@ -155,8 +164,7 @@ export const stackStorageEndpointFor = ( if (storage === undefined) return yield* new StackStorageCapabilityError({ message: "Storage is disabled for this stack.", - suggestion: - "Set [storage] enabled = true in supabase/config.toml, then run supabase start without --exclude storage.", + suggestion: `${enableStorageAdvice(configFile)}, then run supabase start without --exclude storage.`, disabled: true, }); const database = members.find((instance) => instance.service === "database"); @@ -203,7 +211,7 @@ export const stackStorageEndpointFor = ( export const stackStorageEndpoint: Effect.Effect< StorageCredentials, StackStorageUnavailableError | StackStorageCapabilityError, - CommandSettings | StackApi | Path.Path + CommandSettings | StackApi | Path.Path | FileSystem.FileSystem > = Effect.gen(function* () { const opened = yield* stackOpenProjectBy( (cause) => new StackStorageUnavailableError({ message: cause.message }), @@ -213,7 +221,9 @@ export const stackStorageEndpoint: Effect.Effect< message: "No stack is registered for this project.", suggestion: "Run supabase start to create it.", }); - return yield* stackStorageEndpointFor(opened.value); + const settings = yield* CommandSettings; + const configFile = yield* findConfigFileRef(settings.workdir); + return yield* stackStorageEndpointFor(opened.value, configFile); }).pipe(Effect.scoped); /** @@ -239,7 +249,8 @@ export const withStackStorageGuidance = ( message: `The stack gateway returned HTTP ${error.status} for Storage${ body.length === 0 ? "." : `: ${body}` }`, - suggestion: "Run supabase stack logs to inspect Storage, then supabase stack restart.", + suggestion: + "Run supabase logs --service storage to inspect Storage, then supabase stack restart.", cause: error, }), ); diff --git a/apps/cli/src/commands/experimental/stack/stack.shared.ts b/apps/cli/src/command-internal/stack-target.ts similarity index 96% rename from apps/cli/src/commands/experimental/stack/stack.shared.ts rename to apps/cli/src/command-internal/stack-target.ts index d27ce4fa6c..7c2d502513 100644 --- a/apps/cli/src/commands/experimental/stack/stack.shared.ts +++ b/apps/cli/src/command-internal/stack-target.ts @@ -5,17 +5,14 @@ import { type StackError, } from "@supabase/stack/effect"; import { Context, Data, Effect, FileSystem, Layer, Option, Path, Schema } from "effect"; -import { CommandSettings } from "../../../config/command-settings.service.ts"; +import { CommandSettings } from "../config/command-settings.service.ts"; import { actionability, type CliErrorActionabilityDeclaration, ErrorActionabilityId, -} from "../../../shared/telemetry/error-actionability.ts"; -import { StackApi, stackApiLayer } from "../../../command-internal/stack-api.ts"; -import { - recordStackRuntimeTelemetry, - type StackRuntime, -} from "../../../command-internal/stack-runtime.ts"; +} from "../shared/telemetry/error-actionability.ts"; +import { StackApi, stackApiLayer } from "./stack-api.ts"; +import { recordStackRuntimeTelemetry, type StackRuntime } from "./stack-runtime.ts"; export { StackApi, stackApiLayer }; diff --git a/apps/cli/src/command-internal/status-values.ts b/apps/cli/src/command-internal/status-values.ts index 530e8146a9..8ffe97b92a 100644 --- a/apps/cli/src/command-internal/status-values.ts +++ b/apps/cli/src/command-internal/status-values.ts @@ -239,6 +239,8 @@ export const resolveStatusLocalState = Effect.fnUntraced(function* ( projectEnvValues?: Readonly>, /** `LoadedCliConfig.document` — see {@link resolveLocalConfigValues}'s doc comment. */ document?: Readonly>, + /** The loaded config file's path, so messages name it; `undefined` reads as `config.toml`. */ + configPath?: string, /** * An already-resolved {@link resolveLocalConfigValues} result to reuse instead of * re-deriving one. Callers that resolved `local` earlier in the same process (e.g. @@ -250,7 +252,8 @@ export const resolveStatusLocalState = Effect.fnUntraced(function* ( precomputedLocal?: LocalConfigValues, ) { const local = - precomputedLocal ?? (yield* resolveLocalConfigValues(config, hostname, workdir, document)); + precomputedLocal ?? + (yield* resolveLocalConfigValues(config, hostname, workdir, document, configPath)); const apiEnabled = config.api.enabled; const studioSectionEnabled = config.studio.enabled; diff --git a/apps/cli/src/command-internal/storage-credentials.integration.test.ts b/apps/cli/src/command-internal/storage-credentials.integration.test.ts index 4a8448d787..e9980637e4 100644 --- a/apps/cli/src/command-internal/storage-credentials.integration.test.ts +++ b/apps/cli/src/command-internal/storage-credentials.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { ConfigProvider, Effect, Layer } from "effect"; @@ -33,6 +33,7 @@ describe("resolveStorageCredentials for a remote project", () => { mockRuntimeInfo(), unusedStackServices, BunServices.layer, + BunCrypto.layer, ), ), ); diff --git a/apps/cli/src/command-internal/storage-gateway.ts b/apps/cli/src/command-internal/storage-gateway.ts index 6190f17106..0a8a3d8995 100644 --- a/apps/cli/src/command-internal/storage-gateway.ts +++ b/apps/cli/src/command-internal/storage-gateway.ts @@ -147,7 +147,7 @@ function localGatewayHint(port: string): string { return ( "The local Supabase API gateway did not return a valid HTTP response. " + `Another process may be listening on the configured API port ${port}. ` + - `Check the port with \`lsof -nP -iTCP:${port} -sTCP:LISTEN\`, then stop the conflicting process or set a different \`api.port\` in supabase/config.toml.` + `Check the port with \`lsof -nP -iTCP:${port} -sTCP:LISTEN\`, then stop the conflicting process or set a different \`api.port\` in your config file.` ); } diff --git a/apps/cli/src/command-internal/storage-gateway.unit.test.ts b/apps/cli/src/command-internal/storage-gateway.unit.test.ts index 0ae48a9be2..48fb95bd7b 100644 --- a/apps/cli/src/command-internal/storage-gateway.unit.test.ts +++ b/apps/cli/src/command-internal/storage-gateway.unit.test.ts @@ -7,7 +7,7 @@ import * as HttpClientResponse from "effect/http/HttpClientResponse"; import type * as HttpClientRequest from "effect/http/HttpClientRequest"; import { withEnvVar } from "../../tests/helpers/command-mocks.ts"; -import caProd2021 from "../commands/gen/types/templates/prod-ca-2021.ts"; +import caProd2021 from "./supabase-ca-templates/prod-ca-2021.ts"; import { KONG_LOCAL_TLS_CERT, KONG_LOCAL_TLS_KEY, diff --git a/apps/cli/src/commands/gen/types/templates/prod-ca-2021.ts b/apps/cli/src/command-internal/supabase-ca-templates/prod-ca-2021.ts similarity index 100% rename from apps/cli/src/commands/gen/types/templates/prod-ca-2021.ts rename to apps/cli/src/command-internal/supabase-ca-templates/prod-ca-2021.ts diff --git a/apps/cli/src/commands/gen/types/templates/prod-ca-2025.ts b/apps/cli/src/command-internal/supabase-ca-templates/prod-ca-2025.ts similarity index 100% rename from apps/cli/src/commands/gen/types/templates/prod-ca-2025.ts rename to apps/cli/src/command-internal/supabase-ca-templates/prod-ca-2025.ts diff --git a/apps/cli/src/commands/gen/types/templates/staging-ca-2021.ts b/apps/cli/src/command-internal/supabase-ca-templates/staging-ca-2021.ts similarity index 100% rename from apps/cli/src/commands/gen/types/templates/staging-ca-2021.ts rename to apps/cli/src/command-internal/supabase-ca-templates/staging-ca-2021.ts diff --git a/apps/cli/src/command-internal/supabase-tls.ts b/apps/cli/src/command-internal/supabase-tls.ts new file mode 100644 index 0000000000..138bdd995d --- /dev/null +++ b/apps/cli/src/command-internal/supabase-tls.ts @@ -0,0 +1,36 @@ +import { getDomain } from "tldts"; + +import type { PgConnInput } from "./db-connection.service.ts"; +import { isDirectDbHost } from "./pooler-fallback.ts"; +import { poolerHost, projectHost } from "./profile.ts"; +import caProd2021 from "./supabase-ca-templates/prod-ca-2021.ts"; +import caProd2025 from "./supabase-ca-templates/prod-ca-2025.ts"; +import caStaging2021 from "./supabase-ca-templates/staging-ca-2021.ts"; + +export function rootCaBundle() { + return `${caStaging2021}${caProd2021}${caProd2025}`; +} + +/** Built-in profiles whose database servers chain to {@link rootCaBundle}. */ +const PINNED_PROFILES: ReadonlySet = new Set(["supabase", "supabase-staging", "snap"]); + +/** + * Pins the bundled Supabase CA (`require` plus an inline root CA, which the driver promotes to + * `verify-ca`) on a connection to a hosted Supabase database of the active built-in profile. A + * root CA set without `sslmode` replaces the bundle but keeps `require`, since the unset default + * (`prefer`) would ignore it. A connection that sets `sslmode` from any source, a host outside + * the profile's project and pooler domains, and every other profile (`supabase-local`, YAML) are + * returned unchanged. + */ +export function pinSupabaseTls(conn: PgConnInput, profile: string): PgConnInput { + if (!PINNED_PROFILES.has(profile) || conn.sslmode !== undefined) return conn; + const host = conn.host.toLowerCase().replace(/\.$/, ""); + const domain = getDomain(host); + const hosted = + isDirectDbHost(host, projectHost(profile)) || + (domain !== null && domain.toLowerCase() === poolerHost(profile)); + if (!hosted) return conn; + return conn.sslrootcert !== undefined || conn.sslrootcertInline !== undefined + ? { ...conn, sslmode: "require" } + : { ...conn, sslmode: "require", sslrootcertInline: rootCaBundle() }; +} diff --git a/apps/cli/src/command-internal/supabase-tls.unit.test.ts b/apps/cli/src/command-internal/supabase-tls.unit.test.ts new file mode 100644 index 0000000000..c3bec69a6e --- /dev/null +++ b/apps/cli/src/command-internal/supabase-tls.unit.test.ts @@ -0,0 +1,55 @@ +import { describe, expect, it } from "vitest"; + +import type { PgConnInput } from "./db-connection.service.ts"; +import { pinSupabaseTls, rootCaBundle } from "./supabase-tls.ts"; + +const conn = (host: string, overrides: Partial = {}): PgConnInput => ({ + host, + port: 5432, + user: "postgres", + password: "pw", + database: "postgres", + ...overrides, +}); + +describe("pinSupabaseTls host matching", () => { + it.each([ + "db.ref.supabase.co", + "DB.REF.SUPABASE.CO", + "db.ref.supabase.co.", + "aws-0-us-east-1.pooler.supabase.com", + "AWS-0-US-EAST-1.POOLER.SUPABASE.COM.", + ])("pins %s", (host) => { + expect(pinSupabaseTls(conn(host), "supabase")).toMatchObject({ + sslmode: "require", + sslrootcertInline: rootCaBundle(), + }); + }); + + it.each([ + "db.x.evilsupabase.co", + "db.ref.supabase.co.evil.com", + "supabase.co", + "localhost", + "127.0.0.1", + "::1", + "2600:1f18::1", + ])("leaves %s unpinned", (host) => { + const input = conn(host); + expect(pinSupabaseTls(input, "supabase")).toBe(input); + }); + + it("leaves a connection that chose its own sslmode unchanged", () => { + const input = conn("db.ref.supabase.co", { sslmode: "disable" }); + expect(pinSupabaseTls(input, "supabase")).toBe(input); + }); + + it("requires TLS verified against a root CA set without sslmode", () => { + const pinned = pinSupabaseTls( + conn("db.ref.supabase.co", { sslrootcert: "/ca.pem" }), + "supabase", + ); + expect(pinned).toMatchObject({ sslmode: "require", sslrootcert: "/ca.pem" }); + expect(pinned.sslrootcertInline).toBeUndefined(); + }); +}); diff --git a/apps/cli/src/command-internal/tabular-output.ts b/apps/cli/src/command-internal/tabular-output.ts new file mode 100644 index 0000000000..26b670e240 --- /dev/null +++ b/apps/cli/src/command-internal/tabular-output.ts @@ -0,0 +1,79 @@ +import { stringWidth } from "./string-width.ts"; + +// Width counts East Asian Wide as 2 and zero-width/combining as 0, so CJK/emoji cells still +// align columns and borders; counting JS code points would under-measure them. +const displayWidth = (text: string): number => stringWidth(text); + +/** + * Render rows as the established box-layout table (header not upper-cased). + * Left aligned, one space of padding each side, Unicode box-drawing borders. + * An empty column set renders nothing (established empty-header output). + */ +export function renderTablewriter( + cols: ReadonlyArray, + data: ReadonlyArray>, + formatCell: (value: unknown, columnIndex: number) => string, +): string { + if (cols.length === 0) return ""; + const rows = data.map((row) => row.map((cell, columnIndex) => formatCell(cell, columnIndex))); + // Column width is the widest visual line, since a multi-line cell splits across stacked lines. + const widths = cols.map((col, i) => { + let width = displayWidth(col); + for (const row of rows) { + for (const line of (row[i] ?? "").split("\n")) width = Math.max(width, displayWidth(line)); + } + return width; + }); + + const segment = (width: number) => "─".repeat(width + 2); + const top = `┌${widths.map(segment).join("┬")}┐`; + const sep = `├${widths.map(segment).join("┼")}┤`; + const bottom = `└${widths.map(segment).join("┴")}┘`; + const renderLine = (cells: ReadonlyArray) => + `│${cells + .map((cell, i) => { + const width = widths[i]; + return ` ${cell}${width === undefined ? "" : " ".repeat(width - displayWidth(cell))} `; + }) + .join("│")}│`; + // A multiline cell splits across stacked bordered lines within the same + // logical row (other columns blank on continuation lines), no per-row separator. + const renderRow = (cells: ReadonlyArray): string => { + const split = cells.map((cell) => cell.split("\n")); + const lineCount = Math.max(1, ...split.map((s) => s.length)); + const visual: string[] = []; + for (let j = 0; j < lineCount; j++) { + visual.push(renderLine(split.map((s) => s[j] ?? ""))); + } + return visual.join("\n"); + }; + + const lines = [top, renderLine(cols), sep, ...rows.map(renderRow), bottom]; + return `${lines.join("\n")}\n`; +} + +/** The established CSV field-quoting rule. */ +function csvFieldNeedsQuotes(field: string): boolean { + if (field === "") return false; + if (field === "\\.") return true; + if (/[\n\r",]/.test(field)) return true; + return /\s/u.test(field[0] ?? ""); +} + +function csvField(field: string): string { + if (!csvFieldNeedsQuotes(field)) return field; + return `"${field.replaceAll('"', '""')}"`; +} + +/** The established CSV output (RFC4180, `\n` line terminator). */ +export function toCsv( + cols: ReadonlyArray, + data: ReadonlyArray>, + formatCell: (value: unknown, columnIndex: number) => string, +): string { + const lines = [cols.map(csvField).join(",")]; + for (const row of data) { + lines.push(row.map((value, columnIndex) => csvField(formatCell(value, columnIndex))).join(",")); + } + return `${lines.join("\n")}\n`; +} diff --git a/apps/cli/src/command-internal/target/resolve-project-target.integration.test.ts b/apps/cli/src/command-internal/target/resolve-project-target.integration.test.ts index 389637ca53..632054924e 100644 --- a/apps/cli/src/command-internal/target/resolve-project-target.integration.test.ts +++ b/apps/cli/src/command-internal/target/resolve-project-target.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Cause, Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; @@ -127,7 +127,9 @@ const runWith = ( resolver, api.layer, mockCommandSettings({ workdir: dir, projectId: Option.none() }), - commandRuntimeLayer(["functions", "deploy"]).pipe(Layer.provide(BunServices.layer)), + commandRuntimeLayer(["functions", "deploy"]).pipe( + Layer.provide([BunServices.layer, BunCrypto.layer]), + ), options.args === undefined ? Layer.empty : Layer.succeed(CliArgs, prepareCliArgs(root, options.args)), @@ -150,7 +152,7 @@ const runWith = ( branchRequests: api.requests.length, requestedUrls: api.requests.map((request) => request.url), }; - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); const run = (input: RunInput, options: RunOptions = {}) => runWith(input, options, (flags) => resolveProjectTarget(flags)); diff --git a/apps/cli/src/command-internal/target/resolve-target.integration.test.ts b/apps/cli/src/command-internal/target/resolve-target.integration.test.ts index 0406d717f4..f8dc4b5d68 100644 --- a/apps/cli/src/command-internal/target/resolve-target.integration.test.ts +++ b/apps/cli/src/command-internal/target/resolve-target.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Cause, @@ -207,7 +207,7 @@ const run = ( kind: attributes?.get("target.kind"), selection: attributes?.get("target.selection"), }; - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }).pipe(Effect.scoped, Effect.provide([BunServices.layer, BunCrypto.layer])); const succeeded = (exit: Exit.Exit): A => { if (!Exit.isSuccess(exit)) throw new Error(`expected success, got ${stringifyJson(exit)}`); diff --git a/apps/cli/src/command-internal/target/target-flags.ts b/apps/cli/src/command-internal/target/target-flags.ts index 00d318ea72..15dbb70973 100644 --- a/apps/cli/src/command-internal/target/target-flags.ts +++ b/apps/cli/src/command-internal/target/target-flags.ts @@ -6,7 +6,7 @@ import { REMOTE_FLAG_NAME } from "./remote-vocabulary.ts"; import type { ResolvedTarget } from "./resolve-target.ts"; import { remoteFlagText } from "./target-deprecation.ts"; -type TargetSubject = "database" | "Storage API"; +type TargetSubject = "database" | "Storage API" | "log source"; interface TargetFlagOptions { readonly subject: TargetSubject; diff --git a/apps/cli/src/command-internal/target/target-password.ts b/apps/cli/src/command-internal/target/target-password.ts index 7cd076ce5c..529aacb612 100644 --- a/apps/cli/src/command-internal/target/target-password.ts +++ b/apps/cli/src/command-internal/target/target-password.ts @@ -1,22 +1,25 @@ import { Effect, Option } from "effect"; +import { configKey } from "../config-advice.ts"; +import { findConfigFileRef } from "../config-file-ref.ts"; + import { SUGGEST_LINKED_PROJECT } from "../connect-errors.ts"; import { TargetPasswordConflictError } from "./target.errors.ts"; /** Fails when a non-empty `--password` accompanies a target that is not a linked project. */ -export const rejectTargetPassword = ( +export const rejectTargetPassword = Effect.fnUntraced(function* ( target: "local" | "db-url", password: Option.Option, - linkedCheckout = false, -): Effect.Effect => - Option.isNone(Option.filter(password, (value) => value.length > 0)) - ? Effect.void - : Effect.fail( - new TargetPasswordConflictError({ - target, - message: - target === "db-url" - ? "--password can't be used with --remote=. Put the password in the connection string: postgres://USER:PASSWORD@HOST:PORT/postgres" - : `--password only applies to a linked project. The local database uses [db].password from supabase/config.toml.${linkedCheckout ? ` ${SUGGEST_LINKED_PROJECT}` : ""}`, - }), - ); + linkedCheckout: boolean, + workdir: string, +) { + if (Option.isNone(Option.filter(password, (value) => value.length > 0))) return; + const configFile = target === "db-url" ? undefined : yield* findConfigFileRef(workdir); + return yield* new TargetPasswordConflictError({ + target, + message: + configFile === undefined + ? "--password can't be used with --remote=. Put the password in the connection string: postgres://USER:PASSWORD@HOST:PORT/postgres" + : `--password only applies to a linked project. The local database uses ${configKey(configFile.format, "db", "password")} from ${configFile.path}.${linkedCheckout ? ` ${SUGGEST_LINKED_PROJECT}` : ""}`, + }); +}); diff --git a/apps/cli/src/command-internal/test-db.handler.ts b/apps/cli/src/command-internal/test-db.handler.ts index 05f8b0ca6d..e8289ed013 100644 --- a/apps/cli/src/command-internal/test-db.handler.ts +++ b/apps/cli/src/command-internal/test-db.handler.ts @@ -10,7 +10,7 @@ import { import { CommandSettings } from "../config/command-settings.service.ts"; import { TelemetryState } from "../telemetry/telemetry-state.service.ts"; import { DbConfigResolver } from "./db-config.service.ts"; -import { readDbToml } from "./db-config.toml-read.ts"; +import { readDbConfig } from "./db-config.read.ts"; import { DbConnection, type PgConnInput } from "./db-connection.service.ts"; import { LocalDbRunningError } from "./db-bootstrap/local-db-running.ts"; import { DockerRun } from "./docker-run.service.ts"; @@ -192,7 +192,7 @@ export const testDb = Effect.fn("test.db")(function* (flags: TestDbFlags) { ? { _tag: "named" as const, name: networkId } : isLocal && backend.kind !== "stack" ? yield* Effect.gen(function* () { - const toml = yield* readDbToml(fs, path, cliSettings.workdir); + const toml = yield* readDbConfig(fs, path, cliSettings.workdir); return { _tag: "named" as const, name: `supabase_network_${toml.projectId}` }; }) : { _tag: "host" as const }; @@ -289,7 +289,7 @@ export const testDb = Effect.fn("test.db")(function* (flags: TestDbFlags) { } if (backend.kind === "stack") { const bundled = yield* BundledPostgresClient; - const toml = yield* readDbToml(fs, path, cliSettings.workdir); + const toml = yield* readDbConfig(fs, path, cliSettings.workdir); const pin = yield* databaseArtifactPin(String(toml.majorVersion)).pipe( Effect.mapError((cause) => new TestDbRunError({ message: cause.message, cause })), ); diff --git a/apps/cli/src/command-internal/test-db.layers.unit.test.ts b/apps/cli/src/command-internal/test-db.layers.unit.test.ts index fd37653934..fa7e111627 100644 --- a/apps/cli/src/command-internal/test-db.layers.unit.test.ts +++ b/apps/cli/src/command-internal/test-db.layers.unit.test.ts @@ -8,7 +8,7 @@ */ import { describe, expect, it } from "@effect/vitest"; -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { Effect, Layer, Option } from "effect"; import { @@ -77,6 +77,7 @@ function ambientStubs() { return Layer.mergeAll( BunServices.layer, + BunCrypto.layer, isolatedHomeLayer(tempRoot.current), mockTty(), mockProcessControl().layer, diff --git a/apps/cli/src/command-internal/test-db.native.integration.test.ts b/apps/cli/src/command-internal/test-db.native.integration.test.ts index e28679863c..df69cbdb4b 100644 --- a/apps/cli/src/command-internal/test-db.native.integration.test.ts +++ b/apps/cli/src/command-internal/test-db.native.integration.test.ts @@ -1,4 +1,4 @@ -import { BunServices } from "@effect/platform-bun"; +import { BunCrypto, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { FetchHttpClient } from "effect/http"; import { Cause, Effect, Exit, FileSystem, Layer, Option, Redacted } from "effect"; @@ -33,7 +33,7 @@ import { cliConfigValuesTestLayer } from "../../tests/helpers/config-values-laye import { selection } from "../../tests/helpers/pins.ts"; const runtimes = ["native", "docker"] as const; -const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); +const liveStackApi = stackApiLayer.pipe(Layer.provide([BunServices.layer, BunCrypto.layer])); const databaseCreation = { service: "database" as const, diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json index 69487a0395..76dff623b1 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json index b3411f8000..8201373c7b 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json index 69487a0395..76dff623b1 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json index 42ed444410..f8219edbe4 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json index a8c7202210..d31753b22f 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json index 42ed444410..f8219edbe4 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json index 5dbaa54e7d..eff666f09d 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json index 3e04de121e..12238a1728 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json index 2f7cf1d7e3..7c86c7eda0 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json index 2f7cf1d7e3..7c86c7eda0 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json index 69487a0395..76dff623b1 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json index 6450d106a5..0f072f9600 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json @@ -12,6 +12,10 @@ "storageEnabled": false, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json index 62551b56f7..1046bb5db1 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json index 73b419dc35..20f0cd22c2 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json @@ -12,6 +12,10 @@ "storageEnabled": true, "vaultNames": [] }, + "configFile": { + "format": "toml", + "path": "supabase/config.toml" + }, "denoVersion": 2, "exposedSchemas": [ "public", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json index 31aa4d291e..42b3019a5c 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/no-overrides.json @@ -64,6 +64,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -319,6 +320,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json index 1ad43f1575..6f4fd64405 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/project-dotenv.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -288,6 +289,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json index 4a2f20f873..4c3abc5ee9 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-ignored.json @@ -64,6 +64,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -319,6 +320,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false @@ -716,6 +718,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json index 4b6d8a4fcf..6962b0ecc0 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context-with-shell-env.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -280,6 +281,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json index da40f6b4d6..d1258fc988 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/remote-block-merged-via-context.json @@ -64,6 +64,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -304,6 +305,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json index e8476e18d0..71ea5e2a13 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-beats-dotenv.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -286,6 +287,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json index 41a7615602..2088819abb 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/stack-config/shell-env.json @@ -55,6 +55,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "template": {} }, @@ -280,6 +281,7 @@ "notification": {}, "otp_expiry": 3600, "otp_length": 6, + "require_current_password": false, "secure_password_change": false, "smtp": { "enabled": false diff --git a/apps/cli/src/command-internal/upgrade-notice.ts b/apps/cli/src/command-internal/upgrade-notice.ts index 9ec18c5941..588cfcdc25 100644 --- a/apps/cli/src/command-internal/upgrade-notice.ts +++ b/apps/cli/src/command-internal/upgrade-notice.ts @@ -9,6 +9,7 @@ import { stripVTControlCharacters } from "node:util"; +import { findCliProjectRoot } from "@supabase/config/effect"; import { Clock, Config, @@ -197,7 +198,6 @@ const resolveNoticeBaseDir = Effect.fnUntraced(function* ( env: Readonly>, isValueTakingFlagToken?: (token: string) => boolean, ) { - const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; // A set flag beats the env even when empty; an empty effective value falls // through to the ancestor walk. @@ -206,16 +206,7 @@ const resolveNoticeBaseDir = Effect.fnUntraced(function* ( if (explicit !== undefined && explicit !== "") { return path.resolve(cwd, explicit); } - let current = cwd; - while (true) { - for (const configFile of ["config.json", "config.toml"]) { - const configPath = path.join(current, "supabase", configFile); - if (yield* fs.exists(configPath).pipe(Effect.orElseSucceed(() => false))) return current; - } - const parent = path.dirname(current); - if (parent === current) return cwd; - current = parent; - } + return (yield* findCliProjectRoot(cwd)) ?? cwd; }); /** diff --git a/apps/cli/src/command-internal/workdir-project.ts b/apps/cli/src/command-internal/workdir-project.ts index 507484650d..81ac899b6c 100644 --- a/apps/cli/src/command-internal/workdir-project.ts +++ b/apps/cli/src/command-internal/workdir-project.ts @@ -34,9 +34,9 @@ export function missingProjectConfigMessage(input: { readonly explicitWorkdir: boolean; }): string { if (!input.explicitWorkdir) { - return "failed to read supabase/config.toml or supabase/config.json: file not found. Run `supabase init` to create one."; + return "failed to read supabase/config.toml, config.json or config.jsonc: file not found. Run `supabase init` to create one."; } - return `failed to read supabase/config.toml or supabase/config.json in ${sanitizeInlineName(input.workdir)}: file not found. --workdir/SUPABASE_WORKDIR is used exactly as given and no ancestor directory is searched, so it must name the directory that contains your supabase/ folder.`; + return `failed to read supabase/config.toml, config.json or config.jsonc in ${sanitizeInlineName(input.workdir)}: file not found. --workdir/SUPABASE_WORKDIR is used exactly as given and no ancestor directory is searched, so it must name the directory that contains your supabase/ folder.`; } /** diff --git a/apps/cli/src/command-internal/workdir-project.unit.test.ts b/apps/cli/src/command-internal/workdir-project.unit.test.ts index d82095139a..956cfc29ad 100644 --- a/apps/cli/src/command-internal/workdir-project.unit.test.ts +++ b/apps/cli/src/command-internal/workdir-project.unit.test.ts @@ -7,7 +7,7 @@ describe("missingProjectConfigMessage", () => { // Pinned byte-for-byte: `diff.e2e.test.ts`/`pull.e2e.test.ts` assert this // exact string when run with no `--workdir`. expect(missingProjectConfigMessage({ workdir: "/repo/sub", explicitWorkdir: false })).toBe( - "failed to read supabase/config.toml or supabase/config.json: file not found. Run `supabase init` to create one.", + "failed to read supabase/config.toml, config.json or config.jsonc: file not found. Run `supabase init` to create one.", ); }); diff --git a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md index 3dfd263281..102f878f26 100644 --- a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md @@ -10,28 +10,29 @@ command — see Notes). ## Files Read -| Path | Format | When | -| ------------------------------------------------------------------ | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `~/.supabase/access-token` | plain text | ensure-login token miss (env unset and keyring unavailable) | -| `/.env.example` | dotenv | optional; merged into the generated `.env` | -| `/supabase/{.env..local,.env.local,.env.,.env}` | dotenv | step I (`loadProjectEnvValues`), before config.toml validation and again inside `checkDbToml`; `` is `SUPABASE_ENV` (default `development`), `.env.local` is skipped when `SUPABASE_ENV=test`; first of the 4 files (in this order) to set a key wins, and this `supabase/` directory tier beats the workdir-root tier below — feeds config.toml `env(VAR)` expansion and the push step's `SUPABASE_YES` auto-confirm default | -| `/{.env..local,.env.local,.env.,.env}` | dotenv | same read as above; lower-precedence fallback tier, only consulted for a key none of the `supabase/` directory's 4 files above already set | -| `/supabase/config.toml` | TOML | native push step (embedded defaults used when absent) | -| `/supabase/.temp/pooler-url` | plain text | native push step's connection resolution, only when the direct `db..:5432` host is unreachable (IPv4-only network) — `resolveLinkedConn` falls back through the saved pooler URL `link.LinkServices` wrote in the earlier link-services step | -| `/supabase/migrations/` | directory | native push step, when `[db.migrations].enabled` (default true) | -| `/supabase/migrations/*.sql` | SQL | native push step, for each pending migration applied | -| seed files from `[db.seed].sql_paths` | SQL | native push step (`--include-seed` is always set; gated on `[db.seed].enabled`) | -| `/supabase/roles.sql` | SQL | native push step (`--include-roles` is always set; existence check + apply) | -| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | native push step, remote project, when set; see `db push` | +| Path | Format | When | +| ------------------------------------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `~/.supabase/access-token` | plain text | ensure-login token miss (env unset and keyring unavailable) | +| `/.env.example` | dotenv | optional; merged into the generated `.env` | +| `/supabase/{.env..local,.env.local,.env.,.env}` | dotenv | step I (`loadProjectEnvValues`), before config.toml validation and again inside `checkDbConfig`; `` is `SUPABASE_ENV` (default `development`), `.env.local` is skipped when `SUPABASE_ENV=test`; first of the 4 files (in this order) to set a key wins, and this `supabase/` directory tier beats the workdir-root tier below — feeds config.toml `env(VAR)` expansion and the push step's `SUPABASE_YES` auto-confirm default | +| `/{.env..local,.env.local,.env.,.env}` | dotenv | same read as above; lower-precedence fallback tier, only consulted for a key none of the `supabase/` directory's 4 files above already set | +| `/supabase/.temp/project-ref` | plain text | link-services step, to tell whether it names the new project | +| `/supabase/config.toml` | TOML | native push step (embedded defaults used when absent) | +| `/supabase/.temp/pooler-url` | plain text | native push step's connection resolution, only when the direct `db..:5432` host is unreachable (IPv4-only network) — `resolveLinkedConn` falls back through the saved pooler URL `link.LinkServices` wrote in the earlier link-services step | +| `/supabase/migrations/` | directory | native push step, when `[db.migrations].enabled` (default true) | +| `/supabase/migrations/*.sql` | SQL | native push step, for each pending migration applied | +| seed files from `[db.seed].sql_paths` | SQL | native push step (`--include-seed` is always set; gated on `[db.seed].enabled`) | +| `/supabase/roles.sql` | SQL | native push step (`--include-roles` is always set; existence check + apply) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | native push step, remote project, when set; see `db push` | ## Files Written | Path | Format | When | | ----------------------------------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `/supabase/config.toml` | TOML | blank/`scratch` path only (via `initProject`) | +| `/supabase/config.toml` | TOML | blank/`scratch` path only (via `initProject` with `force`, which also removes `supabase/config.json`/`config.jsonc`) | | `/