diff --git a/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md b/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md index fb4d3c40e4..dfccd50f28 100644 --- a/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/functions/serve/SIDE_EFFECTS.md @@ -19,13 +19,12 @@ ## Files Written -| Path | Format | When | -| ---------------------------------------------------------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| `~/.supabase/telemetry.json` | JSON | always, at command exit via `Effect.ensuring` | -| `/supabase/.temp/start-secrets/supabase_edge_runtime_/env/docker.env` | dotenv | per start, when single-line container env exists; passed via `--env-file`; mode `0600`; removed after the run | -| `/supabase/.temp/start-secrets/supabase_edge_runtime_/multiline-env/…` | shell + raw | per start, only when an env value contains a newline; bind-mounted read-only into the container; mode `0600`; removed after the run | +| Path | Format | When | +| --------------------------------------------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------- | +| `~/.supabase/telemetry.json` | JSON | always, at command exit via `Effect.ensuring` | +| `/supabase/.temp/start-secrets/supabase_edge_runtime_/env/docker.env` | dotenv | per start, when single-line container env exists; passed via `--env-file`; mode `0600`; removed after the run | -The env files hold secrets (JWT secret, anon/service-role keys, JWKS), so they are +The env file holds secrets (JWT secret, anon/service-role keys, JWKS), so it is written owner-only (`0600`, in `0700` directories) under the project's own `supabase/.temp/` (gitignored) — a deterministic, persistent path rather than `os.tmpdir()`, so `supabase stop` and a failed-start rollback can reclaim it via @@ -176,7 +175,7 @@ per-function `import_map.json` is not read and does not fail the command. - A container that stops on its own with exit code `0`, or that is torn down by an external supervisor (exit `129`/`130`/`131`/`143`, or `15` from edge-runtime's own `SIGTERM` exit, e.g. `supabase stop` or `docker stop` in another terminal), or that is already gone by the time a follow-up inspect runs, all end the command successfully — each prints its own distinct line (see Output above) rather than the user-initiated `Stopped serving …` line, so scrollback can tell "I stopped it" from "the runtime walked out" or "a supervisor tore it down". In a `functions serve &` CI step this means a runtime that exits on its own does not fail the step; the distinct message is the only signal, and a downstream failure otherwise only surfaces later as connection-refused. Exit `137` (SIGKILL) fails the command: `State.OOMKilled` separates a container that hit its memory limit, which reports the memory-allocation remediation, from one killed by something the CLI cannot identify. A non-OOM `137` gets one re-inspect first — if `supabase stop` force-killed a runtime that ignored `SIGTERM` and the follow-up prune has already removed the container, the command still ends successfully instead of failing on a container that is already gone. Any other non-zero container exit fails the command; the error message includes the container id. Only a watched-file change restarts the container itself — none of these outcomes ever restart it. - A `docker logs -f` re-attach (the daemon can close the stream while the container keeps running) resumes with `--since ` instead of replaying the full log history, and is capped at 5 consecutive re-attaches that forward no new output; exceeding the cap fails the command with a tagged error instead of looping forever. - On the log-stream path, the Docker-daemon-unreachable classification comes from a follow-up `docker container inspect` failure, not from `docker logs -f`'s own stderr text. That same follow-up inspect's `State.OOMKilled` also drives the memory-limit remediation on this path, so a `docker logs -f` failure over a container killed for exceeding its memory limit reports the same suggestion as the clean-exit case above, instead of a bare unattributable failure. -- The worker bootstrap template (`serve.main.ts`) is bundled into a single self-contained module with `jose` and the local path/status helpers inlined, so the edge-runtime worker boots without any network access (supabase/supabase#45570). The bundle is embedded at build time for shipped binaries and produced on demand (esbuild) when running from source. It is delivered into the created (not yet started) container as a `docker cp` stdin tar archive at `/root/index.ts` — never a single-file host bind mount, which materializes as an empty directory on daemons that cannot see the client's filesystem (remote `DOCKER_HOST`/Docker-context daemons, podman machines) and breaks bring-up with edge-runtime's "failed to determine entrypoint" (supabase/cli#6254). Only this bootstrap template is daemon-independent: user function sources, import maps, static files, and the multiline-env script directory (present only when an env value contains a newline) still arrive by host bind mounts, so they require a daemon that can see the project directory. +- The worker bootstrap template (`serve.main.ts`) is bundled into a single self-contained module with `jose` and the local path/status helpers inlined, so the edge-runtime worker boots without any network access (supabase/supabase#45570). The bundle is embedded at build time for shipped binaries and produced on demand (esbuild) when running from source. It is delivered into the created (not yet started) container as a `docker cp` stdin tar archive at `/root/index.ts` — never a single-file host bind mount, which materializes as an empty directory on daemons that cannot see the client's filesystem (remote `DOCKER_HOST`/Docker-context daemons, podman machines) and breaks bring-up with edge-runtime's "failed to determine entrypoint" (supabase/cli#6254). The multiline-env script and value files (present only when an env value contains a newline) ride in the same archive under `/root/.supabase/multiline-env/` as root-owned `0644` files that stay in the container until it is removed (supabase/cli#7603). User function sources, import maps, and static files still arrive by host bind mounts, so they require a daemon that can see the project directory. - The aggregated bind mount list is pruned before `docker create`: a bind is dropped when another bind of the same mode already supplies the same content at the same container path — a file bind nested inside an already-bound read-only package directory would otherwise make the bootstrap `docker cp` fail with `destination ":/" must be a directory` (supabase/supabase#50088). Pruned paths remain visible in the container through their covering parent mounts; the `--workdir` gate and the file-watch set are computed from the unpruned aggregate. - Existing local values declared under an import map's `scopes` are explicit read-only Docker mounts and may resolve outside the nearest Git root; each distinct out-of-root host path prints one `WARN` during bring-up, deduplicated across Functions sharing an import map. Such out-of-root mounts are excluded from the file-watch set per Function, so a scope target contributes no watch root of its own and cannot enlarge or destabilise the watcher; a path that another Function reaches through its ordinary binds is still watched. Other file-valued binds are watched through their immediate parent non-recursively, while directory binds remain recursive. Missing targets retain serve's existing skip behavior. - **Spec-strict import-map key matching:** bind mounts are computed by the functions import scanner (`walkImportPaths`/`substituteImportMapValue`, shared with `functions deploy` and `start`'s Edge Runtime bring-up), which matches import-map keys per the import-maps spec Deno/edge-runtime implement — exact match, or prefix match only for a `/`-suffixed key, so a bare-key prefix does not match and bind mounts are limited to spec-matching keys; an unwalkable target (`ENOTDIR` — a value routed through a file) is skipped with a `WARN`. diff --git a/apps/cli/src/commands/functions/serve/serve.integration.test.ts b/apps/cli/src/commands/functions/serve/serve.integration.test.ts index 704d862b73..7e9109fd08 100644 --- a/apps/cli/src/commands/functions/serve/serve.integration.test.ts +++ b/apps/cli/src/commands/functions/serve/serve.integration.test.ts @@ -134,7 +134,7 @@ vi.mock("../../../shared/functions/functions-docker.ts", () => import("effect"), import("@effect/platform-bun"), import("../../../command-internal/docker-registry.ts"), - ]).then(([actual, { Effect, FileSystem, Path }, { BunServices }, { getRegistryImageUrl }]) => ({ + ]).then(([actual, { Effect, FileSystem }, { BunServices }, { getRegistryImageUrl }]) => ({ ...actual, ensureDockerNetwork: (networkMode: string, projectId: string) => Effect.sync(() => { @@ -157,41 +157,15 @@ vi.mock("../../../shared/functions/functions-docker.ts", () => runChildProcess: (command: string, args: ReadonlyArray, options?: unknown) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; const envFile = args.flatMap((value, index) => args[index - 1] === "--env-file" ? [value] : [], )[0]; - const multilineEnvDir = args - .flatMap((value, index) => (args[index - 1] === "-v" ? [value] : [])) - .find((value) => value.endsWith(":/root/.supabase/multiline-env:ro,Z")) - ?.slice(0, -":/root/.supabase/multiline-env:ro,Z".length); const enrichedOptions = - envFile === undefined && multilineEnvDir === undefined + envFile === undefined ? options : { ...(typeof options === "object" && options !== null ? options : {}), - ...(envFile === undefined - ? {} - : { envFileContents: yield* fs.readFileString(envFile) }), - ...(multilineEnvDir === undefined - ? {} - : { - multilineEnvScript: yield* fs.readFileString( - path.join(multilineEnvDir, "multiline-env.sh"), - ), - multilineEnvFiles: Object.fromEntries( - yield* Effect.forEach( - (yield* fs.readDirectory(path.join(multilineEnvDir, "values"))).filter( - (name) => name.startsWith("env-"), - ), - (name) => - Effect.map( - fs.readFileString(path.join(multilineEnvDir, "values", name)), - (contents): readonly [string, string] => [name, contents], - ), - ), - ), - }), + envFileContents: yield* fs.readFileString(envFile), }; deployMockState.runCalls.push({ command, args: [...args], options: enrichedOptions }); const result = deployMockState.runHandler?.(command, args, options) ?? { @@ -1098,7 +1072,7 @@ describe("functions serve integration", () => { }, ); - it.live("mounts multiline env values without placing their contents in docker argv", () => { + it.live("copies multiline env values into the container without a bind or argv", () => { deployMockState.runHandler = (command, args) => { if (command !== "docker") { throw new Error(`unexpected process: ${command}`); @@ -1118,29 +1092,8 @@ describe("functions serve integration", () => { throw new Error(`unexpected docker args: ${args.join(" ")}`); }; - let multilineEnvDirWhenLogsStarted: string | undefined; - let multilineEnvDirExistedWhenLogsStarted = false; const childSpawner = mockDockerLogSpawner([ - { - exitCode: 1, - stderr: "error running container: exit 1", - onSpawn: () => - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const dockerRun = deployMockState.runCalls.find( - (call) => call.command === "docker" && call.args[0] === "create", - ); - if (dockerRun === undefined) { - throw new Error("expected docker create call before docker logs spawn"); - } - multilineEnvDirWhenLogsStarted = extractFlagValues(dockerRun.args, "-v") - .find((value) => value.endsWith(":/root/.supabase/multiline-env:ro,Z")) - ?.slice(0, -":/root/.supabase/multiline-env:ro,Z".length); - multilineEnvDirExistedWhenLogsStarted = - multilineEnvDirWhenLogsStarted !== undefined && - (yield* fs.exists(multilineEnvDirWhenLogsStarted)); - }).pipe(Effect.provide(BunServices.layer), Effect.orDie), - }, + { exitCode: 1, stderr: "error running container: exit 1" }, ]); const multilineValue = ["-----BEGIN KEY-----", "EOF_ENV_0", "line-3", "-----END KEY-----"].join( @@ -1176,40 +1129,52 @@ describe("functions serve integration", () => { expect(dockerRun.args.join(" ")).not.toContain(multilineValue); expect(dockerRun.args.join(" ")).not.toContain("EOF_ENV_0"); - const multilineBind = extractFlagValues(dockerRun.args, "-v").find((value) => - value.endsWith(":/root/.supabase/multiline-env:ro,Z"), - ); - expect(multilineBind).toBeDefined(); - if (multilineBind === undefined) { - throw new Error("expected multiline env bind"); - } + expect( + extractFlagValues(dockerRun.args, "-v").some((bind) => + bind.includes("/root/.supabase/multiline-env"), + ), + ).toBe(false); - const script = - Predicate.hasProperty(dockerRun.options, "multilineEnvScript") && - Predicate.isString(dockerRun.options.multilineEnvScript) - ? dockerRun.options.multilineEnvScript - : undefined; - const files = - Predicate.hasProperty(dockerRun.options, "multilineEnvFiles") && - isStringRecord(dockerRun.options.multilineEnvFiles) - ? dockerRun.options.multilineEnvFiles + const cp = deployMockState.runCalls.find( + (call) => call.command === "docker" && call.args[0] === "cp", + ); + const stdin = + cp !== undefined && Predicate.hasProperty(cp.options, "stdin") + ? cp.options.stdin : undefined; - - expect(script).toBeDefined(); - expect(files).toBeDefined(); + const isCpArchiveStream = (value: unknown): value is Stream.Stream => + Stream.isStream(value); + if (!isCpArchiveStream(stdin)) return yield* Effect.die("docker cp stdin was not a stream"); + const archiveBytes = (yield* Stream.runCollect(stdin))[0]; + if (!(archiveBytes instanceof Uint8Array)) { + return yield* Effect.die("docker cp stdin did not contain archive bytes"); + } + const files = yield* Effect.promise(() => new Bun.Archive(archiveBytes).files()); + const readEntry = (name: string) => { + const file = files.get(name); + return file === undefined ? Effect.void : Effect.promise(() => file.text()); + }; + const script = yield* readEntry("root/.supabase/multiline-env/multiline-env.sh"); expect(script).toContain( 'MULTILINE_SECRET="$(cat /root/.supabase/multiline-env/values/env-0; printf x)"', ); expect(script).toContain('export MULTILINE_SECRET="${MULTILINE_SECRET%x}"'); expect(script).not.toContain(multilineValue); expect(script).not.toContain("EOF_ENV_0"); - expect(files?.["env-0"]).toBe(multilineValue); - expect(multilineEnvDirWhenLogsStarted).toBeDefined(); - if (multilineEnvDirWhenLogsStarted === undefined) { - throw new Error("expected multiline env dir when docker logs started"); - } - expect(multilineEnvDirExistedWhenLogsStarted).toBe(true); - expect(yield* fs.exists(multilineEnvDirWhenLogsStarted)).toBe(false); + expect(yield* readEntry("root/.supabase/multiline-env/values/env-0")).toBe(multilineValue); + expect(files.has("root/index.ts")).toBe(true); + expect(dockerRun.args.at(-1)).toContain(". /root/.supabase/multiline-env/multiline-env.sh\n"); + expect( + yield* fs.exists( + path.join( + tempRoot.current, + "supabase", + ".temp", + "start-secrets", + "supabase_edge_runtime_test-project", + ), + ), + ).toBe(false); }).pipe(Effect.provide(BunServices.layer)); }); @@ -1235,13 +1200,6 @@ describe("functions serve integration", () => { throw new Error(`unexpected docker args: ${args.join(" ")}`); }; - const childSpawner = mockDockerLogSpawner([ - { - exitCode: 1, - stderr: "error running container: exit 1", - }, - ]); - return Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; @@ -1273,25 +1231,27 @@ describe("functions serve integration", () => { "stale secret\n", ); + let staleDirExistedAfterBringUp: boolean | undefined; + const childSpawner = mockDockerLogSpawner([ + { + exitCode: 1, + stderr: "error running container: exit 1", + onSpawn: () => + fs.exists(staleMultilineEnvDir).pipe( + Effect.map((exists) => { + staleDirExistedAfterBringUp = exists; + }), + Effect.orDie, + ), + }, + ]); const { layer } = setupServe({ childSpawner }); const error = yield* functionsServe(baseFlags()).pipe(Effect.provide(layer), Effect.flip); expect(error).toBeInstanceOf(Error); + expect(staleDirExistedAfterBringUp).toBe(false); expect(yield* fs.exists(staleMultilineEnvDir)).toBe(false); - - const dockerRun = deployMockState.runCalls.find( - (call) => call.command === "docker" && call.args[0] === "create", - ); - expect(dockerRun).toBeDefined(); - if (dockerRun === undefined) { - throw new Error("expected docker create call"); - } - expect( - extractFlagValues(dockerRun.args, "-v").some((value) => - value.endsWith(":/root/.supabase/multiline-env:ro,Z"), - ), - ).toBe(false); }).pipe(Effect.provide(BunServices.layer)); }, ); diff --git a/apps/cli/src/commands/start/SIDE_EFFECTS.md b/apps/cli/src/commands/start/SIDE_EFFECTS.md index b614c70e22..93df2b8067 100644 --- a/apps/cli/src/commands/start/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/start/SIDE_EFFECTS.md @@ -210,14 +210,14 @@ command. ## Files Written -| Path | Format | When | -| ---------------------------------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `/supabase/.branches/_current_branch` | text | on every start, only if absent — writes `"main"` | -| `/supabase/snippets/` | — | when Studio is enabled, only if absent — created before any container and kept on rollback; `podman create -v`, unlike Docker, rejects a missing bind source | -| `/supabase/.temp/start-secrets//{env,multiline-env}/` | varies | Edge Runtime's own JWT/service-role-key/secret env artifacts — see below | -| auth email `content_path` files (SELinux security label only, no content change) | xattr | when the container runtime applies SELinux labels (enforcing or permissive): the `rw,z` Kong bind relabels each mounted template to the shared container context; persists after `supabase stop` | -| `$SUPABASE_HOME/locks/docker-lifecycle.lock` | SQLite lock file | held for the whole run: shared for one project, exclusive for `stop --all`; waits up to 10 minutes for a conflicting command (skipped, with a debug line, when `$SUPABASE_HOME/locks` cannot be created or opened) | -| `$SUPABASE_HOME/locks/docker-project-.lock` | SQLite lock file | held for the whole run: serializes lifecycle commands for the same project; waits up to 10 minutes (skipped, with a debug line, when `$SUPABASE_HOME/locks` cannot be created or opened) | +| Path | Format | When | +| -------------------------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `/supabase/.branches/_current_branch` | text | on every start, only if absent — writes `"main"` | +| `/supabase/snippets/` | — | when Studio is enabled, only if absent — created before any container and kept on rollback; `podman create -v`, unlike Docker, rejects a missing bind source | +| `/supabase/.temp/start-secrets//env/` | dotenv | Edge Runtime's own JWT/service-role-key/secret env artifacts — see below | +| auth email `content_path` files (SELinux security label only, no content change) | xattr | when the container runtime applies SELinux labels (enforcing or permissive): the `rw,z` Kong bind relabels each mounted template to the shared container context; persists after `supabase stop` | +| `$SUPABASE_HOME/locks/docker-lifecycle.lock` | SQLite lock file | held for the whole run: shared for one project, exclusive for `stop --all`; waits up to 10 minutes for a conflicting command (skipped, with a debug line, when `$SUPABASE_HOME/locks` cannot be created or opened) | +| `$SUPABASE_HOME/locks/docker-project-.lock` | SQLite lock file | held for the whole run: serializes lifecycle commands for the same project; waits up to 10 minutes (skipped, with a debug line, when `$SUPABASE_HOME/locks` cannot be created or opened) | Kong's `custom_nginx.template`, Vector's `vector.yaml`, and Postgres's own bootstrap script (`postgresql.conf`-equivalent setup) are all rendered in memory and injected @@ -255,27 +255,27 @@ since the content already lives inside the container's own filesystem. Studio reads/writes SQL snippets under `/supabase/snippets/` at its own runtime; `start` only creates the empty directory (see Files Written). -Edge Runtime's own JWT/service-role-key/configured-secret env file and multiline-env -script + value files (`shared/functions/serve.ts`'s -`writeDockerEnvFile`/`writeDockerMultilineEnvScript`) are +Edge Runtime's own JWT/service-role-key/configured-secret env file +(`shared/functions/serve.ts`'s `writeDockerEnvFile`) is staged on host disk under `/supabase/.temp/start-secrets//{env,multiline-env}/` (directory mode `0700`, files mode `0600`) — the -env file is read client-side by `--env-file`, the multiline-env directory is -bind-mounted `:ro,Z` into the container — a deterministic, persistent path rather than +containerName>/env/` (directory mode `0700`, file mode `0600`) and read client-side +by `--env-file` — a deterministic, persistent path rather than `os.tmpdir()` (which is frequently tmpfs and gets wiped on reboot) so `cleanupStartSecrets` (see the Exit Codes/rollback section below) can reclaim -them on `stop` or a failed-start rollback. Each writer removes and -recreates its own subdirectory fresh on every call (self-healing), so a +it on `stop` or a failed-start rollback. The writer removes and +recreates its subdirectory fresh on every call (self-healing), so a shrinking env set never leaves stale files behind. The bootstrap `index.ts` template carries no secret content and, as of supabase/cli#6254, never touches host disk at all: it is streamed via `docker cp` straight into the created (not yet started) Edge Runtime container at `/root/index.ts` — a single-file host bind mount materializes as an empty directory on daemons that cannot see the client's filesystem (remote `DOCKER_HOST`/Docker-context daemons, podman machines), which broke `start` with edge-runtime's "failed to determine -entrypoint". Only the bootstrap template is daemon-independent: user function -sources under `supabase/functions/**` and the multiline-env script directory -(present only when a secret value contains a newline) still arrive by host bind -mounts and require a daemon that can see the project directory. +entrypoint". The multiline-env script and value files (present only when a secret value +contains a newline) ride in that same `docker cp` stream under +`/root/.supabase/multiline-env/` as root-owned `0644` files that stay in the container +until it is removed (supabase/cli#7603). User function sources under +`supabase/functions/**` still arrive by host bind mounts and require a daemon that can +see the project directory. ## API Routes @@ -488,7 +488,7 @@ prose, not structured data. `/supabase/.temp/start-secrets/` directory. As of supabase/cli#6022 this is a no-op for a removed Kong/Postgres/Supavisor container (nothing under its own directory anymore); it still matters for a removed Edge Runtime - container, whose own env-file/multiline-env-script staging is + container, whose own env-file staging is unaffected by that change. - Existing local values declared under a Function import map's `scopes` are mounted read-only into Edge Runtime, and into the Studio container that shares the same resolved Function bind mounts, even when they resolve outside the nearest Git root. The Edge Runtime container additionally prunes its aggregated bind list before `docker create` — a bind already supplied verbatim by a same-mode parent bind is dropped so the bootstrap `docker cp` never sees a file bind nested inside a read-only parent (supabase/supabase#50088) — while Studio keeps the full unpruned set (its bring-up performs no `docker cp`). The mounted target itself is bound as declared; imports reached from inside an out-of-root target are not additionally bound. Edge Runtime bring-up prints a `WARN` naming each distinct out-of-root host path once; Studio's bind resolution stays silent, so with Edge Runtime excluded (`-x edge-runtime`) the mounts still reach Studio and no warning is printed. Missing targets retain Edge Runtime startup's existing skip behavior. This is NOT the same permissiveness as email template `content_path` below — a `scopes` value is an explicit opt-in mechanism, mounted `:ro`, with containment enforced on the upload side elsewhere (`functions deploy`), whereas `content_path` has no opt-in at all and is enforced at resolution time, mounted `:rw,z`. - **Auth email `content_path` project-root containment AND readability apply here regardless of `auth.enabled`.** Kong's mount set (`resolveKongEmailTemplateMounts`, `start.handler.ts`) covers every configured `[auth.email.template.*]` entry, and every `enabled = true` `[auth.email.notification.*]` entry, unconditionally — Kong is the stack's mandatory gateway, independent of whether GoTrue itself is started. `start.handler.ts` resolves, confines (`resolveEmailTemplateContentPath`; symlinks dereferenced using `fs.realPath` before the check), AND read-verifies (a discarded `FileSystem.readFileString`) that same set in one eager pass before any Docker work, in addition to the `auth.enabled`-gated read `resolveLocalConfigValues`'s own validation performs; the resulting resolved path is threaded straight into Kong's bind mount (`buildKongEmailTemplateBind` in `kong.service.ts`, which no longer re-resolves anything) rather than re-derived later, right before Kong's own `docker create`. Each template is bound `rw,z`: when the container runtime applies SELinux labels (enforcing or permissive mode) it relabels the mounted host file in place with the shared container context so Kong can read it under enforcement — a persistent change to the file's security label that survives `supabase stop` — and the option is a no-op where the runtime has no SELinux support. A path that resolves outside the project root aborts the run with `Invalid config for auth.email.
..content_path: resolves outside the project root ()`; a resolved, in-root path that cannot be read as a regular file aborts with `Invalid config for auth.email.
..content_path: ` — both before a single container is created, and both regardless of `auth.enabled` (closing the gap where a missing `content_path` would otherwise reach an unconditional Kong bind mount and the root-privileged Docker daemon would silently create a directory there). diff --git a/apps/cli/src/commands/start/services/edge-runtime.service.ts b/apps/cli/src/commands/start/services/edge-runtime.service.ts index c725d139b5..ca2690e2a4 100644 --- a/apps/cli/src/commands/start/services/edge-runtime.service.ts +++ b/apps/cli/src/commands/start/services/edge-runtime.service.ts @@ -5,8 +5,8 @@ * * Delegates to `functions serve`'s bring-up (`shared/functions/serve.ts`) * rather than the shared `createContainer` spec, since Edge Runtime needs - * `--workdir`/`--ulimit` flags and a secret-delivery mechanism (env-file plus - * a bind-mounted multiline-env script) that spec can't express. + * `--workdir`/`--ulimit` flags and a secret-delivery mechanism (`--env-file`) + * that spec can't express. * `start.handler.ts` must call {@link startStackEdgeRuntimeContainer} directly, the * same way it already special-cases Postgres. * @@ -86,10 +86,9 @@ export interface EdgeRuntimeBringUpInput { * * Sets no Docker restart policy — unlike every other `start` service, Edge * Runtime's lifecycle is reconciled at the CLI level instead. Leave the - * returned `cleanup` unused on success: its bind-mounted host files must - * outlive the call for as long as the container can be reattached to, and - * `startEdgeRuntimeContainer` already runs `cleanup` on any failed or - * interrupted bring-up. + * returned `cleanup` unused on success: `stop` or a failed-start rollback + * reclaims the staged env file, and `startEdgeRuntimeContainer` already runs + * `cleanup` on any failed or interrupted bring-up. */ export const startStackEdgeRuntimeContainer = Effect.fn("start.edgeRuntime")(function* ( input: EdgeRuntimeBringUpInput, diff --git a/apps/cli/src/commands/start/start.handler.ts b/apps/cli/src/commands/start/start.handler.ts index 63bce0dfbf..93b3f9072d 100644 --- a/apps/cli/src/commands/start/start.handler.ts +++ b/apps/cli/src/commands/start/start.handler.ts @@ -1376,11 +1376,9 @@ export const start = Effect.fn("start")(function* (flags: StartFlags) { platform: runtimeInfo.platform, }; const runtime: StartedRuntime = yield* startStackEdgeRuntimeContainer(edgeRuntimeInput); - // Not calling `runtime.cleanup` here: unlike every other service (`createContainer`'s - // `restartPolicy: "unless-stopped"`), Edge Runtime sets no Docker restart policy, so - // its bind-mounted host temp files must still exist for as long as the container can - // be reattached to. `startStackEdgeRuntimeContainer` already runs cleanup internally on - // a failed or interrupted bring-up, so only the success path must leave it alone. + // Not calling `runtime.cleanup` here: `stop` or a failed-start rollback reclaims Edge + // Runtime's staged env file, and `startStackEdgeRuntimeContainer` already runs cleanup + // internally on a failed or interrupted bring-up. started.set(runtime.containerId, edgeRuntimeInput.image); edgeRuntimeGateway = { containerId: runtime.containerId, diff --git a/apps/cli/src/commands/stop/SIDE_EFFECTS.md b/apps/cli/src/commands/stop/SIDE_EFFECTS.md index 69372de140..71d8028fe4 100644 --- a/apps/cli/src/commands/stop/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/stop/SIDE_EFFECTS.md @@ -35,7 +35,7 @@ model (see the CLI-1324 plan's "Critical architectural finding" for why). The `start-secrets` removal is a hygiene step (`cleanupStartSecrets`, `command-internal/start-secrets-cleanup.ts`). Only Edge Runtime's own JWT/service-role-key/secret env artifacts (`shared/functions/serve.ts`'s -`writeDockerEnvFile`/`writeDockerMultilineEnvScript`) still +`writeDockerEnvFile`) still land on host disk this way, because that container's bring-up shells out to the docker CLI directly rather than a struct call over the Docker Engine API; without this cleanup that directory would survive `stop` indefinitely. (Kong's TLS/`kong.yml`, Postgres's pgsodium diff --git a/apps/cli/src/shared/functions/serve.ts b/apps/cli/src/shared/functions/serve.ts index 83b2a0a19f..3541d52bfd 100644 --- a/apps/cli/src/shared/functions/serve.ts +++ b/apps/cli/src/shared/functions/serve.ts @@ -852,65 +852,29 @@ const writeDockerEnvFile = Effect.fn("functions.serve.writeDockerEnvFile")(funct return { path: pathname }; }); -const writeDockerMultilineEnvScript = Effect.fn("functions.serve.writeDockerMultilineEnvScript")( - function* ( - { fs, path }: { readonly fs: FileSystem.FileSystem; readonly path: Path.Path }, - env: ReadonlyArray, - containerDir: string, - dir: string, - ) { - // Self-healing — see the matching comment in `writeDockerEnvFile`. Runs - // unconditionally, before the length check, so a stale directory from an - // earlier invocation that needed multiline secrets is still reclaimed. - yield* fs - .remove(dir, { recursive: true, force: true }) - .pipe(Effect.mapError((error) => nativePlatformFailure(error, dir))); - - if (env.length === 0) { - return undefined; - } - - yield* fs - .makeDirectory(dir, { recursive: true, mode: 0o700 }) - .pipe(Effect.mapError((error) => nativePlatformFailure(error, dir))); - const scriptName = "multiline-env.sh"; - const pathname = path.join(dir, scriptName); - const envDir = path.join(containerDir, "values"); - const hostEnvDir = path.join(dir, "values"); - // Names are validated by `validateDockerMultilineEnvNames` before this runs. - const script = env - .map(([name], index) => { - const valueFile = `env-${index}`; - const valuePath = path.join(envDir, valueFile).replaceAll("\\", "/"); - return `${name}="$(cat ${valuePath}; printf x)" +/* In the `docker cp` archive, not a bind: Docker Desktop fails `docker cp` on a bind source + that was deleted and recreated (supabase/cli#7603). */ +function dockerMultilineEnvFiles( + env: ReadonlyArray, + containerDir: string, +) { + if (env.length === 0) { + return undefined; + } + const files: Record = {}; + // Names are validated by `validateDockerMultilineEnvNames` before this runs. + const script = env + .map(([name, value], index) => { + const valuePath = `${containerDir}/values/env-${index}`; + files[valuePath] = value; + return `${name}="$(cat ${valuePath}; printf x)" export ${name}="\${${name}%x}"`; - }) - .join("\n"); - yield* fs - .makeDirectory(hostEnvDir, { recursive: true, mode: 0o700 }) - .pipe(Effect.mapError((error) => nativePlatformFailure(error, hostEnvDir))); - // The value files hold secret env values, so keep them owner-only. - yield* Effect.all( - env.map(([, value], index) => { - const valuePath = path.join(hostEnvDir, `env-${index}`); - return fs - .writeFileString(valuePath, value, { mode: 0o600 }) - .pipe(Effect.mapError((error) => nativePlatformFailure(error, valuePath))); - }), - { concurrency: "unbounded" }, - ); - yield* fs - .writeFileString(pathname, script, { mode: 0o600 }) - .pipe(Effect.mapError((error) => nativePlatformFailure(error, pathname))); - - return { - // `Z`: private SELinux relabel of this CLI-staged dir (supabase/cli#5989); - // single-consumer bind, no-op without SELinux. - bind: `${dir}:${containerDir}:ro,Z`, - scriptPath: path.join(containerDir, scriptName).replaceAll("\\", "/"), - }; - }, -); + }) + .join("\n"); + const scriptPath = `${containerDir}/multiline-env.sh`; + files[scriptPath] = script; + return { files, scriptPath }; +} function partitionDockerEnvEntries(env: Readonly>) { const singleLine: Record = {}; @@ -1574,8 +1538,7 @@ export const startEdgeRuntimeContainer = Effect.fn("functions.startEdgeRuntimeCo containerId, ); // A single directory-wide `rm` (not per-file cleanup closures) covers the - // whole staging-write window below, including a mid-write failure - // between two `writeDocker*` calls, not just the final docker steps. + // whole staging-write window below, not just the final docker steps. const fs = yield* FileSystem.FileSystem; const removeRuntimeArtifacts = fs .remove(stagingDir, { recursive: true, force: true }) @@ -1713,13 +1676,13 @@ export const startEdgeRuntimeContainer = Effect.fn("functions.startEdgeRuntimeCo singleLineDockerEnv, path.join(stagingDir, "env"), ); + /* Drops the `multiline-env` dir older CLIs bind-mounted. */ + const legacyMultilineEnvDir = path.join(stagingDir, "multiline-env"); + yield* fs + .remove(legacyMultilineEnvDir, { recursive: true, force: true }) + .pipe(Effect.mapError((error) => nativePlatformFailure(error, legacyMultilineEnvDir))); const multilineEnvDir = "/root/.supabase/multiline-env"; - const dockerMultilineEnvScript = yield* writeDockerMultilineEnvScript( - { fs, path }, - multilineDockerEnv, - multilineEnvDir, - path.join(stagingDir, "multiline-env"), - ); + const dockerMultilineEnv = dockerMultilineEnvFiles(multilineDockerEnv, multilineEnvDir); const labels = dockerProjectLabels(projectId); const serveMainFile = `${serveMainDir}/index.ts`; @@ -1739,7 +1702,11 @@ export const startEdgeRuntimeContainer = Effect.fn("functions.startEdgeRuntimeCo // `sh -c` argv hits Windows ENAMETOOLONG (#5711), and a single-file host bind mounts as // an empty directory on daemons that cannot see this host's filesystem (#6254, #4190). const serveMainArchive = yield* Effect.tryPromise({ - try: () => containerArchiveBytes({ [serveMainFile]: serveMainTemplate }), + try: () => + containerArchiveBytes({ + ...dockerMultilineEnv?.files, + [serveMainFile]: serveMainTemplate, + }), catch: nativeFailure, }); const containerProjectRoot = toDockerPath(input.projectRoot, path); @@ -1767,7 +1734,6 @@ export const startEdgeRuntimeContainer = Effect.fn("functions.startEdgeRuntimeCo "--label", `${dockerWorkdirLabel}=${input.projectRoot}`, ...binds.flatMap((bind) => ["-v", formatDockerBind(bind)]), - ...(dockerMultilineEnvScript === undefined ? [] : ["-v", dockerMultilineEnvScript.bind]), ...(dockerEnvFile === undefined ? [] : ["--env-file", dockerEnvFile.path]), ...(input.platform === "linux" ? ["--add-host", "host.docker.internal:host-gateway"] : []), ...(input.inspectMode === undefined @@ -1777,7 +1743,7 @@ export const startEdgeRuntimeContainer = Effect.fn("functions.startEdgeRuntimeCo "sh", input.image, "-c", - buildServeEntrypointCommand(runtimeCommand, dockerMultilineEnvScript?.scriptPath), + buildServeEntrypointCommand(runtimeCommand, dockerMultilineEnv?.scriptPath), ]; // The container must exist for `docker cp` to have a target, and must not be running