From 5828aea475506c435dccc7daafdb2b0cbbd6d7ea Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Thu, 3 Sep 2026 01:23:22 -0400 Subject: [PATCH 1/5] feat: local and single-site production installer (S2) Implements S2 of docs/ghost-cli-replacement.md: a release-selecting bootstrap and a checkout-owned installer, plus the `.ghost-docker.json` reader and writer that S1 deferred to this step. - bootstrap.sh selects a release in semver order (never lexically), clones it, and execs that checkout's install.sh. Bootstrap logic only. - install.sh installs into its own directory: mode-aware preflight, stable project identity, generated passwords, exact Ghost version resolution from the image's own GHOST_VERSION/GHOST_CONTENT/GHOST_INSTALL, private config files, Caddy rendering, readiness verification through the site's real ingress, and a final summary. - Installation never stops or reconfigures anything already running. A chosen port moves out of the way; an explicitly requested busy port, or an occupied 80/443 in production, is an error naming what holds it. - Prompts read /dev/tty and every required input has a flag or environment variable, so --no-prompt is fully scriptable. --no-start starts nothing. - Options for steps that have not landed (--import, --with supervisor, --image-registry, --ghost-channel, --without) exit 3 naming the step rather than being reported as unknown options. - scripts/lib/meta.sh reads and writes .ghost-docker.json at schema v1, refuses a newer schema rather than misreading it, and treats a missing file as a pre-metadata install rather than a broken site. - scripts/site.sh adds list, check/doctor and info. Doctor degrades to useful host-level output when Docker is unreachable. - Read-only Docker probes have deadlines: a wedged daemon answers nothing rather than returning an error, and the check that exists to report that must not be able to hang on it. - Host tool contract recorded in GD_HOST_UTILITIES and enforced by an install run with a PATH built from exactly that list. Docker access is established by asking the daemon, never from docker-group membership. Adds tests/meta.test.mjs, tests/install.test.mjs (no daemon needed) and tests/install-e2e.test.mjs, which installs local and production sites from a candidate release built out of the working tree, runs two independent local sites, and covers port conflicts, an existing proxy, --no-start and --no-prompt. Co-Authored-By: Claude Opus 5 --- .github/workflows/test.yml | 40 +++ CLAUDE.md | 46 ++- README.md | 35 ++- bootstrap.sh | 276 +++++++++++++++++ docs/caddy.md | 4 + docs/configuration.md | 67 ++++- docs/ghost-cli-replacement.md | 9 + docs/install.md | 192 ++++++++++++ help | 10 + install.sh | 543 ++++++++++++++++++++++++++++++++++ scripts/lib/common.sh | 6 + scripts/lib/install.sh | 408 +++++++++++++++++++++++++ scripts/lib/meta.sh | 244 +++++++++++++++ scripts/lib/preflight.sh | 409 +++++++++++++++++++++++++ scripts/site.sh | 162 ++++++++++ tests/helpers.mjs | 92 +++++- tests/install-e2e.test.mjs | 358 ++++++++++++++++++++++ tests/install.test.mjs | 319 ++++++++++++++++++++ tests/meta.test.mjs | 111 +++++++ 19 files changed, 3308 insertions(+), 23 deletions(-) create mode 100755 bootstrap.sh create mode 100644 docs/install.md create mode 100755 install.sh create mode 100644 scripts/lib/install.sh create mode 100644 scripts/lib/meta.sh create mode 100644 scripts/lib/preflight.sh create mode 100755 scripts/site.sh create mode 100644 tests/install-e2e.test.mjs create mode 100644 tests/install.test.mjs create mode 100644 tests/meta.test.mjs diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 2f67935a..464557ca 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -62,3 +62,43 @@ jobs: - name: Show service logs on failure if: failure() run: docker ps -a && docker compose ls || true + + install: + name: Installer, from a candidate release + runs-on: ubuntu-latest + timeout-minutes: 60 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + + - uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0 + with: + node-version: "22" + + # The installer must work for an operator who is not root and is not in + # the docker group in the way the tests assume; access is established by + # asking the daemon, so this only confirms it can be asked. + - name: Confirm the daemon answers as this user + run: docker info --format '{{.ServerVersion}}' + + # 80 and 443 are used by the existing-proxy and production cases, and + # must be free before they run. + - name: Confirm the ingress ports are free + run: | + set -eu + for port in 80 443; do + if (exec 3<>/dev/tcp/127.0.0.1/"$port") 2>/dev/null; then + echo "port $port is already in use on this runner" >&2 + exit 1 + fi + done + + - name: Install local and production sites from a candidate release + env: + GD_TEST_INSTALL: "1" + run: node --test --test-timeout=1800000 tests/install-e2e.test.mjs + + - name: Show what was left behind on failure + if: failure() + run: | + docker ps -a + docker compose ls || true diff --git a/CLAUDE.md b/CLAUDE.md index f60b0d74..2612637a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,6 +37,12 @@ one-shot jobs (`activitypub-migrate`, `tinybird-*`) keep `restart: "no"`. ## Common Commands ```bash +# Installation +curl -fsSL .../bootstrap.sh | bash -s -- --domain example.com # release-selecting shim +./install.sh --local --no-prompt --no-start # checkout-owned installer +scripts/site.sh check # doctor: config, health, DB, ingress +scripts/site.sh list # every managed container on this host + # Core operations docker compose up -d # Start the services for the selected mode docker compose down # Stop all services @@ -71,6 +77,7 @@ scripts/caddy.sh apply # Render, validate, install, reload, ver # docker tests skip without a daemon) node --test --test-timeout=120000 tests/*.test.mjs GD_TEST_INGRESS=1 node --test --test-timeout=900000 tests/ingress.test.mjs +GD_TEST_INSTALL=1 node --test --test-timeout=1800000 tests/install-e2e.test.mjs ``` ## Configuration @@ -97,14 +104,20 @@ safely. Never source an env file; use `scripts/lib/env.sh`. - **Data persistence**: `UPLOAD_LOCATION` and `MYSQL_DATA_LOCATION` ### Key files +- `bootstrap.sh` — curl-able release-selecting shim; bootstrap logic only +- `install.sh` — checkout-owned installer. One checkout is one site - `.env` / `.env.example` — operator configuration - `ghost.env` / `ghost.env.example` — application configuration -- `.ghost-docker.json` — generated installation metadata (schema v1, written from S2) +- `.ghost-docker.json` — generated installation metadata (schema v1, read and + written by `scripts/lib/meta.sh`; a missing file means "pre-metadata install", + not a broken site) - `compose.yml` — service definitions - `caddy/Caddyfile` — tracked generic entry point; site routes are generated into `caddy/sites/`, operator routes live in `caddy/custom/`, global options in `caddy/global/` -- `scripts/lib/*.sh` — shared helpers (env, fs, compose, config, caddy) +- `scripts/lib/*.sh` — shared helpers (env, fs, compose, config, caddy, meta, + preflight, install) +- `scripts/site.sh` — `list`, `check`/doctor, `info` - `mysql-init/create-multiple-databases.sh` — MySQL multi-database initialization ## Migration from Ghost CLI @@ -126,10 +139,37 @@ The repository includes comprehensive migration tools: by default. This is the only host Node dependency, and `install.sh --import` removes it +## Installer + +`install.sh` is checkout-owned and installs into its own directory; `--dir` +elsewhere is refused. `bootstrap.sh` selects a release by semver order (never +lexically), clones it, and `exec`s that checkout's installer. Ghost versions are +resolved to an exact tag by asking the pulled image for its own `GHOST_VERSION`, +`GHOST_CONTENT` and `GHOST_INSTALL`; the digest goes into `.ghost-docker.json`. + +Rules that must not regress: + +- Installation never stops or reconfigures anything already running. A chosen + port moves out of the way; an explicitly requested busy port is an error, and + so is an occupied 80/443 in production. +- Every prompt reads `/dev/tty` and has a flag or environment-variable + equivalent. No prompt has a silent default. +- Docker access is established by asking the daemon, never from `docker` group + membership. Read-only probes have deadlines so a wedged daemon is reported + rather than hung on. +- Host tools are `docker` + `jq` (+ `git` for the bootstrap) plus the POSIX + utilities in `GD_HOST_UTILITIES`; `tests/install-e2e.test.mjs` installs with a + `PATH` of exactly that list. +- Options for steps that have not landed (`--import`, `--with supervisor`, + `--image-registry`, `--ghost-channel`, `--without`) exit 3 naming the step, + not as unknown options. + +See `docs/install.md`. + ## Development Workflow 1. Copy `.env.example` to `.env` and `ghost.env.example` to `ghost.env` - (both mode `0600`) + (both mode `0600`) — or let `install.sh` do it 2. Configure the required variables for the mode; run `scripts/config.sh validate` 3. Production only: `scripts/caddy.sh apply` 4. Run `docker compose up -d` diff --git a/README.md b/README.md index 464f9498..388f0af6 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,30 @@ Configuration to run Ghost and its services with Docker Compose. Requires **bash**, **Docker Engine 25.0+**, **Docker Compose v2.24+** and **jq**. +## Install + +```sh +# A production site with HTTPS +curl -fsSL https://ghost.org/docker/bootstrap.sh | bash -s -- --domain example.com + +# A local development site +curl -fsSL https://ghost.org/docker/bootstrap.sh | bash -s -- --local +``` + +`bootstrap.sh` selects a release, clones it, and runs that checkout's +`install.sh`, which does everything else: preflight, an exact Ghost version pin, +generated passwords, configuration, routing, and verifying that the site answers +through its own ingress before it says it is installed. + +Every prompt has a flag, so `--no-prompt` is fully scriptable. Installation +never stops or reconfigures anything already running on this host: a port that +is in use is an error naming what holds it. See [docs/install.md](docs/install.md). + +```sh +scripts/site.sh check # diagnose this site +scripts/site.sh list # every ghost-docker container on this host +``` + ## Configuration Two files, deliberately separate: @@ -19,8 +43,10 @@ cp ghost.env.example ghost.env && chmod 0600 ghost.env scripts/config.sh validate ``` -See [docs/configuration.md](docs/configuration.md) for the full contract: -value encoding, site modes, profiles, lifecycle, service aliases and metadata. +Both are written for you by `install.sh`; the examples are for hand-built +sites and for reference. See [docs/configuration.md](docs/configuration.md) for +the full contract: value encoding, site modes, profiles, lifecycle, service +aliases and metadata. ## Site modes @@ -109,11 +135,14 @@ dependencies and no `package.json`: Node is a development requirement only, and the repository is not a Node package. ```sh -# helpers, mode matrix, Caddy routes +# helpers, mode matrix, Caddy routes, installer decisions, metadata node --test --test-timeout=120000 tests/*.test.mjs # local and production ingress, against real containers GD_TEST_INGRESS=1 node --test --test-timeout=900000 tests/ingress.test.mjs + +# real installations from a candidate release built out of the working tree +GD_TEST_INSTALL=1 node --test --test-timeout=1800000 tests/install-e2e.test.mjs ``` Docker-dependent tests are skipped when no daemon is reachable. Set diff --git a/bootstrap.sh b/bootstrap.sh new file mode 100755 index 00000000..fbb2eb0d --- /dev/null +++ b/bootstrap.sh @@ -0,0 +1,276 @@ +#!/usr/bin/env bash +# Select a ghost-docker release, put it somewhere, and run its installer. +# +# curl -fsSL https://ghost.org/docker/bootstrap.sh | bash -s -- --domain example.com +# +# bootstrap.sh [--channel stable|beta] [--ref vX.Y.Z] [--dir PATH] +# [installer options...] +# +# --channel CHANNEL stable (default) or beta. Selects the newest release on +# that channel. +# --ref REF Install this exact tag instead of resolving a channel. +# --dir PATH Where to put the checkout. Default: ./ghost-docker +# +# Every other option is passed to the selected release's install.sh unchanged; +# run `bootstrap.sh --help` after the checkout exists, or see install.sh, for +# that list. +# +# This file contains bootstrap logic only. Installation logic belongs to the +# release, so that a site is always installed by the code it is pinned to. This +# shim carries its own version comparison rather than sourcing the repository's +# helpers, because it runs before there is a checkout to source them from. +set -euo pipefail + +GD_BOOTSTRAP_REPO=${GD_BOOTSTRAP_REPO:-https://github.com/TryGhost/ghost-docker.git} + +channel=stable +ref="" +dir="" +declare -a passthrough=() + +die() { + printf 'error: %s\n' "$1" >&2 + exit "${2:-1}" +} + +usage() { + local line + { + read -r line + while IFS= read -r line; do + [[ $line == '#'* ]] || break + line=${line#\#} + printf '%s\n' "${line# }" + done + } <"$0" +} + +# _timeout SECONDS COMMAND... +# Runs a command with a deadline, returning 124 when it is killed. `timeout(1)` +# is GNU coreutils and is absent on macOS, so this is spelled out. A daemon +# that has wedged stops answering rather than returning an error, so the check +# that exists to report that must not be able to hang on it. +_timeout() { + local seconds=$1 pid waited=0 + shift + "$@" & + pid=$! + while kill -0 "$pid" 2>/dev/null; do + if ((waited >= seconds)); then + kill -9 "$pid" 2>/dev/null + wait "$pid" 2>/dev/null + return 124 + fi + sleep 1 + waited=$((waited + 1)) + done + wait "$pid" +} + +# --- Release selection ----------------------------------------------------- + +# _semver_cmp A B -> -1, 0 or 1 +# +# Full semver ordering, including prereleases: 1.2.0-beta.2 sorts before +# 1.2.0, and 1.9.0 after 1.10.0 would be wrong. Lexical sorting gets both +# backwards, which is why this is spelled out. +_semver_cmp() { + local a=${1#v} b=${2#v} ac bc ap bp i x y + ac=${a%%-*} + bc=${b%%-*} + case $a in *-*) ap=${a#*-} ;; *) ap="" ;; esac + case $b in *-*) bp=${b#*-} ;; *) bp="" ;; esac + + local -a av bv + IFS=. read -ra av <<<"$ac" + IFS=. read -ra bv <<<"$bc" + for ((i = 0; i < 3; i++)); do + x=${av[i]:-0} + y=${bv[i]:-0} + [[ $x =~ ^[0-9]+$ ]] || x=0 + [[ $y =~ ^[0-9]+$ ]] || y=0 + ((10#$x > 10#$y)) && { + printf '1\n' + return + } + ((10#$x < 10#$y)) && { + printf -- '-1\n' + return + } + done + + # A release outranks any prerelease of the same version. + [[ -z $ap && -z $bp ]] && { + printf '0\n' + return + } + [[ -z $ap ]] && { + printf '1\n' + return + } + [[ -z $bp ]] && { + printf -- '-1\n' + return + } + + local -a ai bi + IFS=. read -ra ai <<<"$ap" + IFS=. read -ra bi <<<"$bp" + for ((i = 0; i < ${#ai[@]} || i < ${#bi[@]}; i++)); do + x=${ai[i]:-} + y=${bi[i]:-} + [[ -z $x ]] && { + printf -- '-1\n' + return + } + [[ -z $y ]] && { + printf '1\n' + return + } + if [[ $x =~ ^[0-9]+$ && $y =~ ^[0-9]+$ ]]; then + ((10#$x > 10#$y)) && { + printf '1\n' + return + } + ((10#$x < 10#$y)) && { + printf -- '-1\n' + return + } + else + [[ $x > $y ]] && { + printf '1\n' + return + } + [[ $x < $y ]] && { + printf -- '-1\n' + return + } + fi + done + printf '0\n' +} + +# _latest_release CHANNEL +# The newest tag on a channel. Stable is vX.Y.Z; beta also considers +# vX.Y.Z-beta.N, and still prefers a release over a prerelease of the same +# version. Selection is by semver order, never by the order the remote listed +# the tags in. +_latest_release() { + local want=$1 line tag best="" + local stable='^v[0-9]+\.[0-9]+\.[0-9]+$' + local prerelease='^v[0-9]+\.[0-9]+\.[0-9]+-beta\.[0-9]+$' + + while IFS= read -r line; do + tag=${line##*refs/tags/} + tag=${tag%'^{}'} + [[ $tag =~ $stable ]] || { [[ $want == beta && $tag =~ $prerelease ]] || continue; } + if [[ -z $best ]] || [[ $(_semver_cmp "$tag" "$best") == 1 ]]; then + best=$tag + fi + done < <(git ls-remote --tags "$GD_BOOTSTRAP_REPO" 2>/dev/null) + + [[ -n $best ]] || return 1 + printf '%s\n' "$best" +} + +# Sourcing this file with GD_BOOTSTRAP_SOURCED=1 defines the helpers above and +# stops, so that release selection can be tested without cloning anything. +if [[ ${GD_BOOTSTRAP_SOURCED:-0} == 1 ]]; then + return 0 +fi + +# --- Options --------------------------------------------------------------- + +while (($#)); do + case $1 in + --channel) + [[ ${2:-} ]] || die "--channel needs a value" 2 + channel=$2 + shift + ;; + --channel=*) channel=${1#*=} ;; + --ref) + [[ ${2:-} ]] || die "--ref needs a value" 2 + ref=$2 + shift + ;; + --ref=*) ref=${1#*=} ;; + --dir) + [[ ${2:-} ]] || die "--dir needs a value" 2 + dir=$2 + shift + ;; + --dir=*) dir=${1#*=} ;; + -h | --help) + usage + exit 0 + ;; + *) passthrough+=("$1") ;; + esac + shift +done + +case $channel in + stable | beta) ;; + *) die "--channel must be stable or beta" 2 ;; +esac + +# --- Preflight ------------------------------------------------------------- +# +# Only what the bootstrap itself needs, plus the tools the installer will +# require, so a missing prerequisite is reported before anything is cloned. +# Daemon access is established by asking the daemon, not by looking at group +# membership: neither rootless Docker nor a remote DOCKER_HOST involves the +# docker group, and being in it does not mean the daemon is running. + +# The target directory is checked first: it costs nothing, and being told the +# directory is wrong beats waiting on a daemon probe to find that out. +[[ -n $dir ]] || dir=./ghost-docker + +if [[ -e $dir ]]; then + [[ -d $dir ]] || die "$dir exists and is not a directory" + if [[ -n $(ls -A "$dir" 2>/dev/null) ]]; then + die "$dir is not empty. Choose an empty directory with --dir, or install + from inside an existing checkout with its own install.sh." + fi +fi + +missing="" +for cmd in git docker jq; do + command -v "$cmd" >/dev/null 2>&1 || missing="$missing $cmd" +done +[[ -z $missing ]] || die "these are required and not installed:$missing" + +_timeout 20 docker info >/dev/null 2>&1 || + die "the Docker daemon is not reachable, or is not answering. Start Docker, + wait for it to report running, and try again." + +_timeout 20 docker compose version >/dev/null 2>&1 || + die "the Docker Compose v2 plugin is not available. See https://docs.docker.com/compose/install/" + +if [[ -z $ref ]]; then + printf 'Resolving the newest %s release\n' "$channel" + ref=$(_latest_release "$channel") || + die "no $channel release was found in $GD_BOOTSTRAP_REPO" + printf ' %s\n' "$ref" +fi + +# A prerelease tag implies the beta channel, whether or not it was named. +case $ref in *-beta.*) channel=beta ;; esac + +# --- Checkout -------------------------------------------------------------- + +printf 'Cloning %s at %s into %s\n' "$GD_BOOTSTRAP_REPO" "$ref" "$dir" +git clone --quiet --depth 1 --branch "$ref" "$GD_BOOTSTRAP_REPO" "$dir" || + die "could not clone $GD_BOOTSTRAP_REPO at $ref" + +target=$(CDPATH='' cd -- "$dir" && pwd -P) + +[[ -x $target/install.sh ]] || + die "$ref does not contain an executable install.sh, so it cannot be installed by this bootstrap" + +# The installer is the release's, and owns everything from here: preflight, +# configuration, routing and verification. Its exit status is this script's. +printf '\n' +exec "$target/install.sh" --dir "$target" --channel "$channel" --ref "$ref" \ + ${passthrough[@]+"${passthrough[@]}"} diff --git a/docs/caddy.md b/docs/caddy.md index e7c62e5b..419d0bda 100644 --- a/docs/caddy.md +++ b/docs/caddy.md @@ -1,5 +1,9 @@ # Caddy routing +`install.sh` runs `caddy_apply` for a production site, so a fresh installation +already has validated, installed and verified routes. Everything below is for +changing them afterwards. + ## Layout | Path | Tracked | Owner | diff --git a/docs/configuration.md b/docs/configuration.md index 5f9270b9..ab818a86 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -166,17 +166,46 @@ contract. ## Installation metadata `.ghost-docker.json` records the schema version, installation time, mode, -release channel, installed stack version/commit, project identity, resolved -Ghost image and digest, and completed migrations. Its schema is specified in -§2.2 of [the plan](ghost-cli-replacement.md). It is gitignored and machine -generated — do not hand-edit it. Durable operation journals for in-progress -work and recovery state are separate files. +release channel, installed stack version/commit/ref, project identity, resolved +Ghost image and digest, selected profiles, and completed migrations. Its schema +is specified in §2.2 of [the plan](ghost-cli-replacement.md). It is gitignored, +mode `0600`, and machine generated — do not hand-edit it. Durable operation +journals for in-progress work and recovery state are separate files, and land +with backup/restore in S4. + +```json +{ + "schemaVersion": 1, + "installedAt": "2026-09-03T09:12:44Z", + "mode": "production", + "channel": "stable", + "stack": { "version": "v1.2.3", "commit": "…", "ref": "v1.2.3" }, + "site": { + "project": "ghost-example-com", "dir": "/opt/ghost/example.com", + "url": "https://example.com", "domain": "example.com", "adminDomain": null + }, + "ghost": { + "image": "ghost", "tag": "6.62.0-next-alpine", + "version": "6.62.0", "digest": "sha256:…" + }, + "profiles": ["production"], + "migrations": [] +} +``` + +A field that was not supplied is `null` rather than an empty string, so "not +known" and "deliberately empty" stay distinguishable. The digest is the +immutable image identity, recorded so the exact image can be found again during +recovery even after a tag moves. -An installation that predates this file is supported explicitly: readers must -treat a missing file as "unknown, pre-metadata install", not as a broken site. +`scripts/lib/meta.sh` is the reader and writer. It writes atomically, refuses a +document without the right `schemaVersion`, and refuses to *read* one written by +a newer schema rather than misinterpreting it. -**Not implemented yet.** `install.sh` is the first thing that writes this file, -so the reader and writer land with it in S2 rather than sitting unused here. +An installation that predates this file is supported explicitly: `meta_present` +answers that question, and readers must treat a missing file as "unknown, +pre-metadata install", not as a broken site. `scripts/site.sh info` prints that +state in words. ## The Compose invocation contract @@ -222,10 +251,18 @@ Runtime, on the server: compatible with bash 3.2 so macOS's system bash works for local development. - Docker Engine 25.0.0 — for `healthcheck.start_interval` - Docker Compose v2.24.0 — for `env_file` `required` and `depends_on` `required` -- `jq` — used by the helpers for JSON, including `.ghost-docker.json` (S2) +- `jq` — used by the helpers for JSON, including `.ghost-docker.json` + +`install.sh` verifies all three during preflight, and `bootstrap.sh` also needs +`git`. `scripts/migrate.sh` already required `jq`, so this is not a new +prerequisite for existing servers. -`install.sh` verifies all three during preflight (S2). `scripts/migrate.sh` -already required `jq`, so this is not a new prerequisite for existing servers. +Every other host utility the scripts invoke is POSIX and is listed in +`GD_HOST_UTILITIES` in `scripts/lib/preflight.sh`. That list is the tool +contract: `tests/install-e2e.test.mjs` runs a complete installation with a +`PATH` built from exactly it, so a GNU-only or otherwise unusual dependency +fails a test rather than someone's server. Nothing decides Docker access from +`docker` group membership — see [install.md](install.md#host-tools). Development only, not needed on a server: @@ -267,7 +304,7 @@ recovery testing. ## Why `jq` is a prerequisite -The helpers use `jq` for JSON, starting with `.ghost-docker.json` in S2. +The helpers use `jq` for JSON, starting with `.ghost-docker.json`. The implementation plan originally recorded "No host Node or jq requirement". That was changed during S1, deliberately, and §1 of @@ -279,5 +316,5 @@ That was changed during S1, deliberately, and §1 of 240 lines and bought nothing an operator can see. Node.js is **not** a runtime requirement. It is used only to run the test -suite. `install.sh` (S2) must verify `docker`, `docker compose` and `jq` during -preflight, and must not require Node. +suite. `install.sh` verifies `docker`, `docker compose` and `jq` during +preflight, and does not require Node. diff --git a/docs/ghost-cli-replacement.md b/docs/ghost-cli-replacement.md index 5c61142b..bf4a2ded 100644 --- a/docs/ghost-cli-replacement.md +++ b/docs/ghost-cli-replacement.md @@ -823,6 +823,15 @@ blockers for dependent steps. ### S2 — Local and single-site production installer +Status: implemented. `bootstrap.sh` is the release-selecting shim and +`install.sh` the checkout-owned installer; `scripts/lib/meta.sh` is the +`.ghost-docker.json` reader/writer deferred from S1, and `scripts/site.sh` +provides `list` and `check`/doctor. See `docs/install.md` for the contract as +built. Two deliberate notes for dependent steps: the bring-your-own-proxy path +stays the documented manual edit of `compose.yml` from §2.1, and the operation +lock in §2.2 is *not* implemented here — it lands with the other mutating +operations in S4, so S4 must add it to installation as well as to its own. + Repo: ghost-docker. Deps: S1. Implement §2.8 for fresh installs, exact version resolution, stable identity, scriptable prompts, custom proxy use, and optional service setup. `--import`/supervisor behavior may initially fail as unimplemented diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 00000000..0638bd88 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,192 @@ +# Installation + +```bash +curl -fsSL https://ghost.org/docker/bootstrap.sh | bash -s -- --domain example.com +``` + +Two scripts, with a deliberate boundary between them: + +| Script | Owned by | Does | +| --- | --- | --- | +| `bootstrap.sh` | nothing — it is the thing you curl | Selects a release, clones it, runs that release's installer | +| `install.sh` | the checkout it lives in | Everything else: preflight, configuration, routing, verification | + +The split exists so a site is always installed by the code it is pinned to. +`bootstrap.sh` resolves a tag, clones it, and `exec`s that checkout's +`install.sh`; it never installs anything itself. Its exit status is the +installer's. + +**One checkout is one site.** `install.sh` installs into its own directory and +refuses `--dir` pointing anywhere else, naming `bootstrap.sh` as the way to +install into a new one. + +## bootstrap.sh + +```text +bootstrap.sh [--channel stable|beta] [--ref vX.Y.Z] [--dir PATH] + [installer options...] +``` + +| Option | Default | Meaning | +| --- | --- | --- | +| `--channel` | `stable` | Newest release on that channel. `beta` also considers `vX.Y.Z-beta.N`. | +| `--ref` | resolved from the channel | Install this exact tag. A prerelease tag implies the beta channel. | +| `--dir` | `./ghost-docker` | Where the checkout goes. Must be empty or absent. | + +Everything else is passed to the release's `install.sh` unchanged. + +Releases are selected in **semver order**, not lexically: `v1.10.0` is newer +than `v1.9.0`, `v1.2.0-beta.10` is newer than `v1.2.0-beta.2`, and `v1.2.0` is +newer than any `v1.2.0-beta.N`. `GD_BOOTSTRAP_REPO` overrides the source +repository, which is how the tests install from a candidate release built out of +the working tree. + +## install.sh + +```text +install.sh [--local | --domain example.com [--admin-domain admin.example.com]] + [--dir PATH] [--port 2368] [--version 6.3.1] + [--channel stable|beta] [--ref vX.Y.Z] + [--with analytics,activitypub] + [--no-prompt] [--no-start] +``` + +| Option | Meaning | +| --- | --- | +| `--local` | Ghost and MySQL, published on `127.0.0.1:PORT`. `NODE_ENV=development`, `RESTART_POLICY=no`. | +| `--domain DOMAIN` | Production: Ghost, MySQL and Caddy with HTTPS on that domain. | +| `--admin-domain DOMAIN` | A separate Ghost Admin domain. Production only. | +| `--port PORT` | The loopback port. Omitted: the first free port at or above 2368. | +| `--version VERSION` | A Ghost version (`6.3.1`) or a full image tag (`6-alpine`). | +| `--with LIST` | `analytics`, `activitypub`, or both. | +| `--no-prompt` | Never ask. Every required input must then be supplied. | +| `--no-start` | Write the configuration and routes; start no application services. | + +Exit codes: `0` success, `1` failure, `2` usage error, `3` a documented option +whose step has not landed. + +### Prompts + +Prompts read from `/dev/tty`, so an installer piped from `curl` can still ask a +question. **Every required input has a flag or environment variable**, so +`--no-prompt` is fully scriptable, and no prompt has a silent default: without a +terminal, a required answer is an error naming the flag that supplies it. + +### Options that are not implemented yet + +These are part of the documented interface and fail with exit code `3`, naming +the step they belong to, rather than being reported as unknown options: + +| Option | Lands in | +| --- | --- | +| `--import BUNDLE` | S5. `scripts/migrate.sh` is the supported migration path today. | +| `--with supervisor` | S8. The profile is reserved and defines no service. | +| `--image-registry`, `--ghost-channel`, `--without` | S14–S16. | + +## What installation does + +1. **Preflight**, mode aware, entirely in host shell so it still works when + Docker is missing or stopped: platform, required tools, Docker Engine and + Compose versions, a writable site directory, disk, memory, and the ports the + selected mode needs. +2. **Identity.** A stable `COMPOSE_PROJECT_NAME` — `ghost-example-com` in + production, `ghost-local-` locally — kept independent of the + directory name and used as the suffix of every service network alias. +3. **Secrets.** Fresh application and root database passwords, 192 bits each. + Nothing ships with a default credential. +4. **Exact Ghost image.** The requested version is pulled, and the image is + asked for its own `GHOST_VERSION`, `GHOST_CONTENT` and `GHOST_INSTALL`. The + *exact* version tag is written to `.env` — never a moving one — and the + digest is recorded in `.ghost-docker.json` for recovery. `GHOST_CONTENT_PATH` + and `GHOST_TINYBIRD_PATH` come from the image, so the mounted content + directory and the image layout cannot disagree. +5. **Configuration.** `.env` and `ghost.env`, both mode `0600`. `.env` starts + from the tracked example so its comments survive; `ghost.env` is written + fresh, because the example's SMTP block is a placeholder and a site shipping + with `smtp.example.com` configured fails to send mail in a way that looks + like a Ghost bug. +6. **Routing**, in production: routes are rendered, validated, installed and + verified through `scripts/caddy.sh apply`. Files in `caddy/custom/` and + `caddy/global/` are yours and are never touched. +7. **Metadata.** `.ghost-docker.json`, described in + [configuration.md](configuration.md#installation-metadata). +8. **Start and verify**, unless `--no-start`: the database and Ghost must report + *healthy* through their own health checks, and the Admin API must answer + through the ingress the site actually uses. A running container is not + readiness, and `up -d` returning zero is not a working site. + +## Ports, and your existing proxy + +**Installation never stops or reconfigures anything already running.** A server +may proxy other applications, and replacing its web server is not an installer's +decision to make. + +- A port the installer *chooses* moves out of the way: with no `--port`, it + takes the first free one at or above 2368. +- A port you *asked for* does not. `--port` on a busy port is an error, because + silently using a different one produces a site at an address nothing else is + configured for. +- In production, 80 and 443 are required. If something already holds them, the + installation fails and names what holds it — a Docker container by name where + it can. Nothing is stopped, and no configuration is written. + +To run Ghost behind your own nginx or Apache, point it at +`127.0.0.1:${GHOST_PORT}` and edit `compose.yml` to drop the `caddy` service or +move it off 80/443. That is an unsupported manual customization and stack +updates may touch `compose.yml`; see +[configuration.md](configuration.md#site-modes-and-profiles) for why +bring-your-own-proxy is not a mode. + +## Optional services + +`--with analytics` needs Tinybird credentials. They are read from the +environment rather than taken as flags, so an unattended install does not put a +token into the process table or the shell history: + +```bash +TINYBIRD_TRACKER_TOKEN=... TINYBIRD_ADMIN_TOKEN=... TINYBIRD_WORKSPACE_ID=... \ + ./install.sh --domain example.com --with analytics --no-prompt +``` + +Without them, and without a terminal to ask, installation fails rather than +configuring a half-enabled profile. The Tinybird login is interactive and is not +run by the installer; the summary prints the two commands that finish it. See +[TINYBIRD.md](../TINYBIRD.md). + +`--with activitypub` needs no credentials. Either optional profile sets +`labs__publicAPI` in `ghost.env`, which both features require. + +## Host tools + +Beyond `bash`, the installer requires **`docker`** (with Compose v2) and +**`jq`**; `bootstrap.sh` also needs **`git`**. Everything else it invokes is +POSIX and listed in `GD_HOST_UTILITIES` in `scripts/lib/preflight.sh`. That list +is the tool contract, and `tests/install-e2e.test.mjs` runs an install with a +`PATH` containing exactly it — so a GNU-only or unusual dependency added to a +code path fails a test rather than someone's server. + +Node.js is **not** required to install or run a site. It runs the test suite. + +Docker access is established by **asking the daemon**, never by checking +`docker` group membership: neither rootless Docker nor a remote `DOCKER_HOST` +involves that group, and being in it does not mean the daemon is running. A +daemon that has wedged answers nothing rather than returning an error, so every +read-only probe has a deadline and reports that state instead of hanging. + +## After installation + +```bash +scripts/site.sh check # diagnose this site +scripts/site.sh list # every ghost-docker container on this host +scripts/site.sh info # the recorded installation metadata +``` + +`site.sh list` reads Docker's own labels, including stopped containers. There is +no host-wide registry of installations, so a checkout whose containers have +never been created cannot be discovered from outside it — `list` says so rather +than implying the list is complete. + +`site.sh check` validates the configuration for its mode, verifies a real client +connection to the application database, checks service health, and re-runs the +ingress verification. It degrades to useful host-level output when Docker is +unreachable, which is when it matters most. diff --git a/help b/help index 68a9054b..f363ff35 100755 --- a/help +++ b/help @@ -5,6 +5,15 @@ cat << 'HELPEOF' GHOST DOCKER HELP & COMMANDS ════════════════════════════════════════════════════════════════════ +INSTALLATION: + curl -fsSL .../bootstrap.sh | bash -s -- --domain example.com + curl -fsSL .../bootstrap.sh | bash -s -- --local + + ./install.sh --help # every option, in this checkout + scripts/site.sh check # diagnose this site + scripts/site.sh list # every ghost-docker container on this host + scripts/site.sh info # recorded installation metadata + SITE MODES (exactly one, selected in COMPOSE_PROFILES in .env): local ghost + db, published on 127.0.0.1:${GHOST_PORT} production ghost + db + caddy, HTTPS on ${HTTP_PORT}/${HTTPS_PORT} @@ -65,6 +74,7 @@ USEFUL PATHS: Logs: docker compose logs DOCUMENTATION: + docs/install.md Installation, ports, optional services, doctor docs/configuration.md Configuration split, profiles, lifecycle, metadata docs/caddy.md Route generation, custom routes, validation docs/bundle-v1.md Migration bundle contract diff --git a/install.sh b/install.sh new file mode 100755 index 00000000..9be0f868 --- /dev/null +++ b/install.sh @@ -0,0 +1,543 @@ +#!/usr/bin/env bash +# Install a Ghost site into this checkout. One checkout is one site. +# +# install.sh [--local | --domain example.com [--admin-domain admin.example.com]] +# [--dir PATH] [--port 2368] [--version 6.3.1] +# [--channel stable|beta] [--ref vX.Y.Z] +# [--with analytics,activitypub] +# [--no-prompt] [--no-start] +# +# --local Ghost and MySQL, published on 127.0.0.1:PORT. +# --domain DOMAIN Production: Ghost, MySQL and Caddy with HTTPS. +# --admin-domain D Serve Ghost Admin on a separate domain. +# --dir PATH The site directory. Must be this checkout; use +# bootstrap.sh to install into a new one. +# --port PORT The loopback port Ghost is published on. A free one is +# chosen when this is omitted; a busy one is an error. +# --version VERSION Ghost version or image tag. Resolved to an exact pin. +# --channel CHANNEL Release channel to record: stable (default) or beta. +# --ref REF The stack release this checkout is at. +# --with LIST Optional per-site services: analytics, activitypub. +# --no-prompt Never ask. Every required input must then be supplied. +# --no-start Configure the site but start no application services. +# +# Installation never stops or reconfigures anything already running on this +# host. A port that is in use is reported as an error, with what holds it. +set -euo pipefail + +# Everything written here may hold a credential until it is explicitly made +# public. +umask 077 + +# shellcheck source=scripts/lib/common.sh +. "$(dirname -- "$0")/scripts/lib/common.sh" + +readonly EXIT_USAGE=2 +readonly EXIT_UNIMPLEMENTED=3 + +mode="" +domain="" +admin_domain="" +dir="" +port="" +ghost_version="" +channel="" +ref="" +with="" +no_start=0 + +die() { + printf 'error: %s\n' "$1" >&2 + exit "${2:-1}" +} + +# Flags that belong to steps which have not landed. They fail with what they +# will be rather than as an unknown option, so that a script written against +# the documented interface gets a useful answer. +unimplemented() { + printf 'error: %s is not implemented yet (%s).\n' "$1" "$2" >&2 + exit "$EXIT_UNIMPLEMENTED" +} + +while (($#)); do + case $1 in + --local) + mode=local + ;; + --domain) + [[ ${2:-} ]] || die "--domain needs a value" "$EXIT_USAGE" + mode=production + domain=$2 + shift + ;; + --domain=*) mode=production domain=${1#*=} ;; + --admin-domain) + [[ ${2:-} ]] || die "--admin-domain needs a value" "$EXIT_USAGE" + admin_domain=$2 + shift + ;; + --admin-domain=*) admin_domain=${1#*=} ;; + --dir) + [[ ${2:-} ]] || die "--dir needs a value" "$EXIT_USAGE" + dir=$2 + shift + ;; + --dir=*) dir=${1#*=} ;; + --port) + [[ ${2:-} ]] || die "--port needs a value" "$EXIT_USAGE" + port=$2 + shift + ;; + --port=*) port=${1#*=} ;; + --version) + [[ ${2:-} ]] || die "--version needs a value" "$EXIT_USAGE" + ghost_version=$2 + shift + ;; + --version=*) ghost_version=${1#*=} ;; + --channel) + [[ ${2:-} ]] || die "--channel needs a value" "$EXIT_USAGE" + channel=$2 + shift + ;; + --channel=*) channel=${1#*=} ;; + --ref) + [[ ${2:-} ]] || die "--ref needs a value" "$EXIT_USAGE" + ref=$2 + shift + ;; + --ref=*) ref=${1#*=} ;; + --with) + [[ ${2:-} ]] || die "--with needs a value" "$EXIT_USAGE" + with=$2 + shift + ;; + --with=*) with=${1#*=} ;; + --no-prompt) GD_NO_PROMPT=1 ;; + --no-start) no_start=1 ;; + --import | --import=*) + unimplemented "--import" "bundle import lands in S5; scripts/migrate.sh is the supported migration path today" + ;; + --image-registry | --image-registry=* | --ghost-channel | --ghost-channel=* | --without | --without=*) + unimplemented "${1%%=*}" "service image registries, the Ghost nightly channel and Redis land in S14-S16" + ;; + -h | --help | help) + usage + exit 0 + ;; + *) + printf 'error: unknown option: %s\n' "$1" >&2 + usage >&2 + exit "$EXIT_USAGE" + ;; + esac + shift +done + +# --- The site directory ---------------------------------------------------- +# +# Installation logic is owned by the checkout it runs from, so the site is +# installed here. Selecting a release and putting it somewhere else is +# bootstrap.sh's job, and it re-executes this script from the new checkout. + +abspath() { (CDPATH='' cd -- "$1" >/dev/null 2>&1 && pwd -P); } + +checkout=$(abspath "$GD_ROOT_DIR") || die "cannot resolve this checkout's path" +if [[ -n $dir ]]; then + resolved=$(abspath "$dir" 2>/dev/null || printf '%s' "$dir") + if [[ $resolved != "$checkout" ]]; then + die "--dir is $resolved but this installer belongs to $checkout. + One checkout is one site. To install into another directory, use bootstrap.sh, + which selects a release, clones it there, and runs its installer." "$EXIT_USAGE" + fi +fi +dir=$checkout + +# --- Refuse to install over an existing site ------------------------------- + +for existing in "$GD_ENV_FILE_NAME" "$GD_META_FILE_NAME"; do + [[ -e $dir/$existing ]] || continue + die "$dir/$existing already exists, so this checkout already holds a site. + Move it aside, or install into a new directory with bootstrap.sh. Nothing has + been changed." +done + +# --- Mode ------------------------------------------------------------------ + +if [[ -z $mode ]]; then + if answer=$(install_ask "Install a [local] development site or a [production] site on a domain?" local); then + case $answer in + local | l | 1) mode=local ;; + production | prod | p | 2) mode=production ;; + *) die "unrecognised answer: $answer" "$EXIT_USAGE" ;; + esac + else + die "choose a site mode with --local or --domain example.com" "$EXIT_USAGE" + fi +fi + +if [[ $mode == production && -z $domain ]]; then + domain=$(install_require "" "Public domain for this site (example.com)" "--domain") || + exit "$EXIT_USAGE" +fi + +if [[ $mode == local && -n $admin_domain ]]; then + die "--admin-domain applies to production sites only" "$EXIT_USAGE" +fi + +case $domain in + '' | *[!a-zA-Z0-9.-]*) + [[ $mode == local ]] || die "--domain must be a hostname, not a URL: got '$domain'" "$EXIT_USAGE" + ;; +esac + +# --- Optional per-site services -------------------------------------------- + +profiles=$mode +IFS=, read -ra selected <<<"$with" +for choice in ${selected[@]+"${selected[@]}"}; do + choice=${choice#"${choice%%[![:space:]]*}"} + choice=${choice%"${choice##*[![:space:]]}"} + [[ -n $choice ]] || continue + case $choice in + analytics | activitypub) profiles="$profiles,$choice" ;; + supervisor) + unimplemented "--with supervisor" "the upgrade supervisor lands in S8; the profile is reserved and defines no service yet" + ;; + local | production) die "--with selects optional services; the site mode comes from --local or --domain" "$EXIT_USAGE" ;; + *) die "unknown optional service: $choice (analytics, activitypub)" "$EXIT_USAGE" ;; + esac +done +case ",$profiles," in *,analytics,*) want_analytics=1 ;; *) want_analytics=0 ;; esac +case ",$profiles," in *,activitypub,*) want_activitypub=1 ;; *) want_activitypub=0 ;; esac + +# --- Release identity ------------------------------------------------------ + +case ${channel:=stable} in + stable | beta) ;; + *) die "--channel must be stable or beta" "$EXIT_USAGE" ;; +esac + +stack_ref="" +stack_commit="" +if command -v git >/dev/null 2>&1 && git -C "$dir" rev-parse --git-dir >/dev/null 2>&1; then + stack_commit=$(git -C "$dir" rev-parse HEAD 2>/dev/null || printf '') + stack_ref=$(git -C "$dir" describe --tags --exact-match HEAD 2>/dev/null || + git -C "$dir" rev-parse --abbrev-ref HEAD 2>/dev/null || printf '') +fi +if [[ -n $ref && -n $stack_ref && $ref != "$stack_ref" ]]; then + die "--ref is $ref but this checkout is at $stack_ref. + install.sh installs the release it belongs to. Use bootstrap.sh --ref $ref to + select a different one." "$EXIT_USAGE" +fi +[[ -n $ref ]] || ref=$stack_ref + +# --- Ports ----------------------------------------------------------------- + +http_port=80 +https_port=443 + +if [[ -n $port ]]; then + if [[ ! $port =~ ^[0-9]+$ ]] || ((port < 1 || port > 65535)); then + die "--port must be a port number: got '$port'" "$EXIT_USAGE" + fi +else + port=$(free_port 2368) || die "no free port was found at or above 2368" +fi + +# --- Preflight ------------------------------------------------------------- + +printf 'Checking this host\n' +records=$(preflight_site "$dir" "$mode" "$port" "$http_port" "$https_port") +preflight_render "$records" +if preflight_failed "$records"; then + printf '\nerror: preflight failed. Nothing has been changed on this host.\n' >&2 + exit 1 +fi + +# --- Identity and secrets -------------------------------------------------- + +project=$(install_project_name "$mode" "$domain" "$dir") || + die "could not derive a project name from '$domain'" + +if [[ $mode == production ]]; then + url="https://$domain" + node_env=production + restart_policy=unless-stopped +else + url="http://localhost:$port" + node_env=development + restart_policy=no +fi +admin_url="" +[[ -n $admin_domain ]] && admin_url="https://$admin_domain" + +db_password=$(install_secret 24) || die "could not generate a database password" +db_root_password=$(install_secret 24) || die "could not generate a database root password" + +# --- Tinybird credentials for the analytics profile ------------------------ +# +# Read from the environment first so an unattended install can supply them +# without putting a token in a command line, where it would reach the process +# table and the shell history. + +tinybird_tracker_token=${TINYBIRD_TRACKER_TOKEN:-} +tinybird_admin_token=${TINYBIRD_ADMIN_TOKEN:-} +tinybird_workspace_id=${TINYBIRD_WORKSPACE_ID:-} +tinybird_api_url=${TINYBIRD_API_URL:-https://api.tinybird.co} + +if ((want_analytics)); then + if [[ -z $tinybird_tracker_token ]]; then + tinybird_tracker_token=$(install_ask_secret "Tinybird tracker token" || printf '') + fi + if [[ -z $tinybird_admin_token ]]; then + tinybird_admin_token=$(install_ask_secret "Tinybird admin token" || printf '') + fi + if [[ -z $tinybird_workspace_id ]]; then + tinybird_workspace_id=$(install_ask "Tinybird workspace id" || printf '') + fi + if [[ -z $tinybird_tracker_token || -z $tinybird_admin_token || -z $tinybird_workspace_id ]]; then + die "the analytics profile needs TINYBIRD_TRACKER_TOKEN, TINYBIRD_ADMIN_TOKEN and + TINYBIRD_WORKSPACE_ID. Export them, or drop analytics from --with. See TINYBIRD.md." "$EXIT_USAGE" + fi +fi + +# --- Resolve the exact Ghost image ----------------------------------------- + +printf '\nResolving the Ghost image\n' +resolved_ghost=$(install_resolve_ghost "$GD_DEFAULT_GHOST_IMAGE" "$ghost_version") || exit 1 +IFS=$'\t' read -r ghost_tag ghost_exact_version ghost_digest ghost_content_path ghost_tinybird_path \ + <<<"$resolved_ghost" +printf ' ok ghost %s (%s) %s\n' \ + "$GD_DEFAULT_GHOST_IMAGE:$ghost_tag" "$ghost_exact_version" "${ghost_digest:-no digest recorded}" + +# --- Write the configuration ----------------------------------------------- + +printf '\nWriting configuration\n' + +env_file="$dir/$GD_ENV_FILE_NAME" +ghost_env_file="$dir/$GD_GHOST_ENV_FILE_NAME" + +# Start from the tracked example so its comments — the ones that explain value +# encoding and the optional settings — survive into the installed file. +cp "$dir/$GD_ENV_FILE_NAME.example" "$env_file" +chmod 0600 "$env_file" + +set_env() { env_set "$env_file" "$1" "$2" 0600; } + +set_env COMPOSE_PROFILES "$profiles" +set_env SITE_MODE "$mode" +set_env COMPOSE_PROJECT_NAME "$project" +set_env PROJECT_DIR "$dir" +set_env NODE_ENV "$node_env" +set_env URL "$url" +set_env GHOST_IMAGE "$GD_DEFAULT_GHOST_IMAGE" +set_env GHOST_VERSION "$ghost_tag" +set_env GHOST_CONTENT_PATH "$ghost_content_path" +set_env GHOST_TINYBIRD_PATH "$ghost_tinybird_path" +set_env GHOST_PORT "$port" +set_env RESTART_POLICY "$restart_policy" +set_env DATABASE_HOST db +set_env DATABASE_PORT 3306 +set_env DATABASE_NAME ghost +set_env DATABASE_USER ghost +set_env DATABASE_PASSWORD "$db_password" +set_env DATABASE_ROOT_PASSWORD "$db_root_password" + +if [[ $mode == production ]]; then + set_env DOMAIN "$domain" + set_env HTTP_PORT "$http_port" + set_env HTTPS_PORT "$https_port" + if [[ -n $admin_domain ]]; then + set_env ADMIN_DOMAIN "$admin_domain" + set_env ADMIN_URL "$admin_url" + fi +else + # A local site has no Caddy ingress, so leaving the example's DOMAIN in + # place would describe a domain nothing serves. + env_unset "$env_file" DOMAIN +fi + +if ((want_analytics)); then + set_env TINYBIRD_API_URL "$tinybird_api_url" + set_env TINYBIRD_TRACKER_TOKEN "$tinybird_tracker_token" + set_env TINYBIRD_ADMIN_TOKEN "$tinybird_admin_token" + set_env TINYBIRD_WORKSPACE_ID "$tinybird_workspace_id" +fi + +printf ' ok %s\n' "$GD_ENV_FILE_NAME" + +# ghost.env is written fresh rather than copied from the example, whose SMTP +# block is a placeholder: a site that ships with smtp.example.com configured +# fails to send mail in a way that looks like a Ghost bug. +{ + printf '# Ghost application settings for %s.\n' "$project" + printf '#\n' + printf '# This is the only env_file of the ghost service. See ghost.env.example\n' + printf '# and docs/configuration.md. Write values with:\n' + printf '#\n' + printf '# scripts/config.sh set ghost.env KEY VALUE\n' + printf '#\n' + # shellcheck disable=SC2016 # the backticked `$` is literal prose, not an expansion + printf '# which encodes them for Compose. Do not hand-edit a value containing `$`.\n' + printf '\n' + printf '# Transactional email is required for staff logins, invites and password\n' + printf '# resets, separately from newsletters. Configure it before inviting anyone:\n' + printf '#\n' + printf '# scripts/config.sh set ghost.env mail__transport SMTP\n' + printf '# scripts/config.sh set ghost.env mail__options__host smtp.example.com\n' + printf '# scripts/config.sh set ghost.env mail__options__port 465\n' + printf '# scripts/config.sh set ghost.env mail__options__secure true\n' + printf '# scripts/config.sh set ghost.env mail__options__auth__user USER\n' + printf '# scripts/config.sh set ghost.env mail__options__auth__pass PASSWORD\n' + printf "# scripts/config.sh set ghost.env mail__from \"'Site' \"\n" + printf '\n' +} >"$ghost_env_file" +chmod 0600 "$ghost_env_file" + +if ((want_analytics || want_activitypub)); then + env_set "$ghost_env_file" labs__publicAPI true 0600 +fi +printf ' ok %s\n' "$GD_GHOST_ENV_FILE_NAME" + +# Bind mount sources must exist before the daemon resolves them. Ownership +# inside the containers is the images' own business: Ghost's entrypoint takes +# its content directory, and assuming a host uid here would be wrong under +# rootless Docker and userns remapping. +mkdir -p "$dir/data/ghost" "$dir/data/mysql" +chmod 0755 "$dir/data" "$dir/data/ghost" "$dir/data/mysql" +printf ' ok data directories\n' + +if ! config_validate "$dir"; then + die "the generated configuration did not validate. Please report this." +fi +printf ' ok configuration validates\n' + +# --- Routing --------------------------------------------------------------- + +if [[ $mode == production ]]; then + printf '\nRendering routes\n' + if ! caddy_apply "$dir"; then + die "the generated Caddy configuration could not be applied" + fi + printf ' ok caddy/sites/site.caddy\n' +fi + +# --- Metadata -------------------------------------------------------------- + +meta_init "$dir" \ + "mode=$mode" \ + "channel=$channel" \ + "stack.version=${ref:-unknown}" \ + "stack.commit=${stack_commit:-unknown}" \ + "stack.ref=${ref:-unknown}" \ + "site.project=$project" \ + "site.dir=$dir" \ + "site.url=$url" \ + "site.domain=$domain" \ + "site.adminDomain=$admin_domain" \ + "ghost.image=$GD_DEFAULT_GHOST_IMAGE" \ + "ghost.tag=$ghost_tag" \ + "ghost.version=$ghost_exact_version" \ + "ghost.digest=$ghost_digest" \ + "profiles=$profiles" +printf ' ok %s\n' "$GD_META_FILE_NAME" + +# --- Start and verify ------------------------------------------------------ + +started=0 +if ((no_start)); then + printf '\nNot starting: --no-start was given.\n' +else + printf '\nStarting services\n' + compose_run "$dir" up -d + started=1 + + install_wait_healthy "$dir" db "$GD_READY_TIMEOUT_DB" || + die "the database did not become ready. See: docker compose logs db" + printf ' ok database is ready\n' + + install_wait_healthy "$dir" ghost "$GD_READY_TIMEOUT_GHOST" || + die "Ghost did not become ready. See: docker compose logs ghost" + printf ' ok Ghost is ready\n' + + printf '\nVerifying ingress\n' + if ! install_verify_ingress "$dir" "$mode" "$port" "$http_port" "$domain" "$admin_domain"; then + die "the site started but is not reachable through its own ingress" + fi +fi + +# --- Summary --------------------------------------------------------------- + +admin_link=${admin_url:-$url} +cat < `next-alpine`, +# `6.3.1-alpine` -> `alpine`, `next-alpine` -> `next-alpine`, `6` -> ``. +_gd_tag_variant() { + local tag=$1 + if [[ $tag =~ ^[0-9]+(\.[0-9]+)*(-(.*))?$ ]]; then + printf '%s\n' "${BASH_REMATCH[3]}" + return 0 + fi + printf '%s\n' "$tag" +} + +# _gd_image_env IMAGE_REF NAME +# One environment variable declared by an image, without running it. +_gd_image_env() { + local line + while IFS= read -r line; do + if [[ $line == "$2="* ]]; then + printf '%s\n' "${line#"$2"=}" + return 0 + fi + done < <(docker image inspect "$1" --format '{{range .Config.Env}}{{println .}}{{end}}' 2>/dev/null) + return 1 +} + +# _gd_image_digest IMAGE_REF REPOSITORY +_gd_image_digest() { + local line + while IFS= read -r line; do + [[ $line == "$2@"* ]] || continue + printf '%s\n' "${line#*@}" + return 0 + done < <(docker image inspect "$1" --format '{{range .RepoDigests}}{{println .}}{{end}}' 2>/dev/null) + return 1 +} + +# _gd_image_tinybird_path IMAGE_REF INSTALL_PATH +# Where the image keeps the Tinybird datafiles. The two published layouts put +# them in different places, so the image is asked rather than the tag name +# being mapped to a layout — a mapping would drift the next time the layout +# changes. The environment-based fallback keeps this working if the probe +# cannot run. +_gd_image_tinybird_path() { + local image=$1 install=$2 out + out=$(docker run --rm --entrypoint sh "$image" -c ' + for p in "$GHOST_INSTALL/core/server/data/tinybird" \ + "$GHOST_INSTALL/current/core/server/data/tinybird"; do + [ -d "$p" ] && { printf "%s" "$p"; exit 0; } + done + exit 1' 2>/dev/null) && [[ -n $out ]] && { + printf '%s\n' "$out" + return 0 + } + + if _gd_image_env "$image" GHOST_CLI_INSTALL >/dev/null 2>&1; then + printf '%s/current/core/server/data/tinybird\n' "$install" + else + printf '%s/core/server/data/tinybird\n' "$install" + fi +} + +# install_resolve_ghost IMAGE REQUESTED +# Resolves the requested version to one exact image and prints, tab separated: +# +# TAG VERSION DIGEST CONTENT_PATH TINYBIRD_PATH +# +# TAG is the exact pin written to `.env`; a moving tag is never persisted. +# DIGEST is the immutable identity, recorded in `.ghost-docker.json` so the +# image can be identified again during recovery. The paths come from the +# image's own GHOST_CONTENT and GHOST_INSTALL, so the mounted content directory +# and the image layout cannot disagree. +install_resolve_ghost() { + local image=$1 requested=$2 + local tag version content install digest variant exact ref exact_ref tinybird + + tag=$(install_ghost_tag "$requested") + ref="$image:$tag" + + if ! docker pull --quiet "$ref" >/dev/null 2>&1; then + printf 'error: could not pull %s. Check the version and that this host can reach the registry.\n' "$ref" >&2 + return 1 + fi + + version=$(_gd_image_env "$ref" GHOST_VERSION) || { + printf 'error: %s does not declare GHOST_VERSION; it does not look like a Ghost image\n' "$ref" >&2 + return 1 + } + content=$(_gd_image_env "$ref" GHOST_CONTENT) || content=/home/ghost/content + install=$(_gd_image_env "$ref" GHOST_INSTALL) || install=/home/ghost + + # Prefer the immutable tag for that exact version, so the pin does not move + # under the site the next time the registry updates a rolling tag. It is + # accepted only when it is the same image. + variant=$(_gd_tag_variant "$tag") + if [[ -n $variant ]]; then + exact="$version-$variant" + else + exact=$version + fi + exact_ref="$image:$exact" + if [[ $exact != "$tag" ]] && docker pull --quiet "$exact_ref" >/dev/null 2>&1; then + if [[ $(docker image inspect "$exact_ref" --format '{{.Id}}' 2>/dev/null) == \ + $(docker image inspect "$ref" --format '{{.Id}}' 2>/dev/null) ]]; then + tag=$exact + ref=$exact_ref + fi + fi + + digest=$(_gd_image_digest "$ref" "$image") || digest="" + tinybird=$(_gd_image_tinybird_path "$ref" "$install") + + printf '%s\t%s\t%s\t%s\t%s\n' "$tag" "$version" "$digest" "$content" "$tinybird" +} + +# --- Prompting ------------------------------------------------------------- +# +# Prompts read from /dev/tty rather than stdin, so that an installer piped from +# curl can still ask a question. Where there is no terminal, or --no-prompt was +# given, a required answer is an error naming the flag that supplies it: no +# prompt has a silent default. + +GD_NO_PROMPT=${GD_NO_PROMPT:-0} + +# install_can_prompt +install_can_prompt() { + ((GD_NO_PROMPT)) && return 1 + [[ -r /dev/tty && -w /dev/tty ]] +} + +# install_ask PROMPT [DEFAULT] +# Prints the answer. Returns 1 when there is no way to ask. +install_ask() { + local prompt=$1 default=${2:-} answer + install_can_prompt || return 1 + if [[ -n $default ]]; then + printf '%s [%s]: ' "$prompt" "$default" >/dev/tty + else + printf '%s: ' "$prompt" >/dev/tty + fi + IFS= read -r answer /dev/tty + IFS= read -r -s answer /dev/tty + printf '%s\n' "$answer" +} + +# install_require VALUE PROMPT FLAG +# A required input: the value if it was supplied, otherwise a prompt, otherwise +# an error naming the flag or variable that provides it non-interactively. +install_require() { + local value=$1 prompt=$2 flag=$3 + if [[ -n $value ]]; then + printf '%s\n' "$value" + return 0 + fi + if value=$(install_ask "$prompt"); then + [[ -n $value ]] && { + printf '%s\n' "$value" + return 0 + } + fi + printf 'error: %s is required; supply it with %s\n' "$prompt" "$flag" >&2 + return 1 +} + +# --- Readiness ------------------------------------------------------------- + +# install_service_id DIR SERVICE +install_service_id() { + compose_run "$1" ps -q "$2" 2>/dev/null | head -1 +} + +# install_wait_healthy DIR SERVICE TIMEOUT +# Waits for a service's own health check, not for a running container. Fails +# early when the container has exited: waiting out a timeout on a container +# that is already gone tells the operator nothing. +install_wait_healthy() { + local dir=$1 service=$2 timeout=$3 waited=0 id state health + while ((waited < timeout)); do + id=$(install_service_id "$dir" "$service") + if [[ -n $id ]]; then + state=$(docker inspect -f '{{.State.Status}}' "$id" 2>/dev/null || printf '') + health=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$id" 2>/dev/null || printf '') + case $state in + exited | dead) + printf 'error: the %s container exited before it became ready\n' "$service" >&2 + return 1 + ;; + esac + [[ $health == healthy ]] && return 0 + # A service with no health check is ready when it is running. + [[ $health == none && $state == running ]] && return 0 + fi + sleep 3 + waited=$((waited + 3)) + done + printf 'error: %s did not become ready within %ss\n' "$service" "$timeout" >&2 + return 1 +} + +# install_http_head HOST PORT PATH [HOST_HEADER] +# The status line and headers of one HTTP response, using bash's own /dev/tcp +# so that neither curl nor wget has to be installed on the host. +install_http_head() { + local host=$1 port=$2 path=$3 host_header=${4:-$1} line + + exec 3<>"/dev/tcp/$host/$port" 2>/dev/null || return 1 + printf 'GET %s HTTP/1.1\r\nHost: %s\r\nConnection: close\r\nUser-Agent: ghost-docker-install\r\nAccept: */*\r\n\r\n' \ + "$path" "$host_header" >&3 || { + exec 3<&- + exec 3>&- + return 1 + } + while IFS= read -r -t 20 line <&3; do + line=${line%$'\r'} + [[ -z $line ]] && break + printf '%s\n' "$line" + done + exec 3<&- + exec 3>&- +} + +# install_http_status HOST PORT PATH [HOST_HEADER] +install_http_status() { + local head status + head=$(install_http_head "$@") || return 1 + status=$(printf '%s\n' "$head" | head -1) + [[ $status =~ ^HTTP/[0-9.]+[[:space:]]+([0-9]{3}) ]] || return 1 + printf '%s\n' "${BASH_REMATCH[1]}" +} + +# install_https_status DIR DOMAIN +# The status of an Admin API request through Caddy's HTTPS listener, made from +# inside the site network with the right SNI and Host. It runs in the Ghost +# container because TLS is beyond what bash's /dev/tcp can do, and neither curl +# nor openssl is a host requirement — while a Ghost image always has node. +install_https_status() { + local dir=$1 domain=$2 out path script + + path=$(env_get "$dir/.env" GHOST_HEALTHCHECK_PATH 2>/dev/null) || path=/ghost/api/admin/site/ + [[ -n $path ]] || path=/ghost/api/admin/site/ + + script=$( + cat <<'NODE' +const https = require('https'); +const [domain, path] = process.argv.slice(1); +const req = https.request({ + host: 'caddy', port: 443, servername: domain, path, + headers: { Host: domain }, rejectUnauthorized: false, timeout: 20000, +}, (res) => { res.resume(); process.stdout.write(String(res.statusCode)); }); +req.on('timeout', () => { req.destroy(); process.exit(1); }); +req.on('error', () => process.exit(1)); +req.end(); +NODE + ) + + out=$(compose_run "$dir" exec -T ghost node -e "$script" "$domain" "$path" 2>/dev/null) || return 1 + [[ $out =~ ^[0-9]{3}$ ]] || return 1 + printf '%s\n' "$out" +} + +# install_verify_ingress DIR MODE GHOST_PORT HTTP_PORT DOMAIN [ADMIN_DOMAIN] +# Confirms the Admin API answers through the ingress the site actually uses: +# the published loopback port in local mode, and Caddy in production. +# +# The HTTPS leg is a warning rather than a failure, deliberately. Caddy orders a +# certificate on the first request for a name, so a site installed before its +# DNS is pointed has no certificate yet and cannot have one. That is an expected +# state on a fresh production install, not a broken site — the routing checks +# above are the ones that prove the configuration is right. +install_verify_ingress() { + local dir=$1 mode=$2 ghost_port=$3 http_port=$4 domain=$5 admin=${6:-} + local status rc=0 path + + path=$(env_get "$dir/.env" GHOST_HEALTHCHECK_PATH 2>/dev/null) || path=/ghost/api/admin/site/ + [[ -n $path ]] || path=/ghost/api/admin/site/ + + if status=$(install_http_status 127.0.0.1 "$ghost_port" "$path" localhost); then + if [[ $status == 200 ]]; then + printf 'ok Ghost answers on 127.0.0.1:%s%s\n' "$ghost_port" "$path" + else + printf 'ERROR Ghost answered %s on 127.0.0.1:%s%s\n' "$status" "$ghost_port" "$path" >&2 + rc=1 + fi + else + printf 'ERROR nothing answered on 127.0.0.1:%s\n' "$ghost_port" >&2 + rc=1 + fi + + [[ $mode == production ]] || return $rc + + if ! caddy_verify "$dir" "$domain" "$admin"; then + rc=1 + else + printf 'ok Caddy routes %s\n' "$domain" + fi + + if status=$(install_http_status 127.0.0.1 "$http_port" / "$domain"); then + if [[ $status =~ ^3 ]]; then + printf 'ok Caddy redirects http://%s to HTTPS\n' "$domain" + else + printf 'ERROR Caddy answered %s for http://%s instead of a redirect to HTTPS\n' "$status" "$domain" >&2 + rc=1 + fi + else + printf 'ERROR Caddy did not answer on 127.0.0.1:%s\n' "$http_port" >&2 + rc=1 + fi + + if status=$(install_https_status "$dir" "$domain") && [[ $status == 200 ]]; then + printf 'ok Ghost Admin answers over HTTPS at %s\n' "$domain" + else + printf 'warning HTTPS for %s is not serving yet. Caddy orders a certificate on the\n' "$domain" >&2 + printf ' first request for the name, so point this domain at this host if you\n' >&2 + printf ' have not already. Routing itself is configured correctly.\n' >&2 + fi + + return $rc +} diff --git a/scripts/lib/meta.sh b/scripts/lib/meta.sh new file mode 100644 index 00000000..99c97dde --- /dev/null +++ b/scripts/lib/meta.sh @@ -0,0 +1,244 @@ +#!/usr/bin/env bash +# `.ghost-docker.json`: the installation metadata file. +# +# Records what an operation needs to know about a site that its configuration +# does not say: when it was installed, from which stack release and commit, on +# which release channel, the exact Ghost image identity that was resolved, and +# which migrations have completed. The schema is specified in section 2.2 of +# docs/ghost-cli-replacement.md. +# +# The file is machine generated, gitignored, and written privately: it names a +# site and its provenance, and later steps add operation state to it. +# +# An installation that predates this file is a supported state, not a broken +# site. `meta_present` answers that question; every reader must handle a +# missing file rather than failing on it. + +# shellcheck disable=SC2034 +GD_META_LIB_LOADED=1 + +GD_META_FILE_NAME=".ghost-docker.json" + +# The schema version this library writes. A reader must refuse a file whose +# version it does not understand rather than guessing at its shape. +readonly GD_META_SCHEMA_VERSION=1 + +# Keys accepted by meta_init, in `dotted.path` form. Allowlisted so that a +# misspelled key is an error rather than a field nothing ever reads. +readonly GD_META_KEYS=( + mode + channel + installedAt + stack.version + stack.commit + stack.ref + site.project + site.dir + site.url + site.domain + site.adminDomain + ghost.image + ghost.tag + ghost.version + ghost.digest + profiles +) + +# meta_file DIR +meta_file() { + printf '%s/%s\n' "$1" "$GD_META_FILE_NAME" +} + +# meta_present DIR +# True when the site records metadata. False means "installed before metadata +# was recorded", which callers must treat as unknown rather than as an error. +meta_present() { + [[ -f "$1/$GD_META_FILE_NAME" ]] +} + +# meta_read DIR +# Prints the metadata document. Returns 1 when there is none, and 2 when the +# file is present but unreadable or not valid JSON. +meta_read() { + local file=$1/$GD_META_FILE_NAME + [[ -f $file ]] || return 1 + jq -e . "$file" >/dev/null 2>&1 || { + printf 'error: %s is not valid JSON\n' "$file" >&2 + return 2 + } + cat "$file" +} + +# meta_schema_version DIR +# Prints the recorded schema version, or nothing for a pre-metadata install. +meta_schema_version() { + local file=$1/$GD_META_FILE_NAME + [[ -f $file ]] || return 1 + jq -r '.schemaVersion // empty' "$file" 2>/dev/null +} + +# meta_check_schema DIR +# Fails when the file records a schema this library cannot read. A newer file +# is a stack that was downgraded; say so instead of misreading it. +meta_check_schema() { + local version + meta_present "$1" || return 0 + version=$(meta_schema_version "$1") || version= + if [[ -z $version ]]; then + printf 'error: %s/%s has no schemaVersion\n' "$1" "$GD_META_FILE_NAME" >&2 + return 1 + fi + if [[ $version != "$GD_META_SCHEMA_VERSION" ]]; then + printf 'error: %s/%s uses schema version %s; this stack understands version %s\n' \ + "$1" "$GD_META_FILE_NAME" "$version" "$GD_META_SCHEMA_VERSION" >&2 + return 1 + fi +} + +# meta_get DIR FILTER +# Reads one value with a jq filter, for example `.ghost.version`. Returns 1 +# when the file is absent or the filter yields null, so a caller can +# distinguish "not recorded" from an empty string. +meta_get() { + local file=$1/$GD_META_FILE_NAME out + [[ -f $file ]] || return 1 + out=$(jq -r "$2 // empty" "$file" 2>/dev/null) || return 1 + [[ -n $out ]] || return 1 + printf '%s\n' "$out" +} + +# meta_write DIR +# Replaces the metadata with the JSON document on stdin. The document is +# validated and normalised before it is installed, so a failed write can never +# leave an unreadable file in place. Mode 0600: this file names a site and its +# provenance. +meta_write() { + local dir=$1 file=$1/$GD_META_FILE_NAME body + body=$(cat) + if ! printf '%s' "$body" | jq -e . >/dev/null 2>&1; then + printf 'error: refusing to write invalid JSON to %s\n' "$file" >&2 + return 1 + fi + if [[ $(printf '%s' "$body" | jq -r '.schemaVersion // empty') != "$GD_META_SCHEMA_VERSION" ]]; then + printf 'error: refusing to write %s without schemaVersion %s\n' \ + "$file" "$GD_META_SCHEMA_VERSION" >&2 + return 1 + fi + printf '%s' "$body" | jq -S . | fs_atomic_write "$file" 0600 +} + +# meta_update DIR FILTER [JQ_ARG...] +# Applies a jq filter to the existing metadata and writes the result back. +meta_update() { + local dir=$1 filter=$2 + shift 2 + meta_present "$dir" || { + printf 'error: %s/%s does not exist\n' "$dir" "$GD_META_FILE_NAME" >&2 + return 1 + } + meta_check_schema "$dir" || return 1 + jq "$@" "$filter" "$dir/$GD_META_FILE_NAME" | meta_write "$dir" +} + +# meta_record_migration DIR NAME +# Appends a completed migration, without duplicating one already recorded. +meta_record_migration() { + # shellcheck disable=SC2016 # $name is a jq variable, bound by --arg below + meta_update "$1" '.migrations = (.migrations + [$name] | unique)' --arg name "$2" +} + +# meta_has_migration DIR NAME +meta_has_migration() { + meta_present "$1" || return 1 + jq -e --arg name "$2" '.migrations | index($name) != null' \ + "$1/$GD_META_FILE_NAME" >/dev/null 2>&1 +} + +# _gd_meta_known_key KEY +_gd_meta_known_key() { + local known + for known in "${GD_META_KEYS[@]}"; do + [[ $1 == "$known" ]] && return 0 + done + return 1 +} + +# meta_init DIR KEY=VALUE... +# Writes a fresh metadata document. Keys are the dotted paths in GD_META_KEYS; +# `profiles` is a comma separated list and becomes an array. `installedAt` +# defaults to now, in UTC. +meta_init() { + local dir=$1 pair key value + shift + local -a jqargs=() + + for pair in "$@"; do + [[ $pair == *=* ]] || { + printf 'error: metadata argument %s is not KEY=VALUE\n' "$pair" >&2 + return 2 + } + key=${pair%%=*} + value=${pair#*=} + _gd_meta_known_key "$key" || { + printf 'error: unknown metadata key %s\n' "$key" >&2 + return 2 + } + # jq argument names cannot contain a dot. + jqargs+=(--arg "${key//./_}" "$value") + done + + jq -n \ + --argjson schema "$GD_META_SCHEMA_VERSION" \ + --arg now "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ + ${jqargs[@]+"${jqargs[@]}"} ' + ($ARGS.named) as $a + | def val($k): ($a[$k] // "") | if . == "" then null else . end; + { + schemaVersion: $schema, + installedAt: (val("installedAt") // $now), + mode: val("mode"), + channel: val("channel"), + stack: { + version: val("stack_version"), + commit: val("stack_commit"), + ref: val("stack_ref"), + }, + site: { + project: val("site_project"), + dir: val("site_dir"), + url: val("site_url"), + domain: val("site_domain"), + adminDomain: val("site_adminDomain"), + }, + ghost: { + image: val("ghost_image"), + tag: val("ghost_tag"), + version: val("ghost_version"), + digest: val("ghost_digest"), + }, + profiles: ((val("profiles") // "") | split(",") | map(select(length > 0))), + migrations: [], + }' | meta_write "$dir" +} + +# meta_describe DIR +# A human readable summary, including for a site with no metadata at all. +meta_describe() { + local dir=$1 + if ! meta_present "$dir"; then + printf 'installation metadata: none (installed before metadata was recorded)\n' + return 0 + fi + meta_check_schema "$dir" || return 1 + jq -r ' + "installed: \(.installedAt // "unknown")", + "mode: \(.mode // "unknown")", + "channel: \(.channel // "unknown")", + "stack: \(.stack.version // "unknown") (\(.stack.commit // "unknown"))", + "project: \(.site.project // "unknown")", + "url: \(.site.url // "unknown")", + "ghost: \(.ghost.tag // "unknown") \(.ghost.digest // "")", + "profiles: \(.profiles // [] | join(",") )", + "migrations: \(.migrations // [] | join(",") )" + ' "$dir/$GD_META_FILE_NAME" +} diff --git a/scripts/lib/preflight.sh b/scripts/lib/preflight.sh new file mode 100644 index 00000000..f1528e43 --- /dev/null +++ b/scripts/lib/preflight.sh @@ -0,0 +1,409 @@ +#!/usr/bin/env bash +# Host preflight: everything that has to be true before a site can be installed +# or is worth diagnosing when one is broken. +# +# These checks run in host shell rather than in a container, deliberately: they +# have to work on a host where Docker is missing, stopped, or unreachable, so +# they cannot depend on being able to run an image. See section 2.10 of +# docs/ghost-cli-replacement.md. +# +# Every check prints one record, `STATUSLABELDETAIL`, where STATUS is +# `ok`, `warn` or `error`. Callers render the records; `preflight_failed` +# decides the exit status. That keeps the checks free of presentation and makes +# them straightforward to assert on. +# +# Portability: no GNU-only utilities, no `sort -V`, no `ss`/`lsof`/`curl` +# requirement, and daemon access is established by using the daemon rather than +# by inspecting group membership. + +# shellcheck disable=SC2034 +GD_PREFLIGHT_LIB_LOADED=1 + +# Host commands the installer and helpers actually invoke. Anything not on this +# list must not appear in a code path an operator can reach; the minimum-tools +# test runs an install with a PATH containing only these. +readonly GD_REQUIRED_COMMANDS=(docker jq) + +# Every other host utility the installer and helpers invoke. These are POSIX +# and present on any supported host, so they are not preflight checks; they are +# recorded here as the tool contract, and tests/install-e2e.test.mjs runs an +# install with a PATH built from exactly this list plus GD_REQUIRED_COMMANDS. +# Adding a utility to a code path without adding it here fails that test, which +# is the point: it is how a GNU-only or unusual dependency gets noticed. +readonly GD_HOST_UTILITIES=( + awk basename bash cat chmod chown cp cut date df dirname env grep head id + ls mkdir mktemp mv od rm sed sleep sort stat tr uname +) + +# Recommended free space for a site: Ghost and MySQL images, the database, and +# uploaded content. Below this an install will probably succeed and then fail +# later, so it is a warning rather than a refusal. +readonly GD_RECOMMENDED_DISK_MB=5120 +readonly GD_RECOMMENDED_MEMORY_MB=1024 + +# How long a read-only Docker probe may take before it is treated as +# unreachable. A daemon that has wedged does not return an error, it stops +# answering; without a bound, the check that exists to diagnose that hangs +# instead of reporting it. +GD_DOCKER_PROBE_TIMEOUT=${GD_DOCKER_PROBE_TIMEOUT:-20} + +# _gd_timeout SECONDS COMMAND... +# Runs a command with a deadline, returning 124 when it is killed. `timeout(1)` +# is GNU coreutils and is not present on macOS, so this is spelled out. +_gd_timeout() { + local seconds=$1 pid waited=0 + shift + "$@" & + pid=$! + while kill -0 "$pid" 2>/dev/null; do + if ((waited >= seconds)); then + kill -9 "$pid" 2>/dev/null + wait "$pid" 2>/dev/null + return 124 + fi + sleep 1 + waited=$((waited + 1)) + done + wait "$pid" +} + +# _gd_docker SECONDS ARGS... +# `docker ARGS` with a deadline, printing what it wrote to stdout and stderr. +# +# The output goes through a temporary file rather than a pipe, deliberately: a +# docker client can leave a helper process behind holding the write end, and a +# command substitution then waits for that helper long after the client itself +# has been killed — which is exactly the wedged-daemon case this deadline +# exists to report. +_gd_docker() { + local seconds=$1 tmp rc + shift + tmp=$(fs_mktemp_dir gd-docker)/out || return 1 + _gd_timeout "$seconds" docker "$@" >"$tmp" 2>&1 /dev/null + rm -rf "$(dirname "$tmp")" + return $rc +} + +# docker_responsive [SECONDS] +# True when the daemon answers within the deadline. This is the one question +# every other Docker check depends on, and it is answered by asking the daemon, +# never by inspecting group membership: neither rootless Docker nor a remote +# DOCKER_HOST involves the docker group, and being in it does not mean the +# daemon is running. +docker_responsive() { + command -v docker >/dev/null 2>&1 || return 1 + _gd_docker "${1:-$GD_DOCKER_PROBE_TIMEOUT}" info >/dev/null 2>&1 +} + +# _gd_report STATUS LABEL DETAIL +_gd_report() { + printf '%s\t%s\t%s\n' "$1" "$2" "$3" +} + +# preflight_failed RECORDS +# True when any record is an error. Warnings never fail a preflight. +preflight_failed() { + printf '%s\n' "$1" | grep -q '^error ' +} + +# preflight_render RECORDS +# Renders records for a person. Errors and warnings go to stderr so that a +# caller can separate them from a summary on stdout. +preflight_render() { + local status label detail + while IFS=$'\t' read -r status label detail; do + [[ -n $status ]] || continue + case $status in + ok) printf ' ok %-22s %s\n' "$label" "$detail" ;; + warn) printf ' warning %-22s %s\n' "$label" "$detail" >&2 ;; + *) printf ' ERROR %-22s %s\n' "$label" "$detail" >&2 ;; + esac + done <<<"$1" +} + +# version_compare A B +# Prints -1, 0 or 1 for A against B, comparing dot separated numeric +# components. Trailing non-numeric parts (`-beta.1`, `+ce`) are ignored, which +# is what a minimum-version check needs. Implemented here because `sort -V` is +# not portable and lexical comparison gets 2.9.0 and 2.24.0 backwards. +version_compare() { + local a=$1 b=$2 i x y + local -a av bv + a=${a#v} + b=${b#v} + a=${a%%[-+]*} + b=${b%%[-+]*} + IFS=. read -ra av <<<"$a" + IFS=. read -ra bv <<<"$b" + for ((i = 0; i < 4; i++)); do + x=${av[i]:-0} + y=${bv[i]:-0} + # A component that is not a number sorts as zero rather than crashing. + [[ $x =~ ^[0-9]+$ ]] || x=0 + [[ $y =~ ^[0-9]+$ ]] || y=0 + if ((10#$x > 10#$y)); then + printf '1\n' + return 0 + fi + if ((10#$x < 10#$y)); then + printf -- '-1\n' + return 0 + fi + done + printf '0\n' +} + +# version_at_least VERSION MINIMUM +version_at_least() { + [[ $(version_compare "$1" "$2") != "-1" ]] +} + +# preflight_os +preflight_os() { + local os arch + os=$(uname -s 2>/dev/null || printf 'unknown') + arch=$(uname -m 2>/dev/null || printf 'unknown') + case $os in + Linux | Darwin) ;; + *) + _gd_report error "operating system" "$os is not supported; Linux and macOS are" + return + ;; + esac + case $arch in + x86_64 | amd64 | arm64 | aarch64) ;; + *) + _gd_report error architecture "$arch has no published images for the services this stack runs" + return + ;; + esac + _gd_report ok platform "$os/$arch" +} + +# preflight_commands [COMMAND...] +preflight_commands() { + local cmd missing="" + local -a wanted=("$@") + ((${#wanted[@]})) || wanted=("${GD_REQUIRED_COMMANDS[@]}") + for cmd in "${wanted[@]}"; do + command -v "$cmd" >/dev/null 2>&1 || missing="$missing $cmd" + done + if [[ -n $missing ]]; then + _gd_report error "required tools" "missing:${missing}" + return + fi + _gd_report ok "required tools" "${wanted[*]}" +} + +# _gd_daemon_hint +# What to actually do about an unreachable daemon on this host. +_gd_daemon_hint() { + case $(uname -s 2>/dev/null) in + Darwin) printf 'start Docker Desktop or OrbStack and wait for it to report running' ;; + Linux) + if [[ -d /run/systemd/system ]]; then + printf 'start it with: sudo systemctl start docker' + else + printf 'start the Docker daemon for this system' + fi + ;; + *) printf 'start the Docker daemon' ;; + esac +} + +# preflight_docker +# Docker and Compose, in the order a failure should be reported: the binary, +# then daemon access, then versions. Daemon access is tested by asking the +# daemon a question. Membership of the `docker` group is neither necessary +# (rootless, sudo-less contexts, a remote DOCKER_HOST) nor sufficient (a +# stopped daemon), so it is not what gets checked. +preflight_docker() { + local out rc version compose_version + + if ! command -v docker >/dev/null 2>&1; then + _gd_report error docker "not installed; see https://docs.docker.com/engine/install/" + return + fi + + out=$(_gd_docker "$GD_DOCKER_PROBE_TIMEOUT" info --format '{{.ServerVersion}}') + rc=$? + if ((rc == 124)); then + _gd_report error "docker daemon" "did not answer within ${GD_DOCKER_PROBE_TIMEOUT}s; it is running but not responding ($(_gd_daemon_hint))" + return + fi + if ((rc != 0)); then + _gd_report error "docker daemon" "not reachable ($(_gd_daemon_hint)); docker said: $(printf '%s' "$out" | tr '\n' ' ' | cut -c1-160)" + return + fi + version=$out + [[ -n $version ]] || version=$(_gd_docker "$GD_DOCKER_PROBE_TIMEOUT" version --format '{{.Server.Version}}' || printf '') + + if [[ -z $version ]]; then + _gd_report warn "docker engine" "reachable, but the version could not be determined" + elif version_at_least "$version" "$GD_MIN_DOCKER_VERSION"; then + _gd_report ok "docker engine" "$version" + else + _gd_report error "docker engine" "$version is older than the required $GD_MIN_DOCKER_VERSION (healthcheck start_interval)" + fi + + if ! compose_version=$(_gd_docker "$GD_DOCKER_PROBE_TIMEOUT" compose version --short); then + _gd_report error "docker compose" "the Compose v2 plugin is not available; see https://docs.docker.com/compose/install/" + return + fi + if version_at_least "$compose_version" "$GD_MIN_COMPOSE_VERSION"; then + _gd_report ok "docker compose" "$compose_version" + else + _gd_report error "docker compose" "$compose_version is older than the required $GD_MIN_COMPOSE_VERSION (env_file required:, depends_on required:)" + fi +} + +# port_in_use PORT +# True when something already accepts connections on the loopback interface. +# bash's own /dev/tcp is used so that no probing utility has to be installed; +# a container publishing on 0.0.0.0 answers here too. +port_in_use() { + local port=$1 + [[ $port =~ ^[0-9]+$ ]] || return 1 + (exec 3<>"/dev/tcp/127.0.0.1/$port") >/dev/null 2>&1 && return 0 + return 1 +} + +# port_holder PORT +# Best effort description of what holds a port, for the error message only. +# Docker is asked first because a published container port is the case an +# operator most needs named; host tools are used when they happen to exist. +port_holder() { + local port=$1 line out + + if command -v docker >/dev/null 2>&1; then + while IFS= read -r line; do + case $line in + *":$port->"*) + printf 'docker container %s\n' "${line%%$'\t'*}" + return 0 + ;; + esac + done <<<"$(_gd_docker "$GD_DOCKER_PROBE_TIMEOUT" ps --format '{{.Names}} {{.Ports}}')" + fi + + if command -v lsof >/dev/null 2>&1; then + out=$(lsof -nP -iTCP:"$port" -sTCP:LISTEN -Fc 2>/dev/null | sed -n 's/^c//p' | head -1) + [[ -n $out ]] && { + printf 'process %s\n' "$out" + return 0 + } + fi + if command -v ss >/dev/null 2>&1; then + out=$(ss -ltnH "sport = :$port" 2>/dev/null | head -1) + [[ -n $out ]] && { + printf 'a listener (ss: %s)\n' "$out" + return 0 + } + fi + + printf 'another process\n' +} + +# preflight_port PORT PURPOSE +# A port that is already in use is an error, never something to resolve by +# stopping whatever holds it: on a server that may be the operator's own proxy, +# serving other applications. +preflight_port() { + local port=$1 purpose=$2 + if port_in_use "$port"; then + _gd_report error "port $port" "already in use by $(port_holder "$port"); $purpose needs it. Free it, or choose another port. Nothing was stopped." + return + fi + _gd_report ok "port $port" "free ($purpose)" +} + +# free_port [START] [COUNT] +# The first free port at or above START. Used only when no port was requested: +# an explicitly requested port that is busy is an error, not something to work +# around silently. +free_port() { + local port=${1:-2368} limit=${2:-200} tried=0 + while ((tried < limit)); do + if ! port_in_use "$port"; then + printf '%s\n' "$port" + return 0 + fi + port=$((port + 1)) + tried=$((tried + 1)) + done + return 1 +} + +# preflight_disk DIR +preflight_disk() { + local dir=$1 probe avail_kb avail_mb + probe=$dir + while [[ -n $probe && ! -d $probe ]]; do probe=$(dirname "$probe"); done + [[ -d $probe ]] || probe=. + + # -P is POSIX output; -k is kibibytes on both GNU and BSD df. + avail_kb=$(df -Pk "$probe" 2>/dev/null | awk 'NR==2 {print $4}') + [[ $avail_kb =~ ^[0-9]+$ ]] || { + _gd_report warn "disk space" "could not be determined for $probe" + return + } + avail_mb=$((avail_kb / 1024)) + if ((avail_mb < GD_RECOMMENDED_DISK_MB)); then + _gd_report warn "disk space" "${avail_mb} MB free on $probe; ${GD_RECOMMENDED_DISK_MB} MB is recommended for images, the database and content" + return + fi + _gd_report ok "disk space" "${avail_mb} MB free on $probe" +} + +# preflight_memory +preflight_memory() { + local total_mb="" + if [[ -r /proc/meminfo ]]; then + total_mb=$(awk '/^MemTotal:/ {print int($2/1024)}' /proc/meminfo 2>/dev/null) + elif command -v sysctl >/dev/null 2>&1; then + total_mb=$(sysctl -n hw.memsize 2>/dev/null) + [[ $total_mb =~ ^[0-9]+$ ]] && total_mb=$((total_mb / 1024 / 1024)) + fi + [[ $total_mb =~ ^[0-9]+$ ]] || { + _gd_report warn memory "could not be determined" + return + } + if ((total_mb < GD_RECOMMENDED_MEMORY_MB)); then + _gd_report warn memory "${total_mb} MB; Ghost and MySQL together want at least ${GD_RECOMMENDED_MEMORY_MB} MB" + return + fi + _gd_report ok memory "${total_mb} MB" +} + +# preflight_writable DIR +# The site directory has to be writable by the user running this, and its +# bind-mounted data directories have to exist at daemon start. +preflight_writable() { + local dir=$1 probe=$1 + while [[ -n $probe && ! -d $probe ]]; do probe=$(dirname "$probe"); done + if [[ ! -w $probe ]]; then + _gd_report error "site directory" "$probe is not writable by $(id -un 2>/dev/null || printf 'this user')" + return + fi + _gd_report ok "site directory" "$dir" +} + +# preflight_site DIR MODE GHOST_PORT [HTTP_PORT] [HTTPS_PORT] +# The whole mode-aware preflight, as one record stream. +preflight_site() { + local dir=$1 mode=$2 ghost_port=$3 http_port=${4:-80} https_port=${5:-443} + + preflight_os + preflight_commands "${GD_REQUIRED_COMMANDS[@]}" + preflight_docker + preflight_writable "$dir" + preflight_disk "$dir" + preflight_memory + preflight_port "$ghost_port" "Ghost on the loopback interface" + if [[ $mode == production ]]; then + preflight_port "$http_port" "Caddy HTTP" + preflight_port "$https_port" "Caddy HTTPS" + fi +} diff --git a/scripts/site.sh b/scripts/site.sh new file mode 100755 index 00000000..93743382 --- /dev/null +++ b/scripts/site.sh @@ -0,0 +1,162 @@ +#!/usr/bin/env bash +# Inspect the sites on this host. +# +# scripts/site.sh list every container this stack manages +# scripts/site.sh check [DIR] diagnose one site +# scripts/site.sh info [DIR] print the recorded installation metadata +# +# `list` reads Docker's own labels, including stopped containers, so a site +# that is down still appears. There is no registry of installations: a checkout +# whose containers have never been created is not discoverable from here, and +# `list` says so rather than implying the list is complete. +set -euo pipefail + +# shellcheck source=scripts/lib/common.sh +. "$(dirname -- "$0")/lib/common.sh" + +readonly MANAGED_LABEL="org.ghost.docker.managed=true" + +site_list() { + local out + if ! docker_responsive; then + printf 'The Docker daemon is not reachable, so no containers can be listed.\n' >&2 + return 1 + fi + + out=$(docker ps -a \ + --filter "label=$MANAGED_LABEL" \ + --format '{{.Label "org.ghost.docker.site"}} {{.Label "org.ghost.docker.mode"}} {{.Label "org.ghost.docker.role"}} {{.Label "org.ghost.docker.lifecycle"}} {{.Status}}' \ + 2>/dev/null | sort) + + if [[ -z $out ]]; then + printf 'No ghost-docker containers exist on this host.\n' + else + printf '%-28s %-11s %-20s %-11s %s\n' SITE MODE SERVICE LIFECYCLE STATUS + local site mode role lifecycle status + while IFS=$'\t' read -r site mode role lifecycle status; do + printf '%-28s %-11s %-20s %-11s %s\n' "$site" "$mode" "$role" "$lifecycle" "$status" + done <<<"$out" + fi + + printf '\nOnly sites whose containers exist are listed. A checkout that has never\n' + printf 'been started has no containers and no host-wide registry to be found in;\n' + # shellcheck disable=SC2016 # backticks are prose here + printf 'run `scripts/site.sh check` from inside it instead.\n' +} + +# _db_reachable DIR +# A real client connection to the application database, not a running +# container. The password goes through MYSQL_PWD so that it does not appear in +# the container's process list. +_db_reachable() { + local dir=$1 env=$1/.env user database password + user=$(env_get "$env" DATABASE_USER 2>/dev/null) || user=ghost + database=$(env_get "$env" DATABASE_NAME 2>/dev/null) || database=ghost + password=$(env_get "$env" DATABASE_PASSWORD 2>/dev/null) || return 1 + MYSQL_PWD=$password compose_run "$dir" exec -T \ + -e MYSQL_PWD \ + db mysql -h 127.0.0.1 -u "$user" -e 'SELECT 1' "$database" >/dev/null 2>&1 +} + +site_check() { + local dir=$1 rc=0 records mode profiles port http_port domain admin id health + + printf 'Site directory\n %s\n\n' "$dir" + + if [[ ! -f $dir/$GD_ENV_FILE_NAME ]]; then + printf 'error: %s/%s does not exist; this directory does not hold a site.\n' \ + "$dir" "$GD_ENV_FILE_NAME" >&2 + return 1 + fi + + profiles=$(env_get "$dir/$GD_ENV_FILE_NAME" COMPOSE_PROFILES 2>/dev/null) || profiles="" + mode=$(compose_site_mode "$profiles" 2>/dev/null) || mode="" + port=$(env_get "$dir/$GD_ENV_FILE_NAME" GHOST_PORT 2>/dev/null) || port=2368 + http_port=$(env_get "$dir/$GD_ENV_FILE_NAME" HTTP_PORT 2>/dev/null) || http_port=80 + domain=$(env_get "$dir/$GD_ENV_FILE_NAME" DOMAIN 2>/dev/null) || domain="" + admin=$(env_get "$dir/$GD_ENV_FILE_NAME" ADMIN_DOMAIN 2>/dev/null) || admin="" + + printf 'Installation\n' + meta_describe "$dir" | sed 's/^/ /' || rc=1 + printf '\n' + + # Host checks first: they are the ones that still work when Docker is + # broken, which is when a doctor command matters most. The site's own ports + # are expected to be in use here, so they are not re-checked as conflicts. + printf 'Host\n' + records=$( + preflight_os + preflight_commands "${GD_REQUIRED_COMMANDS[@]}" + preflight_docker + preflight_disk "$dir" + preflight_memory + ) + preflight_render "$records" + preflight_failed "$records" && rc=1 + + printf '\nConfiguration\n' + if config_validate "$dir" | sed 's/^/ /'; then + printf ' ok .env and ghost.env are valid for mode %s\n' "${mode:-unknown}" + else + rc=1 + fi + + if ! docker_responsive; then + printf '\nServices\n skipped: the Docker daemon is not reachable.\n' + return 1 + fi + + printf '\nServices\n' + local service + for service in db ghost caddy; do + [[ $service != caddy || $mode == production ]] || continue + id=$(install_service_id "$dir" "$service" || printf '') + if [[ -z $id ]]; then + printf ' ERROR %-8s no container. Start it with: docker compose up -d\n' "$service" >&2 + rc=1 + continue + fi + health=$(docker inspect -f '{{.State.Status}}{{if .State.Health}}/{{.State.Health.Status}}{{end}}' "$id" 2>/dev/null || printf 'unknown') + case $health in + running | running/healthy) printf ' ok %-8s %s\n' "$service" "$health" ;; + *) + printf ' ERROR %-8s %s\n' "$service" "$health" >&2 + rc=1 + ;; + esac + done + + if _db_reachable "$dir"; then + printf ' ok database accepts a client connection to the application database\n' + else + printf ' ERROR database could not be reached with the configured credentials\n' >&2 + rc=1 + fi + + printf '\nIngress\n' + install_verify_ingress "$dir" "${mode:-local}" "$port" "$http_port" "$domain" "$admin" | + sed 's/^/ /' || rc=1 + + printf '\n' + if ((rc == 0)); then + printf 'This site looks healthy.\n' + else + printf 'Problems were reported above.\n' >&2 + fi + return $rc +} + +cmd=${1:-} +(($#)) && shift || true + +case "$cmd" in + list) site_list ;; + check | doctor) site_check "${1:-$GD_ROOT_DIR}" ;; + info) meta_describe "${1:-$GD_ROOT_DIR}" ;; + '' | -h | --help | help) usage ;; + *) + printf 'unknown command: %s\n' "$cmd" >&2 + usage >&2 + exit 2 + ;; +esac diff --git a/tests/helpers.mjs b/tests/helpers.mjs index 367f7f64..68483dbf 100644 --- a/tests/helpers.mjs +++ b/tests/helpers.mjs @@ -1,6 +1,6 @@ import { execFileSync, execFile, spawnSync } from 'node:child_process'; import { promisify } from 'node:util'; -import { mkdtempSync, mkdirSync, rmSync, cpSync, writeFileSync, chmodSync } from 'node:fs'; +import { mkdtempSync, mkdirSync, rmSync, cpSync, writeFileSync, chmodSync, readdirSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -10,7 +10,7 @@ const execFileAsync = promisify(execFile); export const TESTS_DIR = dirname(fileURLToPath(import.meta.url)); export const REPO_DIR = join(TESTS_DIR, '..'); -const LIBS = ['fs', 'env', 'compose', 'config', 'caddy']; +const LIBS = ['fs', 'env', 'compose', 'config', 'caddy', 'meta', 'preflight', 'install']; /** * Run a bash snippet with every ghost-docker library sourced. @@ -175,3 +175,91 @@ function composeVersion(bin) { const argv = bin ? ['version', '--short'] : ['compose', 'version', '--short']; return execFileSync(bin ?? 'docker', argv, { encoding: 'utf8' }).trim().replace(/^v/, ''); } + +// --- Installation ---------------------------------------------------------- +// +// The installer is exercised the way an operator reaches it: a candidate +// release is built from the working tree, tagged, and installed through +// bootstrap.sh or the checkout's own install.sh. + +/** Never travels with a release: local state, secrets, and generated routes. */ +const RELEASE_EXCLUDE = new Set([ + '.git', 'data', 'node_modules', '.env', 'ghost.env', '.ghost-docker.json', +]); + +/** Copy the working tree into `dest` as a release would ship it. */ +export function copyWorktree(dest) { + mkdirSync(dest, { recursive: true }); + for (const entry of readdirSync(REPO_DIR)) { + if (RELEASE_EXCLUDE.has(entry)) continue; + cpSync(join(REPO_DIR, entry), join(dest, entry), { recursive: true }); + } + // Generated and operator-owned routes are per-site, not part of a release. + for (const sub of ['sites', 'custom', 'global']) { + const routes = join(dest, 'caddy', sub); + for (const file of readdirSync(routes)) { + if (file.endsWith('.caddy')) rmSync(join(routes, file), { force: true }); + } + } + rmSync(join(dest, 'caddy', '.staging'), { recursive: true, force: true }); + return dest; +} + +const GIT_IDENTITY = { + GIT_AUTHOR_NAME: 'ghost-docker tests', + GIT_AUTHOR_EMAIL: 'tests@example.com', + GIT_COMMITTER_NAME: 'ghost-docker tests', + GIT_COMMITTER_EMAIL: 'tests@example.com', +}; + +export function git(repo, args) { + return execFileSync('git', ['-C', repo, ...args], { + encoding: 'utf8', + env: { ...process.env, ...GIT_IDENTITY }, + stdio: ['pipe', 'pipe', 'pipe'], + }); +} + +/** + * A candidate release of the working tree: a real git repository with real + * tags, so bootstrap.sh resolves and clones it exactly as it would the + * published one. + */ +export function makeCandidateRelease(dir, tags = ['v9.9.9']) { + const repo = join(dir, 'release'); + copyWorktree(repo); + git(repo, ['init', '-q', '-b', 'main']); + git(repo, ['add', '-A']); + git(repo, ['commit', '-q', '-m', 'candidate release']); + for (const tag of tags) git(repo, ['tag', tag]); + return { repo, tags }; +} + +/** Run a script with a captured status, without throwing. */ +export function run(command, args, { cwd, env = {}, input, timeout } = {}) { + const result = spawnSync(command, args, { + cwd, + input, + timeout, + encoding: 'utf8', + env: { ...process.env, ...env }, + }); + if (result.error && result.error.code !== 'ETIMEDOUT') throw result.error; + return { + stdout: result.stdout ?? '', + stderr: result.stderr ?? '', + status: result.status, + output: `${result.stdout ?? ''}${result.stderr ?? ''}`, + }; +} + +/** Occupy a TCP port for the duration of a test, so a conflict is real. */ +export async function occupyPort(port, host = '127.0.0.1') { + const { createServer } = await import('node:net'); + const server = createServer(() => {}); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(port, host, resolve); + }); + return () => new Promise((resolve) => server.close(resolve)); +} diff --git a/tests/install-e2e.test.mjs b/tests/install-e2e.test.mjs new file mode 100644 index 00000000..88978b4d --- /dev/null +++ b/tests/install-e2e.test.mjs @@ -0,0 +1,358 @@ +// Real installations, from a candidate release, against real containers. +// +// A candidate release is built from the working tree: a git repository with +// real tags, so bootstrap.sh resolves, clones and delegates exactly as it +// would against the published repository. Nothing here reads the developer's +// own checkout state. +// +// These pull images and start containers. Set GD_TEST_INSTALL=1 to run them; +// they are skipped otherwise. The port-conflict and existing-proxy cases bind +// real host ports, including 80 and 443. +import { test, describe, before, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { existsSync, statSync, readFileSync, writeFileSync, mkdirSync, symlinkSync } from 'node:fs'; +import { execFileSync } from 'node:child_process'; +import { join } from 'node:path'; +import { delimiter } from 'node:path'; +import { + tempDir, cleanup, makeCandidateRelease, git, run, occupyPort, compose, + dockerAvailable, shOk, q, REPO_DIR, +} from './helpers.mjs'; + +const enabled = process.env.GD_TEST_INSTALL === '1' && dockerAvailable(); +const skip = enabled ? false : 'set GD_TEST_INSTALL=1 with a working Docker daemon'; + +// The candidate release is a prerelease, so the beta channel has to select it: +// that exercises the channel path rather than only an explicit --ref. +const CANDIDATE_TAG = 'v9.9.9-beta.1'; +const PROXY_CONTAINER = 'ghost-docker-test-proxy'; + +let dir; +let repo; + +/** Clone the candidate release into a directory, as bootstrap.sh would. */ +const clone = (name) => { + const target = join(dir, name); + execFileSync('git', ['clone', '--quiet', '--depth', '1', '--branch', CANDIDATE_TAG, repo, target]); + return target; +}; + +const install = (site, args, options = {}) => + run(join(site, 'install.sh'), args, { cwd: site, timeout: 900_000, ...options }); + +const env = (site, key) => shOk(`env_get ${q(join(site, '.env'))} ${q(key)}`).trim(); +const meta = (site) => JSON.parse(readFileSync(join(site, '.ghost-docker.json'), 'utf8')); + +/** Ghost's Admin API through a host port, with an explicit Host header. */ +const adminSite = async (port, host = 'localhost') => { + const response = await fetch(`http://127.0.0.1:${port}/ghost/api/admin/site/`, { + headers: { Host: host }, + redirect: 'manual', + }); + return response; +}; + +const down = (site) => { + if (existsSync(join(site, '.env'))) compose(site, ['down', '-v', '--remove-orphans']); +}; + +/** + * Caddy issues from its own internal CA rather than attempting a real ACME + * order for a name that does not resolve. caddy/global/ is operator owned and + * the installer never overwrites it, so this survives installation. + */ +const useInternalCerts = (site) => { + mkdirSync(join(site, 'caddy', 'global'), { recursive: true }); + writeFileSync(join(site, 'caddy', 'global', 'tls.caddy'), 'local_certs\n'); +}; + +describe('installing from a candidate release', { skip, concurrency: 1 }, () => { + before(() => { + dir = tempDir('install-e2e'); + ({ repo } = makeCandidateRelease(dir, ['v1.0.0', CANDIDATE_TAG])); + }); + after(() => cleanup(dir)); + + describe('a local site, through the bootstrap', () => { + let site; + before(() => { + site = join(dir, 'local-a'); + }); + after(() => down(site)); + + test('bootstrap resolves the release, clones it and runs its installer', () => { + const result = run(join(REPO_DIR, 'bootstrap.sh'), + ['--channel', 'beta', '--dir', site, '--local', '--no-prompt'], + { env: { GD_BOOTSTRAP_REPO: repo }, timeout: 900_000 }); + assert.equal(result.status, 0, result.output); + assert.match(result.stdout, new RegExp(CANDIDATE_TAG.replace(/\./g, '\\.'))); + assert.match(result.stdout, /Ghost is installed/); + }); + + test('configuration and metadata are written privately', () => { + for (const file of ['.env', 'ghost.env', '.ghost-docker.json']) { + const path = join(site, file); + assert.ok(existsSync(path), `${file} was not written`); + assert.equal(statSync(path).mode & 0o777, 0o600, `${file} is not private`); + } + }); + + test('the metadata records the release, channel and resolved image', () => { + const recorded = meta(site); + assert.equal(recorded.schemaVersion, 1); + assert.equal(recorded.mode, 'local'); + // A prerelease tag implies the beta channel whether or not it was named. + assert.equal(recorded.channel, 'beta'); + assert.equal(recorded.stack.ref, CANDIDATE_TAG); + assert.match(recorded.stack.commit, /^[0-9a-f]{40}$/); + assert.equal(recorded.site.dir, site); + assert.match(recorded.ghost.version, /^\d+\.\d+\.\d+$/); + assert.match(recorded.ghost.digest, /^sha256:[0-9a-f]{64}$/, 'no digest recorded for recovery'); + assert.deepEqual(recorded.profiles, ['local']); + }); + + // A moving tag would let the site change Ghost version under the operator + // on the next `docker compose pull`. The pin has to be exact. + test('the Ghost version is pinned exactly, and the paths come from the image', () => { + assert.match(env(site, 'GHOST_VERSION'), /^\d+\.\d+\.\d+(-.+)?$/); + assert.equal(env(site, 'GHOST_VERSION').startsWith(meta(site).ghost.version), true); + const content = env(site, 'GHOST_CONTENT_PATH'); + assert.ok(content.endsWith('/content'), content); + assert.match(env(site, 'GHOST_TINYBIRD_PATH'), /\/core\/server\/data\/tinybird$/); + }); + + test('generated credentials are unique to the site and are not the example ones', () => { + const password = env(site, 'DATABASE_PASSWORD'); + const root = env(site, 'DATABASE_ROOT_PASSWORD'); + assert.ok(password.length >= 32, 'the application password is short'); + assert.notEqual(password, root); + assert.ok(!password.includes('change-me'), 'the example password survived'); + assert.ok(!root.includes('change-me'), 'the example root password survived'); + }); + + test('Ghost answers through the ingress the site actually uses', async () => { + const response = await adminSite(env(site, 'GHOST_PORT')); + assert.equal(response.status, 200); + assert.ok((await response.json()).site, 'no site payload'); + }); + + test('the site is published on the loopback interface only', () => { + const config = JSON.parse(compose(site, ['ps', '--format', 'json']).stdout.trim().split('\n')[0] ?? '{}'); + assert.ok(config, 'no containers'); + const bindings = execFileSync('docker', ['inspect', '-f', '{{json .HostConfig.PortBindings}}', + compose(site, ['ps', '-q', 'ghost']).stdout.trim()], { encoding: 'utf8' }); + const hosts = Object.values(JSON.parse(bindings)).flat().map((b) => b.HostIp); + assert.deepEqual(hosts, ['127.0.0.1']); + }); + + test('site.sh check passes, and site.sh list finds the site', () => { + const check = run(join(site, 'scripts', 'site.sh'), ['check', site], { timeout: 300_000 }); + assert.equal(check.status, 0, check.output); + const list = run(join(site, 'scripts', 'site.sh'), ['list'], { timeout: 120_000 }); + assert.equal(list.status, 0, list.output); + assert.match(list.stdout, new RegExp(env(site, 'COMPOSE_PROJECT_NAME'))); + }); + }); + + describe('a second local site alongside the first', () => { + let first; + let second; + before(() => { + first = join(dir, 'local-a'); + second = clone('local-b'); + }); + after(() => down(second)); + + test('installs without disturbing the first', () => { + const result = install(second, ['--local', '--no-prompt']); + assert.equal(result.status, 0, result.output); + }); + + // Two sites on one host share a Docker daemon and the loopback interface, + // so their identities and ports must differ. The project name is also the + // suffix of every service network alias. + test('the two sites have distinct identities and ports', () => { + assert.notEqual(env(first, 'COMPOSE_PROJECT_NAME'), env(second, 'COMPOSE_PROJECT_NAME')); + assert.equal(env(first, 'COMPOSE_PROJECT_NAME'), 'ghost-local-local-a'); + assert.equal(env(second, 'COMPOSE_PROJECT_NAME'), 'ghost-local-local-b'); + assert.notEqual(env(first, 'GHOST_PORT'), env(second, 'GHOST_PORT')); + assert.notEqual(env(first, 'DATABASE_PASSWORD'), env(second, 'DATABASE_PASSWORD')); + }); + + test('both sites answer at the same time', async () => { + for (const site of [first, second]) { + const response = await adminSite(env(site, 'GHOST_PORT')); + assert.equal(response.status, 200, `${site} did not answer`); + } + }); + }); + + describe('an explicitly requested port that is in use', () => { + let site; + let release; + before(async () => { + site = clone('port-conflict'); + release = await occupyPort(24771); + }); + after(async () => { + if (release) await release(); + }); + + // A port chosen by the installer moves out of the way; a port the operator + // asked for does not. Silently using a different one would produce a site + // on an address nothing else is configured for. + test('fails, names what holds the port, and writes nothing', () => { + const result = install(site, ['--local', '--no-prompt', '--port', '24771']); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /port 24771/); + assert.match(result.stderr, /already in use/); + assert.match(result.stderr, /Nothing has been changed/); + assert.ok(!existsSync(join(site, '.env')), 'configuration was written anyway'); + assert.ok(!existsSync(join(site, '.ghost-docker.json')), 'metadata was written anyway'); + }); + }); + + describe('a server that already runs a proxy on 80 and 443', () => { + let site; + before(() => { + site = clone('existing-proxy'); + const image = readFileSync(join(REPO_DIR, 'compose.yml'), 'utf8') + .match(/image: (caddy:[^\s@]+@sha256:[0-9a-f]+)/)[1]; + execFileSync('docker', ['rm', '-f', PROXY_CONTAINER], { stdio: 'ignore' }); + execFileSync('docker', [ + 'run', '-d', '--name', PROXY_CONTAINER, '-p', '80:80', '-p', '443:443', + image, 'caddy', 'respond', '--listen', ':80', 'the operator\'s own proxy', + ], { stdio: 'ignore' }); + }); + after(() => { + execFileSync('docker', ['rm', '-f', PROXY_CONTAINER], { stdio: 'ignore' }); + }); + + // A server may proxy other applications. Taking its ports, or stopping it, + // is not the installer's call to make. + test('installation fails on the port conflict and leaves the proxy running', () => { + const result = install(site, ['--domain', 'ghost.test', '--no-prompt']); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /port 80/); + assert.match(result.stderr, new RegExp(PROXY_CONTAINER)); + assert.match(result.stderr, /Nothing was stopped/); + + const running = execFileSync('docker', ['inspect', '-f', '{{.State.Running}}', PROXY_CONTAINER], + { encoding: 'utf8' }).trim(); + assert.equal(running, 'true', 'the operator\'s proxy was stopped'); + assert.ok(!existsSync(join(site, '.env')), 'configuration was written anyway'); + }); + }); + + describe('a production site', () => { + let site; + before(() => { + site = clone('production'); + useInternalCerts(site); + }); + after(() => down(site)); + + test('installs, renders routes, and verifies its own ingress', () => { + const result = install(site, ['--domain', 'ghost.test', '--no-prompt']); + assert.equal(result.status, 0, result.output); + assert.match(result.stdout, /Caddy routes ghost\.test/); + assert.match(result.stdout, /redirects http:\/\/ghost\.test to HTTPS/); + }); + + test('the mode, URL and restart policy match production', () => { + assert.equal(env(site, 'COMPOSE_PROFILES'), 'production'); + assert.equal(env(site, 'SITE_MODE'), 'production'); + assert.equal(env(site, 'NODE_ENV'), 'production'); + assert.equal(env(site, 'URL'), 'https://ghost.test'); + assert.equal(env(site, 'DOMAIN'), 'ghost.test'); + assert.equal(env(site, 'RESTART_POLICY'), 'unless-stopped'); + assert.equal(meta(site).mode, 'production'); + }); + + test('the generated routes are installed, and the operator files are untouched', () => { + const generated = readFileSync(join(site, 'caddy', 'sites', 'site.caddy'), 'utf8'); + assert.match(generated, /^ghost\.test \{/m); + assert.match(generated, new RegExp(`reverse_proxy ghost-${env(site, 'COMPOSE_PROJECT_NAME')}:2368`)); + assert.equal(readFileSync(join(site, 'caddy', 'global', 'tls.caddy'), 'utf8'), 'local_certs\n'); + }); + + test('HTTP on the ingress port redirects to HTTPS for the site domain', async () => { + const response = await fetch(`http://127.0.0.1:${env(site, 'HTTP_PORT')}/`, { + headers: { Host: 'ghost.test' }, + redirect: 'manual', + }); + assert.ok(response.status >= 300 && response.status < 400, `got ${response.status}`); + assert.match(response.headers.get('location') ?? '', /^https:\/\/ghost\.test/); + }); + + // The whole intended ingress, end to end: TLS terminated by Caddy for this + // name, proxied to Ghost, Admin API answering. The installer reports this + // as a warning rather than a failure, because a real production install may + // legitimately run before DNS is pointed and no certificate can exist yet; + // here the internal CA removes that variable, so it must actually work. + test('Ghost Admin answers over HTTPS through Caddy', () => { + const status = run('bash', ['-c', + `. ${JSON.stringify(join(site, 'scripts', 'lib', 'common.sh'))}\n` + + `install_https_status ${JSON.stringify(site)} ghost.test`], { timeout: 180_000 }); + assert.equal(status.stdout.trim(), '200', status.output); + }); + }); + + describe('--no-start', () => { + let site; + before(() => { + site = clone('no-start'); + }); + after(() => down(site)); + + test('configures the site and starts no application services', () => { + const result = install(site, ['--local', '--no-prompt', '--no-start']); + assert.equal(result.status, 0, result.output); + assert.ok(existsSync(join(site, '.env'))); + assert.ok(existsSync(join(site, '.ghost-docker.json'))); + assert.match(result.stdout, /Nothing is running/); + assert.equal(compose(site, ['ps', '-a', '-q']).stdout.trim(), '', 'containers were created'); + }); + + test('the configuration it wrote is valid and startable', () => { + const result = compose(site, ['config', '--quiet']); + assert.equal(result.status, 0, result.stderr); + }); + }); + + // The installer must not acquire a dependency an operator does not have. + // The allowlist is the tool contract recorded in scripts/lib/preflight.sh, + // so adding a utility to a code path without recording it fails here. + describe('the declared minimum host tools', () => { + let site; + before(() => { + site = clone('minimum-tools'); + }); + after(() => down(site)); + + test('an install completes with only the recorded utilities on PATH', () => { + const declared = [ + ...shOk('printf "%s\\n" "${GD_REQUIRED_COMMANDS[@]}"').trim().split('\n'), + ...shOk('printf "%s\\n" "${GD_HOST_UTILITIES[@]}"').trim().split('\n'), + ]; + const bin = join(dir, 'minimum-bin'); + mkdirSync(bin, { recursive: true }); + const missing = []; + for (const tool of declared) { + const found = run('sh', ['-c', `command -v ${tool}`]).stdout.trim(); + if (!found) { + missing.push(tool); + continue; + } + symlinkSync(found, join(bin, tool)); + } + assert.deepEqual(missing, [], 'a declared utility is not installed on this host'); + + const result = install(site, ['--local', '--no-prompt', '--no-start'], { + env: { PATH: [bin, '/nonexistent'].join(delimiter) }, + }); + assert.equal(result.status, 0, result.output); + assert.ok(existsSync(join(site, '.ghost-docker.json'))); + }); + }); +}); diff --git a/tests/install.test.mjs b/tests/install.test.mjs new file mode 100644 index 00000000..1b1e28d4 --- /dev/null +++ b/tests/install.test.mjs @@ -0,0 +1,319 @@ +// The installer's decisions, and the release selection in front of it. +// +// Everything here runs without starting a container; tests/install-e2e.test.mjs +// installs real sites. The split matters: these are the checks that must fail +// *before* anything on the host is touched, so they must not need a daemon to +// be exercised. +import { test, describe, before, after, beforeEach, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { existsSync, writeFileSync, mkdirSync, readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { + tempDir, cleanup, copyWorktree, makeCandidateRelease, git, run, occupyPort, + sh, shOk, shSucceeds, q, REPO_DIR, +} from './helpers.mjs'; + +// A checkout to run install.sh from. Copied rather than used in place so a +// failing test cannot write a .env into the developer's working tree. +const checkout = (dir, name = 'site') => copyWorktree(join(dir, name)); + +/** install.sh from a scratch checkout. Never starts anything. */ +const install = (site, args, options = {}) => + run(join(site, 'install.sh'), args, { cwd: site, timeout: 120_000, ...options }); + +describe('install.sh options', () => { + let dir; + let site; + beforeEach(() => { + dir = tempDir('install-flags'); + site = checkout(dir); + }); + afterEach(() => cleanup(dir)); + + test('--help prints the interface and exits zero', () => { + const result = install(site, ['--help']); + assert.equal(result.status, 0); + assert.match(result.stdout, /--local/); + assert.match(result.stdout, /--no-prompt/); + }); + + test('an unknown option fails as a usage error', () => { + const result = install(site, ['--local', '--nonsense']); + assert.equal(result.status, 2); + assert.match(result.stderr, /unknown option: --nonsense/); + }); + + // Steps that have not landed must say so. A script written against the + // documented interface gets an answer it can act on, and nothing advertises + // support that does not exist. + test('--import fails as unimplemented, naming the step and the path that works today', () => { + const result = install(site, ['--local', '--import', '/tmp/bundle.tar.gz']); + assert.equal(result.status, 3); + assert.match(result.stderr, /--import is not implemented yet/); + assert.match(result.stderr, /S5/); + assert.match(result.stderr, /migrate\.sh/); + }); + + test('--with supervisor fails as unimplemented rather than enabling an empty profile', () => { + const result = install(site, ['--local', '--with', 'supervisor']); + assert.equal(result.status, 3); + assert.match(result.stderr, /supervisor is not implemented yet/); + assert.match(result.stderr, /S8/); + assert.ok(!existsSync(join(site, '.env')), 'wrote configuration anyway'); + }); + + test('the final-phase flags are refused as unimplemented, not as unknown', () => { + for (const flag of [['--image-registry', 'ghcr'], ['--ghost-channel', 'nightly'], ['--without', 'redis']]) { + const result = install(site, ['--local', ...flag]); + assert.equal(result.status, 3, flag[0]); + assert.match(result.stderr, /not implemented yet/); + } + }); + + test('--no-prompt with no mode names the flags that supply one', () => { + const result = install(site, ['--no-prompt']); + assert.equal(result.status, 2); + assert.match(result.stderr, /--local or --domain/); + }); + + test('an unknown optional service is rejected', () => { + const result = install(site, ['--local', '--with', 'redis']); + assert.equal(result.status, 2); + assert.match(result.stderr, /unknown optional service: redis/); + }); + + test('--admin-domain is production only', () => { + const result = install(site, ['--local', '--admin-domain', 'admin.example.com']); + assert.equal(result.status, 2); + assert.match(result.stderr, /production sites only/); + }); + + test('--domain takes a hostname, not a URL', () => { + const result = install(site, ['--domain', 'https://example.com', '--no-prompt']); + assert.equal(result.status, 2); + assert.match(result.stderr, /must be a hostname/); + }); + + test('--port must be a port number', () => { + const result = install(site, ['--local', '--no-prompt', '--port', '99999']); + assert.equal(result.status, 2); + assert.match(result.stderr, /--port must be a port number/); + }); + + test('--dir elsewhere points at the bootstrap rather than installing the wrong tree', () => { + const elsewhere = join(dir, 'elsewhere'); + mkdirSync(elsewhere); + const result = install(site, ['--local', '--no-prompt', '--dir', elsewhere]); + assert.equal(result.status, 2); + assert.match(result.stderr, /bootstrap\.sh/); + assert.ok(!existsSync(join(elsewhere, '.env')), 'wrote into the other directory'); + }); + + test('an existing site is never installed over', () => { + writeFileSync(join(site, '.env'), 'COMPOSE_PROFILES="local"\n', { mode: 0o600 }); + const result = install(site, ['--local', '--no-prompt']); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /already holds a site/); + assert.equal(readFileSync(join(site, '.env'), 'utf8'), 'COMPOSE_PROFILES="local"\n'); + }); + + test('--ref that disagrees with the checkout is refused', () => { + git(site, ['init', '-q', '-b', 'main']); + git(site, ['add', '-A']); + git(site, ['commit', '-q', '-m', 'candidate']); + git(site, ['tag', 'v1.0.0']); + const result = install(site, ['--local', '--no-prompt', '--ref', 'v2.0.0']); + assert.equal(result.status, 2); + assert.match(result.stderr, /this checkout is at v1\.0\.0/); + }); + + // A prompt has to reach a terminal, and a required answer must never have a + // silent default. Without a tty, --no-prompt is the only behaviour available + // and the installer must fail rather than block. + test('with no terminal a required input fails instead of hanging', () => { + const result = install(site, [], { input: '', timeout: 30_000 }); + assert.equal(result.status, 2, result.output); + assert.match(result.stderr, /--local or --domain/); + }); +}); + +describe('port selection', () => { + test('an occupied port is detected, and a free one is chosen above it', async () => { + const release = await occupyPort(24681); + try { + assert.ok(shSucceeds('port_in_use 24681'), 'an occupied port was reported free'); + assert.ok(!shSucceeds('port_in_use 24682'), 'a free port was reported busy'); + assert.equal(shOk('free_port 24681').trim(), '24682'); + } finally { + await release(); + } + }); + + test('an occupied port is an error naming what holds it, never something to stop', async () => { + const release = await occupyPort(24683); + try { + const record = shOk('preflight_port 24683 "Ghost"'); + assert.match(record, /^error\t/); + assert.match(record, /already in use/); + assert.match(record, /Nothing was stopped/); + } finally { + await release(); + } + }); +}); + +describe('preflight', () => { + test('records are STATUS, LABEL, DETAIL, and only errors fail a run', () => { + const records = shOk('preflight_os; preflight_commands docker jq'); + for (const line of records.trim().split('\n')) { + assert.equal(line.split('\t').length, 3, line); + } + assert.ok(!shSucceeds(`preflight_failed ${q(records)}`), 'ok records failed the run'); + assert.ok(shSucceeds(`preflight_failed ${q('error\tx\ty')}`), 'an error record passed'); + assert.ok(!shSucceeds(`preflight_failed ${q('warn\tx\ty')}`), 'a warning failed the run'); + }); + + test('a missing tool is reported by name', () => { + const record = shOk('preflight_commands definitely-not-a-real-command'); + assert.match(record, /^error\trequired tools\tmissing: definitely-not-a-real-command/); + }); + + // Daemon access is established by asking the daemon. Group membership is + // neither necessary (rootless, a remote DOCKER_HOST) nor sufficient (a + // stopped daemon), so nothing here may look at it. + test('an unreachable daemon is an error with what to do about it', () => { + const record = shOk('preflight_docker', { env: { DOCKER_HOST: 'tcp://127.0.0.1:1' } }); + assert.match(record, /^error\tdocker daemon\tnot reachable/); + assert.match(record, /start Docker|systemctl start docker|Docker Desktop/); + }); + + test('nothing decides access from docker-group membership', () => { + const sources = ['preflight.sh', 'install.sh', 'common.sh'] + .map((f) => readFileSync(join(REPO_DIR, 'scripts', 'lib', f), 'utf8')) + .concat(readFileSync(join(REPO_DIR, 'install.sh'), 'utf8')) + .concat(readFileSync(join(REPO_DIR, 'bootstrap.sh'), 'utf8')) + .join('\n') + // The rule is stated in a comment in each file; only code is at issue. + .split('\n') + .filter((line) => !line.trim().startsWith('#')) + .join('\n'); + assert.ok(!/\bgroups\b|\bid -nG\b|getent group/.test(sources), 'a group check crept in'); + }); + + test('version comparison is numeric, not lexical', () => { + assert.equal(shOk('version_compare 2.24.0 2.9.0').trim(), '1'); + assert.equal(shOk('version_compare 2.9.0 2.24.0').trim(), '-1'); + assert.equal(shOk('version_compare 25.0.0 25.0.0').trim(), '0'); + // A build suffix does not make an engine older than its own version. + assert.ok(shSucceeds('version_at_least 25.0.0+ce 25.0.0')); + assert.ok(shSucceeds('version_at_least 2.24.0 2.24.0')); + assert.ok(!shSucceeds('version_at_least 2.23.9 2.24.0')); + }); + + test('the declared minimums are the ones the checks enforce', () => { + assert.equal(shOk('printf %s "$GD_MIN_COMPOSE_VERSION"'), '2.24.0'); + assert.equal(shOk('printf %s "$GD_MIN_DOCKER_VERSION"'), '25.0.0'); + }); +}); + +describe('site identity and secrets', () => { + test('a production project name is derived from the domain', () => { + assert.equal(shOk('install_project_name production Example.COM /tmp/x').trim(), 'ghost-example-com'); + assert.equal(shOk('install_project_name production blog.example.com /tmp/x').trim(), 'ghost-blog-example-com'); + }); + + // Two local sites on one host share no state, so their identities must + // differ; the alias every generated route uses is suffixed with this name. + test('two local sites in different directories get different identities', () => { + const a = shOk('install_project_name local "" /srv/site-a').trim(); + const b = shOk('install_project_name local "" /srv/site-b').trim(); + assert.equal(a, 'ghost-local-site-a'); + assert.equal(b, 'ghost-local-site-b'); + assert.notEqual(a, b); + }); + + test('generated secrets are long, random, and free of dotenv metacharacters', () => { + const first = shOk('install_secret 24').trim(); + const second = shOk('install_secret 24').trim(); + assert.equal(first.length, 48); + assert.notEqual(first, second); + assert.match(first, /^[0-9a-f]+$/); + }); +}); + +describe('Ghost version resolution', () => { + test('a bare version selects the default variant; a tag is taken as given', () => { + assert.equal(shOk('install_ghost_tag 6.3.1').trim(), '6.3.1-next-alpine'); + assert.equal(shOk('install_ghost_tag v6.3.1').trim(), '6.3.1-next-alpine'); + assert.equal(shOk('install_ghost_tag ""').trim(), '6-next-alpine'); + assert.equal(shOk('install_ghost_tag 6-alpine').trim(), '6-alpine'); + assert.equal(shOk('install_ghost_tag 6.3.1-alpine').trim(), '6.3.1-alpine'); + }); + + test('the variant is separated from the version so an exact pin can be built', () => { + assert.equal(shOk('_gd_tag_variant 6-next-alpine').trim(), 'next-alpine'); + assert.equal(shOk('_gd_tag_variant 6.3.1-alpine').trim(), 'alpine'); + assert.equal(shOk('_gd_tag_variant next-alpine').trim(), 'next-alpine'); + assert.equal(shOk('_gd_tag_variant 6').trim(), ''); + }); +}); + +describe('release selection', () => { + let dir; + let repo; + + // bootstrap.sh is sourced in library mode so that ordering can be tested + // without cloning anything. + const bootstrap = (script, env = {}) => + run('bash', ['-c', `GD_BOOTSTRAP_SOURCED=1 . ${JSON.stringify(join(REPO_DIR, 'bootstrap.sh'))}\n${script}`], { env }); + + before(() => { + dir = tempDir('release'); + ({ repo } = makeCandidateRelease(dir, [ + 'v1.9.0', 'v1.10.0', 'v1.11.0-beta.2', 'v1.11.0-beta.10', 'v0.1.0', 'not-a-release', + ])); + }); + after(() => cleanup(dir)); + + test('semver ordering, including prereleases', () => { + const cmp = (a, b) => bootstrap(`_semver_cmp ${a} ${b}`).stdout.trim(); + assert.equal(cmp('v1.10.0', 'v1.9.0'), '1', '1.10.0 must be newer than 1.9.0'); + assert.equal(cmp('v1.2.0-beta.1', 'v1.2.0'), '-1', 'a prerelease precedes its release'); + assert.equal(cmp('v1.2.0-beta.10', 'v1.2.0-beta.2'), '1', 'beta.10 must be newer than beta.2'); + assert.equal(cmp('v1.2.3', 'v1.2.3'), '0'); + }); + + test('stable selects the newest release and ignores prereleases', () => { + const result = bootstrap('_latest_release stable', { GD_BOOTSTRAP_REPO: repo }); + assert.equal(result.stdout.trim(), 'v1.10.0'); + }); + + test('beta also considers prereleases, in semver order', () => { + const result = bootstrap('_latest_release beta', { GD_BOOTSTRAP_REPO: repo }); + assert.equal(result.stdout.trim(), 'v1.11.0-beta.10'); + }); + + test('a repository with no releases fails rather than guessing', () => { + const empty = join(dir, 'empty'); + copyWorktree(empty); + git(empty, ['init', '-q', '-b', 'main']); + git(empty, ['add', '-A']); + git(empty, ['commit', '-q', '-m', 'no tags']); + const result = bootstrap('_latest_release stable', { GD_BOOTSTRAP_REPO: empty }); + assert.notEqual(result.status, 0); + }); + + test('bootstrap.sh refuses a non-empty target directory', () => { + const occupied = join(dir, 'occupied'); + mkdirSync(occupied, { recursive: true }); + writeFileSync(join(occupied, 'something'), 'x'); + const result = run(join(REPO_DIR, 'bootstrap.sh'), ['--dir', occupied, '--ref', 'v1.10.0', '--local'], { + env: { GD_BOOTSTRAP_REPO: repo }, + timeout: 60_000, + }); + assert.notEqual(result.status, 0); + // Checked before the daemon probe: a wrong directory should be reported + // straight away, not after waiting on Docker. + assert.match(result.stderr, /is not empty/); + }); +}); diff --git a/tests/meta.test.mjs b/tests/meta.test.mjs new file mode 100644 index 00000000..b634e153 --- /dev/null +++ b/tests/meta.test.mjs @@ -0,0 +1,111 @@ +// `.ghost-docker.json`: the installation metadata reader and writer. +// +// The schema is specified in section 2.2 of docs/ghost-cli-replacement.md. +// install.sh is its first writer, which is why it lands here rather than with +// the S1 helpers. +import { test, describe, beforeEach, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync, writeFileSync, statSync } from 'node:fs'; +import { join } from 'node:path'; +import { tempDir, cleanup, sh, shOk, shSucceeds, q } from './helpers.mjs'; + +let dir; +const file = () => join(dir, '.ghost-docker.json'); +const read = () => JSON.parse(readFileSync(file(), 'utf8')); + +const init = (extra = '') => + shOk(`meta_init ${q(dir)} mode=production channel=stable \\ + stack.version=v1.2.3 stack.commit=abc123 stack.ref=v1.2.3 \\ + site.project=ghost-example-com site.dir=${q(dir)} site.url=https://example.com \\ + site.domain=example.com \\ + ghost.image=ghost ghost.tag=6.62.0-next-alpine ghost.version=6.62.0 \\ + ghost.digest=sha256:deadbeef profiles=production,analytics ${extra}`); + +describe('installation metadata', () => { + beforeEach(() => { + dir = tempDir('meta'); + }); + afterEach(() => cleanup(dir)); + + test('a site with no metadata is unknown, not broken', () => { + assert.ok(!shSucceeds(`meta_present ${q(dir)}`), 'reported present'); + // The pre-metadata case has to describe itself rather than fail: an + // installation that predates the file is a supported state. + const described = shOk(`meta_describe ${q(dir)}`); + assert.match(described, /installed before metadata was recorded/); + assert.ok(!shSucceeds(`meta_get ${q(dir)} .mode`), 'read a value from nothing'); + // And an absent file is not a schema error. + assert.ok(shSucceeds(`meta_check_schema ${q(dir)}`), 'absent file failed the schema check'); + }); + + test('records the schema, identity, provenance and resolved image', () => { + init(); + const meta = read(); + assert.equal(meta.schemaVersion, 1); + assert.equal(meta.mode, 'production'); + assert.equal(meta.channel, 'stable'); + assert.deepEqual(meta.stack, { version: 'v1.2.3', commit: 'abc123', ref: 'v1.2.3' }); + assert.equal(meta.site.project, 'ghost-example-com'); + assert.equal(meta.site.url, 'https://example.com'); + assert.equal(meta.ghost.tag, '6.62.0-next-alpine'); + assert.equal(meta.ghost.version, '6.62.0'); + assert.equal(meta.ghost.digest, 'sha256:deadbeef'); + assert.deepEqual(meta.profiles, ['production', 'analytics']); + assert.deepEqual(meta.migrations, []); + assert.match(meta.installedAt, /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$/); + // Unsupplied fields are recorded as null rather than as an empty string, so + // "not known" and "deliberately empty" stay distinguishable. + assert.equal(meta.site.adminDomain, null); + }); + + test('the file is private: it names a site and its provenance', () => { + init(); + assert.equal(statSync(file()).mode & 0o777, 0o600); + }); + + test('an unknown key is an error, not a field nothing reads', () => { + const result = sh(`meta_init ${q(dir)} mode=local nonsense=1`); + assert.equal(result.status, 2); + assert.match(result.stderr.toString(), /unknown metadata key nonsense/); + }); + + test('migrations are recorded once and are queryable', () => { + init(); + shOk(`meta_record_migration ${q(dir)} 0001-compose-profiles`); + shOk(`meta_record_migration ${q(dir)} 0001-compose-profiles`); + assert.deepEqual(read().migrations, ['0001-compose-profiles']); + assert.ok(shSucceeds(`meta_has_migration ${q(dir)} 0001-compose-profiles`)); + assert.ok(!shSucceeds(`meta_has_migration ${q(dir)} 0002-nothing`)); + }); + + test('invalid JSON is never installed over a good file', () => { + init(); + const before = readFileSync(file(), 'utf8'); + const result = sh(`printf 'not json' | meta_write ${q(dir)}`); + assert.notEqual(result.status, 0); + assert.equal(readFileSync(file(), 'utf8'), before); + }); + + test('a document without the schema version is refused', () => { + const result = sh(`printf '{"mode":"local"}' | meta_write ${q(dir)}`); + assert.notEqual(result.status, 0); + assert.match(result.stderr.toString(), /schemaVersion/); + }); + + test('a newer schema is refused rather than misread', () => { + writeFileSync(file(), JSON.stringify({ schemaVersion: 99, mode: 'local' })); + const result = sh(`meta_check_schema ${q(dir)}`); + assert.equal(result.status, 1); + assert.match(result.stderr.toString(), /schema version 99.*understands version 1/s); + // And an update refuses too, rather than rewriting it at the old schema. + assert.notEqual(sh(`meta_record_migration ${q(dir)} x`).status, 0); + }); + + test('describes a recorded installation', () => { + init(); + const described = shOk(`meta_describe ${q(dir)}`); + assert.match(described, /mode: *production/); + assert.match(described, /ghost: *6\.62\.0-next-alpine sha256:deadbeef/); + assert.match(described, /profiles: *production,analytics/); + }); +}); From e0167409da8405ae78200cde18da8e7ae212364c Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Thu, 3 Sep 2026 10:21:56 -0400 Subject: [PATCH 2/5] fix(install): set -e-safe env writes, production ingress verify, ref reconciliation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Surfaced by the S2 install e2e suite (now 21/21 against real containers). - env.sh/compose.sh: `((n++))`/`((count++))` return exit 1 on the first iteration (post-increment from 0), aborting under `set -e`. install.sh is the first `set -e` caller to write a multi-line env file, so this latent S1 bug only surfaced now. Use `n=$((n + 1))`. Without this, every install died at "Writing configuration". - install_verify_ingress: a plain http probe straight to the loopback Ghost is 301-redirected to HTTPS by design in production (canonical URL is https), so the direct-loopback check now accepts a 3xx there and sends the site's Host header. Caddy's own checks remain the authoritative production verification. Without this, every production install failed its final verification. - install.sh: a release and its prerelease can share a commit, so `git describe --exact-match` can report a different tag than the one bootstrap pinned. Accept --ref when it is among `git tag --points-at HEAD`. - preflight.sh: record `sysctl` in the host-utility contract (macOS memory probe). Test harness: - Central teardown so two local sites stay running together for the simultaneous-answer and distinct-port assertions; no site is downed while a sibling suite still asserts. - makeCandidateRelease tags each release on its own commit and exposes a file:// URL, so shallow clone is honoured without the local-clone warning and `git describe` is unambiguous. - HTTP-redirect check drives the installer's /dev/tcp helper, since Node fetch forbids overriding Host and Caddy would otherwise redirect to 127.0.0.1. - minimum-tools wraps docker/jq so their own subprocess needs run with the full PATH while the shell's coreutil lookups stay restricted to the contract. - realpathSync for the recorded site.dir (macOS /var -> /private/var). docs/ghost-cli-replacement.md: amend §2.10 with when the manager image lands (S4, first tenant backup/restore) and why env/caddy/preflight stay host shell; note the deferred operation lock and the manager-image timing in the S4 step. Co-Authored-By: Claude Opus 4.8 --- docs/ghost-cli-replacement.md | 61 ++++++++++++++++++++++++ install.sh | 14 +++++- scripts/lib/compose.sh | 2 +- scripts/lib/env.sh | 4 +- scripts/lib/install.sh | 12 ++++- scripts/lib/preflight.sh | 2 +- tests/helpers.mjs | 15 ++++-- tests/install-e2e.test.mjs | 90 ++++++++++++++++++++++------------- tests/install.test.mjs | 5 +- 9 files changed, 159 insertions(+), 46 deletions(-) diff --git a/docs/ghost-cli-replacement.md b/docs/ghost-cli-replacement.md index bf4a2ded..9d68231b 100644 --- a/docs/ghost-cli-replacement.md +++ b/docs/ghost-cli-replacement.md @@ -725,6 +725,57 @@ whether `install.sh` is the installer or a bootstrap that runs one. Steps already shipped in host shell (S1's `config.sh` and `caddy.sh`) stay where they are; the boundary does not run through them. +#### When the manager image lands, and what its first tenant is + +Revised 2026-09-03, after S2. Two questions kept coming up — *should the image +ship sooner?* and *should env validation and Caddy generation move into it?* — +so the boundary above is stated as two rules rather than left to inference. + +**The image's first tenant is the first stateful operation, at S4, not S8.** +§2.6 already publishes a privileged supervisor image from this repo and requires +it to "follow §2.5 rather than inventing a second upgrade/recovery algorithm." +If S4–S7 implement backup, restore, upgrade and recovery in host shell and S8 +then re-implements them in an image, that algorithm exists twice, and the second +copy is the one under the privileged supervisor. So the manager image is +introduced in **S4**, with backup/restore as its first entrypoint; S5 (import), +S7 (upgrade) and S8 (supervisor) are further entrypoints on the same image, not +parallel codebases. This is a scheduling clarification, not a new component: +§2.10 already defined the image and the dispatcher. It does **not** move S4's +deliverable — S4 still ships backup/restore — it fixes the language they are +written in so they are not rewritten at S8. The host dispatcher (this section's +mount, identity and exit-code rules) is written in S4 alongside its first +`docker run` target. + +**Env validation, Caddy generation, preflight and version resolution do not +move into the image.** They are not stateful operations; they are the pure logic +that runs *before and without* a working container, and the boundary is set by +exactly that. Concretely: + +- The bootstrap writes `.env` before any image exists. A container cannot be the + only writer of the file that configures the container. +- `config.sh validate` runs inside preflight, before anything starts, and inside + `site.sh check` on a host whose daemon is missing or wedged. Behind `docker + run` it could not validate offline — the case it exists for — and would add an + image pull to every invocation. (S2 verified this matters: a wedged daemon + mid-session was still diagnosed by host-shell preflight because it does not + depend on the image.) +- Containerizing them does **not** solve the one hard problem, Compose's `$$` + interpolation: anything writing `.env` encodes a literal `$` as `$$` + regardless of implementation language, as recorded above. +- Moving them to JS forces a lose-lose: JS-in-container breaks offline validate + and still needs a host writer for the bootstrap; JS-on-host reintroduces the + host Node requirement S1 deliberately removed with `config-to-env.js`. Host + requirements stay `bash`, `docker`, `docker compose`, `jq`. + +So the dispatcher-plus-image model applies to the **stateful** commands (backup, +restore, import, upgrade, and the supervisor), which become thin host wrappers +around `docker run` of the manager image. Preflight, doctor, bootstrap, config +validation and Caddy rendering stay in host shell — not because bash is +preferable, but because they must run when there is no usable image. If shared +serialization between host and image is ever wanted, the host side stays the +jq-backed bash helpers; the image reimplements what it needs rather than the +host taking a language runtime. + ## 3. Implementation steps The numbering is revised from the original plan; use names as well as numbers when @@ -933,6 +984,16 @@ and journals. Backups include required local application/configuration state and describe optional-service limitations. Define supported backup formats independently of migration bundles; a portable export is not a lossless recovery checkpoint. +This is where the §2.10 manager image lands (see "When the manager image lands, +and what its first tenant is"). Backup/restore is its first entrypoint, so the +recovery algorithm S7 and S8 reuse exists once, in the image, not in host shell +awaiting a rewrite. Write the host dispatcher here — the `docker run` mount, +identity and exit-code rules from §2.10 — and route backup/restore through it. +The S2 host-shell helpers that must run without the image (preflight, config +validation, Caddy rendering, `site.sh check`) stay in host shell. Also add the +shared operation lock to installation/reconfigure retroactively: §2.2 requires +install to take it, and S2 deferred it to this step. + Acceptance: restore a representative site to a fresh destination and verify database, assets/theme/configuration; inject interrupted backup/restore, full disk, stale lock, and SQL pipeline failure. Recover without exposing an incomplete destination. diff --git a/install.sh b/install.sh index 9be0f868..3a638f17 100755 --- a/install.sh +++ b/install.sh @@ -220,15 +220,25 @@ esac stack_ref="" stack_commit="" +tags_here="" if command -v git >/dev/null 2>&1 && git -C "$dir" rev-parse --git-dir >/dev/null 2>&1; then stack_commit=$(git -C "$dir" rev-parse HEAD 2>/dev/null || printf '') + tags_here=$(git -C "$dir" tag --points-at HEAD 2>/dev/null || printf '') stack_ref=$(git -C "$dir" describe --tags --exact-match HEAD 2>/dev/null || git -C "$dir" rev-parse --abbrev-ref HEAD 2>/dev/null || printf '') fi -if [[ -n $ref && -n $stack_ref && $ref != "$stack_ref" ]]; then - die "--ref is $ref but this checkout is at $stack_ref. +# When --ref names a tag that points at this commit, it is authoritative even +# if `git describe` reported a different tag on the same commit — a release and +# its prerelease can share a commit. Only a --ref that is nowhere near this +# checkout is a real disagreement. +if [[ -n $ref ]]; then + if [[ -n $tags_here ]] && printf '%s\n' "$tags_here" | grep -qxF "$ref"; then + stack_ref=$ref + elif [[ -n $stack_ref && $ref != "$stack_ref" ]]; then + die "--ref is $ref but this checkout is at $stack_ref. install.sh installs the release it belongs to. Use bootstrap.sh --ref $ref to select a different one." "$EXIT_USAGE" + fi fi [[ -n $ref ]] || ref=$stack_ref diff --git a/scripts/lib/compose.sh b/scripts/lib/compose.sh index 7a6c2ae3..e4c3d094 100644 --- a/scripts/lib/compose.sh +++ b/scripts/lib/compose.sh @@ -75,7 +75,7 @@ compose_site_mode() { for mode in "${GD_SITE_MODES[@]}"; do if [[ $profile == "$mode" ]]; then found=$profile - ((count++)) + count=$((count + 1)) fi done done < <(_gd_split_profiles "$1") diff --git a/scripts/lib/env.sh b/scripts/lib/env.sh index 0a85afbd..c938d4e9 100644 --- a/scripts/lib/env.sh +++ b/scripts/lib/env.sh @@ -41,7 +41,7 @@ _gd_env_scan() { local trailing_comment='^(.*)[[:space:]]#' while IFS= read -r line || [[ -n $line ]]; do - ((n++)) + n=$((n + 1)) # Inside a multi-line value: skip it rather than mistaking one of its # lines for an assignment. @@ -188,7 +188,7 @@ _gd_env_write() { local file=$1 target=${2:-0} replacement=${3:-} mode=${4:-} line n=0 { while IFS= read -r line || [[ -n $line ]]; do - ((n++)) + n=$((n + 1)) if ((n == target)); then if [[ -n $replacement ]]; then printf '%s\n' "$replacement" diff --git a/scripts/lib/install.sh b/scripts/lib/install.sh index 0c030cbc..5ff5127f 100644 --- a/scripts/lib/install.sh +++ b/scripts/lib/install.sh @@ -364,8 +364,16 @@ install_verify_ingress() { path=$(env_get "$dir/.env" GHOST_HEALTHCHECK_PATH 2>/dev/null) || path=/ghost/api/admin/site/ [[ -n $path ]] || path=/ghost/api/admin/site/ - if status=$(install_http_status 127.0.0.1 "$ghost_port" "$path" localhost); then - if [[ $status == 200 ]]; then + # The direct loopback probe proves the Ghost process is answering. In local + # mode the site's canonical URL is that loopback address, so a 200 is + # expected. In production the canonical URL is https://DOMAIN, so a plain + # http request straight to the container — bypassing Caddy, without + # X-Forwarded-Proto — is redirected to HTTPS by design; a 3xx there is Ghost + # working, not a fault. The authoritative production checks are Caddy's. + local host_header=localhost + [[ $mode == production ]] && host_header=$domain + if status=$(install_http_status 127.0.0.1 "$ghost_port" "$path" "$host_header"); then + if [[ $status == 200 || ( $mode == production && $status =~ ^3 ) ]]; then printf 'ok Ghost answers on 127.0.0.1:%s%s\n' "$ghost_port" "$path" else printf 'ERROR Ghost answered %s on 127.0.0.1:%s%s\n' "$status" "$ghost_port" "$path" >&2 diff --git a/scripts/lib/preflight.sh b/scripts/lib/preflight.sh index f1528e43..7e33739f 100644 --- a/scripts/lib/preflight.sh +++ b/scripts/lib/preflight.sh @@ -32,7 +32,7 @@ readonly GD_REQUIRED_COMMANDS=(docker jq) # is the point: it is how a GNU-only or unusual dependency gets noticed. readonly GD_HOST_UTILITIES=( awk basename bash cat chmod chown cp cut date df dirname env grep head id - ls mkdir mktemp mv od rm sed sleep sort stat tr uname + ls mkdir mktemp mv od rm sed sleep sort stat sysctl tr uname ) # Recommended free space for a site: Ghost and MySQL images, the database, and diff --git a/tests/helpers.mjs b/tests/helpers.mjs index 68483dbf..d2f26a3a 100644 --- a/tests/helpers.mjs +++ b/tests/helpers.mjs @@ -223,7 +223,13 @@ export function git(repo, args) { /** * A candidate release of the working tree: a real git repository with real * tags, so bootstrap.sh resolves and clones it exactly as it would the - * published one. + * published one. Each tag lands on its own commit — a release and a prerelease + * do not share one — so `git describe` on a checkout is unambiguous, matching a + * real published history rather than a pile of tags on one commit. + * + * `url` is a file:// form of the repository. `--depth` is honoured over that, + * unlike a bare local path, so the shallow-clone path the bootstrap really uses + * is exercised without the "depth ignored in local clones" warning. */ export function makeCandidateRelease(dir, tags = ['v9.9.9']) { const repo = join(dir, 'release'); @@ -231,8 +237,11 @@ export function makeCandidateRelease(dir, tags = ['v9.9.9']) { git(repo, ['init', '-q', '-b', 'main']); git(repo, ['add', '-A']); git(repo, ['commit', '-q', '-m', 'candidate release']); - for (const tag of tags) git(repo, ['tag', tag]); - return { repo, tags }; + tags.forEach((tag, index) => { + if (index > 0) git(repo, ['commit', '-q', '--allow-empty', '-m', `release ${tag}`]); + git(repo, ['tag', tag]); + }); + return { repo, tags, url: `file://${repo}` }; } /** Run a script with a captured status, without throwing. */ diff --git a/tests/install-e2e.test.mjs b/tests/install-e2e.test.mjs index 88978b4d..cb3fe61a 100644 --- a/tests/install-e2e.test.mjs +++ b/tests/install-e2e.test.mjs @@ -10,7 +10,7 @@ // real host ports, including 80 and 443. import { test, describe, before, after } from 'node:test'; import assert from 'node:assert/strict'; -import { existsSync, statSync, readFileSync, writeFileSync, mkdirSync, symlinkSync } from 'node:fs'; +import { existsSync, statSync, readFileSync, writeFileSync, mkdirSync, symlinkSync, realpathSync } from 'node:fs'; import { execFileSync } from 'node:child_process'; import { join } from 'node:path'; import { delimiter } from 'node:path'; @@ -29,11 +29,13 @@ const PROXY_CONTAINER = 'ghost-docker-test-proxy'; let dir; let repo; +let repoUrl; +const installed = new Set(); /** Clone the candidate release into a directory, as bootstrap.sh would. */ const clone = (name) => { const target = join(dir, name); - execFileSync('git', ['clone', '--quiet', '--depth', '1', '--branch', CANDIDATE_TAG, repo, target]); + execFileSync('git', ['clone', '--quiet', '--depth', '1', '--branch', CANDIDATE_TAG, repoUrl, target]); return target; }; @@ -56,6 +58,11 @@ const down = (site) => { if (existsSync(join(site, '.env'))) compose(site, ['down', '-v', '--remove-orphans']); }; +// Sites are torn down together at the very end, not in each suite's own after: +// two of the checks need an earlier suite's site still running alongside a +// later one, so no site may be downed while a sibling suite is still asserting. +const track = (site) => { installed.add(site); return site; }; + /** * Caddy issues from its own internal CA rather than attempting a real ACME * order for a name that does not resolve. caddy/global/ is operator owned and @@ -69,21 +76,23 @@ const useInternalCerts = (site) => { describe('installing from a candidate release', { skip, concurrency: 1 }, () => { before(() => { dir = tempDir('install-e2e'); - ({ repo } = makeCandidateRelease(dir, ['v1.0.0', CANDIDATE_TAG])); + ({ repo, url: repoUrl } = makeCandidateRelease(dir, ['v1.0.0', CANDIDATE_TAG])); + }); + after(() => { + for (const site of installed) down(site); + cleanup(dir); }); - after(() => cleanup(dir)); describe('a local site, through the bootstrap', () => { let site; before(() => { - site = join(dir, 'local-a'); + site = track(join(dir, 'local-a')); }); - after(() => down(site)); test('bootstrap resolves the release, clones it and runs its installer', () => { const result = run(join(REPO_DIR, 'bootstrap.sh'), ['--channel', 'beta', '--dir', site, '--local', '--no-prompt'], - { env: { GD_BOOTSTRAP_REPO: repo }, timeout: 900_000 }); + { env: { GD_BOOTSTRAP_REPO: repoUrl }, timeout: 900_000 }); assert.equal(result.status, 0, result.output); assert.match(result.stdout, new RegExp(CANDIDATE_TAG.replace(/\./g, '\\.'))); assert.match(result.stdout, /Ghost is installed/); @@ -105,7 +114,9 @@ describe('installing from a candidate release', { skip, concurrency: 1 }, () => assert.equal(recorded.channel, 'beta'); assert.equal(recorded.stack.ref, CANDIDATE_TAG); assert.match(recorded.stack.commit, /^[0-9a-f]{40}$/); - assert.equal(recorded.site.dir, site); + // §2.10: the recorded dir is the resolved real host path (pwd -P), so + // compare against the realpath, not the possibly-symlinked test path. + assert.equal(recorded.site.dir, realpathSync(site)); assert.match(recorded.ghost.version, /^\d+\.\d+\.\d+$/); assert.match(recorded.ghost.digest, /^sha256:[0-9a-f]{64}$/, 'no digest recorded for recovery'); assert.deepEqual(recorded.profiles, ['local']); @@ -159,9 +170,8 @@ describe('installing from a candidate release', { skip, concurrency: 1 }, () => let second; before(() => { first = join(dir, 'local-a'); - second = clone('local-b'); + second = track(clone('local-b')); }); - after(() => down(second)); test('installs without disturbing the first', () => { const result = install(second, ['--local', '--no-prompt']); @@ -247,10 +257,9 @@ describe('installing from a candidate release', { skip, concurrency: 1 }, () => describe('a production site', () => { let site; before(() => { - site = clone('production'); + site = track(clone('production')); useInternalCerts(site); }); - after(() => down(site)); test('installs, renders routes, and verifies its own ingress', () => { const result = install(site, ['--domain', 'ghost.test', '--no-prompt']); @@ -276,13 +285,16 @@ describe('installing from a candidate release', { skip, concurrency: 1 }, () => assert.equal(readFileSync(join(site, 'caddy', 'global', 'tls.caddy'), 'utf8'), 'local_certs\n'); }); - test('HTTP on the ingress port redirects to HTTPS for the site domain', async () => { - const response = await fetch(`http://127.0.0.1:${env(site, 'HTTP_PORT')}/`, { - headers: { Host: 'ghost.test' }, - redirect: 'manual', - }); - assert.ok(response.status >= 300 && response.status < 400, `got ${response.status}`); - assert.match(response.headers.get('location') ?? '', /^https:\/\/ghost\.test/); + // Node's fetch forbids overriding the Host header, so Caddy would see + // 127.0.0.1 and redirect there; the request has to carry Host: ghost.test. + // The installer's own /dev/tcp helper sets it, which is what it verifies + // with too, so drive the check through that rather than fetch. + test('HTTP on the ingress port redirects to HTTPS for the site domain', () => { + const head = run('bash', ['-c', + `. ${JSON.stringify(join(site, 'scripts', 'lib', 'common.sh'))}\n` + + `install_http_head 127.0.0.1 ${env(site, 'HTTP_PORT')} / ghost.test`], { timeout: 60_000 }); + assert.match(head.stdout, /^HTTP\/[0-9.]+ 3\d\d/m, head.output); + assert.match(head.stdout, /^location: https:\/\/ghost\.test/im, head.output); }); // The whole intended ingress, end to end: TLS terminated by Caddy for this @@ -301,9 +313,8 @@ describe('installing from a candidate release', { skip, concurrency: 1 }, () => describe('--no-start', () => { let site; before(() => { - site = clone('no-start'); + site = track(clone('no-start')); }); - after(() => down(site)); test('configures the site and starts no application services', () => { const result = install(site, ['--local', '--no-prompt', '--no-start']); @@ -326,28 +337,41 @@ describe('installing from a candidate release', { skip, concurrency: 1 }, () => describe('the declared minimum host tools', () => { let site; before(() => { - site = clone('minimum-tools'); + site = track(clone('minimum-tools')); }); - after(() => down(site)); test('an install completes with only the recorded utilities on PATH', () => { - const declared = [ - ...shOk('printf "%s\\n" "${GD_REQUIRED_COMMANDS[@]}"').trim().split('\n'), - ...shOk('printf "%s\\n" "${GD_HOST_UTILITIES[@]}"').trim().split('\n'), - ]; + const utilities = shOk('printf "%s\\n" "${GD_HOST_UTILITIES[@]}"').trim().split('\n'); + const commands = shOk('printf "%s\\n" "${GD_REQUIRED_COMMANDS[@]}"').trim().split('\n'); + const realPath = process.env.PATH; + const bin = join(dir, 'minimum-bin'); mkdirSync(bin, { recursive: true }); + + // `docker` (and `jq`) are declared *commands*, not POSIX utilities: their + // own subprocess needs are their business, not the installer's. What this + // test isolates is whether our shell reaches for a coreutil that is not + // in the contract — so the utilities are the only things symlinked bare, + // and the commands are wrapped to run with the full PATH restored. A + // command that shelled out to an unlisted tool would still pass; a script + // of ours that did would not, which is the line this test draws. const missing = []; - for (const tool of declared) { + for (const tool of utilities) { const found = run('sh', ['-c', `command -v ${tool}`]).stdout.trim(); - if (!found) { - missing.push(tool); - continue; - } - symlinkSync(found, join(bin, tool)); + if (!found) missing.push(tool); + else symlinkSync(found, join(bin, tool)); } assert.deepEqual(missing, [], 'a declared utility is not installed on this host'); + for (const command of commands) { + const found = run('sh', ['-c', `command -v ${command}`]).stdout.trim(); + assert.ok(found, `${command} is not installed on this host`); + const wrapper = join(bin, command); + writeFileSync(wrapper, + `#!/bin/sh\nexec env PATH=${JSON.stringify(realPath)} ${JSON.stringify(found)} "$@"\n`, + { mode: 0o755 }); + } + const result = install(site, ['--local', '--no-prompt', '--no-start'], { env: { PATH: [bin, '/nonexistent'].join(delimiter) }, }); diff --git a/tests/install.test.mjs b/tests/install.test.mjs index 1b1e28d4..6ba943b3 100644 --- a/tests/install.test.mjs +++ b/tests/install.test.mjs @@ -261,6 +261,7 @@ describe('Ghost version resolution', () => { describe('release selection', () => { let dir; let repo; + let repoUrl; // bootstrap.sh is sourced in library mode so that ordering can be tested // without cloning anything. @@ -269,7 +270,7 @@ describe('release selection', () => { before(() => { dir = tempDir('release'); - ({ repo } = makeCandidateRelease(dir, [ + ({ repo, url: repoUrl } = makeCandidateRelease(dir, [ 'v1.9.0', 'v1.10.0', 'v1.11.0-beta.2', 'v1.11.0-beta.10', 'v0.1.0', 'not-a-release', ])); }); @@ -308,7 +309,7 @@ describe('release selection', () => { mkdirSync(occupied, { recursive: true }); writeFileSync(join(occupied, 'something'), 'x'); const result = run(join(REPO_DIR, 'bootstrap.sh'), ['--dir', occupied, '--ref', 'v1.10.0', '--local'], { - env: { GD_BOOTSTRAP_REPO: repo }, + env: { GD_BOOTSTRAP_REPO: repoUrl }, timeout: 60_000, }); assert.notEqual(result.status, 0); From 21f7f3780163d0c4084d6cfeee990c8d8bdb1fcf Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Thu, 3 Sep 2026 13:25:38 -0400 Subject: [PATCH 3/5] =?UTF-8?q?docs(plan):=20refine=20=C2=A72.10=20?= =?UTF-8?q?=E2=80=94=20which=20config=20helpers=20move=20into=20the=20mana?= =?UTF-8?q?ger=20CLI=20at=20S4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects the earlier "env/caddy never move into the image" framing. The dividing line is daemon-dependency, not config-vs-stateful: - Move onto the manager CLI at S4 (image + dispatcher exist there anyway): env.sh, meta.sh, and caddy_render — pure functions of on-disk files, a straight bash-deletion substitution. - Stay host shell permanently: bootstrap shim, preflight, site.sh check/doctor (must run without the image), and config validate + caddy validate/reload/ verify (they derive answers by asking `docker compose config`; moving them means bundling the Docker CLI in the manager or reimplementing Compose interpolation — the drift "ask Compose" exists to avoid). Records the accepted split (config get/set in the CLI, validate host shell) and why the move waits for S4 rather than S2: net line count is a wash once the Dockerfile, publish pipeline and privileged dispatcher are counted, so it only pays off riding a PR that builds them anyway. Updates the S4 step accordingly. Co-Authored-By: Claude Opus 4.8 --- docs/ghost-cli-replacement.md | 112 +++++++++++++++++++++++----------- 1 file changed, 78 insertions(+), 34 deletions(-) diff --git a/docs/ghost-cli-replacement.md b/docs/ghost-cli-replacement.md index 9d68231b..9772a1b2 100644 --- a/docs/ghost-cli-replacement.md +++ b/docs/ghost-cli-replacement.md @@ -729,7 +729,8 @@ are; the boundary does not run through them. Revised 2026-09-03, after S2. Two questions kept coming up — *should the image ship sooner?* and *should env validation and Caddy generation move into it?* — -so the boundary above is stated as two rules rather than left to inference. +so the boundary above is stated concretely rather than left to inference: when +the image lands, and exactly which helpers move onto it versus stay host shell. **The image's first tenant is the first stateful operation, at S4, not S8.** §2.6 already publishes a privileged supervisor image from this repo and requires @@ -746,35 +747,67 @@ written in so they are not rewritten at S8. The host dispatcher (this section's mount, identity and exit-code rules) is written in S4 alongside its first `docker run` target. -**Env validation, Caddy generation, preflight and version resolution do not -move into the image.** They are not stateful operations; they are the pure logic -that runs *before and without* a working container, and the boundary is set by -exactly that. Concretely: - -- The bootstrap writes `.env` before any image exists. A container cannot be the - only writer of the file that configures the container. -- `config.sh validate` runs inside preflight, before anything starts, and inside - `site.sh check` on a host whose daemon is missing or wedged. Behind `docker - run` it could not validate offline — the case it exists for — and would add an - image pull to every invocation. (S2 verified this matters: a wedged daemon - mid-session was still diagnosed by host-shell preflight because it does not - depend on the image.) -- Containerizing them does **not** solve the one hard problem, Compose's `$$` - interpolation: anything writing `.env` encodes a literal `$` as `$$` - regardless of implementation language, as recorded above. -- Moving them to JS forces a lose-lose: JS-in-container breaks offline validate - and still needs a host writer for the bootstrap; JS-on-host reintroduces the - host Node requirement S1 deliberately removed with `config-to-env.js`. Host - requirements stay `bash`, `docker`, `docker compose`, `jq`. - -So the dispatcher-plus-image model applies to the **stateful** commands (backup, -restore, import, upgrade, and the supervisor), which become thin host wrappers -around `docker run` of the manager image. Preflight, doctor, bootstrap, config -validation and Caddy rendering stay in host shell — not because bash is -preferable, but because they must run when there is no usable image. If shared -serialization between host and image is ever wanted, the host side stays the -jq-backed bash helpers; the image reimplements what it needs rather than the -host taking a language runtime. +**Some config helpers move into the manager CLI at S4; a specific subset must +not.** The dividing line is not "config versus stateful" — it is whether the +helper can run *before and without* a working daemon, and whether it derives its +answer by asking Docker. This was worked out concretely after S2, against the +real files, because "put the helpers in a node container so the scripts get +smaller" is a reasonable instinct that turns out to be right for two files and +wrong for two others. + +The manager image exists from S4 for backup/restore, and the dispatcher (§2.10's +mount/identity/exit-code rules) is written there. Once both exist, moving the +*pure* config logic onto them is close to free and is a real simplification, so +S4 (or S5, whichever first needs them container-side) does it: + +- **Moves into the manager CLI.** `env.sh` (the `$$` serializer/parser, ~260 + lines of bash regex state machine → ~60 lines of `JSON.parse`/stringify plus + one encoder) and `meta.sh` (`.ghost-docker.json`, ~245 lines of `jq` → native + JSON). Caddy *rendering* (`caddy_render` and `_caddy_site_block`, pure template + emission) moves with them. These are pure functions of files on disk; nothing + in them asks the daemon a question. The bash versions are deleted, not + wrapped — a straight substitution, which is the easy-to-review kind of diff. + +- **Stays host shell, permanently.** Two reasons, each disqualifying on its own: + + - *Runs before/without the image.* The bootstrap resolves the manager from the + release tag and can have it write the first `.env`, but the bootstrap shim + itself, preflight, and `site.sh check`/doctor must diagnose a host where + Docker is missing, stopped, or wedged. They cannot be `docker run`. (S2 + verified this the hard way: a wedged daemon mid-session was still diagnosed + by host-shell preflight precisely because it does not depend on the image.) + - *Derives its answer by asking Docker.* `config.sh validate` establishes the + container-owned keys by asking `docker compose config` what the container + receives (`config_ghost_environment`, the "derived, not listed" guarantee). + `caddy_validate`/`reload`/`verify` drive the running caddy container through + `compose_run`. Moving these into the manager would mean either bundling the + Docker/Compose CLI inside the manager and running compose-in-a-container over + a mounted socket, or reimplementing Compose interpolation in node — which is + exactly the drift that "ask Compose" was chosen to avoid. Neither is worth + it, so validation and caddy orchestration stay where they can call Compose + directly. + +Consequences to accept deliberately: config logic ends up split — `config +get/set/unset` in the manager CLI, `config validate` in host shell — because the +two halves have different daemon dependencies. That split is the honest cost, and +it is smaller than the cost of dragging a Docker CLI into the manager image to +avoid it. Containerizing the movable helpers also does **not** solve Compose's +`$$` interpolation: a literal `$` is encoded `$$` regardless of implementation +language, as recorded above. And the net line count of the move, counting the +Dockerfile, publish pipeline and privileged dispatcher it rides on, is roughly a +wash — the reason to do it is that env/meta stop being bash, not that the repo +gets shorter. Which is why it waits for S4: on a PR that builds the image and +dispatcher anyway, the env/meta deletion is pure upside; as a standalone change +it would stand up a privileged image and publish pipeline to save ~130 counted +lines, which does not clear the bar. + +So the dispatcher-plus-image model covers the **stateful** commands (backup, +restore, import, upgrade, supervisor) and, from S4, the **pure** config helpers +(`env`, `meta`, caddy render). Preflight, doctor, the bootstrap shim, `config +validate` and caddy orchestration stay host shell — the first three because they +must run when there is no usable image, the last two because they answer by +asking Docker. Host requirements stay `bash`, `docker`, `docker compose`, `jq`; +no host language runtime is added. ## 3. Implementation steps @@ -989,10 +1022,21 @@ and what its first tenant is"). Backup/restore is its first entrypoint, so the recovery algorithm S7 and S8 reuse exists once, in the image, not in host shell awaiting a rewrite. Write the host dispatcher here — the `docker run` mount, identity and exit-code rules from §2.10 — and route backup/restore through it. -The S2 host-shell helpers that must run without the image (preflight, config -validation, Caddy rendering, `site.sh check`) stay in host shell. Also add the -shared operation lock to installation/reconfigure retroactively: §2.2 requires -install to take it, and S2 deferred it to this step. + +With the image and dispatcher built, migrate the **pure** config helpers onto +them: `env.sh` and `meta.sh` become manager-CLI entrypoints (`config get/set/ +unset`, `meta ...`), and `caddy_render` moves with them; the bash versions are +deleted, a straight substitution. This is optional to S4's backup/restore +deliverable and may slip to S5 if it competes for review attention, but it is +cheapest here because the image already exists. The helpers that must stay host +shell do not move: preflight, the bootstrap shim and `site.sh check`/doctor +(they run without the image), and `config validate` plus caddy +orchestration (`caddy_validate`/`reload`/`verify`, which answer by asking +`docker compose`). Expect config logic to end up split — `config get/set` in the +manager, `config validate` in host shell — as §2.10 records. + +Also add the shared operation lock to installation/reconfigure retroactively: +§2.2 requires install to take it, and S2 deferred it to this step. Acceptance: restore a representative site to a fresh destination and verify database, assets/theme/configuration; inject interrupted backup/restore, full disk, stale lock, From 3dcbb205584503148ada4a16171df77f31cae228 Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Thu, 3 Sep 2026 13:34:00 -0400 Subject: [PATCH 4/5] fix(lint): shellcheck 0.9 cleanliness in site.sh and preflight.sh CI's shellcheck (Ubuntu 0.9.0) flags what local 0.11 does not: - site.sh: `(($#)) && shift || true` (SC2015) -> `if (($#)); then shift; fi` - docker_responsive: its SECONDS arg is optional and bare calls are the norm, so silence SC2120/SC2119 at the definition rather than every call site. Verified with koalaman/shellcheck:v0.9.0. Co-Authored-By: Claude Opus 4.8 --- scripts/lib/preflight.sh | 5 +++++ scripts/site.sh | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/lib/preflight.sh b/scripts/lib/preflight.sh index 7e33739f..92b6af14 100644 --- a/scripts/lib/preflight.sh +++ b/scripts/lib/preflight.sh @@ -92,6 +92,11 @@ _gd_docker() { # never by inspecting group membership: neither rootless Docker nor a remote # DOCKER_HOST involves the docker group, and being in it does not mean the # daemon is running. +# +# The SECONDS argument is optional; a bare call using the default deadline is +# the common one, so the "argument never passed" lint is silenced here rather +# than at every call site. +# shellcheck disable=SC2120 docker_responsive() { command -v docker >/dev/null 2>&1 || return 1 _gd_docker "${1:-$GD_DOCKER_PROBE_TIMEOUT}" info >/dev/null 2>&1 diff --git a/scripts/site.sh b/scripts/site.sh index 93743382..957b5f67 100755 --- a/scripts/site.sh +++ b/scripts/site.sh @@ -147,7 +147,7 @@ site_check() { } cmd=${1:-} -(($#)) && shift || true +if (($#)); then shift; fi case "$cmd" in list) site_list ;; From 4964099816e682a6b93d103e98f58c6ea339be7c Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Thu, 3 Sep 2026 13:39:34 -0400 Subject: [PATCH 5/5] fix(preflight): detect an unreachable daemon by output, not exit code On the Linux CI runner, `docker info --format '{{.ServerVersion}}'` prints "Cannot connect to the Docker daemon ..." and still exits 0, so preflight_docker took the version path and reported the connect-error string as an old engine version instead of "docker daemon not reachable". Decide reachability from the output: the server version is a dotted number when the daemon answers, so extract it and treat its absence as unreachable, whatever the exit code. Keeps the timeout (124) path for a wedged daemon. Co-Authored-By: Claude Opus 4.8 --- scripts/lib/preflight.sh | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/scripts/lib/preflight.sh b/scripts/lib/preflight.sh index 92b6af14..80fa73cc 100644 --- a/scripts/lib/preflight.sh +++ b/scripts/lib/preflight.sh @@ -238,16 +238,19 @@ preflight_docker() { _gd_report error "docker daemon" "did not answer within ${GD_DOCKER_PROBE_TIMEOUT}s; it is running but not responding ($(_gd_daemon_hint))" return fi - if ((rc != 0)); then + + # Reachability is decided by the output, not the exit code: with + # `info --format`, some docker CLIs print "Cannot connect to the Docker + # daemon ..." and still exit 0, so a non-zero rc cannot be relied on. The + # server version is a dotted number when the daemon answers; extract it, and + # its absence means the daemon did not answer whatever the exit code said. + version=$(printf '%s\n' "$out" | sed -n 's/^\([0-9][0-9]*\(\.[0-9][0-9]*\)*\).*/\1/p' | head -1) + if [[ -z $version ]]; then _gd_report error "docker daemon" "not reachable ($(_gd_daemon_hint)); docker said: $(printf '%s' "$out" | tr '\n' ' ' | cut -c1-160)" return fi - version=$out - [[ -n $version ]] || version=$(_gd_docker "$GD_DOCKER_PROBE_TIMEOUT" version --format '{{.Server.Version}}' || printf '') - if [[ -z $version ]]; then - _gd_report warn "docker engine" "reachable, but the version could not be determined" - elif version_at_least "$version" "$GD_MIN_DOCKER_VERSION"; then + if version_at_least "$version" "$GD_MIN_DOCKER_VERSION"; then _gd_report ok "docker engine" "$version" else _gd_report error "docker engine" "$version is older than the required $GD_MIN_DOCKER_VERSION (healthcheck start_interval)"