From 0c30b1502b261b974b4af4f91e364fba67e23dad Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Mon, 14 Sep 2026 20:39:22 -0400 Subject: [PATCH 1/3] docs: aligned bundle v1 with S3 exporter and cutover guarantees --- docs/bundle-v1.md | 89 +++++++++++++++---- docs/configuration.md | 5 ++ docs/ghost-cli-replacement.md | 11 ++- .../migration-bundle-v1/mysql-dump.json | 19 ++++ .../migration-bundle-v1/portable.json | 20 +++++ 5 files changed, 124 insertions(+), 20 deletions(-) create mode 100644 tests/fixtures/migration-bundle-v1/mysql-dump.json create mode 100644 tests/fixtures/migration-bundle-v1/portable.json diff --git a/docs/bundle-v1.md b/docs/bundle-v1.md index acdf1b8e..fc7a80e0 100644 --- a/docs/bundle-v1.md +++ b/docs/bundle-v1.md @@ -36,10 +36,13 @@ stand as written. "sourceInstallType": "production", "kind": "mysql-dump", "ghost": { - "version": "5.130.3" + "version": "6.2.0" }, + "url": "https://example.com", + "adminUrl": "https://admin.example.com", + "database": {"path": "database.sql"}, + "content": "content/", "config": { - "url": "https://example.com", "mail__options__auth__pass": "p$ssword", "mail__from": "'Acme Support' " } @@ -52,9 +55,30 @@ stand as written. | --- | --- | | `bundleVersion` | Must be `1`. | | `bundleCreatedAt` | **Required.** RFC 3339 timestamp in UTC. | -| `sourceInstallType` | **Required.** Exactly `local` or `production`. The importer infers the installation mode from this field. | -| `kind` | Bundle kind. Validated against the kinds the importer supports. | -| `ghost.version` | Exact source Ghost version. Validated as supported; the import happens *at* this version, and upgrading is a separate operation. | +| `sourceInstallType` | **Required.** Exactly `local` or `production`. Derived from the source instance’s actual local/production process classification; the importer selects its mode from this field. | +| `kind` | Required `mysql-dump` or `portable`. MySQL/mysql2 or local SQLite development respectively. | +| `ghost.version` | Exact source Ghost 6.x version. Validated as supported; the import happens *at* this version, and upgrading is a separate operation. | + +| `url` | Required public URL, unchanged from source config. | +| `adminUrl` | Optional separate admin URL, unchanged. | +| `database.path` | Required relative path: `database.sql` for MySQL, content JSON for portable. | +| `database.members` | Required for portable only; relative members CSV path. | +| `content` | Required `content/` asset root. | +| `config` | Required raw string map, described below. | + +Portable `database` example (filenames can vary; always read the manifest): + +```json +{ + "path": "content/data/content-from-v6.2.0-on-2026-09-14-12-00-00.json", + "members": "content/data/members-from-v6.2.0-on-2026-09-14-12-00-00.csv" +} +``` + +`kind` appears only at the top level and the version only at `ghost.version`. +There are no `database.kind`, `ghostVersion`, or `sourceEnvironment` aliases. +Matching exporter fixtures are in `tests/fixtures/migration-bundle-v1/` and +Ghost-CLI's `test/fixtures/migration-bundle-v1/`. A bundle missing `bundleCreatedAt` or `sourceInstallType`, or carrying a `sourceInstallType` outside that set, is rejected. There is no inference @@ -78,7 +102,7 @@ including in directory bundles. in [configuration.md](configuration.md) and at the top of [scripts/lib/env.sh](../scripts/lib/env.sh). -So a mail password of `p$ssword` appears in the manifest as the four-character +So a mail password of `p$ssword` appears in the manifest as the JSON string `"p$ssword"`, and reaches `ghost.env` as `mail__options__auth__pass="p$$ssword"`. An exporter that pre-quotes or pre-escapes a value produces a corrupted import, and the round trip is tested through real Docker Compose containers @@ -119,13 +143,46 @@ journals. It is about isolating untrusted bundle content: path traversal, absolute member paths, escaping links, and unbounded expansion are all properties of a file someone else produced. -## Not settled in this step - -The following are defined by their own steps and are **not** promised here: - -- Exporter behaviour, including the documented final-export mode that leaves - the source stopped for cutover, and the preserved restart behaviour for - ordinary exports (S3). -- The portable-import fidelity matrix and the explicit list of losses, which - is established from fixtures and real exporter/importer behaviour (S3/S5). -- The import sequence, isolated destination, verification and cutover (S5). +## Source consistency and cutover (S3) + +Ghost-CLI's `ghost migrate-export --leave-stopped` deliberately leaves the source +stopped after successful export and attempts to stop it after export failure. +Preflight rejection leaves the source unchanged. Ordinary exports restore its +original running state, including a stopped portable source temporarily started +for the API. Failed exports remove partial outputs. Recovery is `ghost start` +in the original installation; verify `ghost ls` if a lifecycle operation failed. + +Portable sources are local SQLite development sites only. Capture order is content +JSON → members CSV → stop Ghost → copy assets. Users must avoid editing throughout +export. This is sequential capture, **not an atomic snapshot or write freeze**; +`bundleCreatedAt` is manifest creation time. Production SQLite and unsupported +clients are rejected. MySQL is stopped before copying assets and dumping its DB; +external database writers must also be quiescent. + +Directories/files/archives are private from creation (`0700`/`0600`). Existing +outputs, source overlap (including symlink aliases), and archive collisions are +refused before source lifecycle changes. Supported content includes hidden files, +full themes, settings, files/images/media, and data redirects. Runtime logs/apps, +SQLite files, external storage and custom adapters are omitted. Links/special +files inside copied content are rejected explicitly by the exporter. + +Portable content/member files preserve Ghost API response bytes, not a complete +database. Author data travels but reusable staff authentication does not; IDs may +be remapped by import. Default content export omits integrations/API keys/webhooks, +member and subscription relationship tables, comments and event/email history. +CSV carries member fields and customer/tier references, not complete paid +subscription or per-newsletter relationships. Re-establish staff access and +integrations; reconnect/reconcile Stripe using a supported importer. + +See the exporter's [fidelity and recovery documentation](https://github.com/TryGhost/Ghost-CLI/blob/claude/ghost-cli-migration-export-c00253/docs/migration-bundle.md). +S3 verifies schema, source lifecycle, private output, system-tar extraction and +real Compose value transport. S5 must qualify destination owner setup, ID mapping, +member imports and subscription reconciliation end to end before promising their +fidelity. S3 does not implement the Docker importer. + +## Remaining S5 work + +The importer, isolated destination, verification, and ingress cutover are S5. +Keep the final source stopped and intact until the destination is accepted; +restarting it permits writes that invalidate the final snapshot. Real production +cutover must prevent writes before the final MySQL export. diff --git a/docs/configuration.md b/docs/configuration.md index c0487c13..42597d0b 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -30,6 +30,11 @@ atomically with a restrictive umask and preserve the mode of an existing file. ## Value encoding +Migration bundle v1 config values are raw strings, with no dotenv encoding. The +importer applies the rules below exactly once when writing `ghost.env`; public +and admin URLs are separate manifest fields mapped to `.env`. See +[bundle-v1.md](bundle-v1.md) for the agreed S3 schema and source guarantees. + Compose interpolates dotenv values, **including inside double quotes**, and `env_file` values are no exception. A literal dollar sign must be written `$$`. There is no quoting a person naturally reaches for that avoids this, and the diff --git a/docs/ghost-cli-replacement.md b/docs/ghost-cli-replacement.md index 8f4285c9..6306d7da 100644 --- a/docs/ghost-cli-replacement.md +++ b/docs/ghost-cli-replacement.md @@ -253,8 +253,10 @@ Before freezing the contract: - Record the consistency/cutover behavior: the exporter currently restarts Ghost. Add a documented final-export mode that leaves the source stopped, with explicit operator selection, and preserve the current restart behavior for ordinary exports. - Portable exports require a running source; final export must arrange a write freeze - before taking API/content snapshots, then leave it stopped for cutover. + Portable exports support local SQLite development sites only: content API export, + then members CSV, then stop Ghost and copy assets. Captures are sequential; users + must avoid editing during export. Do not implement write-freeze machinery or + require Ghost changes. Final export leaves the source stopped for cutover. Import sequence: @@ -855,8 +857,9 @@ quoted-config decoding, or a sourceEnvironment fallback for mode selection: this format has not shipped. Implement deliberate final-export/cutover behavior while preserving ordinary -export restart semantics. Address write-freeze and snapshot consistency for -portable exports; document precisely which data/relationships the portable +export restart semantics, including originally stopped portable sources. Keep +portable capture sequential for local SQLite development sites without write-freeze +machinery or Ghost changes; document precisely which data/relationships the portable format cannot preserve. Use lib/tasks/import/ as the API reference. Keep the beta warning until qualification. diff --git a/tests/fixtures/migration-bundle-v1/mysql-dump.json b/tests/fixtures/migration-bundle-v1/mysql-dump.json new file mode 100644 index 00000000..0c9f754b --- /dev/null +++ b/tests/fixtures/migration-bundle-v1/mysql-dump.json @@ -0,0 +1,19 @@ +{ + "bundleVersion": 1, + "bundleCreatedAt": "2026-09-14T12:00:00.000Z", + "sourceInstallType": "production", + "kind": "mysql-dump", + "ghost": { + "version": "6.2.0" + }, + "url": "https://example.com", + "adminUrl": "https://admin.example.com", + "database": { + "path": "database.sql" + }, + "content": "content/", + "config": { + "mail__from": "Ghost Blog ", + "mail__options__auth__pass": "pa$$word \\\" #\n" + } +} diff --git a/tests/fixtures/migration-bundle-v1/portable.json b/tests/fixtures/migration-bundle-v1/portable.json new file mode 100644 index 00000000..d3b16675 --- /dev/null +++ b/tests/fixtures/migration-bundle-v1/portable.json @@ -0,0 +1,20 @@ +{ + "bundleVersion": 1, + "bundleCreatedAt": "2026-09-14T12:00:00.000Z", + "sourceInstallType": "local", + "kind": "portable", + "ghost": { + "version": "6.2.0" + }, + "url": "https://example.com", + "adminUrl": "https://admin.example.com", + "database": { + "path": "content/data/content.json", + "members": "content/data/members.csv" + }, + "content": "content/", + "config": { + "mail__from": "Ghost Blog ", + "mail__options__auth__pass": "pa$$word \\\" #\n" + } +} From c8d5bb7b4f383f99a9aae51b4d1bee9bb00a1b3f Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Mon, 14 Sep 2026 21:04:46 -0400 Subject: [PATCH 2/3] docs: clarified empty members CSV in portable bundles --- docs/bundle-v1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/bundle-v1.md b/docs/bundle-v1.md index fc7a80e0..a622714c 100644 --- a/docs/bundle-v1.md +++ b/docs/bundle-v1.md @@ -62,7 +62,7 @@ stand as written. | `url` | Required public URL, unchanged from source config. | | `adminUrl` | Optional separate admin URL, unchanged. | | `database.path` | Required relative path: `database.sql` for MySQL, content JSON for portable. | -| `database.members` | Required for portable only; relative members CSV path. | +| `database.members` | Required for portable only; relative members CSV path. A successful zero-byte export means no members; skip member import for that file. | | `content` | Required `content/` asset root. | | `config` | Required raw string map, described below. | From bd156bcbb99e9ad9052e9bb43c72f18b06e923ba Mon Sep 17 00:00:00 2001 From: Austin Burdine Date: Mon, 14 Sep 2026 21:19:27 -0400 Subject: [PATCH 3/3] docs: clarified linked theme materialization in migration bundles --- docs/bundle-v1.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/bundle-v1.md b/docs/bundle-v1.md index a622714c..efa98017 100644 --- a/docs/bundle-v1.md +++ b/docs/bundle-v1.md @@ -163,8 +163,12 @@ Directories/files/archives are private from creation (`0700`/`0600`). Existing outputs, source overlap (including symlink aliases), and archive collisions are refused before source lifecycle changes. Supported content includes hidden files, full themes, settings, files/images/media, and data redirects. Runtime logs/apps, -SQLite files, external storage and custom adapters are omitted. Links/special -files inside copied content are rejected explicitly by the exporter. +SQLite files, external storage and custom adapters are omitted. Individual theme +directory links under `content/themes/` are resolved and materialized as regular +directories, including CLI defaults linked through `current` and external +development themes. Output may not overlap their resolved targets. Broken/cyclic +theme links, links to non-directories, nested links and other content links/special +files are rejected explicitly by the exporter. Portable content/member files preserve Ghost API response bytes, not a complete database. Author data travels but reusable staff authentication does not; IDs may