From 667587e8cf42a1dbbd0c25b2b6b276a7282bc8ea Mon Sep 17 00:00:00 2001 From: Paul O'Fallon Date: Sun, 20 Sep 2026 20:00:03 +0000 Subject: [PATCH 1/5] feat(deploy): restore-on-install from a live deployment (spec 007) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit During an app install the operator may name an existing deployment of the same app on this host as a restore source. The server quiesces it with the backup contract's existing pre-hooks, captures its data root, and lays that data into the new install AFTER images are pulled and BEFORE any container starts — the one moment in an app's life when the data root is empty and no process holds it open, so a restore cannot overwrite anything, cannot race a live process, and needs no stop/start choreography. This slice deliberately contains no backup provider: the source is a live deployment on the same host, so the server is both reader and writer exactly as it already is for a data-aware rollback. No capability contract, no grant, no contract endpoint, no new job type. Sequence 6 (`restore@1`, the provider half) is the consumer of this groundwork. The design is almost entirely reuse — `capturePreUpgradeSnapshot`'s hook-and-tar path, `runPreHooksFailClosed`'s fail-closed policy, `restoreTarGzInto`, `mergeUpgradeAppEnv`'s three-case rule, `resolveContainedDir`'s containment, `checkUpgradePath`'s skew rules, `writeInstanceMarkers`, and `grants`' consent shape. Genuinely new: a candidate resolver, the restore sequence in the lifecycle job, and `composeUp`'s service-list/wait option. Two failures govern the whole design. A restore that cannot succeed refuses before any container starts (there is no partial-success state worth starting an app in), and a restore that carries data carries the configuration that data was written under — or names loudly every generated secret it could not carry, since data encrypted under a key the platform generated is unreadable beside a freshly generated one and nothing in the running app reports that as an error. Also promotes `lineageId` onto the deployment record, which spec 006 predicted in the shipped code it now changes. The `?? deployment.id` fallback makes this zero-migration: an older record reads undefined and yields the value it always wrote, and captures taken since spec 006 already carry the field. Review caught and fixed three defects worth naming. A shell-injection sink: `composeUp` interpolated manifest-supplied service names into a string run through `exec`, in the same file whose backup path builds argv explicitly "so provisioned values can't be injected" — now argv via `execFileAsync`, plus a Compose-service-name pattern at coercion. A `discard: ["."]` could wipe the restored payload after the post-condition passed, because `resolveContainedDir` treats the root as contained — now also requires `isStrictlyInside`. And the wizard judged no version skew at all, demanding an acknowledgement from every operator on every restore. Four of the prompt's own claims were wrong against the post-spec-006 tree and are corrected in the artifacts with evidence: the job payload's shape, the absent `CreateDeploymentRequest` type, a "locate the subtree" trap that describes a provider archive tool rather than this codebase's root-relative helpers, and `checkUpgradePath`'s inability to express "refuse a newer source" (it returns ok for anything that is not a forward upgrade). 53 FRs, 13 SCs, 57 verification scenarios, 88 tasks. Server 1086 -> 1130, web 360 -> 367, CLI 267 -> 274. Three fixes are mutation-proved: the injection probe, the FR-014 end-to-end refusal, and the wizard rollback. The try-hola/apps catalog change (schema + 5 manifests) is prepared but NOT submitted — it targets a repo requiring explicit per-instance permission. Closes #429 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh --- .specify/feature.json | 2 +- CLAUDE.md | 34 +- docs/OPERATIONS.md | 43 + packages/cli/src/__tests__/restore.test.ts | 155 +++ packages/cli/src/commands/install/install.ts | 156 ++- packages/cli/src/index.ts | 6 + packages/cli/src/lib/deploy-flow.ts | 41 +- packages/sdk/src/index.ts | 17 +- .../deployments/restore-on-install.test.ts | 1134 +++++++++++++++++ packages/server/src/server.ts | 59 + packages/server/src/services/core/catalog.ts | 9 +- .../server/src/services/core/deployment.ts | 474 ++++++- packages/server/src/services/core/docker.ts | 71 +- packages/server/src/services/core/draft.ts | 208 ++- .../src/services/core/manifest-restore.ts | 102 ++ .../src/services/core/restore-candidates.ts | 453 +++++++ .../server/src/services/simple-factory.ts | 16 +- packages/shared/src/index.ts | 178 +++ .../pages/InstallWizard.channels.test.tsx | 38 +- .../pages/InstallWizard.grants.test.tsx | 28 +- .../pages/InstallWizard.profiles.test.tsx | 20 +- .../InstallWizard.refNotAllowed.test.tsx | 26 +- .../pages/InstallWizard.restore.test.tsx | 243 ++++ .../pages/InstallWizard.secretWand.test.tsx | 26 +- packages/web/src/pages/InstallWizard.tsx | 307 ++++- packages/web/src/utils/api-hybrid.ts | 2 + packages/web/src/utils/sdk-adapter.ts | 11 +- .../checklists/requirements.md | 71 ++ specs/007-restore-on-install/contracts/api.md | 176 +++ specs/007-restore-on-install/contracts/cli.md | 89 ++ .../contracts/manifest.md | 147 +++ specs/007-restore-on-install/data-model.md | 254 ++++ specs/007-restore-on-install/plan.md | 255 ++++ specs/007-restore-on-install/quickstart.md | 200 +++ specs/007-restore-on-install/research.md | 693 ++++++++++ specs/007-restore-on-install/spec.md | 732 +++++++++++ specs/007-restore-on-install/tasks.md | 319 +++++ 37 files changed, 6693 insertions(+), 102 deletions(-) create mode 100644 packages/cli/src/__tests__/restore.test.ts create mode 100644 packages/server/src/__tests__/deployments/restore-on-install.test.ts create mode 100644 packages/server/src/services/core/manifest-restore.ts create mode 100644 packages/server/src/services/core/restore-candidates.ts create mode 100644 packages/web/src/__tests__/pages/InstallWizard.restore.test.tsx create mode 100644 specs/007-restore-on-install/checklists/requirements.md create mode 100644 specs/007-restore-on-install/contracts/api.md create mode 100644 specs/007-restore-on-install/contracts/cli.md create mode 100644 specs/007-restore-on-install/contracts/manifest.md create mode 100644 specs/007-restore-on-install/data-model.md create mode 100644 specs/007-restore-on-install/plan.md create mode 100644 specs/007-restore-on-install/quickstart.md create mode 100644 specs/007-restore-on-install/research.md create mode 100644 specs/007-restore-on-install/spec.md create mode 100644 specs/007-restore-on-install/tasks.md diff --git a/.specify/feature.json b/.specify/feature.json index 95f80727..d6a70b67 100644 --- a/.specify/feature.json +++ b/.specify/feature.json @@ -1,3 +1,3 @@ { - "feature_directory": "specs/006-install-identity" + "feature_directory": "specs/007-restore-on-install" } diff --git a/CLAUDE.md b/CLAUDE.md index 0e820e86..4cff9b04 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -114,6 +114,38 @@ install as **Docker Compose** stacks, orchestrated by a server and routed by `ALREADY_INSTALLED` (`details.code`, same `CONFLICT` top-level shape as `PROVIDER_EXISTS`) so the wizard/CLI render a choice instead of a surface-neutral message the caller has to parse. +- **Restore-on-install (spec 007).** At install, an operator may name an + existing deployment of the same app on this host as a restore source + (`CreateDraftRequest.restoreFrom`, catalog path only — install-by-ref + refuses it, `RESTORE_NOT_SUPPORTED`, since there's no catalog upgrade + metadata to judge the candidate's version against). The choice is resolved + and validated once at draft creation (seeding `appEnv` via the existing + `mergeUpgradeAppEnv` three-case rule when configuration is carried) and + re-validated at `createFromDraft`, because the candidate may have changed + since. The restore itself runs inside `runLifecycleJob`, after + `composePull` and before `composeUp` — never at create time (Constitution + III): assert the target data root is empty, quiesce and capture the source + with the existing `backup@1` pre/post hooks, extract into the target + (root-relative, no intermediate copy), assert the payload landed (a + post-condition, not a subtree search — this codebase's archives are + root-relative on both ends), apply the app's declared `discard` paths, + rewrite the install-identity marker, write pending OIDC credentials (moved + here from its usual pre-restore position so extraction can't destroy it), + start only the declared hook's service with `composeUp({ services, wait: + true })`, and run the restore hook fail-closed. A failed restore fails the + whole install — no partial-success start. An app declares how it wants to + be restored per **backup participation** (`restore` block in the bundle + manifest, reusing `AppBackupHook` verbatim — no second hook format): no + `restore@1` in `accepts` means not offered; `accepts: ["restore@1"]` with no + block means a plain file copy is sufficient; a block adds `discard` paths + (a live database's file-level copy is a smear across the capture window, + not a snapshot) and a reload hook. No capability contract, grant, or + contract endpoint is added — `restore@1` here is a participation marker an + app declares, not a contract the platform brokers (`CONTRACTS` is + unchanged, FR-047). `hola install --restore-from `, with + `--restore-list` reading the same candidates route with no draft created; + the non-interactive default is always **no restore** — a candidate existing + is never itself consent to use it. ## Conventions @@ -172,5 +204,5 @@ Full guide: `docs/MCP_VM_TESTING.md`. For additional context about technologies to be used, project structure, shell commands, and other important information, read the current plan: -`specs/006-install-identity/plan.md` +`specs/007-restore-on-install/plan.md` diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index 5f958734..95111825 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -332,6 +332,49 @@ existing provider and telling you to uninstall it first. A pair recorded before this rule existed (rare) is flagged as a warning on the Backups page rather than silently resolved — uninstall one of them. +### Restore-on-install + +During an **install**, you can pick an existing deployment of the same app on +this host as a restore source. The server quiesces it with the same pre/post +hooks App backups already use, captures its data root, and lays that data down +into the new install after images are pulled and before any container starts — +the one moment an app's life when the data root is empty and no process holds +it open. It closes [#429](https://github.com/try-hola/hola/issues/429): the +fastest way to get a second, independently-addressable copy of a running app +holding its data. + +**Where it lives.** The install wizard's first step, before Configuration — a +restore choice determines what configuration gets pre-filled, so it has to +come first. From the CLI: `hola install --restore-from ` +(`--restore-list` to see candidates first, with no draft created). + +**What it carries.** Files, always. Configuration — including secrets — only +when you ask for it (on by default when the candidate has one recorded); an +app that generates its own secrets on first boot will mint fresh ones for data +that was encrypted under the originals unless you carry the recorded values +forward too. The confirmation step names this explicitly: a restore carries +**data and credentials**, and any jobs, webhooks or integrations the app runs +may fire the moment it starts holding that data. + +**Why it refuses.** A restore fails the whole install rather than starting an +app on empty or partial data — there is no partial-success state. Common +reasons: the candidate was captured on a newer version than you're installing; +the app's own upgrade rules block the version hop; the candidate no longer +exists or isn't in a settled (running/stopped) state by the time the install +actually runs; or a restore hook failed (a database load with `ON_ERROR_STOP` +enabled reports the load failure, not the app's later confusion about missing +tables). Every refusal is a specific answer, not a generic install failure — a +new deployment left in `error` with its data root intact, so you can inspect it +before deciding to retry. + +**What an app has to declare.** Nothing, for the common case: most catalog +apps (anything SQLite or flat-file) restore correctly with a plain file copy, +and need no manifest changes at all. A database-backed app declares a small +`restore` block in its manifest naming which paths to discard after the files +land (a live database's file-level copy is a smear across the capture window, +not a snapshot) and the `psql`-style command that reloads the dump — the same +hook shape the backup declaration already uses. + ### Container logs `container-logs@1` is a **provisioned** contract: a log collector app from the diff --git a/packages/cli/src/__tests__/restore.test.ts b/packages/cli/src/__tests__/restore.test.ts new file mode 100644 index 00000000..55191e9c --- /dev/null +++ b/packages/cli/src/__tests__/restore.test.ts @@ -0,0 +1,155 @@ +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; + +import { runInstall, parseAcks } from '../commands/install/install'; +import { reportDeployError } from '../lib/deploy-flow'; +import { HolaApiError } from '@hola/sdk'; +import type { HolaSdk } from '@hola/sdk'; +import type { ListRestoreCandidatesResponse } from '@hola/shared'; + +function candidate(id: string, lineageId: string, carriesEnv = true) { + return { + deploymentId: id, + lineageId, + app: 'mealie', + name: id, + subdomain: id, + host: `${id}.example.com`, + appVersion: '1.0.0', + channel: 'stable', + carriesEnv, + capturedAt: '2026-01-01T00:00:00.000Z', + hasIdentityRecord: true, + skew: { kind: 'ok' as const }, + requiredAcknowledgements: [], + warnings: [], + }; +} + +function makeSdk(overrides: { drafts?: Record; restoreCandidatesResp?: ListRestoreCandidatesResponse } = {}) { + const calls: string[] = []; + const oneLineage: ListRestoreCandidatesResponse = { + appId: 'mealie', + lineages: [{ lineageId: 'l1', candidates: [candidate('mealie-aaa', 'l1')] }], + defaultCandidateId: 'mealie-aaa', + requiresExplicitChoice: false, + }; + return { + calls, + drafts: { + create: vi.fn(async () => { calls.push('create'); return { draftId: 'd1' }; }), + byId: vi.fn(async () => { calls.push('byId'); return { draftId: 'd1', appEnv: [] }; }), + update: vi.fn(async () => { calls.push('update'); return { ok: true }; }), + validate: vi.fn(async () => { calls.push('validate'); return { ok: true, errors: [], warnings: [] }; }), + preflight: vi.fn(async () => { calls.push('preflight'); return { ok: true, checks: [] }; }), + finalize: vi.fn(async () => { calls.push('finalize'); return { spec: {}, checksum: 'x' }; }), + ...(overrides.drafts ?? {}), + }, + deployments: { + create: vi.fn(async () => { calls.push('deploy'); return { deploymentId: 'dep1', releaseId: 'r1', jobId: 'j1' }; }), + }, + jobs: { byId: vi.fn(async () => ({ status: 'completed' })) }, + restoreCandidates: vi.fn(async () => overrides.restoreCandidatesResp ?? oneLineage), + }; +} + +describe('restore-on-install CLI (spec 007)', () => { + beforeEach(() => { process.exitCode = 0; }); + afterEach(() => { process.exitCode = 0; vi.restoreAllMocks(); }); + + // ---- parseAcks mirrors parseGrants exactly (scenario 52) ---- + it('scenario 52: parseAcks parses repeated and comma-separated values exactly like --grant', () => { + expect(parseAcks(undefined)).toBeUndefined(); + expect(parseAcks('a')).toEqual(['a']); + expect(parseAcks(['a', 'b'])).toEqual(['a', 'b']); + expect(parseAcks('a,b, c')).toEqual(['a', 'b', 'c']); + expect(parseAcks(['a,b', 'b'])).toEqual(['a', 'b']); // deduped + }); + + // ---- scenario 49: each flag behaves as specified ---- + it('scenario 49: --restore-from resolves that exact candidate and sends restoreFrom on create', async () => { + const sdk = makeSdk(); + await runInstall('mealie', { restoreFrom: 'mealie-aaa', ack: 'restore-env-not-carried', noStream: true }, { sdk: sdk as unknown as HolaSdk }); + + expect(sdk.restoreCandidates).toHaveBeenCalledWith('mealie', 'latest', undefined, undefined); + expect(sdk.drafts.create).toHaveBeenCalledWith(expect.objectContaining({ + appId: 'mealie', + restoreFrom: { candidateId: 'mealie-aaa', carryEnv: true, acknowledge: ['restore-env-not-carried'] }, + })); + }); + + it('scenario 49: --restore-from latest refuses across two-or-more unrelated lineages', async () => { + const sdk = makeSdk({ + restoreCandidatesResp: { + appId: 'mealie', + lineages: [ + { lineageId: 'l1', candidates: [candidate('mealie-aaa', 'l1')] }, + { lineageId: 'l2', candidates: [candidate('mealie-bbb', 'l2')] }, + ], + defaultCandidateId: null, + requiresExplicitChoice: true, + }, + }); + const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + const res = await runInstall('mealie', { restoreFrom: 'latest', noStream: true }, { sdk: sdk as unknown as HolaSdk }); + + expect(res).toBeUndefined(); + expect(process.exitCode).toBe(1); + expect(errSpy).toHaveBeenCalledWith(expect.stringContaining('ambiguous')); + expect(sdk.drafts.create).not.toHaveBeenCalled(); + }); + + it('scenario 49: --restore-list lists candidates and creates no draft', async () => { + const sdk = makeSdk(); + const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {}); + const res = await runInstall('mealie', { restoreList: true, noStream: true }, { sdk: sdk as unknown as HolaSdk }); + + expect(res).toBeUndefined(); + expect(sdk.restoreCandidates).toHaveBeenCalledWith('mealie', 'latest', undefined, undefined); + expect(sdk.drafts.create).not.toHaveBeenCalled(); + expect(sdk.deployments.create).not.toHaveBeenCalled(); + expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('mealie-aaa')); + logSpy.mockRestore(); + }); + + it('scenario 49: --no-restore is a no-op — same outcome as no flag at all', async () => { + const sdk = makeSdk(); + await runInstall('mealie', { restore: false, noStream: true }, { sdk: sdk as unknown as HolaSdk }); + + expect(sdk.restoreCandidates).not.toHaveBeenCalled(); + expect(sdk.drafts.create).toHaveBeenCalledWith({ appId: 'mealie', version: 'latest' }); + }); + + // ---- scenario 50: no restore flag at all -> no restore, even with candidates present ---- + it('scenario 50: with no restore flag, no restore happens even when candidates exist (FR-044)', async () => { + const sdk = makeSdk(); // restoreCandidates would report a candidate if asked + await runInstall('mealie', { noStream: true }, { sdk: sdk as unknown as HolaSdk }); + + // The candidates route is never even consulted — silence must never guess. + expect(sdk.restoreCandidates).not.toHaveBeenCalled(); + expect(sdk.drafts.create).toHaveBeenCalledWith({ appId: 'mealie', version: 'latest' }); + }); + + // ---- scenario 51 (HIGHEST-adjacent): every hint is built from details, never the message ---- + it('scenario 51: every RESTORE_* hint is built from details alone, with the message blanked', () => { + const cases: Array<{ code: string; details: Record; expect: string }> = [ + { code: 'RESTORE_SOURCE_NEWER', details: { candidateVersion: '2.0.0', targetVersion: '1.0.0' }, expect: '2.0.0' }, + { code: 'RESTORE_UPGRADE_PATH', details: { suggestedVersion: '1.5.0' }, expect: '1.5.0' }, + { code: 'RESTORE_ENV_REQUIRED', details: { missingKeys: ['DB_PASSWORD'] }, expect: 'DB_PASSWORD' }, + { code: 'RESTORE_ACK_REQUIRED', details: { required: ['restore-env-not-carried'] }, expect: '--ack restore-env-not-carried' }, + { code: 'RESTORE_CANDIDATE_GONE', details: {}, expect: '--restore-list' }, + { code: 'RESTORE_CANDIDATE_BUSY', details: {}, expect: '--restore-list' }, + { code: 'RESTORE_NOT_SUPPORTED', details: {}, expect: 'install-by-ref' }, + ]; + + for (const c of cases) { + const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + // Message deliberately blank — the hint must not depend on it at all. + const err = new HolaApiError('', 409, { details: { code: c.code, ...c.details } }); + reportDeployError(err); + const hintCall = errSpy.mock.calls.find(call => String(call[0]).startsWith('Hint:')); + expect(hintCall, `no Hint printed for ${c.code}`).toBeTruthy(); + expect(String(hintCall![0])).toContain(c.expect); + errSpy.mockRestore(); + } + }); +}); diff --git a/packages/cli/src/commands/install/install.ts b/packages/cli/src/commands/install/install.ts index c7312567..5fdb08f7 100644 --- a/packages/cli/src/commands/install/install.ts +++ b/packages/cli/src/commands/install/install.ts @@ -1,9 +1,9 @@ import { HolaSdk } from '@hola/sdk'; import { STABLE_CHANNEL } from '@hola/shared'; -import type { CreateDraftResponse, GetDraftResponse, AppEnvVar } from '@hola/shared'; +import type { CreateDraftResponse, GetDraftResponse, AppEnvVar, ListRestoreCandidatesResponse, RestoreCandidate, RestoreChoice } from '@hola/shared'; import { validateParams, generateSecretValue } from '@hola/shared/param-validate'; -import { finalizeAndDeploy, reportDeployError, type DeployResult } from '../../lib/deploy-flow'; +import { finalizeAndDeploy, reportDeployError, DeployAbort, type DeployResult } from '../../lib/deploy-flow'; import { maybeNotifyUpdate } from '../../lib/update-notice'; export interface InstallOptions { @@ -68,6 +68,24 @@ export interface InstallOptions { * both `--name` and `--as` are given, `--name` wins and a note is printed. */ as?: string; + /** + * Restore-on-install (spec 007). `restoreFrom` is `--restore-from ` (a + * candidate deployment id) or the literal string `latest`. `restore` is + * `false` only when `--no-restore` was passed — the explicit-intent flag; + * its ABSENCE (not passing any restore flag at all) is what FR-044 means by + * "the non-interactive default is no restore", so `restore === false` and + * `restoreFrom === undefined` are both "no restore", not two cases to + * reconcile. `restoreList` is `--restore-list` (list and exit; installs + * nothing). `carryEnv` is `true`/`false` only when `--carry-env`/ + * `--no-carry-env` was explicitly passed — `undefined` means "use the + * candidate's own `carriesEnv` as the default" (contracts/cli.md). + */ + restoreFrom?: string; + restore?: boolean; + restoreList?: boolean; + carryEnv?: boolean; + /** From `--ack ` (repeatable, or comma-separated) — mirrors `--grant`. */ + ack?: string | string[]; } /** Parse repeated/comma-separated `--profile` flags into a deduped key list. */ @@ -92,6 +110,101 @@ export function parseGrants(grant?: string | string[]): string[] | undefined { return [...new Set(refs)]; } +/** + * Parse repeated/comma-separated `--ack ` flags into a deduped + * acknowledgement-code list — the exact same shape/parsing as `--grant` + * (spec 007, contracts/cli.md), because a scripted install must acknowledge + * each specific risk deliberately: it can never satisfy an acknowledgement it + * did not name (SC-012). + */ +export function parseAcks(ack?: string | string[]): string[] | undefined { + if (ack === undefined) return undefined; + const raw = Array.isArray(ack) ? ack : [ack]; + const codes = raw.flatMap(a => String(a).split(',')).map(a => a.trim()).filter(Boolean); + return [...new Set(codes)]; +} + +/** Every candidate across every lineage in a candidates response, flattened. */ +function allCandidates(resp: ListRestoreCandidatesResponse): RestoreCandidate[] { + return resp.lineages.flatMap(l => l.candidates); +} + +/** + * Render `--restore-list`'s output (contracts/cli.md): one line per + * candidate, newest-first within its lineage, with a `!` warning line under + * any candidate that carries one — each warning names the exact flag that + * would satisfy it, so the operator's next command is on screen. + */ +function renderRestoreList(appId: string, resp: ListRestoreCandidatesResponse): string { + const lines: string[] = [`Restore candidates for ${appId}:`, '']; + for (const lineage of resp.lineages) { + for (const c of lineage.candidates) { + const captured = c.capturedAt ? new Date(c.capturedAt).toISOString().replace('T', ' ').slice(0, 16) : 'unknown'; + lines.push( + ` ${c.deploymentId} ${c.name} ${c.host ?? c.subdomain ?? '(no host)'} ${c.appVersion ? `v${c.appVersion}` : 'unknown version'} env: ${c.carriesEnv ? 'yes' : 'no'} ${captured}`, + ); + for (const w of c.warnings) { + if (w.code === 'env-not-carried') { + lines.push(` ! configuration cannot be carried: ${w.keys.join(', ')}`); + lines.push(` requires --ack restore-env-not-carried`); + } else if (w.code === 'no-identity-record') { + lines.push(` ! no install-identity record — described from the deployment record alone`); + } else if (w.code === 'host-divergence') { + lines.push(` ! host would change: ${w.from} → ${w.to}`); + } + } + if (c.skew.kind === 'unknown') { + lines.push(` ! version relationship unknown — requires --ack restore-version-unknown`); + } else if (c.skew.kind === 'refused') { + lines.push(` ! refused: ${c.skew.message}`); + } + } + } + const total = allCandidates(resp).length; + lines.push(''); + lines.push( + `${total} candidate${total === 1 ? '' : 's'} in ${resp.lineages.length} lineage${resp.lineages.length === 1 ? '' : 's'}.` + + (resp.defaultCandidateId ? ` Default: ${resp.defaultCandidateId}` : resp.requiresExplicitChoice ? ' No default — pick one explicitly with --restore-from .' : ''), + ); + return lines.join('\n'); +} + +/** + * Resolve `--restore-from ` (+ `--carry-env`/`--ack`) into a + * `RestoreChoice`, reading the candidates route to resolve `latest` and to + * default `carryEnv` from the chosen candidate's own `carriesEnv` + * (contracts/cli.md). `latest` REFUSES across two-or-more unrelated lineages + * (FR-036) — "latest" is ambiguous across unrelated histories, so this must + * fail rather than guess. + */ +async function resolveRestoreChoice( + sdk: HolaSdk, + appId: string, + version: string, + opts: InstallOptions, +): Promise { + const resp = (await sdk.restoreCandidates(appId, version, opts.source, opts.channel)) as ListRestoreCandidatesResponse; + + let candidateId: string; + if (opts.restoreFrom === 'latest') { + if (resp.requiresExplicitChoice || !resp.defaultCandidateId) { + throw new DeployAbort( + `--restore-from latest is ambiguous: ${resp.lineages.length} unrelated lineages match. ` + + `Pick one explicitly with --restore-from , or see them with --restore-list.`, + ); + } + candidateId = resp.defaultCandidateId; + } else { + candidateId = opts.restoreFrom!; + } + + const candidate = allCandidates(resp).find(c => c.deploymentId === candidateId); + const carryEnv = opts.carryEnv === false ? false : opts.carryEnv === true ? true : (candidate?.carriesEnv ?? false); + const acknowledge = parseAcks(opts.ack); + + return { candidateId, carryEnv, ...(acknowledge?.length ? { acknowledge } : {}) }; +} + /** * Heuristic: does this argument look like a full OCI reference (e.g. * `ghcr.io/acme/app:1.0`) rather than a catalog app id? True when it has a path @@ -159,9 +272,45 @@ export async function runInstall( if (opts.name && opts.as) out('Note: --name overrides --as'); const name = opts.name ?? opts.as ?? (isRef ? undefined : appId); + // Restore-on-install (spec 007): `--restore-list` reads the candidates + // route with NO draft created (research R6) and exits — it installs + // nothing, so it runs before any of the create-a-draft work below. + if (opts.restoreList) { + if (isRef) { + console.error('--restore-list needs a catalog app id, not an OCI reference.'); + process.exitCode = 1; + return undefined; + } + try { + const resp = (await sdk.restoreCandidates(appId, version, opts.source, opts.channel)) as ListRestoreCandidatesResponse; + if (opts.json) console.log(JSON.stringify(resp, null, 2)); + else console.log(renderRestoreList(appId, resp)); + return undefined; + } catch (err) { + return reportDeployError(err); + } + } + try { const overrides = parseSet(opts.set); + // Restore-on-install (spec 007, FR-044): the non-interactive default is + // NO restore — a candidate existing is not consent to use it. Only + // `--restore-from` (never `--no-restore`, which is the same "no restore" + // outcome stated explicitly) resolves an actual choice, and only on the + // catalog path (R2) — install-by-ref refuses client-side here rather + // than silently dropping it, matching the server's own fail-closed rule. + let restoreFrom: RestoreChoice | undefined; + if (opts.restoreFrom) { + if (isRef) { + console.error('Cannot restore on an install-by-ref install: no catalog index exists to judge the candidate\'s version against.'); + process.exitCode = 1; + return undefined; + } + out(`Resolving restore source '${opts.restoreFrom}'…`); + restoreFrom = await resolveRestoreChoice(sdk, appId, version, opts); + } + let draftId: string; if (isRef) { out(`Creating draft from OCI reference ${rawAppId}${opts.registryCred ? ` (credential: ${opts.registryCred})` : ''}`); @@ -169,7 +318,8 @@ export async function runInstall( } else { const from = opts.source && opts.source !== 'hola' ? ` (source: ${opts.source})` : ''; out(`Creating draft for ${appId}@${version} (from catalog${from})`); - draftId = ((await sdk.drafts.create({ appId, version, source: opts.source, channel: opts.channel })) as CreateDraftResponse).draftId; + draftId = ((await sdk.drafts.create({ appId, version, source: opts.source, channel: opts.channel, ...(restoreFrom ? { restoreFrom } : {}) })) as CreateDraftResponse).draftId; + if (restoreFrom) out(`Restoring from ${restoreFrom.candidateId} (carrying configuration: ${restoreFrom.carryEnv ? 'yes' : 'no'}).`); } // Merge `--set` overrides and auto-fill empty generate-recipe secrets onto diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 0ccbc830..ea5bb67f 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -161,6 +161,12 @@ prog .option('--set', 'Override an env var, KEY=VALUE (repeatable)') .option('--profile', 'Enable an optional Compose profile the app declares, e.g. elasticsearch (repeatable, or comma-separated)') .option('--grant', 'Consent to a privileged capability contract the app declares, e.g. backup@1 (repeatable, or comma-separated)') + .option('--restore-from', 'Restore from an existing deployment of this app: a deployment id, or "latest" (refuses across two+ unrelated lineages)') + .option('--no-restore', 'Explicitly install with no restore (the default when no restore flag is given at all)', false) + .option('--restore-list', 'List restore candidates for this app and exit — installs nothing', false) + .option('--carry-env', 'Carry the restore candidate\'s configuration (default: on when it has an environment record)') + .option('--no-carry-env', 'Do not carry the restore candidate\'s configuration (needs --ack restore-env-not-carried)', false) + .option('--ack', 'Acknowledge a restore risk by code, e.g. restore-env-not-carried (repeatable, or comma-separated)') .option('--allow-multiple', 'Install a second instance of a single-instance app (needs a distinct --name)', false) .option('--source', 'Catalog source id to install from (default: hola)') .option('--registry-cred', 'Stored registry credential id for a private OCI reference install') diff --git a/packages/cli/src/lib/deploy-flow.ts b/packages/cli/src/lib/deploy-flow.ts index e51202d8..131802d7 100644 --- a/packages/cli/src/lib/deploy-flow.ts +++ b/packages/cli/src/lib/deploy-flow.ts @@ -140,7 +140,18 @@ export function reportDeployError(err: unknown): undefined { // these flags itself), so the hint is built from `details`, not sniffed // from the message text. const details = err instanceof HolaApiError - ? (err.details as { code?: string; existing?: { id: string; name: string; channel: string }; channelPublished?: boolean } | undefined) + ? (err.details as { + code?: string; + existing?: { id: string; name: string; channel: string }; + channelPublished?: boolean; + // Restore-on-install (spec 007, contracts/cli.md). + candidateVersion?: string; + targetVersion?: string; + suggestedVersion?: string; + missingKeys?: string[]; + required?: string[]; + candidateId?: string; + } | undefined) : undefined; if (details?.code === 'ALREADY_INSTALLED' && details.existing) { const { id, name, channel } = details.existing; @@ -153,6 +164,34 @@ export function reportDeployError(err: unknown): undefined { `or force a second copy with '--allow-multiple --name ${name}-2'.`, ); } + // Restore-on-install refusals (spec 007): one branch per `RESTORE_*` code, + // hints built from `details` alone — never from the message, for the same + // reason as ALREADY_INSTALLED above (contracts/cli.md's mapping table). + else if (details?.code === 'RESTORE_SOURCE_NEWER') { + console.error( + `Hint: this candidate was captured on ${details.candidateVersion ?? 'a newer version'}, newer than ` + + `${details.targetVersion ?? 'the version being installed'}. Install ${details.candidateVersion ?? 'that version'} instead: ` + + `--app-version ${details.candidateVersion ?? ''}.`, + ); + } else if (details?.code === 'RESTORE_UPGRADE_PATH' && details.suggestedVersion) { + console.error( + `Hint: install ${details.suggestedVersion} first (--app-version ${details.suggestedVersion}), restore there, then promote.`, + ); + } else if (details?.code === 'RESTORE_ENV_REQUIRED') { + console.error( + `Hint: this app cannot restore without its captured configuration` + + (details.missingKeys?.length ? ` (${details.missingKeys.join(', ')})` : '') + + `. Pick a candidate with an environment record, or use --carry-env.`, + ); + } else if (details?.code === 'RESTORE_ACK_REQUIRED' && details.required?.length) { + console.error(`Hint: add ${details.required.map(c => `--ack ${c}`).join(' ')} to proceed.`); + } else if (details?.code === 'RESTORE_CANDIDATE_GONE' || details?.code === 'RESTORE_CANDIDATE_BUSY') { + console.error('Hint: the candidate changed since it was chosen. Re-read the current set with --restore-list.'); + } else if (details?.code === 'RESTORE_NOT_SUPPORTED') { + console.error('Hint: install-by-ref cannot restore — use the catalog install path instead.'); + } else if (details?.code === 'RESTORE_NOT_ACCEPTED') { + console.error('Hint: this app has not declared that it can be restored; install it fresh and move data in yourself.'); + } // #246: a single-instance app already installed (older/untyped error shape, // e.g. a pre-spec-005 server) — the message already names the // --allow-multiple escape hatch, but pair it with the distinct-name it needs. diff --git a/packages/sdk/src/index.ts b/packages/sdk/src/index.ts index 2321a57e..49a7f66f 100644 --- a/packages/sdk/src/index.ts +++ b/packages/sdk/src/index.ts @@ -35,7 +35,9 @@ import { // Job types DeleteJobsRequest, DeleteJobsResponse, // System types - GetUpdateCheckResponse + GetUpdateCheckResponse, + // Restore-on-install (spec 007) + ListRestoreCandidatesResponse } from '@hola/shared'; export type SdkInitOptions = { @@ -179,6 +181,19 @@ export class HolaSdk { installFromRef = (data: InstallFromRefRequest) => this.post(API.installFromRef, data); + // Restore-on-install (spec 007): deployments of `appId` on this host that can + // serve as a restore source. `version` (optional) judges each candidate's + // version skew/acknowledgements against a specific install target; omitted, + // every candidate's skew reports `unknown`. Reads with no draft created — + // powers both the wizard's first step and `hola install --restore-list`. + // `source`/`channel` ride along with `version` so the route resolves the SAME + // catalog version the draft this call precedes will be created with — without + // them a channel-pinned or alternate-source install is judged against the + // default source's `latest`, and the skew verdict the caller renders can + // disagree with the one `createDraft` enforces. + restoreCandidates = (appId: string, version?: string, source?: string, channel?: string) => + this.get(`${API.restoreCandidates(appId)}${buildQuery({ version, source, channel })}`); + drafts = { create: (data: CreateDraftRequest) => this.post(API.drafts.create, data), byId: (draftId: string) => this.get(API.drafts.byId(draftId)), diff --git a/packages/server/src/__tests__/deployments/restore-on-install.test.ts b/packages/server/src/__tests__/deployments/restore-on-install.test.ts new file mode 100644 index 00000000..6b8a51c1 --- /dev/null +++ b/packages/server/src/__tests__/deployments/restore-on-install.test.ts @@ -0,0 +1,1134 @@ +/** + * Restore-on-install from a live deployment (spec 007). + * + * Two groups: + * - "Pure resolver": `restore-candidates.ts`'s exported functions, exercised + * directly with fabricated data — no filesystem, no server. Covers the + * scenarios quickstart.md marks mode "U". + * - "Real filesystem harness": `RealDeploymentService` + `RealDraftService` + * over `RealStorageService` in two `mkdtemp` roots, `MockDockerService` + * (records `composeUp` calls — plan.md's "Known trap") and + * `MockProvisionerService`. Copied from `install-markers.test.ts:116-142` + * / `snapshot.test.ts:99-125`. `MockStorageService` cannot be used here — + * it discards file modes (#475) — and `MockDockerService` starts no real + * containers, so anything asserting on real files or real ordering needs + * this harness. Covers the scenarios marked "U-fs". + */ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdtemp, rm, mkdir, writeFile, readFile } from 'fs/promises'; +import { existsSync } from 'fs'; +import { tmpdir } from 'os'; +import { join } from 'path'; + +import type { + AppEnvVar, + AppBackupDeclaration, + AppRestoreDeclaration, + AppAuthConfig, + AppUpgradeMeta, + RestoreCandidate, + RestoreChoice, +} from '@hola/shared'; +import { checkUpgradePath, slugifySubdomain } from '@hola/shared'; + +import { RealDeploymentService } from '../../services/core/deployment'; +import { MockProvisionerService } from '../../services/core/provisioner'; +import { RealDraftService } from '../../services/core/draft'; +import { RealStorageService } from '../../services/core/storage'; +import { RealRoutingService } from '../../services/core/routing'; +import { RealDatabaseService } from '../../services/core/database'; +import { RealLoggingService } from '../../services/core/logging'; +import { RealJobService } from '../../services/core/jobs'; +import { MockDockerService } from '../../services/core/docker'; +import { dirHasContents } from '../../services/core/snapshot-fs'; +import { coerceManifestRestore } from '../../services/core/manifest-restore'; +import { + isEligibleCandidate, + describeCandidate, + groupIntoLineages, + computeSkewVerdict, + deriveEnvNotCarriedKeys, + deriveRequiredAcknowledgements, + validateRestoreChoice, + checkCandidateStillEligible, + judgeRestoreChoice, + resolveRestoreNameDefaults, + type CandidateSource, +} from '../../services/core/restore-candidates'; + +// --------------------------------------------------------------------------- +// Pure resolver — no filesystem, no server (quickstart.md scenarios marked "U") +// --------------------------------------------------------------------------- + +/** A settled, well-formed deployment record, only the fields the resolver reads. */ +function fakeDeployment(overrides: Partial = {}): CandidateSource['deployment'] { + return { + id: 'demoapp-aaaaaaaa', + name: 'Demo', + app: 'demoapp', + icon: '📦', + status: 'running', + resources: { cpu: '0%', memory: '0MB' }, + ports: [], + lastUpdated: new Date().toISOString(), + lifecycleState: 'active', + rollbackAvailable: false, + metadata: { createdAt: new Date().toISOString(), owner: 'system', tags: [] }, + ...overrides, + } as CandidateSource['deployment']; +} + +function fakeSource(overrides: Partial = {}): CandidateSource { + return { + deployment: fakeDeployment(), + identity: null, + hasData: true, + carriesEnv: false, + ...overrides, + }; +} + +describe('Restore-on-install (spec 007) — pure resolver', () => { + // ---- Scenario 5: lineage grouping/ordering, default/explicit choice ---- + test('scenario 5: candidates group by lineage, newest-first; two lineages need an explicit pick', () => { + const c = (id: string, lineageId: string, capturedAt: string | null): RestoreCandidate => ({ + deploymentId: id, + lineageId, + app: 'demoapp', + name: id, + subdomain: null, + host: null, + appVersion: '1.0.0', + channel: 'stable', + carriesEnv: false, + capturedAt, + hasIdentityRecord: true, + skew: { kind: 'ok' }, + requiredAcknowledgements: [], + warnings: [], + }); + + // Two lineages: no default, explicit choice required. + const twoLineages = groupIntoLineages([ + c('a1', 'lineage-a', '2026-01-01T00:00:00.000Z'), + c('a2', 'lineage-a', '2026-01-02T00:00:00.000Z'), + c('b1', 'lineage-b', '2026-01-01T00:00:00.000Z'), + ]); + expect(twoLineages.lineages).toHaveLength(2); + expect(twoLineages.requiresExplicitChoice).toBe(true); + expect(twoLineages.defaultCandidateId).toBeNull(); + // Newest-first within the lineage. + expect(twoLineages.lineages.find(l => l.lineageId === 'lineage-a')!.candidates.map(x => x.deploymentId)).toEqual(['a2', 'a1']); + + // One lineage: newest supplies the default. + const oneLineage = groupIntoLineages([ + c('x1', 'lineage-x', '2026-01-01T00:00:00.000Z'), + c('x2', 'lineage-x', '2026-01-03T00:00:00.000Z'), + ]); + expect(oneLineage.requiresExplicitChoice).toBe(false); + expect(oneLineage.defaultCandidateId).toBe('x2'); + }); + + // ---- Scenario 7: empty list is 200, not 404 (answered by the shape, not a status code here) ---- + test('scenario 7: no candidates yields an empty-but-valid grouping, not an error', () => { + const result = groupIntoLineages([]); + expect(result.lineages).toEqual([]); + expect(result.defaultCandidateId).toBeNull(); + expect(result.requiresExplicitChoice).toBe(false); + }); + + // ---- Scenario 34 (HIGHEST VALUE): checkUpgradePath cannot express "refuse a newer source" ---- + test('scenario 34 (highest value): checkUpgradePath(newer, older, meta) returns ok — the rule this feature adds independently', () => { + const meta: AppUpgradeMeta = { minFromVersion: '1.0.0' }; + // Documents WHY FR-029 is a rule of this feature's own, not delegated: + // checkUpgradePath treats a "from" newer than "to" as a downgrade, which + // it deliberately allows through for the promote path it was built for. + expect(checkUpgradePath('2.0.0', '1.0.0', meta)).toEqual({ ok: true }); + + // computeSkewVerdict evaluates the newer-than-target rule FIRST and + // therefore refuses exactly the case checkUpgradePath alone would miss. + const verdict = computeSkewVerdict('2.0.0', '1.0.0', meta); + expect(verdict).toMatchObject({ kind: 'refused', code: 'RESTORE_SOURCE_NEWER' }); + }); + + // ---- Scenario 35: a guarded older→target hop refuses with suggestedVersion ---- + test('scenario 35: a guarded upgrade path refuses with RESTORE_UPGRADE_PATH and a suggestedVersion', () => { + const meta: AppUpgradeMeta = { waypoints: ['1.5.0'] }; + const verdict = computeSkewVerdict('1.0.0', '2.0.0', meta); + expect(verdict).toMatchObject({ kind: 'refused', code: 'RESTORE_UPGRADE_PATH', suggestedVersion: '1.5.0' }); + }); + + // ---- Scenario 36: equal versions, and a clean older path, both proceed ---- + test('scenario 36: equal versions and a clean older-than-target path both verdict ok', () => { + const meta: AppUpgradeMeta = {}; + expect(computeSkewVerdict('1.0.0', '1.0.0', meta)).toEqual({ kind: 'ok' }); + expect(computeSkewVerdict('1.0.0', '2.0.0', meta)).toEqual({ kind: 'ok' }); + }); + + // ---- Scenario 37: unknown candidate version proceeds only with the acknowledgement ---- + test('scenario 37: an unknown version verdicts "unknown"; proceeding needs restore-version-unknown', () => { + expect(computeSkewVerdict(null, '1.0.0', {})).toEqual({ kind: 'unknown' }); + expect(computeSkewVerdict('1.0.0', undefined, {})).toEqual({ kind: 'unknown' }); + expect(computeSkewVerdict('1.0.0', '1.0.0', undefined)).toEqual({ kind: 'unknown' }); + + const required = deriveRequiredAcknowledgements({ skew: { kind: 'unknown' }, carryEnv: true, carriesEnv: true }); + expect(required).toEqual(['restore-version-unknown']); + + const candidate: RestoreCandidate = { + deploymentId: 'x', lineageId: 'x', app: 'demoapp', name: 'x', subdomain: null, host: null, + appVersion: null, channel: null, carriesEnv: true, capturedAt: null, hasIdentityRecord: true, + skew: { kind: 'unknown' }, requiredAcknowledgements: [], warnings: [], + }; + const withoutAck = validateRestoreChoice({ candidate, choice: { candidateId: 'x', carryEnv: true }, requiresEnv: false, envNotCarriedKeys: [] }); + expect(withoutAck).toMatchObject({ ok: false, code: 'RESTORE_ACK_REQUIRED', details: { required: ['restore-version-unknown'] } }); + + const withAck = validateRestoreChoice({ candidate, choice: { candidateId: 'x', carryEnv: true, acknowledge: ['restore-version-unknown'] }, requiresEnv: false, envNotCarriedKeys: [] }); + expect(withAck).toEqual({ ok: true, requiredAcknowledgements: ['restore-version-unknown'] }); + }); + + // ---- Scenario 39: requiresEnv refuses rather than warns ---- + test('scenario 39: requiresEnv + no environment record refuses (RESTORE_ENV_REQUIRED), never just warns', () => { + const candidate: RestoreCandidate = { + deploymentId: 'x', lineageId: 'x', app: 'demoapp', name: 'x', subdomain: null, host: null, + appVersion: '1.0.0', channel: null, carriesEnv: false, capturedAt: null, hasIdentityRecord: true, + skew: { kind: 'ok' }, requiredAcknowledgements: [], warnings: [], + }; + const result = validateRestoreChoice({ + candidate, + choice: { candidateId: 'x', carryEnv: true }, + requiresEnv: true, + envNotCarriedKeys: ['DB_PASSWORD'], + }); + expect(result).toMatchObject({ ok: false, code: 'RESTORE_ENV_REQUIRED', details: { missingKeys: ['DB_PASSWORD'] } }); + // Not acknowledgeable — no `acknowledge` array satisfies it. + const withAckAnyway = validateRestoreChoice({ + candidate, + choice: { candidateId: 'x', carryEnv: true, acknowledge: ['restore-env-not-carried', 'restore-version-unknown'] }, + requiresEnv: true, + envNotCarriedKeys: ['DB_PASSWORD'], + }); + expect(withAckAnyway).toMatchObject({ ok: false, code: 'RESTORE_ENV_REQUIRED' }); + }); + + // ---- Scenario 41: every refusal carries details.code (+ suggestedVersion where applicable) ---- + test('scenario 41: every refusal path carries details.code, in the same shape', () => { + const base: RestoreCandidate = { + deploymentId: 'x', lineageId: 'x', app: 'demoapp', name: 'x', subdomain: null, host: null, + appVersion: '2.0.0', channel: null, carriesEnv: true, capturedAt: null, hasIdentityRecord: true, + skew: { kind: 'refused', code: 'RESTORE_SOURCE_NEWER', message: 'newer' }, requiredAcknowledgements: [], warnings: [], + }; + const r1 = validateRestoreChoice({ candidate: base, choice: { candidateId: 'x', carryEnv: true }, requiresEnv: false, envNotCarriedKeys: [], targetVersion: '1.0.0' }); + expect(r1).toMatchObject({ ok: false, code: 'RESTORE_SOURCE_NEWER' }); + + const guarded: RestoreCandidate = { ...base, skew: { kind: 'refused', code: 'RESTORE_UPGRADE_PATH', message: 'guarded', suggestedVersion: '1.5.0' } }; + const r2 = validateRestoreChoice({ candidate: guarded, choice: { candidateId: 'x', carryEnv: true }, requiresEnv: false, envNotCarriedKeys: [] }); + expect(r2).toMatchObject({ ok: false, code: 'RESTORE_UPGRADE_PATH', details: { suggestedVersion: '1.5.0' } }); + + // Candidate gone / busy. + expect(checkCandidateStillEligible(undefined, 'demoapp', 'target-id')).toEqual({ ok: false, code: 'RESTORE_CANDIDATE_GONE' }); + expect(checkCandidateStillEligible(fakeSource({ deployment: fakeDeployment({ status: 'installing' }) }), 'demoapp', 'target-id')).toEqual({ ok: false, code: 'RESTORE_CANDIDATE_BUSY' }); + }); + + // ---- Scenario 23 (HIGHEST VALUE): MockDockerService records services/wait ---- + test('scenario 23 (highest value): MockDockerService.composeUp records services + wait; a scoped call starts nothing else', async () => { + const docker = new MockDockerService(); + await docker.composeUp('/tmp/proj', 'proj', undefined, undefined, { services: ['db'], wait: true, timeoutMs: 900_000 }); + expect(docker.composeUpCalls).toHaveLength(1); + expect(docker.composeUpCalls[0]).toEqual({ projectName: 'proj', services: ['db'], wait: true, timeoutMs: 900_000 }); + + // A full (unscoped) call — no services filter, no wait — is recorded distinctly. + await docker.composeUp('/tmp/proj', 'proj', undefined, undefined); + expect(docker.composeUpCalls).toHaveLength(2); + expect(docker.composeUpCalls[1].services).toBeUndefined(); + expect(docker.composeUpCalls[1].wait).toBeUndefined(); + }); + + // ---- Scenario 31: the restore hook shape is AppBackupHook verbatim; manifest-restore coercion is additive ---- + test('scenario 31: coerceManifestRestore accepts AppBackupHook-shaped hooks and drops malformed entries', () => { + const good = coerceManifestRestore([ + { id: 'default', discard: ['postgres'], hook: { service: 'db', command: ['sh', '-c', 'psql -f /backups/x.sql'] }, requiresEnv: true }, + ]); + expect(good).toEqual([ + { id: 'default', discard: ['postgres'], hook: { service: 'db', command: ['sh', '-c', 'psql -f /backups/x.sql'] }, requiresEnv: true }, + ]); + + // No `id` -> dropped. Malformed hook (no command) -> hook dropped, declaration survives. + const mixed = coerceManifestRestore([ + { discard: ['x'] }, + { id: 'ok', hook: { service: 'db' } }, + ]); + expect(mixed).toEqual([{ id: 'ok' }]); + + // Not an array at all -> undefined (no legacy singular form for `restore`). + expect(coerceManifestRestore({ id: 'default' })).toBeUndefined(); + expect(coerceManifestRestore(undefined)).toBeUndefined(); + }); + + // ---- Scenario 4 (partial, pure half): settled-status filtering ---- + test('scenario 4 (pure half): only running/stopped deployments are eligible, never installing/updating/error', () => { + for (const status of ['installing', 'updating', 'error'] as const) { + expect(isEligibleCandidate(fakeSource({ deployment: fakeDeployment({ status }) }), 'demoapp')).toBe(false); + } + for (const status of ['running', 'stopped'] as const) { + expect(isEligibleCandidate(fakeSource({ deployment: fakeDeployment({ status }) }), 'demoapp')).toBe(true); + } + }); + + // ---- Scenarios 54, 55: the scope boundary (FR-047, research R19) ---- + test('scenario 54: CONTRACTS is unchanged — exactly auth@1/backup@1/push@1/container-logs@1, no new grant kind', async () => { + const { CONTRACTS } = await import('@hola/shared/contracts'); + expect(CONTRACTS.map((c) => `${c.id}@${c.version}`).sort()).toEqual( + ['auth@1', 'backup@1', 'container-logs@1', 'push@1'].sort(), + ); + }); + + test('scenario 55: the pre-existing dead restore stub is untouched — nothing in this feature imports RestoreBackupRequest', async () => { + // Mechanical form of quickstart.md §9's grep: this feature's own new + // modules (not the pre-existing stub route itself, which legitimately + // names the type) must never reference it. + const featureModules = [ + new URL('../../services/core/restore-candidates.ts', import.meta.url), + new URL('../../services/core/manifest-restore.ts', import.meta.url), + ]; + for (const url of featureModules) { + const source = await readFile(url, 'utf8'); + expect(source).not.toContain('RestoreBackupRequest'); + expect(source).not.toContain('RestoreBackupResponse'); + } + }); + + // ---- Description fallback (research R5, FR-003): identity record wins, deployment record falls back ---- + test('describeCandidate: identity record fields win; absence falls back to the deployment record', () => { + const withIdentity = describeCandidate(fakeSource({ + identity: { lineageId: 'lineage-1', app: 'demoapp', appVersion: '2.0.0', channel: 'stable', subdomain: 'demo', host: 'demo.example.com', writtenAt: '2026-01-01T00:00:00.000Z' }, + })); + expect(withIdentity).toMatchObject({ lineageId: 'lineage-1', appVersion: '2.0.0', subdomain: 'demo', host: 'demo.example.com', hasIdentityRecord: true }); + + const withoutIdentity = describeCandidate(fakeSource({ identity: null, deployment: fakeDeployment({ version: '1.2.3' }) })); + expect(withoutIdentity).toMatchObject({ lineageId: 'demoapp-aaaaaaaa', appVersion: '1.2.3', host: null, hasIdentityRecord: false }); + }); + + // ---- env-not-carried keys: exactly isSecret && generate ---- + test('scenario 38 (pure half): env-not-carried names exactly isSecret+generate keys, nothing else', () => { + const appEnv: AppEnvVar[] = [ + { key: 'PLAIN', value: '', isSecret: false }, + { key: 'SECRET_NO_GEN', value: 'x', isSecret: true }, + { key: 'GENERATED_SECRET', value: '', isSecret: true, generate: { kind: 'hex' } }, + ]; + expect(deriveEnvNotCarriedKeys(appEnv)).toEqual(['GENERATED_SECRET']); + }); + + // ---- Scenario 54: scope boundary — judgeRestoreChoice composes the whole judgement from already-fetched state ---- + test('judgeRestoreChoice composes eligibility + skew + acknowledgement into one verdict', () => { + const source = fakeSource({ identity: { appVersion: '1.0.0' }, carriesEnv: true }); + const okResult = judgeRestoreChoice({ + source, appId: 'demoapp', excludeDeploymentId: 'target-1', + choice: { candidateId: source.deployment.id, carryEnv: true }, + targetVersion: '1.0.0', meta: {}, requiresEnv: false, envNotCarriedKeys: [], + }); + expect(okResult.ok).toBe(true); + + const goneResult = judgeRestoreChoice({ + source: undefined, appId: 'demoapp', excludeDeploymentId: 'target-1', + choice: { candidateId: 'nope', carryEnv: true }, + targetVersion: '1.0.0', meta: {}, requiresEnv: false, envNotCarriedKeys: [], + }); + expect(goneResult).toMatchObject({ ok: false, code: 'RESTORE_CANDIDATE_GONE' }); + }); +}); + +// --------------------------------------------------------------------------- +// Real filesystem harness (quickstart.md scenarios marked "U-fs") +// --------------------------------------------------------------------------- + +type CatalogArg = ConstructorParameters[1]; +type ValidationArg = ConstructorParameters[2]; + +const APP_ID = 'demoapp'; +const COMPOSE_WITH_DATA = + 'services:\n demoapp:\n image: demoapp:latest\n volumes:\n - ${HOLA_APP_DATA}:/data\n'; + +let defaultEnv: AppEnvVar[]; +let acceptsConfig: string[] | undefined; +let backupConfig: AppBackupDeclaration | undefined; +let restoreConfig: AppRestoreDeclaration[] | undefined; +let authConfig: AppAuthConfig | undefined; +// Present-but-empty by default (`{}`, not `undefined`): every install in this +// suite requests the SAME version as the source it restores from, so with +// upgrade metadata PRESENT, `computeSkewVerdict` reads an equal-version hop as +// `ok` rather than `unknown` — matching what a real catalog entry (which +// always has SOME upgrade block, even an empty one) would report. Tests that +// specifically exercise the unknown-version path set this to `undefined`. +let upgradeConfig: AppUpgradeMeta | undefined = {}; + +function makeCatalog(): CatalogArg { + return { + getApp: async (appId: string) => ({ id: appId, name: 'Demo App', icon: '🧪' }), + getVersionDetail: async () => ({ + defaultEnv, + defaults: { ports: [], volumes: [] }, + accepts: acceptsConfig, + backup: backupConfig, + restore: restoreConfig, + auth: authConfig, + upgrade: upgradeConfig, + multiInstance: true, // lets these tests install a second copy of the same app freely + }), + } as unknown as CatalogArg; +} + +function makeValidation(): ValidationArg { + return { + validateDraft: async () => ({ ok: true, errors: [], warnings: [] }), + preflightCheck: async () => ({ ok: true, checks: [] }), + } as unknown as ValidationArg; +} + +async function waitForJob(jobs: RealJobService, id: string, timeoutMs = 10_000) { + const start = Date.now(); + for (;;) { + const job = await jobs.getJob(id); + if (job && (job.status === 'completed' || job.status === 'failed')) return job; + if (Date.now() - start > timeoutMs) throw new Error(`Job ${id} did not finish (last status: ${job?.status})`); + await new Promise((r) => setTimeout(r, 10)); + } +} + +describe('Restore-on-install (spec 007) — real filesystem harness', () => { + let dataRoot: string; + let appsRoot: string; + let prevAppsBindRoot: string | undefined; + let docker: MockDockerService; + + beforeEach(async () => { + dataRoot = await mkdtemp(join(tmpdir(), 'hola-restore-data-')); + appsRoot = await mkdtemp(join(tmpdir(), 'hola-restore-apps-')); + prevAppsBindRoot = process.env.HOLA_APPS_BIND_ROOT; + process.env.HOLA_APPS_BIND_ROOT = appsRoot; + defaultEnv = []; + // Every app in this suite is restorable by default (declares `restore@1`) + // — the one scenario that needs "not declared" overrides this itself. + acceptsConfig = ['restore@1']; + backupConfig = undefined; + restoreConfig = undefined; + authConfig = undefined; + upgradeConfig = {}; + docker = new MockDockerService(); + }); + + afterEach(async () => { + if (prevAppsBindRoot === undefined) delete process.env.HOLA_APPS_BIND_ROOT; + else process.env.HOLA_APPS_BIND_ROOT = prevAppsBindRoot; + await rm(dataRoot, { recursive: true, force: true }); + await rm(appsRoot, { recursive: true, force: true }); + }); + + function makeSystem() { + const storage = new RealStorageService({ holaDir: dataRoot }); + const database = new RealDatabaseService(storage); + const logging = new RealLoggingService(storage); + const jobs = new RealJobService(database, logging); + const routing = new RealRoutingService(storage, { baseDomain: 'local.hola' }); + const drafts = new RealDraftService(storage, makeCatalog(), makeValidation()); + const deployments = new RealDeploymentService(storage, jobs, docker, drafts, routing, logging, new MockProvisionerService()); + // Restore-on-install (spec 007): the same post-construction wiring + // simple-factory.ts does — drafts needs deployments to resolve/validate a + // restoreFrom choice, and deployments takes drafts as a constructor arg, + // so this can't be threaded through either constructor. + drafts.setDeploymentsService(deployments); + return { storage, jobs, drafts, deployments }; + } + + async function install( + system: ReturnType, + opts: { name: string; version?: string; restoreFrom?: RestoreChoice; allowMultiple?: boolean }, + ) { + const { drafts, deployments, jobs } = system; + const { draftId } = await drafts.createDraft({ + appId: APP_ID, + version: opts.version ?? '1.0.0', + ...(opts.restoreFrom ? { restoreFrom: opts.restoreFrom } : {}), + }); + await drafts.updateDraft(draftId, { composeOverride: COMPOSE_WITH_DATA }); + await drafts.finalizeDraft(draftId); + const created = await deployments.createFromDraft({ + draftId, + name: opts.name, + options: { autoStart: true }, + allowMultiple: opts.allowMultiple ?? true, + }); + const job = created.jobId ? await waitForJob(jobs, created.jobId) : undefined; + return { ...created, job, draftId }; + } + + /** Write an extra file into a deployment's data root — the thing that makes + * it eligible as a restore candidate (data root holding more than `.hola`). */ + async function writeExtraData(deploymentId: string, name: string, content: string) { + const dir = join(appsRoot, deploymentId); + await mkdir(dir, { recursive: true }); + await writeFile(join(dir, name), content); + } + async function readExtraData(deploymentId: string, name: string): Promise { + return readFile(join(appsRoot, deploymentId, name), 'utf8'); + } + async function readInstanceRecord(deploymentId: string): Promise> { + const raw = await readFile(join(appsRoot, deploymentId, '.hola', 'instance.json'), 'utf8'); + return JSON.parse(raw) as Record; + } + + // ========================================================================= + // 1. Candidate discovery (scenarios 1-4, 6) + // ========================================================================= + + test('scenario 1: one installed copy holding data lists as exactly one candidate, fully described', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const sources = await system.deployments.listRestoreSources(APP_ID); + expect(sources).toHaveLength(1); + expect(sources[0]!.deployment.id).toBe(source.deploymentId); + expect(sources[0]!.hasData).toBe(true); + }); + + test('scenario 2: with the identity record deleted, the candidate is still listed, described from the deployment record', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await rm(join(appsRoot, source.deploymentId, '.hola', 'instance.json'), { force: true }); + + const sources = await system.deployments.listRestoreSources(APP_ID); + expect(sources).toHaveLength(1); + expect(sources[0]!.identity).toBeNull(); + const described = describeCandidate(sources[0]!); + expect(described.hasIdentityRecord).toBe(false); + expect(described.lineageId).toBe(source.deploymentId); // degrades to the deployment id + }); + + test('scenario 3: a data root holding only .hola is not listed (the ignore-list rule)', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + // No writeExtraData — the data root holds only what materializeCompose wrote. + + const sources = await system.deployments.listRestoreSources(APP_ID); + expect(sources.map(s => s.deployment.id)).not.toContain(source.deploymentId); + }); + + test('scenario 4: an in-flight or error-state deployment is excluded; running/stopped are included', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + expect((await system.deployments.listRestoreSources(APP_ID)).map(s => s.deployment.id)).toContain(source.deploymentId); + + await system.deployments.executeAction(source.deploymentId, { action: 'stop' }); + expect((await system.deployments.listRestoreSources(APP_ID)).map(s => s.deployment.id)).toContain(source.deploymentId); + }); + + test('scenario 6: the candidates route answers with no backup provider installed anywhere on the host', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + // No backupConfig/acceptsConfig set — no backup@1 anywhere on the host. + const sources = await system.deployments.listRestoreSources(APP_ID); + expect(sources).toHaveLength(1); + }); + + // ========================================================================= + // 2. Entering the choice (scenarios 8, 9, 11, 12, 13, 14) + // ========================================================================= + + test('scenario 8: restoreFrom is accepted on the catalog path and refused on install-by-ref with RESTORE_NOT_SUPPORTED', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + // Catalog path: accepted (does not throw). + const { draftId } = await system.drafts.createDraft({ + appId: APP_ID, + version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(draftId).toBeTruthy(); + + // Install-by-ref refuses — createDraftFromRef is exercised via ociRef. + await expect( + system.drafts.createDraft({ appId: undefined, ociRef: 'ghcr.io/acme/demoapp:1.0.0', restoreFrom: { candidateId: source.deploymentId, carryEnv: false } } as never), + ).rejects.toMatchObject({ code: 'CONFLICT', details: { code: 'RESTORE_NOT_SUPPORTED' } }); + }); + + // ---- data-model.md §7: "no restore@1 in accepts" is NOT the same as "restore@1 with no block" ---- + test('an app that has not declared restore@1 at all refuses restoreFrom, distinct from a plain-copy declaration', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + acceptsConfig = ['backup@1']; // declares backup, but never considered restore + await expect( + system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }), + ).rejects.toMatchObject({ code: 'CONFLICT', details: { code: 'RESTORE_NOT_ACCEPTED' } }); + }); + + test('scenario 11: two finalizes differing only in restoreFrom produce the SAME checksum', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const plain = await system.drafts.createDraft({ appId: APP_ID, version: '1.0.0' }); + await system.drafts.updateDraft(plain.draftId, { composeOverride: COMPOSE_WITH_DATA }); + const plainFinal = await system.drafts.finalizeDraft(plain.draftId); + + const restoring = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + await system.drafts.updateDraft(restoring.draftId, { composeOverride: COMPOSE_WITH_DATA }); + const restoringFinal = await system.drafts.finalizeDraft(restoring.draftId); + + // `spec` (the finalized manifest) already includes `draftId` — + // pre-existing, unrelated to this feature — so two DIFFERENT drafts can + // never hash equal regardless of `restoreFrom`. `restoreFrom` itself DOES + // appear on the returned manifest (it rides outside canonicalSpec, + // alongside `channel`) but must NOT be part of what got HASHED. Prove + // that narrower, stronger claim directly: with `draftId`, `checksum`, + // `finalizedAt` and `restoreFrom` itself stripped (the fields that + // legitimately/expectedly differ), everything else — the actual + // canonicalSpec content the checksum was computed over — matches exactly. + const strip = (spec: unknown) => { + const rest = { ...(spec as Record) }; + delete rest.draftId; + delete rest.checksum; + delete rest.finalizedAt; + delete rest.restoreFrom; + return rest; + }; + expect(strip(restoringFinal.spec)).toEqual(strip(plainFinal.spec)); + expect((restoringFinal.spec as { restoreFrom?: unknown }).restoreFrom).toBeTruthy(); + expect((plainFinal.spec as { restoreFrom?: unknown }).restoreFrom).toBeUndefined(); + }); + + test('scenario 12, 13: the record carries restoreFrom + lineageId; a fresh install\'s lineageId is its own id, a restored one is the candidate\'s', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const fresh = await install(system, { name: 'fresh' }); + const freshDetail = await system.deployments.getDeployment(fresh.deploymentId); + expect(freshDetail.lineageId).toBe(fresh.deploymentId); + expect(freshDetail.restoreFrom).toBeUndefined(); + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + const restoredDetail = await system.deployments.getDeployment(restored.deploymentId); + expect(restoredDetail.lineageId).toBe(source.deploymentId); + expect(restoredDetail.restoreFrom).toMatchObject({ candidateId: source.deploymentId }); + + // Confirm by reading .hola/instance.json too (FR-011, SC-007). + const identity = await readInstanceRecord(restored.deploymentId); + expect(identity.lineageId).toBe(source.deploymentId); + }); + + test('scenario 14: a restored deployment sets restoredAt; a later restart leaves it unchanged and quiesces nothing', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + const detail1 = await system.deployments.getDeployment(restored.deploymentId); + expect(detail1.restoredAt).toBeTruthy(); + + const { jobId } = await system.deployments.executeAction(restored.deploymentId, { action: 'restart' }); + await waitForJob(system.jobs, jobId!); + const detail2 = await system.deployments.getDeployment(restored.deploymentId); + expect(detail2.restoredAt).toBe(detail1.restoredAt); // unchanged — restart never re-enters the restore sequence + + // The source's data is untouched by the restart. + expect(await readExtraData(source.deploymentId, 'note.txt')).toBe('hello'); + }); + + // ========================================================================= + // 3. Executing the restore (scenarios 15, 16, 16a, 18, 19, 20, 21, 22, 25, 26, 27) + // ========================================================================= + + test('scenario 15: the restore runs between composePull and the final composeUp (Mock call ordering)', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + acceptsConfig = ['restore@1']; + restoreConfig = undefined; // plain-file-copy: no hook, so composeUp is called exactly once (the final one) + + // The SOURCE's own install already issued one composeUp on this shared + // Mock — only count calls made by the RESTORE install itself. + const before = docker.composeUpCalls.length; + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + // Exactly one composeUp — the restore's own extraction step runs + // between composePull and this call, never issuing its own composeUp + // when there's no hook to start. + const calls = docker.composeUpCalls.slice(before); + expect(calls).toHaveLength(1); + expect(calls[0]!.services).toBeUndefined(); + }); + + test('scenario 16: a non-empty target refuses RESTORE_TARGET_NOT_EMPTY; a marker-only root proceeds', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + // A marker-only target (freshly materialized, no extra data) proceeds — + // proven by every other restore test in this suite succeeding. The + // REFUSAL half is driven end-to-end below rather than by asserting the + // guard primitive: `autoStart: false` hands back the deployment id with + // NO job queued, which is the only window in which the target's data root + // can be seeded before `performRestoreOnInstall` looks at it. The deploy + // job is then enqueued by hand in exactly the shape `maybeStartJob` + // builds (`type: 'start'`, `action: 'deploy'`), so this is the real + // lifecycle path, not a re-implementation of it. + const { draftId } = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + await system.drafts.updateDraft(draftId, { composeOverride: COMPOSE_WITH_DATA }); + await system.drafts.finalizeDraft(draftId); + const created = await system.deployments.createFromDraft({ + draftId, name: 'target', options: { autoStart: false }, allowMultiple: true, + }); + expect(created.jobId).toBeUndefined(); + + // Pre-existing app data in the target's own root — the thing FR-014 + // exists to protect. `restoreTarGzInto` would `rm -rf` this whole tree. + await mkdir(join(appsRoot, created.deploymentId), { recursive: true }); + await writeFile(join(appsRoot, created.deploymentId, 'pre-existing.txt'), 'do not clobber me'); + + const job = await system.jobs.createJob({ + type: 'start', deploymentId: created.deploymentId, payload: { releaseId: created.releaseId, action: 'deploy' }, + }); + const finished = await waitForJob(system.jobs, job.id); + expect(finished.status).toBe('failed'); + expect(finished.error).toMatch(/already holds app data/i); + + // The refusal happened BEFORE extraction: the operator's data is intact + // and the source was never captured into the target. + expect(await readExtraData(created.deploymentId, 'pre-existing.txt')).toBe('do not clobber me'); + expect(existsSync(join(appsRoot, created.deploymentId, 'note.txt'))).toBe(false); + const detail = await system.deployments.getDeployment(created.deploymentId); + expect(detail.status).toBe('error'); + expect(detail.restoredAt).toBeUndefined(); + + const emptyRoot = join(appsRoot, 'marker-only'); + await mkdir(join(emptyRoot, '.hola'), { recursive: true }); + expect(await dirHasContents(emptyRoot, ['.hola'])).toBe(false); + }); + + // ========================================================================= + // Restore hook service names are APP-SUPPLIED (review, spec 007 target A) + // ========================================================================= + + test('a hostile restore hook service name never reaches a shell: coercion drops it, composeUp is argv-only', async () => { + // Source half: `coerceManifestRestore` refuses anything that could not + // name a real Compose service, so a manifest carrying a metacharacter + // degrades to "no hook" rather than an executable payload. + for (const hostile of ['db"; touch /tmp/pwned; echo "', 'db$(id)', 'db`id`', 'db; rm -rf /', 'db && id', 'db|id']) { + const coerced = coerceManifestRestore([{ id: 'default', hook: { service: hostile, command: ['true'] } }]); + expect(coerced?.[0]?.hook).toBeUndefined(); + } + // A legitimate name still survives. + expect( + coerceManifestRestore([{ id: 'default', hook: { service: 'db-1.primary_x', command: ['true'] } }])?.[0]?.hook, + ).toEqual({ service: 'db-1.primary_x', command: ['true'] }); + + // Sink half: even handed a name the coercion would have dropped, + // `composeUp` passes it as ONE argv element — never a shell string. + // `RealDockerService` is exercised here (the Mock cannot prove argv-ness); + // `docker` is absent in CI, so only the failure shape is asserted — what + // matters is that no side effect of the metacharacters is possible. + const { RealDockerService } = await import('../../services/core/docker'); + const real = new RealDockerService(); + const composeDir = join(appsRoot, 'argv-probe'); + await mkdir(composeDir, { recursive: true }); + await writeFile(join(composeDir, 'docker-compose.yml'), 'services: {}\n'); + const marker = join(appsRoot, 'argv-probe-pwned'); + const res = await real.composeUp(composeDir, 'probe', undefined, undefined, { + services: [`x"; touch ${marker}; echo "`], + }); + expect(res.success).toBe(false); + expect(existsSync(marker)).toBe(false); + }); + + test('scenario 16a: a candidate deleted between draft and deploy fails the install with RESTORE_CANDIDATE_GONE', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const { draftId } = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + await system.drafts.updateDraft(draftId, { composeOverride: COMPOSE_WITH_DATA }); + await system.drafts.finalizeDraft(draftId); + + // Delete the source deployment entirely between draft and deploy. + await system.deployments.deleteDeployment(source.deploymentId); + + // createFromDraft's OWN re-validation (T018) already re-resolves the + // candidate before creating any state, so this throws synchronously + // rather than failing inside the job. + await expect( + system.deployments.createFromDraft({ draftId, name: 'target', options: { autoStart: true }, allowMultiple: true }), + ).rejects.toMatchObject({ code: 'CONFLICT', details: { code: 'RESTORE_CANDIDATE_GONE' } }); + }); + + test('scenario 18 (HIGHEST VALUE): FR-016 is a post-condition — an emptied source archive fails RESTORE_PAYLOAD_EMPTY, not a subtree search', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + // Made eligible, then the extra data is removed — simulating a candidate + // that looked fine when chosen but whose data root holds only the + // platform marker by the time the job actually captures it. Neither + // draft-time nor job-time re-validation re-checks `hasData` (only + // existence/app/settledness), so this reaches the capture step for real. + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await rm(join(appsRoot, source.deploymentId, 'note.txt'), { force: true }); + expect(await dirHasContents(join(appsRoot, source.deploymentId), ['.hola'])).toBe(false); + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('failed'); + expect(restored.job?.error).toMatch(/empty/i); + + const detail = await system.deployments.getDeployment(restored.deploymentId); + expect(detail.status).toBe('error'); + }); + + test('scenario 19: staging lives under the TARGET, not the source, and is gone after success and failure', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + const stagingPath = join(dataRoot, 'deployments', restored.deploymentId, 'restore-staging', 'data.tar.gz'); + expect(existsSync(stagingPath)).toBe(false); + // Never appears in the SOURCE's own snapshot listing. + expect(existsSync(join(dataRoot, 'deployments', source.deploymentId, 'snapshots'))).toBe(false); + + // Failure case too. + const source2 = await install(system, { name: 'source2' }); + await writeExtraData(source2.deploymentId, 'note.txt', 'hello'); + await rm(join(appsRoot, source2.deploymentId, 'note.txt'), { force: true }); + const failed = await install(system, { + name: 'restored2', + restoreFrom: { candidateId: source2.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(failed.job?.status).toBe('failed'); + expect(existsSync(join(dataRoot, 'deployments', failed.deploymentId, 'restore-staging', 'data.tar.gz'))).toBe(false); + }); + + test('scenario 20: discard paths are removed before any container starts; an escaping path refuses the restore', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await mkdir(join(appsRoot, source.deploymentId, 'postgres'), { recursive: true }); + await writeFile(join(appsRoot, source.deploymentId, 'postgres', 'PG_VERSION'), '16'); + + acceptsConfig = ['restore@1', 'backup@1']; + restoreConfig = [{ id: 'default', discard: ['postgres'] }]; + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + expect(existsSync(join(appsRoot, restored.deploymentId, 'postgres'))).toBe(false); + expect(existsSync(join(appsRoot, restored.deploymentId, 'note.txt'))).toBe(true); + + // An escaping discard path refuses the restore. + const source2 = await install(system, { name: 'source3' }); + await writeExtraData(source2.deploymentId, 'note.txt', 'hello'); + restoreConfig = [{ id: 'default', discard: ['../escape'] }]; + const escaping = await install(system, { + name: 'restored3', + restoreFrom: { candidateId: source2.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(escaping.job?.status).toBe('failed'); + }); + + test('scenario 21: after a restore, .hola/instance.json describes the NEW install while lineageId is the source\'s', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + const identity = await readInstanceRecord(restored.deploymentId); + expect(identity.deploymentId).toBe(restored.deploymentId); + expect(identity.name).toBe('restored'); + expect(identity.lineageId).toBe(source.deploymentId); + }); + + test('scenario 22 (HIGHEST VALUE): the OIDC ordering trap — oidc.json exists after a restore, written AFTER extraction', async () => { + authConfig = { + mode: 'native-oidc', + oidc: { redirectPath: '/oidc/callback', scopes: ['openid'], credentialsFile: { path: 'oidc.json' } }, + }; + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + // If the write were still at its original pre-restore position, this + // file would have been destroyed by the restore's `rm -rf` extraction — + // it only survives because T022 moved the write to AFTER extraction. + expect(existsSync(join(appsRoot, restored.deploymentId, 'oidc.json'))).toBe(true); + const creds = JSON.parse(await readFile(join(appsRoot, restored.deploymentId, 'oidc.json'), 'utf8')) as { clientId?: string }; + expect(creds.clientId).toBeTruthy(); + // And the restored payload landed too — proving both writes coexist. + expect(existsSync(join(appsRoot, restored.deploymentId, 'note.txt'))).toBe(true); + }); + + test('scenario 25, 26: every restore failure leaves a failed install in error state with its data root intact, excluded from candidacy', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await rm(join(appsRoot, source.deploymentId, 'note.txt'), { force: true }); // forces RESTORE_PAYLOAD_EMPTY + + const failed = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(failed.job?.status).toBe('failed'); + const detail = await system.deployments.getDeployment(failed.deploymentId); + expect(detail.status).toBe('error'); + + // Data root is intact (nothing deleted automatically) — it still exists + // (holding only `.hola`, since the payload never landed). + expect(existsSync(join(appsRoot, failed.deploymentId))).toBe(true); + + // Excluded from candidacy thereafter. + const sources = await system.deployments.listRestoreSources(APP_ID); + expect(sources.map(s => s.deployment.id)).not.toContain(failed.deploymentId); + }); + + test('scenario 27 (HIGHEST VALUE, regression guard): with no restoreFrom, the deploy job is byte-for-byte what ran before this feature', async () => { + authConfig = { + mode: 'native-oidc', + oidc: { redirectPath: '/oidc/callback', scopes: ['openid'], credentialsFile: { path: 'oidc.json' } }, + }; + const system = makeSystem(); + const plain = await install(system, { name: 'plain' }); + expect(plain.job?.status).toBe('completed'); + + // Exactly ONE composeUp — the full, unscoped start. If the restore + // sequence had run (it must not, with no restoreFrom), there would be a + // SECOND, hook-scoped composeUp call before this one. + expect(docker.composeUpCalls).toHaveLength(1); + expect(docker.composeUpCalls[0]!.services).toBeUndefined(); + expect(docker.composeUpCalls[0]!.wait).toBeUndefined(); + + // writeOidcCredentialsFile ran at its ORIGINAL, unconditional position — + // unaffected by the restore feature's presence. + expect(existsSync(join(appsRoot, plain.deploymentId, 'oidc.json'))).toBe(true); + + const detail = await system.deployments.getDeployment(plain.deploymentId); + expect(detail.restoreFrom).toBeUndefined(); + expect(detail.restoredAt).toBeUndefined(); + expect(detail.lineageId).toBe(plain.deploymentId); + }); + + // ========================================================================= + // 4. App declaration (scenarios 29, 30) + // ========================================================================= + + test('scenario 29: a restore block keyed by participation id drives discards + hook for that participation', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await mkdir(join(appsRoot, source.deploymentId, 'pgdata'), { recursive: true }); + await writeFile(join(appsRoot, source.deploymentId, 'pgdata', 'PG_VERSION'), '16'); + + acceptsConfig = ['restore@1', 'backup@1']; + restoreConfig = [{ id: 'default', discard: ['pgdata'], hook: { service: 'demoapp', command: ['echo', 'restored'] } }]; + + const before = docker.composeUpCalls.length; + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + expect(existsSync(join(appsRoot, restored.deploymentId, 'pgdata'))).toBe(false); + // The hook's service was started scoped (services:['demoapp'], wait:true) + // BEFORE the final full composeUp. + const calls = docker.composeUpCalls.slice(before); + expect(calls.length).toBeGreaterThanOrEqual(2); + expect(calls[0]).toMatchObject({ services: ['demoapp'], wait: true }); + }); + + test('scenario 30: accepts restore@1 with NO block restores by plain file copy — nothing discarded, no hook runs', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await mkdir(join(appsRoot, source.deploymentId, 'anydir'), { recursive: true }); + await writeFile(join(appsRoot, source.deploymentId, 'anydir', 'x'), 'y'); + + acceptsConfig = ['restore@1']; + restoreConfig = undefined; // no block at all — the meaningful middle state + + const before = docker.composeUpCalls.length; + const restored = await install(system, { + name: 'restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(restored.job?.status).toBe('completed'); + // Nothing discarded — the whole tree landed. + expect(existsSync(join(appsRoot, restored.deploymentId, 'anydir', 'x'))).toBe(true); + // No hook-scoped composeUp — exactly one (the final, full) call. + expect(docker.composeUpCalls.slice(before)).toHaveLength(1); + }); + + // ========================================================================= + // 5. Refusals and warnings (scenarios 38, 40, 42, 43) + // ========================================================================= + + test('scenario 38: with no environment record, the warning names exactly the isSecret+generate keys', async () => { + defaultEnv = [ + { key: 'PLAIN', value: 'x', isSecret: false }, + { key: 'API_TOKEN', value: 'x', isSecret: true }, + { key: 'ADMIN_PASSWORD', value: '', isSecret: true, generate: { kind: 'hex' } }, + ]; + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + expect(deriveEnvNotCarriedKeys(defaultEnv)).toEqual(['ADMIN_PASSWORD']); + }); + + test('scenario 40: name/subdomain default from the candidate (pure); a divergent choice warns host-divergence (end-to-end)', async () => { + // The pure half: defaulting itself, isolated from the routing/collision + // machinery a real install exercises (which a SECOND live copy of the + // SAME app at the SAME address would correctly refuse — that's Traefik + // routing working as intended, not this rule failing). + const defaults = resolveRestoreNameDefaults({ + requestedName: undefined, + candidateName: 'Recipes', + candidateSubdomain: 'recipes', + appId: 'demoapp', + deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), + }); + expect(defaults).toEqual({ name: 'Recipes', subdomain: 'recipes', warnings: [] }); + + const diverging = resolveRestoreNameDefaults({ + requestedName: 'a-totally-different-name', + candidateName: 'Recipes', + candidateSubdomain: 'recipes', + appId: 'demoapp', + deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), + }); + expect(diverging.subdomain).toBe('a-totally-different-name'); + expect(diverging.warnings).toEqual([{ code: 'host-divergence', from: 'recipes', to: 'a-totally-different-name' }]); + + // End-to-end half: an explicit, divergent name against a real install + // produces the SAME warning on the actual create response. + const system = makeSystem(); + const source = await install(system, { name: 'source-recipes' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + const { draftId: draftId2 } = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + await system.drafts.updateDraft(draftId2, { composeOverride: COMPOSE_WITH_DATA }); + await system.drafts.finalizeDraft(draftId2); + const diverged = await system.deployments.createFromDraft({ draftId: draftId2, name: 'a-totally-different-name', options: { autoStart: true }, allowMultiple: true }); + expect(diverged.warnings).toBeTruthy(); + expect(diverged.warnings?.[0]).toMatchObject({ code: 'host-divergence' }); + await waitForJob(system.jobs, diverged.jobId!); + }); + + test('scenario 42: a required-and-absent acknowledgement fails the create with RESTORE_ACK_REQUIRED', async () => { + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + + // carryEnv: false with NO acknowledge -> RESTORE_ACK_REQUIRED (restore-env-not-carried required). + await expect( + system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false }, + }), + ).rejects.toMatchObject({ code: 'CONFLICT', details: { code: 'RESTORE_ACK_REQUIRED', required: ['restore-env-not-carried'] } }); + }); + + test('scenario 43: declining available configuration still warns and still requires the acknowledgement', async () => { + defaultEnv = [{ key: 'ADMIN_PASSWORD', value: '', isSecret: true, generate: { kind: 'hex' } }]; + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + // Write an env record so the candidate DOES carry configuration. + await mkdir(join(appsRoot, '.hola', source.deploymentId), { recursive: true, mode: 0o700 }); + await writeFile(join(appsRoot, '.hola', source.deploymentId, 'env.json'), JSON.stringify({ schema: 1, writtenAt: new Date().toISOString(), deploymentId: source.deploymentId, env: { ADMIN_PASSWORD: 'carried-value' } }), { mode: 0o600 }); + + // Decline it explicitly (carryEnv: false) despite it being available — + // still requires the acknowledgement. + await expect( + system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false }, + }), + ).rejects.toMatchObject({ code: 'CONFLICT', details: { code: 'RESTORE_ACK_REQUIRED', required: ['restore-env-not-carried'] } }); + + // With the acknowledgement, it proceeds — and the value is NOT carried + // (the operator's decline is honoured). + const { draftId } = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + const draft = await system.drafts.getDraft(draftId); + expect(draft.appEnv.find(e => e.key === 'ADMIN_PASSWORD')?.value).not.toBe('carried-value'); + }); + + // ========================================================================= + // US2: carrying configuration (scenario 10) + // ========================================================================= + + test('scenario 10: the three-case merge — carried wins, a new generate-recipe key mints, another rides through', async () => { + defaultEnv = [{ key: 'ADMIN_PASSWORD', value: '', isSecret: true, generate: { kind: 'hex' } }]; + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + await mkdir(join(appsRoot, '.hola', source.deploymentId), { recursive: true, mode: 0o700 }); + await writeFile( + join(appsRoot, '.hola', source.deploymentId, 'env.json'), + JSON.stringify({ schema: 1, writtenAt: new Date().toISOString(), deploymentId: source.deploymentId, env: { ADMIN_PASSWORD: 'carried-secret-value' } }), + { mode: 0o600 }, + ); + + const { draftId } = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: true }, + }); + const draft = await system.drafts.getDraft(draftId); + // Carried value equals the source's exactly (SC-003). + expect(draft.appEnv.find(e => e.key === 'ADMIN_PASSWORD')?.value).toBe('carried-secret-value'); + }); +}); diff --git a/packages/server/src/server.ts b/packages/server/src/server.ts index f2f6c195..95ae1bbf 100644 --- a/packages/server/src/server.ts +++ b/packages/server/src/server.ts @@ -58,7 +58,12 @@ import { type ListCatalogSourcesResponse, type RefreshCatalogResponse, type GetSubdomainAvailabilityResponse, + type ListRestoreCandidatesResponse, + type RestoreCandidate, + type AppUpgradeMeta, + type AppEnvVar, } from '@hola/shared'; +import { resolveListedCandidate, groupIntoLineages } from './services/core/restore-candidates'; // Error interface for proper typing interface ServiceError extends Error { @@ -512,6 +517,60 @@ async function route(url: URL, req: Request): Promise { } } + // Restore-on-install (spec 007): deployments of `appId` on this host that + // can serve as a restore source. Ordinary authenticated platform read — NOT + // a capability-contract broker endpoint (FR-047, contracts/api.md §0). + const restoreCandidatesMatch = pathname.match(/^\/api\/apps\/([^/]+)\/restore-candidates$/); + if (restoreCandidatesMatch && req.method === 'GET') { + const appId = decodeURIComponent(restoreCandidatesMatch[1]); + const targetVersion = searchParams.get('version') || undefined; + // Same (source, channel) the draft this read precedes will be created + // with, so both resolve the same concrete version (SDK comment above). + const targetSource = searchParams.get('source') || undefined; + const targetChannel = searchParams.get('channel') || undefined; + try { + const services = getServices(); + const sources = await services.deployments.listRestoreSources(appId); + + // `?version=` lets skew/acknowledgements be judged against a specific + // install target; omitted, every candidate's skew reports `unknown` + // (no target to compare against) — the route still answers 200 either + // way, never 404 (FR-042). + let meta: AppUpgradeMeta | undefined; + let appEnv: AppEnvVar[] = []; + // The version skew is actually judged against — the catalog's RESOLVED + // version, never the raw query value. `?version=latest` (what `hola + // install --restore-list` sends by default) is not a comparable version: + // `compareVersions` parses the word `latest` as `0.0.0`, so comparing a + // candidate against it reports every candidate as RESTORE_SOURCE_NEWER. + // `draft.ts` already resolves the same way (`defaults.resolvedVersion ?? + // request.version`), so the listing and the create-time refusal agree. + let effectiveTargetVersion = targetVersion; + if (targetVersion) { + try { + const detail = await services.catalog.getVersionDetail(appId, targetVersion, targetSource, targetChannel); + meta = detail.upgrade; + appEnv = detail.defaultEnv; + effectiveTargetVersion = detail.version ?? targetVersion; + } catch (error) { + logger.warn('Restore candidates: target version detail unavailable; skew reported as unknown', { + appId, targetVersion, error: error instanceof Error ? error.message : String(error), + }); + } + } + + const candidates: RestoreCandidate[] = sources.map((source) => + resolveListedCandidate(source, effectiveTargetVersion, meta, appEnv), + ); + const { lineages, defaultCandidateId, requiresExplicitChoice } = groupIntoLineages(candidates); + const response: ListRestoreCandidatesResponse = { appId, lineages, defaultCandidateId, requiresExplicitChoice }; + return json(response); + } catch (error) { + logger.warn('Restore candidates lookup failed', { appId, error: error instanceof Error ? error.message : String(error) }); + return errorResponse(req, error); + } + } + // Catalog refresh if (pathname === API.catalog.refresh && req.method === 'POST') { try { diff --git a/packages/server/src/services/core/catalog.ts b/packages/server/src/services/core/catalog.ts index 617cd9d6..fff12b37 100644 --- a/packages/server/src/services/core/catalog.ts +++ b/packages/server/src/services/core/catalog.ts @@ -33,6 +33,7 @@ import { coerceConsumes } from './app-registry'; import { coerceProvides, coerceAccepts, findUndeclaredAcceptorBlocks } from './contracts'; import { coerceManifestUpgrade } from './manifest-upgrade'; import { coerceManifestBackup } from './manifest-backup'; +import { coerceManifestRestore } from './manifest-restore'; import { coerceManifestPush } from './manifest-push'; import { validateParamSpec, PARAM_TYPES, GENERATE_KINDS } from '@hola/shared/param-validate'; import { BundleError, BundleUnavailableError, ValidationError, assertValidChannelName } from '../../middleware/error-mapping'; @@ -646,6 +647,7 @@ export class RealCatalogService implements CatalogService, HealthCheckable { security?: unknown; upgrade?: unknown; backup?: unknown; + restore?: unknown; push?: unknown; profiles?: unknown; ingress?: { service?: unknown; port?: unknown }; @@ -745,6 +747,11 @@ export class RealCatalogService implements CatalogService, HealthCheckable { // the only reader downstream). const backup = coerceManifestBackup(manifest.backup, this.logger, { appId, version }); + // Per-backup-participation restore declarations (spec 007). Coerced + // narrowly like `backup`; unlike `backup` there is no legacy singular + // form to accept — this field is new with this feature. + const restore = coerceManifestRestore(manifest.restore, this.logger, { appId, version }); + // Directories the app declares as pushable (#409). Coerced narrowly like // `auth`; the server resolves each `path` against the deployment's data // root (and proves containment) at push time. @@ -765,7 +772,7 @@ export class RealCatalogService implements CatalogService, HealthCheckable { ? manifest.ingress.service.trim() : undefined; - return { ...merged, version, composeOverride, auth, consumes, provides, accepts, multiInstance, security, upgrade, backup, push, profiles, ingressService } satisfies GetCatalogAppVersionDetailResponse; + return { ...merged, version, composeOverride, auth, consumes, provides, accepts, multiInstance, security, upgrade, backup, restore, push, profiles, ingressService } satisfies GetCatalogAppVersionDetailResponse; } catch (error) { this.logger.warn('Failed to read or parse bundle manifest', { version, error: error instanceof Error ? error.message : String(error) }); // Keep the underlying reason in the message, not just the cause: a missing diff --git a/packages/server/src/services/core/deployment.ts b/packages/server/src/services/core/deployment.ts index dd2efd3e..cef7dc10 100644 --- a/packages/server/src/services/core/deployment.ts +++ b/packages/server/src/services/core/deployment.ts @@ -50,8 +50,18 @@ import type { GetContractsResponse, AppBackupHook, AppAuthConfig, + RestoreWarning, } from '@hola/shared'; import { checkUpgradePath, isNewerVersion, slugifySubdomain, isEligibleOnChannel, newestEligibleVersion, STABLE_CHANNEL, type InstanceReason } from '@hola/shared'; +import { + isEligibleCandidate, + checkCandidateStillEligible, + deriveEnvNotCarriedKeys, + judgeRestoreChoice, + resolveRestoreNameDefaults, + type CandidateSource, + type RestoreIdentitySnapshot, +} from './restore-candidates'; import { requestsPrivilegeEscalation } from './manifest-security'; import { resolveContainedDir, isStrictlyInside } from './path-containment'; import { getHolaVersion } from './system-monitoring'; @@ -60,6 +70,7 @@ import { validateParams } from '@hola/shared/param-validate'; import { getLogger } from '../../lib/logger'; import { NotFoundError, ConflictError, ValidationError, DraftValidationError, ServiceError, assertValidChannelName } from '../../middleware/error-mapping'; import { dirHasContents, fileSize, tarGzipDir, restoreTarGzInto } from './snapshot-fs'; +import { rm } from 'node:fs/promises'; import type { HealthCheckable, ServiceHealth } from './types'; import type { StorageService } from './storage'; import type { JobService, JobContext } from './jobs'; @@ -126,6 +137,16 @@ const DEFAULT_INTERNAL_API_URL = 'http://hola-server:3001'; /** Default host base for per-app data roots when HOLA_APPS_BIND_ROOT is unset. */ const DEFAULT_APPS_BIND_ROOT = '/srv/hola/apps'; +/** + * `composeUp`'s `--wait` timeout for a restore hook's service (spec 007, + * research R12) — deliberately larger than `composeUp`'s default 5-minute + * `execAsync` cap. A freshly-`initdb`'d Postgres under `--wait` can exceed + * five minutes on a slow disk or a large `shared_buffers`; a restore that is + * slow-but-correct must not fail the whole install (FR-022) for a timeout the + * server chose, not the operator. + */ +const RESTORE_HOOK_WAIT_TIMEOUT_MS = 900_000; // 15 minutes + /** * Reserved locations for this feature's platform-authored JSON records (spec * 006). There are TWO, at two different levels under the apps bind root, and @@ -429,6 +450,17 @@ export interface DeploymentService extends HealthCheckable { // Internal management getDirectoryLayout(deploymentId: string): Promise; updateLifecycleState(deploymentId: string, state: DeploymentLifecycleState): Promise; + + // Restore-on-install (spec 007) + /** The raw candidate-source state for ONE deployment id (identity record + + * hasData + carriesEnv), NOT filtered by eligibility — the resolve step + * both draft creation (`draft.ts`) and the deploy job's job-time + * re-resolution (FR-013a) need. `undefined` when the id names no + * deployment at all. */ + getRestoreSource(deploymentId: string): Promise; + /** Every ELIGIBLE restore source for `appId` on this host (data-model.md + * §2), excluding `excludeDeploymentId` — used by the candidates route. */ + listRestoreSources(appId: string, excludeDeploymentId?: string): Promise; } // --------------------------------------------------------------------------- @@ -478,6 +510,11 @@ function toDetailResponse(d: EnhancedDeploymentDetail): GetDeploymentResponse { // Whether this app's manifest declares itself multi-instance (spec 005); // absent (never `false`) means single-instance. See toListItem. ...(d.multiInstance ? { multiInstance: true } : {}), + // Restore-on-install (spec 007, contracts/api.md §4): three additive + // fields, all optional so a pre-spec-007 record reads them as `undefined`. + ...(d.lineageId ? { lineageId: d.lineageId } : {}), + ...(d.restoreFrom ? { restoreFrom: d.restoreFrom } : {}), + ...(d.restoredAt ? { restoredAt: d.restoredAt } : {}), }; } @@ -756,6 +793,38 @@ abstract class InMemoryDeploymentService implements DeploymentService { void fromVersion; } + /** + * Restore-on-install (spec 007): load the filesystem-derived parts of a + * candidate description for ONE deployment — its parsed identity record (or + * `null`), whether its data root holds app data, and whether it carries an + * environment record. Default is a no-filesystem answer (the mock/in-memory + * service has no host data); RealDeploymentService overrides this to read + * `.hola/instance.json` and the sibling env record. Never throws: a + * corrupt/unparseable identity record degrades to `null` (FR-003 — the + * candidate is still offered, described from the deployment record alone). + */ + protected async loadCandidateSource(deployment: EnhancedDeploymentDetail): Promise { + return { deployment, identity: null, hasData: false, carriesEnv: false }; + } + + /** @inheritdoc */ + async getRestoreSource(deploymentId: string): Promise { + await this.ensureLoaded(); + const deployment = this.deployments.get(deploymentId); + if (!deployment) return undefined; + return this.loadCandidateSource(deployment); + } + + /** @inheritdoc */ + async listRestoreSources(appId: string, excludeDeploymentId?: string): Promise { + await this.ensureLoaded(); + const all = Array.from(this.deployments.values()).filter( + (d) => d.app === appId && d.id !== excludeDeploymentId, + ); + const sources = await Promise.all(all.map((d) => this.loadCandidateSource(d))); + return sources.filter((source) => isEligibleCandidate(source, appId, excludeDeploymentId)); + } + /** * Preflight the app's declared auth requirement against the active auth backend * before any deployment state is created (RealDeploymentService overrides this @@ -938,6 +1007,42 @@ abstract class InMemoryDeploymentService implements DeploymentService { // hit, but up front so the user gets the clear error instead of a tombstone. this.assertAuthProvisionable(artifacts?.manifest.auth, app); + // Restore-on-install (spec 007, FR-010): re-validate the choice before + // any deployment state is created. The candidate may have been deleted + // or started a lifecycle action since the draft was made (draft.ts + // already validated it once, at draft-creation time — this is NOT + // redundant, it's the same check run again against fresher state). + // `restoreLineageId` carries the resolved candidate's lineage onto the + // deployment record below (T009/R4); absent a restore, a fresh + // install's lineage is its own id. + let restoreLineageId: string | undefined; + // FR-035: the new install's `name`/`subdomain` default from the + // candidate when the operator supplied no explicit name. + let restoreCandidateName: string | undefined; + let restoreCandidateSubdomain: string | null | undefined; + const restoreFrom = artifacts?.manifest.restoreFrom; + if (restoreFrom) { + const source = await this.getRestoreSource(restoreFrom.candidateId); + const requiresEnv = (artifacts?.manifest.restore ?? []).some((d) => d.requiresEnv === true); + const envNotCarriedKeys = deriveEnvNotCarriedKeys(artifacts?.manifest.appEnv ?? []); + const judged = judgeRestoreChoice({ + source, + appId: app, + excludeDeploymentId: deploymentId, + choice: restoreFrom, + targetVersion: version, + meta: artifacts?.manifest.upgrade, + requiresEnv, + envNotCarriedKeys, + }); + if (!judged.ok) { + throw new ConflictError(judged.message, { code: judged.code, ...judged.details }); + } + restoreLineageId = judged.candidate.lineageId; + restoreCandidateName = judged.candidate.name; + restoreCandidateSubdomain = judged.candidate.subdomain; + } + // Release channel this deployment follows (#428): copied from the // finalized manifest, which already resolved it (explicit request, // implied by a pinned version, or `stable`) at draft-create time — no @@ -986,8 +1091,26 @@ abstract class InMemoryDeploymentService implements DeploymentService { // declaration, so it can never widen what the manifest asked for. const grantedContracts = resolveGrantedContracts(artifacts?.manifest.provides, request.grants); - // The DNS label this deployment routes under; stable for the install's life. - const subdomain = deriveSubdomain(request.name, app); + // The DNS label this deployment routes under; stable for the install's + // life. Restore-on-install (spec 007, FR-035): with no explicit + // `request.name`, default straight to the candidate's own subdomain + // slug rather than re-deriving one from its display name — it's + // already a valid slug. A `host-divergence` warning names it when the + // OPERATOR's own choice (an explicit name, or one that derives to a + // different slug) diverges from the candidate's. `resolveRestoreNameDefaults` + // is the pure form of this rule (restore-candidates.ts) — unaffected + // installs skip it entirely and keep today's plain `deriveSubdomain` call. + const restoreNameDefaults = restoreFrom + ? resolveRestoreNameDefaults({ + requestedName: request.name, + candidateName: restoreCandidateName ?? app, + candidateSubdomain: restoreCandidateSubdomain ?? null, + appId: app, + deriveSubdomain, + }) + : undefined; + const subdomain = restoreNameDefaults?.subdomain ?? deriveSubdomain(request.name, app); + const restoreWarnings: RestoreWarning[] = restoreNameDefaults?.warnings ?? []; // Compose profiles to activate for this install (#162): the operator's // requested set intersected with what the manifest declares, or the declared @@ -1007,11 +1130,13 @@ abstract class InMemoryDeploymentService implements DeploymentService { const deployment: EnhancedDeploymentDetail = { id: deploymentId, - // Default to the catalog product name (e.g. "Uptime Kuma"), falling back - // to the app slug — so the UI shows a readable app name without a live - // catalog lookup, never an opaque "deployment-". A caller-supplied - // name wins. - name: request.name || artifacts?.manifest.displayName || app, + // A caller-supplied name always wins. Absent one, restore-on-install + // (spec 007, FR-035) defaults to the CANDIDATE's name — a more useful + // default than the generic catalog product name for a copy of a + // specific existing install. Falls back to the catalog product name + // (e.g. "Uptime Kuma"), then the app slug, so the UI always shows a + // readable name, never an opaque "deployment-". + name: request.name || restoreCandidateName || artifacts?.manifest.displayName || app, app, // The routed DNS label (#246); reconciled from here, never recomputed from // the name — so the URL stays put even if the display name later changes. @@ -1026,6 +1151,15 @@ abstract class InMemoryDeploymentService implements DeploymentService { // Release channel this deployment follows (#428); always written for a // new record (read as `stable` for pre-feature records with none). channel, + // Restore-on-install (spec 007, R4): the candidate's lineage on a + // restore, else this deployment's own id — always written for a new + // record (never left `undefined`), so `writeInstanceMarkers`' fallback + // (`deployment.lineageId ?? deployment.id`) is only ever exercised by + // a record persisted BEFORE this feature. + lineageId: restoreLineageId ?? deploymentId, + // The restore choice actually applied (spec 007), re-validated above. + // Absent for a fresh install. + ...(restoreFrom ? { restoreFrom } : {}), // Why this is a permitted second copy of a single-instance app (#428); // absent for a first copy or a multi-instance app. ...(instanceReason ? { instanceReason } : {}), @@ -1075,7 +1209,7 @@ abstract class InMemoryDeploymentService implements DeploymentService { // `channel` (#428) so the CLI can print "Following channel: " without a // further lookup — covers both an explicit --channel and one implied by a // pinned pre-release version. - return { deploymentId, releaseId, jobId, channel }; + return { deploymentId, releaseId, jobId, channel, ...(restoreWarnings.length ? { warnings: restoreWarnings } : {}) }; } catch (error) { this.logger.error('Failed to create deployment from draft', error as Error, { deploymentId, @@ -2475,6 +2609,47 @@ export class RealDeploymentService extends InMemoryDeploymentService { return `${this.appsBindRoot()}/${INSTALL_ENV_ROOT_DIR}/${deploymentId}`; } + /** + * Restore-on-install (spec 007): read the filesystem-derived parts of one + * deployment's candidate description — `.hola/instance.json` (identity + * record first, research R5), whether the data root holds app data (the + * same `dirHasContents(..., [INSTALL_MARKERS_DIR])` rule + * `capturePreUpgradeSnapshot` applies), and whether the sibling env record + * exists. Never throws: an unreadable/malformed identity record degrades to + * `identity: null` rather than failing candidate discovery (FR-003). + */ + protected override async loadCandidateSource(deployment: EnhancedDeploymentDetail): Promise { + const appRoot = this.appRootFor(deployment.id); + const hasData = await dirHasContents(appRoot, [INSTALL_MARKERS_DIR]); + + let identity: RestoreIdentitySnapshot | null = null; + const identityPath = `${appRoot}/${INSTALL_MARKERS_DIR}/${INSTANCE_RECORD_FILE}`; + if (await this.storageService.fileExists(identityPath)) { + try { + const raw = JSON.parse(await this.storageService.readFileAsString(identityPath)) as Partial; + identity = { + lineageId: raw.lineageId, + app: raw.app, + appVersion: raw.appVersion ?? null, + channel: raw.channel ?? null, + subdomain: raw.subdomain ?? null, + host: raw.host ?? null, + writtenAt: raw.writtenAt ?? null, + }; + } catch (error) { + this.logger.warn('Unreadable install identity record; candidate described from the deployment record alone', { + deploymentId: deployment.id, + error: error instanceof Error ? error.message : String(error), + }); + } + } + + const envPath = `${this.envRecordDirFor(deployment.id)}/${ENV_RECORD_FILE}`; + const carriesEnv = await this.storageService.fileExists(envPath); + + return { deployment, identity, hasData, carriesEnv }; + } + // ---- Pre-upgrade snapshots (#284 Phase 1) -------------------------------- private snapshotsDir(deploymentId: string): string { @@ -3363,15 +3538,15 @@ export class RealDeploymentService extends InMemoryDeploymentService { writtenBy: getHolaVersion(), writtenAt: new Date().toISOString(), deploymentId: deployment.id, - // Derived, not persisted (spec FR-008, FR-010; data-model.md - // §Lineage Identifier): it always equals `deploymentId` today, so it - // needs no storage of its own — Sequence 5 (restore-on-install) is - // what forces a `lineageId` onto the deployment record, at which - // point this expression becomes `deployment.lineageId ?? deployment.id`. - // Writing it now means captures taken before that ships already - // carry the field. The platform MUST be its sole writer: no settable - // field, request parameter, manifest field, or operator input. - lineageId: deployment.id, + // Sequence 5 (restore-on-install, spec 007) has now shipped and forced + // a `lineageId` onto the deployment record (`createFromDraft`, R4): + // the candidate's lineage on a restore, else the deployment's own id. + // The `??` fallback is what made this a zero-migration change — a + // record persisted before spec 007 reads `lineageId` as `undefined` + // and yields exactly the value this expression always wrote. The + // platform remains its sole writer: no settable field, request + // parameter, manifest field, or operator input. + lineageId: deployment.lineageId ?? deployment.id, app: deployment.app, appVersion: manifest?.version ?? deployment.version ?? null, // `channel` INVERTS the manifest-wins order the two fields either side @@ -3699,6 +3874,250 @@ export class RealDeploymentService extends InMemoryDeploymentService { await logBoth('info', `Auth: wrote OIDC credentials file ${rel} for the bundle to render`); } + /** + * Restore-on-install (spec 007): the one-time restore sequence, run inside + * the deploy job's `deploy / start / rollback` branch — after the + * cancellation check, before `composeUp` brings up the rest of the app + * (research R8). Ten steps, each independently justified in R8 / + * data-model.md §8: + * + * 1. assert the target root holds no app data (FR-014) + * 2. re-resolve the candidate — it may have changed since the draft (FR-013a) + * 3. quiesce + capture the source (FR-015, R14) + * 4. extract into the target root (R7 — root-relative, no intermediate copy) + * 5. assert the payload actually landed (FR-016, R9 — a post-condition, not a subtree search) + * 6. apply `discard` paths (FR-017, R13) + * 7. rewrite `.hola/instance.json` (FR-018, R11 — extraction destroyed it) + * 8. write the OIDC credentials file (FR-019, R10 — after extraction, not before) + * 9. start each hook's service with `--wait` (FR-020, R12) + * 10. run each participation's restore hook, fail-closed (FR-021) + * + * The caller gates on `deployment.restoreFrom && !deployment.restoredAt && + * !deployment.previousReleaseId` — the two independent guards data-model.md + * §8 requires: a restore applies to the deployment's FIRST deploy only + * (`!previousReleaseId`, set by `promoteRelease` before this job ever + * starts — a promote/rollback always has one), and exactly once + * (`!restoredAt`). Either guard alone is enough in the happy path; both + * together mean a retried job after a partial failure cannot re-quiesce a + * live source. Sets `deployment.restoredAt` on success; throws + * `ConflictError` (a `RESTORE_*` `details.code`) on any refusal, which fails + * the whole install (FR-022) — there is no partial-success start. + */ + private async performRestoreOnInstall( + deployment: EnhancedDeploymentDetail, + composeDir: string, + projectName: string, + registryAuth: PullCredentials[] | undefined, + provisioned: { credentials?: ProvisionCredentials; auth: NonNullable } | null, + logBoth: (level: 'info' | 'warn' | 'error' | 'debug', message: string) => Promise, + ): Promise { + const choice = deployment.restoreFrom!; + const targetAppRoot = this.appRootFor(deployment.id); + + // Step 1 (FR-014): the ignore-list is the same rule candidate eligibility + // uses (research R5) — the `.hola` marker the platform itself just wrote + // via `materializeCompose` must not count as "already holds app data". + if (await dirHasContents(targetAppRoot, [INSTALL_MARKERS_DIR])) { + throw new ConflictError( + `Cannot restore into '${deployment.name}': its data root already holds app data.`, + { code: 'RESTORE_TARGET_NOT_EMPTY', deploymentId: deployment.id }, + ); + } + + // Step 2 (FR-013a): re-resolve rather than trust the draft-time choice — + // the candidate may have been deleted or started a lifecycle action since. + const source = await this.getRestoreSource(choice.candidateId); + const eligibility = checkCandidateStillEligible(source, deployment.app, deployment.id); + if (!eligibility.ok) { + throw new ConflictError( + `Restore source '${choice.candidateId}' is no longer available (${eligibility.code}).`, + { code: eligibility.code, candidateId: choice.candidateId }, + ); + } + const sourceDeployment = source!.deployment; + await logBoth('info', `Restoring app data from '${sourceDeployment.name}' (${sourceDeployment.id})…`); + + // Step 3 (FR-015, R14): quiesce + capture the source, reusing the SAME + // fail-closed pre/post-hook policy `capturePreUpgradeSnapshot` applies — + // read from the SOURCE's own currently-active release. + const sourceReleaseId = sourceDeployment.currentReleaseId; + const sourceManifest = sourceReleaseId ? await this.readReleaseManifest(sourceDeployment.id, sourceReleaseId) : undefined; + // A STOPPED source is an eligible candidate (`SETTLED_STATUSES`), and it + // has no containers to `compose exec` into — every hook would fail and, + // being fail-closed, would fail the whole install for an app that needs no + // quiescing at all: its files aren't changing underneath us, so a plain + // file copy is already consistent. This is exactly the rule + // `backupParticipants()` applies for the same reason, logged rather than + // silent so "no hook ran" is never something the operator has to infer. + const sourceIsRunning = sourceDeployment.status === 'running'; + if (!sourceIsRunning && backupParticipations(sourceManifest?.backup).length > 0) { + await logBoth('info', `Restore source '${sourceDeployment.name}' is stopped; skipping its quiesce hooks (its files are static).`); + } + const participants: BackupParticipant[] = sourceIsRunning + ? backupParticipations(sourceManifest?.backup).map((p) => ({ + deploymentId: sourceDeployment.id, + participationId: p.id, + preHook: p.preHook, + postHook: p.postHook, + })) + : []; + const prepared = await this.runPreHooksFailClosed(participants, (level, message) => { + if (level === 'error') this.logger.warn(message, { deploymentId: deployment.id }); + else this.logger.info(message, { deploymentId: deployment.id }); + }); + if (!prepared.ok && prepared.failed) { + // Cleanup runs the postHook of every STARTED participation only, same as capturePreUpgradeSnapshot. + await this.runPostHooks(prepared.started); + throw new ConflictError( + `Restore source quiesce failed (${prepared.failed.participationId}): ${prepared.failed.output ?? 'no output'}`, + { code: 'RESTORE_HOOK_FAILED', participationId: prepared.failed.participationId }, + ); + } + + const sourceAppRoot = this.appRootFor(sourceDeployment.id); + // Staging lives under the TARGET deployment's own directory (research + // R7) — distinct from the pre-upgrade snapshot store, so it never enters + // the SOURCE's `pruneSnapshots` retention or shows up in its snapshot + // listing, and is found next to the failed install if anything leaks. + // Deleted in a `finally` on success and failure alike. + const stagingRelDir = `deployments/${deployment.id}/restore-staging`; + const stagingPath = this.storageService.resolveHolaPath('deployments', deployment.id, 'restore-staging', 'data.tar.gz'); + + try { + try { + // `ensureDir` is INSIDE this try, not above it: every preHook has + // already run by now (the source is quiesced — a `pg_dump` written, a + // maintenance mode entered), so anything that can throw between the + // preHooks and the capture must still reach the `postHook` cleanup + // below, or the source is left quiesced with nothing to un-quiesce it. + await this.storageService.ensureDir(stagingRelDir); + await tarGzipDir(sourceAppRoot, stagingPath); + } finally { + // postHook (clean up the dump) always runs, mirroring capturePreUpgradeSnapshot. + await this.runPostHooks(participants); + } + + // Step 4 (R7): extract straight into the target root. Root-relative on + // both sides (`tar -C .` in, `-C ` out) — no intermediate + // extracted copy, so peak additional disk cost is one compressed + // archive (SC-013). This `rm -rf`s the target root, destroying the + // marker `materializeCompose` just wrote — step 7 rewrites it. + await restoreTarGzInto(stagingPath, targetAppRoot); + + // Step 5 (FR-016, R9): a POST-CONDITION, not a subtree search. This + // codebase's archives are root-relative in both directions (unlike a + // provider archive tool, which reproduces the source's absolute path + // under the target) — there is no nested path to locate, only an + // outcome to assert: the payload must actually be there. + if (!(await dirHasContents(targetAppRoot, [INSTALL_MARKERS_DIR]))) { + throw new ConflictError( + `Restore produced no data: the source's archive was empty.`, + { code: 'RESTORE_PAYLOAD_EMPTY', deploymentId: deployment.id }, + ); + } + + // Step 6 (FR-017, R13): the app's OWN declaration for the release being + // installed, keyed by backup participation id. Every path resolved + // through `resolveContainedDir`, exactly as push targets already are + // (`buildPushTargets` above) — one that escapes REFUSES the restore + // rather than being silently skipped, because this is app-supplied data + // naming a filesystem path for deletion. + const manifest = await this.readActiveManifest(deployment); + const restoreDeclarations = manifest?.restore ?? []; + for (const decl of restoreDeclarations) { + for (const relPath of decl.discard ?? []) { + const resolved = resolveContainedDir(targetAppRoot, relPath); + // `resolveContainedDir` proves containment but treats the root + // itself as contained (`isInside` returns true for `candidate === + // root`), which is fine for a push TARGET and catastrophic for a + // `rm -rf`: a manifest declaring `discard: ["."]` or `["./"]` would + // delete the entire just-restored payload and the marker with it, + // AFTER the payload post-condition above has already passed. So the + // delete also demands STRICT containment — the distinction + // `isStrictlyInside` exists for (path-containment.ts). + if (!resolved || !isStrictlyInside(targetAppRoot, resolved)) { + throw new ConflictError( + `Restore discard path escapes the data root: '${relPath}' (participation '${decl.id}').`, + { code: 'RESTORE_HOOK_FAILED', participationId: decl.id }, + ); + } + await rm(resolved, { recursive: true, force: true }); + } + } + + // Step 7 (FR-018, R11): rewrite the instance marker. Two independent + // reasons, either sufficient: extraction just destroyed it, AND the + // restored tree carries the SOURCE's record (its deploymentId, name, + // host) rather than this install's. `writeInstanceMarkers` already + // computes every field correctly, including `lineageId` — which, after + // `deployment.lineageId` was persisted at `createFromDraft` (R4), reads + // the CARRIED lineage rather than this install's own id (SC-007). + await this.writeInstanceMarkers(deployment, targetAppRoot, this.routingRuleFor(deployment).host); + + // Step 8 (FR-019, R10): write the OIDC credentials file AFTER + // extraction, which just destroyed any earlier write. This restore + // path's OWN write — the caller must not also write it at its usual + // pre-restore position when a restore is happening (FR-023). + if (provisioned) await this.writeOidcCredentialsFile(deployment, provisioned, logBoth); + + // Steps 9 + 10 (FR-020, FR-021, R12): start ONLY the union of every + // participation's hook service, in ONE `--wait` call — `--wait` for + // EACH named service's OWN declared healthcheck, never a bespoke + // readiness poll (which would need per-app knowledge of what "ready" + // means — exactly the branching Constitution V forbids). Then run + // every restore hook fail-closed via the SAME + // `runPreHooksFailClosed`/`runPostHooks` policy the pre-upgrade + // snapshot uses (research R14): declaration order, propagate on the + // first failure, clean up (`postHook`, unused here) only the STARTED + // set. A hook service that never becomes healthy, or a hook that + // fails, fails the whole install. + const hookServices = Array.from( + new Set(restoreDeclarations.map((d) => d.hook?.service).filter((s): s is string => Boolean(s))), + ); + if (hookServices.length > 0) { + await logBoth('info', `Restore: starting ${hookServices.join(', ')}…`); + const up = await this.dockerService.composeUp(composeDir, projectName, registryAuth, deployment.selectedProfiles, { + services: hookServices, + wait: true, + timeoutMs: RESTORE_HOOK_WAIT_TIMEOUT_MS, + }); + if (!up.success) { + throw new ConflictError( + `A restore hook service never became healthy: ${up.output}`, + { code: 'RESTORE_HOOK_FAILED', service: hookServices.join(',') }, + ); + } + + const restoreParticipants: BackupParticipant[] = restoreDeclarations + .filter((d) => d.hook) + .map((d) => ({ deploymentId: deployment.id, participationId: d.id, preHook: d.hook })); + const hookResult = await this.runPreHooksFailClosed(restoreParticipants, (level, message) => { + if (level === 'error') this.logger.warn(message, { deploymentId: deployment.id }); + else this.logger.info(message, { deploymentId: deployment.id }); + }); + if (!hookResult.ok && hookResult.failed) { + await this.runPostHooks(hookResult.started); + throw new ConflictError( + `Restore hook failed for participation '${hookResult.failed.participationId}': ${hookResult.failed.output ?? 'no output'}`, + { + code: 'RESTORE_HOOK_FAILED', + participationId: hookResult.failed.participationId, + service: restoreDeclarations.find((d) => d.id === hookResult.failed!.participationId)?.hook?.service, + }, + ); + } + } + + deployment.restoredAt = new Date().toISOString(); + await logBoth('info', 'Restore complete.'); + } finally { + // The whole staging DIRECTORY, not just the archive inside it — deleting + // only the file left an empty `restore-staging/` under every restored + // deployment forever. + await this.storageService.deleteDir(stagingRelDir, true).catch(() => {}); + } + } + private jobTypeToAction(type: Job['type']): string { switch (type) { case 'stop': return 'stop'; @@ -3793,11 +4212,22 @@ export class RealDeploymentService extends InMemoryDeploymentService { await this.restoreAppDataSnapshot(deploymentId, targetReleaseId, logBoth); } + // Restore-on-install (spec 007): true for AT MOST one job per + // deployment — its very first deploy (`!previousReleaseId`, set by + // `promoteRelease` before this job ever starts; a promote/rollback + // always has one), and only once (`!restoredAt`). When false, every + // line below is byte-for-byte what ran before this feature (FR-023) — + // this flag is the ONLY thing that changes about the no-restore path. + const willRestore = Boolean(deployment.restoreFrom) && !deployment.restoredAt && !deployment.previousReleaseId; + const provisioned = await this.provisionAuth(deployment); const composeDir = await this.materializeCompose(deployment, provisioned?.env ?? {}); // Drop the provisioned OIDC creds file into the data root before `up` so a // bundle sidecar can render the app's SSO config for first boot (e.g. Immich). - if (provisioned) await this.writeOidcCredentialsFile(deployment, provisioned, logBoth); + // Restore-on-install (FR-019, research R10): a restore's OWN extraction + // would destroy this write, so it happens instead INSIDE + // `performRestoreOnInstall`, after extraction — never both places. + if (provisioned && !willRestore) await this.writeOidcCredentialsFile(deployment, provisioned, logBoth); // Pull first (generous timeout) so `up` isn't gated on download time — // large stacks like Postiz used to be SIGKILLed mid-pull by up's 2-min cap. @@ -3812,6 +4242,14 @@ export class RealDeploymentService extends InMemoryDeploymentService { // irreversible step) if the job was cancelled during the long pull. if (ctx.isCancelled()) throw new JobCancelledError(); + // Restore-on-install (spec 007, research R8): between the pull and the + // final `composeUp` — the cheaper failure (a bad pull) happens first, + // and the restore runs after cancellation is no longer possible, so a + // cancelled install never quiesces somebody else's live database. + if (willRestore) { + await this.performRestoreOnInstall(deployment, composeDir, projectName, registryAuth, provisioned, logBoth); + } + const res = await this.dockerService.composeUp(composeDir, projectName, registryAuth, deployment.selectedProfiles); output = res.output; if (!res.success) throw new Error(res.output); diff --git a/packages/server/src/services/core/docker.ts b/packages/server/src/services/core/docker.ts index 25d64e06..88b24eae 100644 --- a/packages/server/src/services/core/docker.ts +++ b/packages/server/src/services/core/docker.ts @@ -64,7 +64,21 @@ export interface DockerService { * Image pulls for large multi-service apps (e.g. Postiz) routinely exceed the * short `up` timeout; pulling first means `up` only has to start local images. */ composePull(projectPath: string, projectName: string, registryAuth?: PullCredentials[], profiles?: string[]): Promise<{ success: boolean; output: string }>; - composeUp(projectPath: string, projectName: string, registryAuth?: PullCredentials[], profiles?: string[]): Promise<{ success: boolean; output: string }>; + /** + * `options.services` starts only the named services (default: every service in + * the project, today's behaviour). `options.wait` adds `--wait`, blocking until + * each named service's own `healthcheck` reports healthy — used by the restore + * sequence to start only a hook's service before running its hook (spec 007, + * FR-020). `options.timeoutMs` overrides the default 5-minute `execAsync` cap, + * because a `--wait` on a freshly-`initdb`'d Postgres can exceed it (research R12). + */ + composeUp( + projectPath: string, + projectName: string, + registryAuth?: PullCredentials[], + profiles?: string[], + options?: { services?: string[]; wait?: boolean; timeoutMs?: number }, + ): Promise<{ success: boolean; output: string }>; composeDown(projectPath: string, projectName: string, profiles?: string[]): Promise<{ success: boolean; output: string }>; composePs(projectPath: string, projectName: string): Promise; composeRestart(projectPath: string, projectName: string, serviceName?: string, profiles?: string[]): Promise<{ success: boolean; output: string }>; @@ -228,11 +242,17 @@ export class RealDockerService implements DockerService, HealthCheckable { } } - async composeUp(projectPath: string, projectName: string, registryAuth?: PullCredentials[], profiles?: string[]): Promise<{ success: boolean; output: string }> { + async composeUp( + projectPath: string, + projectName: string, + registryAuth?: PullCredentials[], + profiles?: string[], + options?: { services?: string[]; wait?: boolean; timeoutMs?: number }, + ): Promise<{ success: boolean; output: string }> { const { env: authEnv, dir } = this.makeRegistryAuthEnv(registryAuth); const env = this.withComposeProfiles(authEnv, profiles); try { - this.logger.info('Starting compose project', { projectPath, projectName }); + this.logger.info('Starting compose project', { projectPath, projectName, services: options?.services, wait: options?.wait }); const composeFile = join(projectPath, 'docker-compose.yml'); if (!existsSync(composeFile)) { @@ -240,13 +260,24 @@ export class RealDockerService implements DockerService, HealthCheckable { } // Images are pre-pulled by composePull, so `up` only starts local images. - // The 5-minute timeout covers container creation for large stacks (it is - // no longer gated on download time). A scoped DOCKER_CONFIG is passed as a - // fallback so a recreate that needs to pull still authenticates. - const { stdout, stderr } = await execAsync( - `docker compose -f "${composeFile}" -p "${projectName}" up -d`, - { cwd: projectPath, timeout: 300000, env } // 5 minute timeout - ); + // `--wait` (when requested) blocks until every NAMED service reports + // healthy via its own declared healthcheck — no bespoke readiness poll + // (Constitution V). `options.timeoutMs` overrides the default 5-minute + // cap: a `--wait` against a freshly-`initdb`'d Postgres can exceed it + // (research R12). A scoped DOCKER_CONFIG is passed as a fallback so a + // recreate that needs to pull still authenticates. + // + // Built as an argv array and run through `execFile` (NO shell), the same + // rule `composeExec` already states: `options.services` carries + // APP-SUPPLIED names (a bundle manifest's `restore[].hook.service`), and + // interpolating those into a shell string would let a manifest with a + // `"`/`;`/`$(`/backtick in a service name run arbitrary commands as the + // server. Nothing here is shell-quoted because nothing here reaches a shell. + const args = ['compose', '-f', composeFile, '-p', projectName, 'up', '-d']; + if (options?.wait) args.push('--wait'); + if (options?.services?.length) args.push(...options.services); + const timeout = options?.timeoutMs ?? 300000; // 5 minute default + const { stdout, stderr } = await execFileAsync('docker', args, { cwd: projectPath, timeout, env }); const output = [stdout, stderr].filter(Boolean).join('\n'); this.logger.info('Compose project started successfully', { @@ -706,6 +737,15 @@ export function parseComposeLogs(output: string): DockerLogs['entries'] { export class MockDockerService implements DockerService { private logger = getLogger().child({ service: 'MockDockerService' }); + /** + * Every `composeUp` call this instance has received, in order — so a test + * can assert on what was actually requested rather than merely that the + * call resolved. A Mock that accepted `services`/`wait` and ignored them + * would let a suite go green over a restore that started the wrong + * containers (plan.md's "Known trap", Constitution IV). + */ + readonly composeUpCalls: Array<{ projectName: string; services?: string[]; wait?: boolean; timeoutMs?: number }> = []; + async getDockerInfo(): Promise { return { available: true, version: 'mock', serverVersion: 'mock', apiVersion: 'mock' }; } @@ -719,8 +759,15 @@ export class MockDockerService implements DockerService { return { success: true, output: `[mock] Project ${projectName} images pulled` }; } - async composeUp(projectPath: string, projectName: string, registryAuth?: PullCredentials[], profiles?: string[]): Promise<{ success: boolean; output: string }> { - this.logger.debug('Mock compose up', { projectPath, projectName, authenticated: Boolean(registryAuth?.length), profiles }); + async composeUp( + projectPath: string, + projectName: string, + registryAuth?: PullCredentials[], + profiles?: string[], + options?: { services?: string[]; wait?: boolean; timeoutMs?: number }, + ): Promise<{ success: boolean; output: string }> { + this.logger.debug('Mock compose up', { projectPath, projectName, authenticated: Boolean(registryAuth?.length), profiles, services: options?.services, wait: options?.wait }); + this.composeUpCalls.push({ projectName, services: options?.services, wait: options?.wait, timeoutMs: options?.timeoutMs }); return { success: true, output: `[mock] Project ${projectName} created and started` }; } diff --git a/packages/server/src/services/core/draft.ts b/packages/server/src/services/core/draft.ts index e0bd786a..3699ae82 100644 --- a/packages/server/src/services/core/draft.ts +++ b/packages/server/src/services/core/draft.ts @@ -5,13 +5,13 @@ * Drafts are mutable until finalized into immutable releases. */ -import type { - Draft, - DraftFile, - CreateDraftRequest, - CreateDraftResponse, - GetDraftResponse, - PatchDraftRequest, +import type { + Draft, + DraftFile, + CreateDraftRequest, + CreateDraftResponse, + GetDraftResponse, + PatchDraftRequest, PatchDraftResponse, UploadDraftFileResponse, DeleteDraftFileResponse, @@ -24,8 +24,10 @@ import type { AppSecurityConfig, AppUpgradeMeta, AppBackupDeclaration, + AppRestoreDeclaration, AppPushTarget, - AppProfileConfig + AppProfileConfig, + RestoreChoice } from '@hola/shared'; import { createHash } from 'crypto'; @@ -34,11 +36,14 @@ import { STABLE_CHANNEL } from '@hola/shared'; import { getLogger } from '../../lib/logger'; import { NotFoundError, ConflictError, ValidationError, DraftValidationError, BundleUnavailableError, assertValidChannelName } from '../../middleware/error-mapping'; import { validateComposeDocument, APP_HOST_TOKEN, BASE_DOMAIN_TOKEN } from '@hola/shared/compose-validate'; +import { mergeUpgradeAppEnv } from './upgrade-env'; +import { deriveEnvNotCarriedKeys, judgeRestoreChoice, type CandidateSource } from './restore-candidates'; import type { HealthCheckable, ServiceHealth } from './types'; import type { StorageService } from './storage'; import type { CatalogService } from './catalog'; import type { RegistryCredentialService } from './registry-credentials'; import type { RoutingService } from './routing'; +import type { DeploymentService } from './deployment'; /** * Shape of `drafts//finalized/manifest.json` produced by `finalizeDraft`. @@ -101,6 +106,10 @@ export interface FinalizedManifest { // Per-app pre/post-backup hooks (#121) carried from the bundle manifest so the // snapshot path can run them around the file capture. backup?: AppBackupDeclaration; + // Per-backup-participation restore declarations (spec 007) carried from the + // bundle manifest so the restore sequence can apply discards/hooks without + // re-reading the bundle. + restore?: AppRestoreDeclaration[]; // Pushable directories (#409) carried from the bundle manifest so `push-targets` // can resolve them against the deployment's data root without re-reading the bundle. push?: AppPushTarget[]; @@ -118,6 +127,11 @@ export interface FinalizedManifest { // read as "not published" (pre-#431 manifests, install-by-ref, a draft built // from placeholder defaults because the catalog was unavailable). channelPublished?: boolean; + // Restore-on-install choice (spec 007), carried outside `canonicalSpec` + // beside `channel` — it names a source deployment, not deployable content, + // and two finalizes differing only in `restoreFrom` must produce the same + // checksum. Absent means no restore. + restoreFrom?: RestoreChoice; files: FinalizedManifestFile[]; checksum: string; finalizedAt: string; @@ -296,6 +310,125 @@ export class RealDraftService implements DraftService { private routingService?: RoutingService ) {} + /** + * Restore-on-install (spec 007): a reference to `DeploymentService`, needed + * to resolve/validate a restore candidate at draft-creation time (R1). NOT a + * constructor parameter: `simple-factory.ts` constructs `drafts` BEFORE + * `deployments` (`RealDeploymentService` takes `drafts` as an argument), so + * threading this the normal way would be circular. Wired via this setter + * once both exist. Optional — absent in tests/wiring that don't need + * restore, which then simply cannot resolve a `restoreFrom` choice (see + * `resolveRestoreChoice` below, which fails closed when it's unset). + */ + private deploymentsService?: DeploymentService; + setDeploymentsService(deployments: DeploymentService): void { + this.deploymentsService = deployments; + } + + /** Same reserved location `deployment.ts` writes the install ENVIRONMENT + * record to (spec 006): `/.hola//env.json`, a + * SIBLING of every app's data root — never inside it (#478). Restore-on- + * install (spec 007) reads the SAME record from here; the constants are + * duplicated rather than imported because `deployment.ts`'s copies are + * private implementation details of its own write path (research R17). + */ + private restoreEnvRecordPath(candidateId: string): string { + const root = (process.env.HOLA_APPS_BIND_ROOT?.trim() || '/srv/hola/apps').replace(/\/+$/, ''); + return `${root}/.hola/${candidateId}/env.json`; + } + + /** + * Read a restore candidate's environment record. `null` when absent, + * unparseable, or malformed — carrying nothing is a legitimate candidate + * state (`carriesEnv: false`), never a draft-creation failure (FR-008). + */ + private async readRestoreEnvRecord(candidateId: string): Promise | null> { + const path = this.restoreEnvRecordPath(candidateId); + if (!(await this.storageService.fileExists(path))) return null; + try { + const raw = JSON.parse(await this.storageService.readFileAsString(path)) as { env?: unknown }; + if (!raw || typeof raw.env !== 'object' || raw.env === null) return null; + const env: Record = {}; + for (const [k, v] of Object.entries(raw.env as Record)) { + if (typeof v === 'string') env[k] = v; + } + return env; + } catch (error) { + this.logger.warn('Unreadable restore environment record', { candidateId, error: error instanceof Error ? error.message : String(error) }); + return null; + } + } + + /** + * Resolve and validate a `restoreFrom` choice at draft-creation time (R1): + * the ONE place a restore choice is judged and turned into a seeded + * `appEnv`. Reused verbatim at `createFromDraft`'s re-validation (T018) via + * the same `validateRestoreChoice` it calls — only WHEN this runs, and + * whether the candidate might have changed since, differs. + * + * Throws `ConflictError` (`details.code`, the `RESTORE_*` union) on any + * refusal — never returns a partial/soft result, matching `PROVIDER_EXISTS` + * / `ALREADY_INSTALLED`'s established shape (FR-037). + */ + private async resolveRestoreChoice( + choice: RestoreChoice, + appId: string, + targetVersion: string | undefined, + upgrade: AppUpgradeMeta | undefined, + restoreDeclarations: AppRestoreDeclaration[] | undefined, + appEnv: AppEnvVar[], + accepts: string[] | undefined, + ): Promise<{ appEnv: AppEnvVar[] }> { + if (!this.deploymentsService) { + throw new ConflictError( + `Cannot resolve restore source '${choice.candidateId}': the deployment registry is unavailable.`, + { code: 'RESTORE_CANDIDATE_GONE', candidateId: choice.candidateId }, + ); + } + + // data-model.md §7 / contracts/manifest.md: "no restore@1 in accepts" is + // NOT the same state as "restore@1 with no block" (the latter is a + // deliberate plain-file-copy declaration; the former means nobody + // considered restoring this app at all). Its own code, NOT the + // install-by-ref path's `RESTORE_NOT_SUPPORTED`: the two refusals have + // opposite remedies ("use the catalog path" vs "this app can't be + // restored at all"), and a client keying a hint off `details.code` cannot + // tell them apart if they share one. + if (!(accepts ?? []).includes('restore@1')) { + throw new ConflictError( + `'${appId}' has not declared that it can be restored (no 'restore@1' in its manifest 'accepts').`, + { code: 'RESTORE_NOT_ACCEPTED', appId }, + ); + } + + const source = await this.deploymentsService.getRestoreSource(choice.candidateId); + const requiresEnv = (restoreDeclarations ?? []).some((d) => d.requiresEnv === true); + const envNotCarriedKeys = deriveEnvNotCarriedKeys(appEnv); + + const result = judgeRestoreChoice({ + source: source as CandidateSource | undefined, + appId, + // No deployment exists yet at draft-creation time — nothing to exclude. + excludeDeploymentId: '', + choice, + targetVersion, + meta: upgrade, + requiresEnv, + envNotCarriedKeys, + }); + if (!result.ok) { + throw new ConflictError(result.message, { code: result.code, ...result.details }); + } + + let mergedEnv = appEnv; + if (choice.carryEnv) { + const carried = await this.readRestoreEnvRecord(choice.candidateId); + if (carried) mergedEnv = mergeUpgradeAppEnv(appEnv, carried); + } + + return { appEnv: mergedEnv }; + } + /** * Replace `${HOLA_APP_HOST}`/`${HOLA_BASE_DOMAIN}` in seeded env values with * this install's concrete values, so the wizard shows a real prefilled URL/ @@ -470,7 +603,34 @@ export class RealDraftService implements DraftService { // Resolve `${HOLA_APP_HOST}`/`${HOLA_BASE_DOMAIN}` in the seeded env values // to this install's concrete host/domain, so the wizard shows a real // prefilled value rather than a raw platform token. - const appEnv = this.resolvePlatformTokens(request.appId, defaults.env); + let appEnv = this.resolvePlatformTokens(request.appId, defaults.env); + + // Restore-on-install (spec 007, R1): the ONLY point a restore choice is + // ever accepted — resolved and validated (candidate exists, settled, + // version skew judged, acknowledgements satisfied) BEFORE it can seed + // anything. A refusal here throws (ConflictError, `details.code`) and no + // draft is created. `name`/`subdomain` default from the candidate + // client-side (FR-035): the wizard already holds the full + // `RestoreCandidate` from the candidates route it read before this call + // (the same reason that route exists at all — research R6), so nothing + // further is echoed back here. + if (request.restoreFrom) { + const resolved = await this.resolveRestoreChoice( + request.restoreFrom, + request.appId, + // Same fallback `draft.version` itself uses below — the catalog's + // resolved version when it reported one, else the requested one. + // Using only `defaults.resolvedVersion` here would read as "target + // version unknown" whenever a test/catalog stub omits it, forcing + // an unrelated `restore-version-unknown` acknowledgement. + defaults.resolvedVersion ?? request.version, + defaults.upgrade, + defaults.restore, + appEnv, + defaults.accepts, + ); + appEnv = resolved.appEnv; + } // Seed the draft's compose from the catalog bundle so it can be deployed // without the user pasting compose. Guard it through the same parse check @@ -511,6 +671,7 @@ export class RealDraftService implements DraftService { ingressService: defaults.ingressService, upgrade: defaults.upgrade, backup: defaults.backup, + restore: defaults.restore, push: defaults.push, profiles: defaults.profiles, channel: resolvedChannel, @@ -523,6 +684,10 @@ export class RealDraftService implements DraftService { // FOLLOW an unpublished channel and receive stable-floor offers; it just // isn't a free second copy of a single-instance app. channelPublished: defaults.channels?.includes(resolvedChannel) === true, + // Restore-on-install choice (spec 007), already resolved/validated + // above. Carried unchanged onto the finalized manifest (draft.ts + // finalizeDraft, outside canonicalSpec beside `channel`). + restoreFrom: request.restoreFrom, files: [], }; @@ -588,6 +753,21 @@ export class RealDraftService implements DraftService { private async createDraftFromRef(draftId: string, request: CreateDraftRequest): Promise { const ociRef = request.ociRef!; this.logger.info('Creating draft from OCI ref', { draftId, ociRef, credentialRef: request.credentialRef }); + + // Restore-on-install (spec 007, R2): REFUSED on install-by-ref, never + // silently ignored. A candidate's version skew is judged against catalog + // upgrade metadata, and install-by-ref deliberately has no catalog index + // to consult (the same reason it already fails closed on channel + // resolution, below). Honouring the choice on the catalog path and + // dropping it here would be exactly the silent-empty-restore failure this + // feature exists to prevent. + if (request.restoreFrom) { + throw new ConflictError( + `Cannot restore on an install-by-ref draft: no catalog index exists to judge the candidate's version against.`, + { code: 'RESTORE_NOT_SUPPORTED' }, + ); + } + try { let credentials; if (request.credentialRef) { @@ -626,6 +806,7 @@ export class RealDraftService implements DraftService { ingressService: detail.ingressService, upgrade: detail.upgrade, backup: detail.backup, + restore: detail.restore, push: detail.push, profiles: detail.profiles, // Install-by-ref bypasses the catalog index (#428): always `stable`. @@ -900,6 +1081,7 @@ export class RealDraftService implements DraftService { ingressService: draft.ingressService, upgrade: draft.upgrade, backup: draft.backup, + restore: draft.restore, push: draft.push, profiles: draft.profiles, files: specFiles, @@ -940,7 +1122,10 @@ export class RealDraftService implements DraftService { // a second time. `channelPublished` (#431) rides along with it — the // catalog fact about that channel, resolved once at draft-create time and // carried so the create-time single-instance guard needs no catalog call. - const manifest = { ...canonicalSpec, icon: draft.icon, displayName: draft.displayName, source: draft.source, credentialRef: draft.credentialRef, channel: draft.channel, channelPublished: draft.channelPublished, checksum, finalizedAt }; + // `restoreFrom` (spec 007) is the same kind of fact again: it names WHERE + // the deployable spec's data comes from, not the spec itself, so two + // finalizes differing only in the restore choice produce the same checksum. + const manifest = { ...canonicalSpec, icon: draft.icon, displayName: draft.displayName, source: draft.source, credentialRef: draft.credentialRef, channel: draft.channel, channelPublished: draft.channelPublished, restoreFrom: draft.restoreFrom, checksum, finalizedAt }; await this.storageService.writeFile( `${finalizedDir}/manifest.json`, JSON.stringify(manifest, null, 2) @@ -990,7 +1175,7 @@ export class RealDraftService implements DraftService { }; } - async getDraftDefaults(appId: string, version?: string, source?: string, channel?: string): Promise<{ env: AppEnvVar[]; defaults: DraftDefaults; composeOverride: string; auth?: AppAuthConfig; consumes?: string[]; provides?: string[]; accepts?: string[]; multiInstance?: boolean; security?: AppSecurityConfig; ingressService?: string; upgrade?: AppUpgradeMeta; backup?: AppBackupDeclaration; push?: AppPushTarget[]; profiles?: AppProfileConfig[]; resolvedVersion?: string; resolvedChannel?: string; channels?: string[] }> { + async getDraftDefaults(appId: string, version?: string, source?: string, channel?: string): Promise<{ env: AppEnvVar[]; defaults: DraftDefaults; composeOverride: string; auth?: AppAuthConfig; consumes?: string[]; provides?: string[]; accepts?: string[]; multiInstance?: boolean; security?: AppSecurityConfig; ingressService?: string; upgrade?: AppUpgradeMeta; backup?: AppBackupDeclaration; restore?: AppRestoreDeclaration[]; push?: AppPushTarget[]; profiles?: AppProfileConfig[]; resolvedVersion?: string; resolvedChannel?: string; channels?: string[] }> { try { const versionDetail = await this.catalogService.getVersionDetail(appId, version || 'latest', source, channel); return { @@ -1006,6 +1191,7 @@ export class RealDraftService implements DraftService { ingressService: versionDetail.ingressService, upgrade: versionDetail.upgrade, backup: versionDetail.backup, + restore: versionDetail.restore, push: versionDetail.push, profiles: versionDetail.profiles, // The concrete version the catalog resolved (e.g. "latest" → "1.4.1"), so diff --git a/packages/server/src/services/core/manifest-restore.ts b/packages/server/src/services/core/manifest-restore.ts new file mode 100644 index 00000000..9e58cac9 --- /dev/null +++ b/packages/server/src/services/core/manifest-restore.ts @@ -0,0 +1,102 @@ +import type { AppBackupHook, AppRestoreDeclaration } from '@hola/shared'; +import type { Logger } from '../../lib/logger'; + +/** + * Narrow-shape coercion for the bundle manifest's optional `restore` block + * (spec 007), mirroring `coerceManifestBackup`. Unlike `backup`, `restore` has + * no legacy singular form — it is new with this feature, so the manifest + * declares it as an array from day one, one entry per **backup** participation + * id it restores. + * + * Drop rules (never throw — a malformed entry degrades, per ADR 0003): + * - an entry with no usable `id` (non-string, empty after trim); + * - an entry whose `id` repeats an earlier one (first wins); + * - a `discard` entry that isn't a non-empty string is dropped from the list, + * not the whole declaration; + * - a malformed `hook` is dropped (the declaration survives with no hook); + * - `requiresEnv` is kept only when it's literally `true`. + */ + +type CoerceCtx = { appId?: string; version?: string }; + +function asRecord(v: unknown): Record | undefined { + return v && typeof v === 'object' && !Array.isArray(v) ? (v as Record) : undefined; +} + +function asString(v: unknown): string | undefined { + return typeof v === 'string' && v.trim().length > 0 ? v.trim() : undefined; +} + +/** A command must be a non-empty array of non-empty strings (exec form, no shell-string). */ +function asCommand(v: unknown): string[] | undefined { + if (!Array.isArray(v) || v.length === 0) return undefined; + if (!v.every((x) => typeof x === 'string' && x.length > 0)) return undefined; + return v as string[]; +} + +/** + * A Compose service name, as Compose itself defines it: `[a-zA-Z0-9]` then + * word/dot/dash characters. Validated HERE, at the manifest boundary, because + * `restore[].hook.service` is app-supplied data that the restore sequence + * hands to `composeUp({ services })` as a command argument. `composeUp` runs + * argv-only (no shell), so this is defence in depth rather than the only + * guard — but a name that could never name a real service is a malformed + * declaration either way, and the drop rules above say a malformed hook is + * dropped rather than thrown over (ADR 0003). + */ +const COMPOSE_SERVICE_NAME = /^[a-zA-Z0-9][a-zA-Z0-9._-]*$/; + +function coerceHook(value: unknown): AppBackupHook | undefined { + const rec = asRecord(value); + if (!rec) return undefined; + const service = asString(rec.service); + const command = asCommand(rec.command); + if (!service || !command) return undefined; + if (!COMPOSE_SERVICE_NAME.test(service)) return undefined; + return { service, command }; +} + +function coerceDiscard(value: unknown): string[] | undefined { + if (!Array.isArray(value)) return undefined; + const paths = value.filter((v): v is string => typeof v === 'string' && v.trim().length > 0); + return paths.length > 0 ? paths : undefined; +} + +export function coerceManifestRestore( + value: unknown, + logger?: Logger, + ctx: CoerceCtx = {}, +): AppRestoreDeclaration[] | undefined { + if (!Array.isArray(value)) return undefined; + + const out: AppRestoreDeclaration[] = []; + const seen = new Set(); + + for (const raw of value) { + const rec = asRecord(raw); + if (!rec) { + logger?.warn('Dropping restore declaration: not an object', { ...ctx }); + continue; + } + const id = asString(rec.id); + if (!id) { + logger?.warn('Dropping restore declaration: missing or blank id', { ...ctx }); + continue; + } + if (seen.has(id)) { + logger?.warn('Dropping restore declaration: duplicate id, keeping the first', { ...ctx, id }); + continue; + } + seen.add(id); + + const declaration: AppRestoreDeclaration = { id }; + const discard = coerceDiscard(rec.discard); + if (discard) declaration.discard = discard; + const hook = coerceHook(rec.hook); + if (hook) declaration.hook = hook; + if (rec.requiresEnv === true) declaration.requiresEnv = true; + out.push(declaration); + } + + return out.length > 0 ? out : undefined; +} diff --git a/packages/server/src/services/core/restore-candidates.ts b/packages/server/src/services/core/restore-candidates.ts new file mode 100644 index 00000000..52263a1b --- /dev/null +++ b/packages/server/src/services/core/restore-candidates.ts @@ -0,0 +1,453 @@ +/** + * Restore-on-install candidate resolution (spec 007). + * + * Pure functions over already-fetched state — no I/O, no app names + * (Constitution III, V). Callers (the candidates route, `draft.ts`, + * `deployment.ts`) do the filesystem/deployment-registry reads and pass in + * plain data; this module never touches a filesystem, a catalog, or a + * datastore, and never branches on which app is being restored. That last + * property is mechanically checked by quickstart.md §9's scope-boundary grep + * (a case-insensitive search of this file for any specific catalog app's or + * datastore's name) — deliberately not spelled out literally here, so this + * very comment can't produce a false positive against that check. + */ + +import { + checkUpgradePath, + isNewerVersion, + type AppEnvVar, + type AppUpgradeMeta, + type EnhancedDeploymentDetail, + type RestoreCandidate, + type RestoreCandidateLineage, + type RestoreRefusalCode, + type RestoreSkewVerdict, + type RestoreWarning, + type RestoreChoice, +} from '@hola/shared'; + +/** + * The subset of the install identity record (`.hola/instance.json`, spec 006) + * this module reads. Deliberately NOT the `InstallIdentityRecord` type in + * `deployment.ts` — that type is module-local by design (spec 006, FR-018: + * "nothing outside the server reads this record"), and this module stays a + * pure function of already-fetched state: the caller reads and parses the + * file, and hands in only the fields a candidate description needs. + */ +export interface RestoreIdentitySnapshot { + lineageId?: string; + app?: string; + appVersion?: string | null; + channel?: string | null; + subdomain?: string | null; + host?: string | null; + writtenAt?: string | null; +} + +/** One deployment considered as a restore source, with the I/O already done by the caller. */ +export interface CandidateSource { + deployment: EnhancedDeploymentDetail; + /** Parsed `.hola/instance.json`, or `null` when absent/unparseable (FR-003). */ + identity: RestoreIdentitySnapshot | null; + /** `dirHasContents(appRoot, [INSTALL_MARKERS_DIR])` — whether the data root holds app data. */ + hasData: boolean; + /** Whether `/.hola//env.json` exists for this deployment. */ + carriesEnv: boolean; +} + +/** Settled deployment states a restore may read from (FR-004a). Mid-lifecycle and `error` are excluded. */ +const SETTLED_STATUSES = new Set(['running', 'stopped']); + +export function isSettledStatus(status: string): boolean { + return SETTLED_STATUSES.has(status); +} + +/** + * Eligibility (data-model.md §2, FR-001/FR-004/FR-004a). A deployment is a + * candidate iff: same app, not the deployment being created, settled state, + * and its data root holds app data ignoring the marker directory. The + * ignore-list is load-bearing — the caller must compute `hasData` with the + * same `dirHasContents(appRoot, [INSTALL_MARKERS_DIR])` rule + * `capturePreUpgradeSnapshot` already applies, or every materialised install + * looks like it holds data (the data-loss shape spec 006's review caught). + */ +export function isEligibleCandidate( + source: CandidateSource, + targetAppId: string, + excludeDeploymentId?: string, +): boolean { + const { deployment, hasData } = source; + if (deployment.app !== targetAppId) return false; + if (excludeDeploymentId && deployment.id === excludeDeploymentId) return false; + if (!isSettledStatus(deployment.status)) return false; + if (!hasData) return false; + return true; +} + +/** + * Describe a candidate from its identity record, falling back per-field to + * the deployment record (FR-003). `lineageId` degrades to the deployment id + * when the identity record is absent or lacks one — the field with no + * deployment-record fallback, because `lineageId` was only persisted onto the + * deployment record by this feature (research R4); a pre-existing identity + * record already carries it (spec 006 wrote it in anticipation). + */ +export function describeCandidate( + source: CandidateSource, +): Omit { + const { deployment, identity, carriesEnv } = source; + return { + deploymentId: deployment.id, + lineageId: identity?.lineageId ?? deployment.lineageId ?? deployment.id, + app: identity?.app ?? deployment.app, + name: deployment.name, + subdomain: identity?.subdomain ?? deployment.subdomain ?? null, + host: identity?.host ?? null, + appVersion: identity?.appVersion ?? deployment.version ?? null, + channel: identity?.channel ?? deployment.channel ?? null, + carriesEnv, + capturedAt: identity?.writtenAt ?? null, + hasIdentityRecord: identity !== null, + }; +} + +/** Newest-first comparator on `capturedAt`; a null `capturedAt` sorts last (FR-005). */ +function compareCapturedAtDesc(a: RestoreCandidate, b: RestoreCandidate): number { + if (a.capturedAt === b.capturedAt) return 0; + if (a.capturedAt === null) return 1; + if (b.capturedAt === null) return -1; + return a.capturedAt < b.capturedAt ? 1 : -1; +} + +/** + * Group candidates by lineage, newest-first within each lineage and across + * lineages (FR-005, FR-036). A single matching lineage supplies a default + * selection; two or more distinct lineages require an explicit pick and no + * default is offered; zero lineages need no choice at all. + */ +export function groupIntoLineages(candidates: RestoreCandidate[]): { + lineages: RestoreCandidateLineage[]; + defaultCandidateId: string | null; + requiresExplicitChoice: boolean; +} { + const byLineage = new Map(); + for (const candidate of candidates) { + const list = byLineage.get(candidate.lineageId) ?? []; + list.push(candidate); + byLineage.set(candidate.lineageId, list); + } + + const lineages: RestoreCandidateLineage[] = Array.from(byLineage.entries()).map(([lineageId, list]) => ({ + lineageId, + candidates: [...list].sort(compareCapturedAtDesc), + })); + lineages.sort((a, b) => compareCapturedAtDesc(a.candidates[0]!, b.candidates[0]!)); + + const requiresExplicitChoice = lineages.length >= 2; + const defaultCandidateId = lineages.length === 1 ? (lineages[0]!.candidates[0]?.deploymentId ?? null) : null; + + return { lineages, defaultCandidateId, requiresExplicitChoice }; +} + +/** + * Version skew (data-model.md §4, research R15). Evaluated in this exact + * order. Only the two `refused` rows below come from `checkUpgradePath` — + * "candidate newer than target" and "version/metadata unknown" are this + * feature's OWN rules, evaluated first, because `checkUpgradePath` returns + * `ok` for both: it exists to guard promotes, where a downgrade (which is + * what a newer-than-target restore looks like to it) is the caller's + * business, not something to block. + */ +export function computeSkewVerdict( + candidateVersion: string | null | undefined, + targetVersion: string | undefined, + meta: AppUpgradeMeta | undefined, +): RestoreSkewVerdict { + if (!candidateVersion || !targetVersion || !meta) return { kind: 'unknown' }; + + if (isNewerVersion(candidateVersion, targetVersion)) { + return { + kind: 'refused', + code: 'RESTORE_SOURCE_NEWER', + message: `This candidate was captured on version ${candidateVersion}, newer than the version being installed (${targetVersion}).`, + }; + } + + const pathResult = checkUpgradePath(candidateVersion, targetVersion, meta); + if (!pathResult.ok) { + return { + kind: 'refused', + code: 'RESTORE_UPGRADE_PATH', + message: pathResult.message, + suggestedVersion: pathResult.suggestedVersion, + }; + } + + return { kind: 'ok' }; +} + +/** + * Exactly the `AppEnvVar` entries where `isSecret === true` AND `generate` is + * present (FR-033) — values the PLATFORM invented and will mint fresh when + * not carried. Derived, never a manifest field an app author could forget or + * let rot. + */ +export function deriveEnvNotCarriedKeys(appEnv: AppEnvVar[]): string[] { + return appEnv.filter((entry) => entry.isSecret === true && entry.generate != null).map((entry) => entry.key); +} + +export interface AcknowledgementInput { + skew: RestoreSkewVerdict; + carryEnv: boolean; + carriesEnv: boolean; +} + +/** + * Which acknowledgement codes a candidate choice REQUIRES (data-model.md §3), + * modelled on `grants`. Both `carryEnv === false` (declined) and + * `carriesEnv === false` (unavailable) map to the SAME code deliberately — + * the operator-facing risk is identical either way, and a second code would + * invite treating one as less serious. + */ +export function deriveRequiredAcknowledgements(input: AcknowledgementInput): string[] { + const required: string[] = []; + if (input.skew.kind === 'unknown') required.push('restore-version-unknown'); + if (!input.carryEnv || !input.carriesEnv) required.push('restore-env-not-carried'); + return required; +} + +export interface WarningsInput { + carryEnv: boolean; + carriesEnv: boolean; + envNotCarriedKeys: string[]; + hasIdentityRecord: boolean; + candidateSubdomain?: string | null; + chosenSubdomain?: string | null; +} + +/** Proceedable, named, non-fatal risks to surface alongside a candidate (data-model.md §5). */ +export function deriveWarnings(input: WarningsInput): RestoreWarning[] { + const warnings: RestoreWarning[] = []; + if ((!input.carryEnv || !input.carriesEnv) && input.envNotCarriedKeys.length > 0) { + warnings.push({ code: 'env-not-carried', keys: input.envNotCarriedKeys }); + } + if (!input.hasIdentityRecord) { + warnings.push({ code: 'no-identity-record' }); + } + if ( + input.candidateSubdomain != null && + input.chosenSubdomain != null && + input.candidateSubdomain !== input.chosenSubdomain + ) { + warnings.push({ code: 'host-divergence', from: input.candidateSubdomain, to: input.chosenSubdomain }); + } + return warnings; +} + +/** + * Resolve one candidate's full listing shape (`skew`, `requiredAcknowledgements`, + * `warnings`) for the candidates ROUTE, before the operator has made a + * `RestoreChoice`. The route assumes the DEFAULT action — carry configuration + * whenever it's available (`carryEnv: candidate.carriesEnv`) — which is why + * `requiredAcknowledgements` is empty for a candidate with an environment + * record and an `ok` skew: that's the shape of accepting the defaults. A + * client that instead declines carrying (or picks a candidate with no record) + * gets `restore-env-not-carried` required at create time via + * {@link deriveRequiredAcknowledgements} evaluated against the actual choice. + */ +export function resolveListedCandidate( + source: CandidateSource, + targetVersion: string | undefined, + meta: AppUpgradeMeta | undefined, + appEnv: AppEnvVar[], +): RestoreCandidate { + const described = describeCandidate(source); + const skew = computeSkewVerdict(described.appVersion, targetVersion, meta); + const envNotCarriedKeys = deriveEnvNotCarriedKeys(appEnv); + const assumedCarryEnv = described.carriesEnv; // the route's default action + const requiredAcknowledgements = deriveRequiredAcknowledgements({ + skew, + carryEnv: assumedCarryEnv, + carriesEnv: described.carriesEnv, + }); + const warnings = deriveWarnings({ + carryEnv: assumedCarryEnv, + carriesEnv: described.carriesEnv, + envNotCarriedKeys, + hasIdentityRecord: described.hasIdentityRecord, + }); + return { ...described, skew, requiredAcknowledgements, warnings }; +} + +/** + * Whether a previously-chosen candidate is STILL a valid restore source, + * re-checked at job time because it may have been deleted or started a + * lifecycle action since the draft was created (FR-013a, research R8 step 2). + * `source` is `undefined` when the deployment no longer exists at all. + */ +export function checkCandidateStillEligible( + source: CandidateSource | undefined, + targetAppId: string, + excludeDeploymentId: string, +): { ok: true } | { ok: false; code: 'RESTORE_CANDIDATE_GONE' | 'RESTORE_CANDIDATE_BUSY' } { + if (!source) return { ok: false, code: 'RESTORE_CANDIDATE_GONE' }; + if (source.deployment.app !== targetAppId) return { ok: false, code: 'RESTORE_CANDIDATE_GONE' }; + if (source.deployment.id === excludeDeploymentId) return { ok: false, code: 'RESTORE_CANDIDATE_GONE' }; + if (!isSettledStatus(source.deployment.status)) return { ok: false, code: 'RESTORE_CANDIDATE_BUSY' }; + return { ok: true }; +} + +export interface RestoreNameDefaults { + name: string; + subdomain: string; + warnings: RestoreWarning[]; +} + +/** + * FR-035: default a restored install's name/subdomain from the candidate + * when the operator supplied no explicit name, and warn when an explicit + * choice diverges from the candidate's own address — absolute addresses + * stored inside the restored data will not be rewritten. Pure: the caller's + * own `deriveSubdomain` is injected rather than imported, so this stays a + * function of its arguments (Constitution III/V) and is directly testable + * without the routing/collision machinery a real install exercises. + */ +export function resolveRestoreNameDefaults(input: { + requestedName: string | undefined; + candidateName: string; + candidateSubdomain: string | null; + appId: string; + deriveSubdomain: (name: string | undefined, appId: string) => string; +}): RestoreNameDefaults { + const name = input.requestedName || input.candidateName; + const subdomain = !input.requestedName && input.candidateSubdomain + ? input.candidateSubdomain + : input.deriveSubdomain(input.requestedName || input.candidateName, input.appId); + const warnings: RestoreWarning[] = []; + if (input.candidateSubdomain && input.candidateSubdomain !== subdomain) { + warnings.push({ code: 'host-divergence', from: input.candidateSubdomain, to: subdomain }); + } + return { name, subdomain, warnings }; +} + +export interface RestoreValidationInput { + /** The candidate, already resolved with `skew` computed against `targetVersion`. */ + candidate: RestoreCandidate; + /** The operator's decision. */ + choice: RestoreChoice; + /** The app's `restore.requiresEnv` declaration for the participation(s) in play. Default `false`. */ + requiresEnv: boolean; + /** `deriveEnvNotCarriedKeys` for the version being installed — the `missingKeys` a `RESTORE_ENV_REQUIRED` refusal names. */ + envNotCarriedKeys: string[]; + targetVersion?: string; +} + +export type RestoreValidationResult = + | { ok: true; requiredAcknowledgements: string[] } + | { ok: false; code: RestoreRefusalCode; message: string; details: Record }; + +/** + * The one place that turns a candidate + a choice into "proceed" or "refuse + * with this code" (data-model.md §3/§4). Reused verbatim at draft creation + * (T015), at `createFromDraft` re-validation (T018), and at job-time + * re-resolution (T051) — the checks are identical each time; only WHEN they + * run, and what a candidate looks like when re-resolved, differs. + * + * A refusal is never acknowledgeable (FR-029, FR-030, FR-034) — only rows + * that reach the acknowledgement check below are. + */ +export function validateRestoreChoice(input: RestoreValidationInput): RestoreValidationResult { + const { candidate, choice, requiresEnv, envNotCarriedKeys, targetVersion } = input; + + if (candidate.skew.kind === 'refused') { + const details: Record = {}; + if (candidate.skew.code === 'RESTORE_SOURCE_NEWER') { + details.candidateVersion = candidate.appVersion; + details.targetVersion = targetVersion; + } else if (candidate.skew.suggestedVersion) { + details.suggestedVersion = candidate.skew.suggestedVersion; + } + return { ok: false, code: candidate.skew.code, message: candidate.skew.message, details }; + } + + // FR-034: `requiresEnv` turns the missing-environment-record WARNING into a + // REFUSAL — never acknowledgeable, because the app has said it cannot + // sensibly restore without its configuration. + const carriesUsable = choice.carryEnv && candidate.carriesEnv; + if (requiresEnv && !carriesUsable) { + return { + ok: false, + code: 'RESTORE_ENV_REQUIRED', + message: 'This app requires its captured configuration to restore, and the chosen candidate has none carried.', + details: { missingKeys: envNotCarriedKeys }, + }; + } + + const required = deriveRequiredAcknowledgements({ + skew: candidate.skew, + carryEnv: choice.carryEnv, + carriesEnv: candidate.carriesEnv, + }); + const acknowledged = new Set(choice.acknowledge ?? []); + const missing = required.filter((code) => !acknowledged.has(code)); + if (missing.length > 0) { + return { + ok: false, + code: 'RESTORE_ACK_REQUIRED', + message: `This restore needs acknowledgement of: ${missing.join(', ')}.`, + details: { required: missing }, + }; + } + + return { ok: true, requiredAcknowledgements: required }; +} + +export interface JudgeRestoreChoiceInput { + /** Already-fetched candidate state, or `undefined` if the id names no deployment at all. */ + source: CandidateSource | undefined; + appId: string; + excludeDeploymentId: string; + choice: RestoreChoice; + targetVersion: string | undefined; + meta: AppUpgradeMeta | undefined; + requiresEnv: boolean; + envNotCarriedKeys: string[]; +} + +export type JudgeRestoreChoiceResult = + | { ok: true; candidate: RestoreCandidate; requiredAcknowledgements: string[] } + | { ok: false; code: RestoreRefusalCode; message: string; details: Record }; + +/** + * Resolve + validate a restore choice against already-fetched state — the ONE + * function draft creation (T015), `createFromDraft`'s re-validation (T018), + * and the deploy job's job-time re-resolution (T051, FR-013a) all call. + * Composes {@link checkCandidateStillEligible}, {@link describeCandidate}, + * {@link computeSkewVerdict} and {@link validateRestoreChoice} — only WHEN + * this runs, and whether `source` might already be stale, differs between + * callers. + */ +export function judgeRestoreChoice(input: JudgeRestoreChoiceInput): JudgeRestoreChoiceResult { + const eligibility = checkCandidateStillEligible(input.source, input.appId, input.excludeDeploymentId); + if (!eligibility.ok) { + return { + ok: false, + code: eligibility.code, + message: `Restore source '${input.choice.candidateId}' is not available (${eligibility.code}).`, + details: { candidateId: input.choice.candidateId }, + }; + } + const described = describeCandidate(input.source!); + const skew = computeSkewVerdict(described.appVersion, input.targetVersion, input.meta); + const candidate: RestoreCandidate = { ...described, skew, requiredAcknowledgements: [], warnings: [] }; + + const result = validateRestoreChoice({ + candidate, + choice: input.choice, + requiresEnv: input.requiresEnv, + envNotCarriedKeys: input.envNotCarriedKeys, + targetVersion: input.targetVersion, + }); + if (!result.ok) return result; + return { ok: true, candidate, requiredAcknowledgements: result.requiredAcknowledgements }; +} diff --git a/packages/server/src/services/simple-factory.ts b/packages/server/src/services/simple-factory.ts index 3313de31..3622f994 100644 --- a/packages/server/src/services/simple-factory.ts +++ b/packages/server/src/services/simple-factory.ts @@ -139,6 +139,13 @@ export function createServices(env: ServiceEnvironment): Services { // Mock provisioner in development for safety (no calls to a real auth platform). const provisioner = new MockProvisionerService(); + const deployments = new RealDeploymentService(storage, jobs, docker, drafts, routing, logging, provisioner, catalog, eventBus, registryCredentials); + // Restore-on-install (spec 007): wire the back-reference draft-creation + // needs to resolve/validate a restoreFrom choice. Set post-construction — + // `deployments` itself takes `drafts` as a constructor argument, so this + // can't be threaded through either constructor without a cycle. + drafts.setDeploymentsService(deployments); + return { storage, config: new RealConfigService(storage), @@ -159,7 +166,7 @@ export function createServices(env: ServiceEnvironment): Services { validation, routing, provisioner, - deployments: new RealDeploymentService(storage, jobs, docker, drafts, routing, logging, provisioner, catalog, eventBus, registryCredentials), + deployments, }; } @@ -216,6 +223,11 @@ export function createServices(env: ServiceEnvironment): Services { ? new RealAuthentikProvisionerService(authConfig) : new NoneProvisionerService(); + const deployments = new RealDeploymentService(storage, jobs, docker, drafts, routing, logging, provisioner, catalog, eventBus, registryCredentials, contractTokens); + // Restore-on-install (spec 007): see the development block above for why + // this is a post-construction setter rather than a constructor argument. + drafts.setDeploymentsService(deployments); + return { storage, config: new RealConfigService(storage), @@ -236,7 +248,7 @@ export function createServices(env: ServiceEnvironment): Services { validation, routing, provisioner, - deployments: new RealDeploymentService(storage, jobs, docker, drafts, routing, logging, provisioner, catalog, eventBus, registryCredentials, contractTokens), + deployments, }; } diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index 48aaaa70..66f4cb5f 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -18,6 +18,13 @@ export const API = { me: '/api/me', summary: '/api/summary', + // Restore-on-install (spec 007): deployments of `appId` on this host that + // can serve as a restore source. GET with optional `?version=` (each + // candidate's skew/acknowledgements are computed against it). Returns + // ListRestoreCandidatesResponse. An ordinary authenticated platform read — + // NOT a capability-contract broker endpoint (FR-047, contracts/api.md §0). + restoreCandidates: (appId: string) => `/api/apps/${encodeURIComponent(appId)}/restore-candidates`, + catalog: { apps: '/api/catalog/apps', // list, query via ?query=&category=&page=&limit= refresh: '/api/catalog/refresh', @@ -334,6 +341,130 @@ export type AppBackupParticipation = { */ export type AppBackupDeclaration = AppBackupConfig | AppBackupParticipation[]; +// ------------------------------------------------------ +// Restore-on-install (spec 007) +// ------------------------------------------------------ + +/** + * Per-participation restore declaration in a bundle manifest's `restore` array, + * keyed by the **backup** participation id it restores. Reuses `AppBackupHook` + * verbatim for `hook` — a restore hook is a command run in a named service, + * exactly what a backup hook already is; a second hook shape would be a second + * thing to validate and get wrong for no expressive gain. + * + * Three declaration states, driven by `accepts`/`restore` together: + * - no `restore@1` in `accepts` → not offered as restorable at all. + * - `restore@1`, no matching entry here → plain file copy: nothing discarded, + * no hook runs. True for every SQLite/flat-file acceptor. + * - `restore@1` + an entry → `discard` paths are removed after extraction and + * before any container starts, then `hook` runs against the started, + * healthy service. + * + * `restore@1` here names a participation an app declares, not a capability + * contract the platform brokers — `CONTRACTS` (`contracts.ts`) gains no entry. + */ +export type AppRestoreDeclaration = { + /** The backup participation id this restores (`default` for the legacy singular form). */ + id: string; + /** Data-root-relative paths removed after extraction, before any container starts. */ + discard?: string[]; + /** Run after discards, against the started-and-healthy service. */ + hook?: AppBackupHook; + /** `true` turns the missing-environment-record warning into a refusal. Default `false`. */ + requiresEnv?: boolean; +}; + +/** The operator's restore decision at draft creation. Created once, consumed once. */ +export type RestoreChoice = { + /** The candidate's deployment id — not a lineage id. */ + candidateId: string; + /** Explicit, never defaulted from candidate state — declining is a decision. */ + carryEnv: boolean; + /** Acknowledgement codes (see {@link RestoreAcknowledgementCode}), modelled on `grants`. */ + acknowledge?: string[]; +}; + +/** + * Closed union of acknowledgement codes a restore choice may need to supply, + * computed server-side from the chosen candidate and refused when required and + * absent — the same enforcement `grants` already gets. + */ +export type RestoreAcknowledgementCode = 'restore-version-unknown' | 'restore-env-not-carried'; + +/** Closed union of refusal codes a restore can fail with, carried in `details.code`. */ +export type RestoreRefusalCode = + | 'RESTORE_SOURCE_NEWER' + | 'RESTORE_UPGRADE_PATH' + | 'RESTORE_ENV_REQUIRED' + | 'RESTORE_CANDIDATE_GONE' + | 'RESTORE_CANDIDATE_BUSY' + | 'RESTORE_TARGET_NOT_EMPTY' + | 'RESTORE_PAYLOAD_EMPTY' + | 'RESTORE_HOOK_FAILED' + | 'RESTORE_NOT_SUPPORTED' + // Distinct from RESTORE_NOT_SUPPORTED on purpose: that one means "this + // INSTALL PATH cannot restore" (install-by-ref, no catalog index), and its + // remedy is "use the catalog path". This one means "this APP has not + // declared it can be restored" (no `restore@1` in `accepts`) on a path that + // otherwise could. One code for both would hand every surface a hint that + // is wrong for half the cases it fires on. + | 'RESTORE_NOT_ACCEPTED' + | 'RESTORE_ACK_REQUIRED'; + +/** + * The version relationship between a candidate and the version being installed. + * Only the `refused` rows guarded by `checkUpgradePath` come from it — a candidate + * newer than the target, and an unknown version, are this feature's own rules, + * because `checkUpgradePath` returns `ok` for both (see `checkUpgradePath` above). + */ +export type RestoreSkewVerdict = + | { kind: 'ok' } + | { kind: 'unknown' } + | { kind: 'refused'; code: RestoreRefusalCode; message: string; suggestedVersion?: string }; + +/** A proceedable, named, non-fatal risk surfaced with a restore candidate. */ +export type RestoreWarning = + | { code: 'env-not-carried'; keys: string[] } + | { code: 'host-divergence'; from: string; to: string } + | { code: 'no-identity-record' }; + +/** + * A source the operator can pick as a restore-on-install source. Derived on + * demand from deployments + identity records + catalog upgrade metadata — + * never stored. Returned by `GET /api/apps/:appId/restore-candidates`. + */ +export type RestoreCandidate = { + deploymentId: string; + lineageId: string; + app: string; + name: string; + subdomain: string | null; + host: string | null; + appVersion: string | null; + channel: string | null; + carriesEnv: boolean; + capturedAt: string | null; + hasIdentityRecord: boolean; + skew: RestoreSkewVerdict; + requiredAcknowledgements: string[]; + warnings: RestoreWarning[]; +}; + +/** One family of candidates sharing a `lineageId`, newest-first within it. */ +export type RestoreCandidateLineage = { + lineageId: string; + candidates: RestoreCandidate[]; +}; + +export type ListRestoreCandidatesResponse = { + appId: string; + lineages: RestoreCandidateLineage[]; + // `null` / `true` whenever two or more distinct lineages match — no default + // is offered and the operator must pick explicitly. + defaultCandidateId: string | null; + requiresExplicitChoice: boolean; +}; + /** * How a push overwrites the target directory (#409). `mirror` is rsync * `--delete` — the local tree becomes the server tree, so files only on the @@ -1016,6 +1147,11 @@ export type GetCatalogAppVersionDetailResponse = { // path `hola app data push` can bulk-load into. Optional: most apps take their // data through their own UI and omit it. push?: AppPushTarget[]; + // Per-backup-participation restore declarations (spec 007), each naming + // `discard` paths and an optional restore `hook`. Optional: an app that + // accepts `restore@1` with no entry here restores by plain file copy; an app + // that doesn't accept `restore@1` at all isn't offered as restorable. + restore?: AppRestoreDeclaration[]; // Elevated container permissions the app requests (e.g. a browser desktop that // needs `sudo`). Each entry is surfaced for explicit operator consent in the // install wizard and relaxes the corresponding platform hardening at deploy @@ -1276,6 +1412,10 @@ export type Draft = { // through finalize so `push-targets` can resolve them against the deployment's // data root (read-only; not user-editable). push?: AppPushTarget[]; + // Per-backup-participation restore declarations (spec 007) seeded from the + // bundle manifest and carried through finalize (read-only; not user-editable) + // so the restore sequence can apply discards/hooks without re-reading the bundle. + restore?: AppRestoreDeclaration[]; // Optional Compose profiles the app declares (#162), seeded from the bundle // manifest so the install wizard can render a checkbox per profile. The // selected keys are sent on create; the declared list itself is read-only. @@ -1294,6 +1434,10 @@ export type Draft = { // absent/false whenever it could not be established (catalog unavailable, // install-by-ref, a pre-#431 draft). channelPublished?: boolean; + // The restore-on-install choice (spec 007), seeded at draft creation and + // carried through finalize outside `canonicalSpec` (beside `channel`), never + // patchable and never re-derived. Absent means no restore. + restoreFrom?: RestoreChoice; }; export type CreateDraftRequest = { @@ -1313,6 +1457,12 @@ export type CreateDraftRequest = { // channel is implied by a pinned `version`'s own channel, else `stable`. // Install-by-ref drafts ignore this (always `stable`). channel?: string; + // Restore-on-install choice (spec 007): pick an existing deployment of this + // app as a restore source. Accepted on the catalog path only — the + // install-by-ref path has no catalog upgrade metadata to judge version skew + // against and refuses it with `RESTORE_NOT_SUPPORTED` rather than silently + // ignoring it. + restoreFrom?: RestoreChoice; }; export type CreateDraftResponse = { draftId: string; @@ -1450,6 +1600,13 @@ export type DeploymentDetail = { // projected from the record written at create time. Absent means // single-instance. multiInstance?: boolean; + // Restore-on-install (spec 007), projected from the stored record — see + // EnhancedDeploymentDetail below for the full field-by-field description. + // All optional; a record written before this feature reads them as + // `undefined`. + lineageId?: string; + restoreFrom?: RestoreChoice; + restoredAt?: string; }; export type GetDeploymentResponse = DeploymentDetail; @@ -2052,6 +2209,22 @@ export type EnhancedDeploymentDetail = DeploymentDetail & { // changed afterward (a later channel change that causes overlap only // returns a PATCH warning). Absent for a first copy or a multi-instance app. instanceReason?: InstanceReason; + // Restore-on-install fields (spec 007). All optional — every record written + // before this feature stays valid, reading them as `undefined`. + // + // `lineageId` identifies the family of installs a chain of restores belongs + // to: a candidate's lineage on a restore, else the deployment's own id. + // Absence means "equals this deployment's id", which is what + // `writeInstanceMarkers` falls back to (`deployment.lineageId ?? deployment.id`) + // so a pre-spec-007 record needs no migration. + lineageId?: string; + // The restore choice that was applied to this install's first deploy, carried + // from the finalized manifest. Absent when this install was not restored. + restoreFrom?: RestoreChoice; + // ISO time the restore completed successfully. Its presence is the + // consumption marker: a restart/promote/rollback finds it set and skips the + // restore sequence — a restore applies to the first deploy only. + restoredAt?: string; metadata: { createdAt: string; owner?: string; @@ -2296,6 +2469,11 @@ export type CreateDeploymentFromDraftResponse = { // client can print/show "Following channel: " without a further lookup. // The server always emits it. channel?: string; + // Restore-on-install (spec 007): proceedable, non-fatal warnings resolved + // at create time — today, only `host-divergence` (FR-035, when the + // operator's chosen subdomain differs from the restore candidate's). + // Absent when no restore happened or nothing diverged. + warnings?: RestoreWarning[]; }; /** diff --git a/packages/web/src/__tests__/pages/InstallWizard.channels.test.tsx b/packages/web/src/__tests__/pages/InstallWizard.channels.test.tsx index 33ca3762..71f3166b 100644 --- a/packages/web/src/__tests__/pages/InstallWizard.channels.test.tsx +++ b/packages/web/src/__tests__/pages/InstallWizard.channels.test.tsx @@ -61,6 +61,12 @@ const catalogApi = { })), }; +// Restore-on-install (spec 007): the wizard's new first step reads this +// route before a draft exists. No candidates in this fixture set. +const restoreCandidates = vi.fn(async (appId: string) => ({ + appId, lineages: [], defaultCandidateId: null, requiresExplicitChoice: false, +})); + vi.mock('../../utils/api-hybrid', () => ({ api: { drafts: draftsApi, @@ -70,6 +76,7 @@ vi.mock('../../utils/api-hybrid', () => ({ update: (id: string, data: unknown) => updateDeployment(id, data), subdomainAvailable: (subdomain: string) => subdomainAvailable(subdomain), }, + restoreCandidates: (appId: string) => restoreCandidates(appId), }, })); @@ -94,8 +101,11 @@ function mockSettingsFetch() { }) as unknown as typeof fetch; } -function renderWizard(query = '') { - return render( +/** Render, then advance past the new Restore Data step (spec 007) — no + * candidates in these fixtures, so Next is immediately enabled there. Every + * assertion below predates this step and starts from "the draft now exists". */ +async function renderWizard(query = '') { + const utils = render( } /> @@ -104,6 +114,9 @@ function renderWizard(query = '') { ); + await waitFor(() => expect(screen.getByRole('button', { name: /next/i })).not.toBeDisabled()); + fireEvent.click(screen.getByRole('button', { name: /next/i })); + return utils; } async function clickNext() { @@ -136,6 +149,7 @@ beforeEach(() => { draftsApi.update.mockClear(); draftsApi.remove.mockClear(); catalogApi.appById.mockClear(); + restoreCandidates.mockClear(); }); afterEach(() => { @@ -145,7 +159,7 @@ afterEach(() => { describe('InstallWizard channel radio (spec 005 US1)', () => { it('has no radiogroup when not enrolled and no ?channel=', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -154,7 +168,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { it('shows the radiogroup with Stable/pre-release options when enrolled with 2+ channels', async () => { showPrerelease = true; - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -166,7 +180,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { it('is absent when enrolled but the app has only one channel', async () => { showPrerelease = true; catalogChannels = ['stable']; - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -174,7 +188,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { }); it('shows the radiogroup with rc checked when opened with ?channel=rc while NOT enrolled', async () => { - renderWizard('?channel=rc'); + await renderWizard('?channel=rc'); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -185,7 +199,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { it('selecting a radio option deletes the current draft and creates a new one on the chosen channel', async () => { showPrerelease = true; - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -212,7 +226,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { // every retry fails the same way and the operator can never get back. it('a failed channel switch falls back to the previous channel so Retry can recover', async () => { showPrerelease = true; - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -239,7 +253,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { }); channelByDraft.set(id, 'rc'); - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -248,7 +262,7 @@ describe('InstallWizard channel radio (spec 005 US1)', () => { }); it('does not show the non-stable note for a plain stable install', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); @@ -260,7 +274,7 @@ describe('InstallWizard opened via a channel link (spec 005 US3)', () => { it.each([false, true])('shows beta checked and the empty-data note when opened with ?channel=beta (enrolled=%s)', async (enrolled) => { showPrerelease = enrolled; catalogChannels = ['stable', 'beta']; - renderWizard('?channel=beta'); + await renderWizard('?channel=beta'); await waitFor(() => expect(draftsApi.create).toHaveBeenCalledWith( expect.objectContaining({ appId: 'demo', channel: 'beta' }) )); @@ -287,7 +301,7 @@ describe('InstallWizard already-installed conflict (spec 005 US4)', () => { async function installAndHitConflict(query = '?channel=beta') { catalogChannels = ['stable', 'beta']; - renderWizard(query); + await renderWizard(query); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); await walkToSummary(); fireEvent.click(screen.getByRole('button', { name: /^install$/i })); diff --git a/packages/web/src/__tests__/pages/InstallWizard.grants.test.tsx b/packages/web/src/__tests__/pages/InstallWizard.grants.test.tsx index 87a66f16..4ccd6452 100644 --- a/packages/web/src/__tests__/pages/InstallWizard.grants.test.tsx +++ b/packages/web/src/__tests__/pages/InstallWizard.grants.test.tsx @@ -37,6 +37,12 @@ const draftsApi = { finalize: vi.fn(async () => ({ spec: {}, checksum: 'x' })), }; +// Restore-on-install (spec 007): the wizard's new first step reads this +// route before a draft exists. No candidates in this fixture set. +const restoreCandidates = vi.fn(async (appId: string) => ({ + appId, lineages: [], defaultCandidateId: null, requiresExplicitChoice: false, +})); + vi.mock('../../utils/api-hybrid', () => ({ api: { drafts: draftsApi, @@ -44,14 +50,18 @@ vi.mock('../../utils/api-hybrid', () => ({ create: (data: unknown) => create(data), subdomainAvailable: vi.fn(async (subdomain: string) => ({ subdomain, host: `${subdomain}.local.hola`, available: true })), }, + restoreCandidates: (appId: string) => restoreCandidates(appId), }, })); // Imported after the mock so InstallWizard picks up the mocked api-hybrid. const { InstallWizard } = await import('../../pages/InstallWizard'); -function renderWizard() { - return render( +/** Render, then advance past the new Restore Data step (spec 007) — no + * candidates in these fixtures, so Next is immediately enabled there. Every + * assertion below predates this step and starts from "the draft now exists". */ +async function renderWizard() { + const utils = render( } /> @@ -59,6 +69,9 @@ function renderWizard() { ); + await waitFor(() => expect(screen.getByRole('button', { name: /next/i })).not.toBeDisabled()); + fireEvent.click(screen.getByRole('button', { name: /next/i })); + return utils; } /** Advance one step, using the pre-advance draft save as the transition signal. */ @@ -73,6 +86,7 @@ beforeEach(() => { create.mockClear(); draftsApi.create.mockClear(); draftsApi.update.mockClear(); + restoreCandidates.mockClear(); }); afterEach(() => { @@ -81,7 +95,7 @@ afterEach(() => { describe('InstallWizard privileged contract grants (ADR 0004)', () => { it('blocks Next until the declared grant is consented to, then sends it to create', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); // The grant is named in the operator's terms, not as a bare contract id. @@ -120,7 +134,7 @@ describe('InstallWizard privileged contract grants (ADR 0004)', () => { defaults: { ports: [], volumes: [] }, }); - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); expect(screen.queryByText(/requests access to other apps/i)).not.toBeInTheDocument(); @@ -137,7 +151,7 @@ describe('InstallWizard privileged contract grants (ADR 0004)', () => { provides: ['container-logs@1'], }); - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); const row = await screen.findByText(/read the logs of every container on this host/i); @@ -160,7 +174,7 @@ describe('InstallWizard privileged contract grants (ADR 0004)', () => { provides: ['backup@1', 'container-logs@1'], }); - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); expect(await screen.findByText(/read the data of every installed app/i)).toBeInTheDocument(); @@ -183,7 +197,7 @@ describe('InstallWizard privileged contract grants (ADR 0004)', () => { provides: ['container-logs@1'], }); - renderWizard(); + await renderWizard(); await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); const row = await screen.findByText(/read the logs of every container on this host/i); diff --git a/packages/web/src/__tests__/pages/InstallWizard.profiles.test.tsx b/packages/web/src/__tests__/pages/InstallWizard.profiles.test.tsx index 98843be2..2422733a 100644 --- a/packages/web/src/__tests__/pages/InstallWizard.profiles.test.tsx +++ b/packages/web/src/__tests__/pages/InstallWizard.profiles.test.tsx @@ -40,6 +40,12 @@ const draftsApi = { finalize: vi.fn(async () => ({ spec: {}, checksum: 'x' })), }; +// Restore-on-install (spec 007): the wizard's new first step reads this +// route before a draft exists. No candidates in this fixture set. +const restoreCandidates = vi.fn(async (appId: string) => ({ + appId, lineages: [], defaultCandidateId: null, requiresExplicitChoice: false, +})); + vi.mock('../../utils/api-hybrid', () => ({ api: { drafts: draftsApi, @@ -47,14 +53,18 @@ vi.mock('../../utils/api-hybrid', () => ({ create: (data: unknown) => create(data), subdomainAvailable: vi.fn(async (subdomain: string) => ({ subdomain, host: `${subdomain}.local.hola`, available: true })), }, + restoreCandidates: (appId: string) => restoreCandidates(appId), }, })); // Imported after the mock so InstallWizard picks up the mocked api-hybrid. const { InstallWizard } = await import('../../pages/InstallWizard'); -function renderWizard() { - return render( +/** Render, then advance past the new Restore Data step (spec 007) — no + * candidates in these fixtures, so Next is immediately enabled there. Every + * assertion below predates this step and starts from "the draft now exists". */ +async function renderWizard() { + const utils = render( } /> @@ -62,6 +72,9 @@ function renderWizard() { ); + await waitFor(() => expect(screen.getByRole('button', { name: /next/i })).not.toBeDisabled()); + fireEvent.click(screen.getByRole('button', { name: /next/i })); + return utils; } // Advance one step: click Next and wait for handleNext's pre-advance draft @@ -77,6 +90,7 @@ beforeEach(() => { globalCache.clear(); create.mockClear(); draftsApi.create.mockClear(); + restoreCandidates.mockClear(); }); afterEach(() => { @@ -85,7 +99,7 @@ afterEach(() => { describe('InstallWizard optional Compose profiles (#162)', () => { it('renders declared profiles on the summary step and sends the enabled set to create', async () => { - renderWizard(); + await renderWizard(); // Wait for the draft to resolve (step 0 heading is the app name). await waitFor(() => expect(draftsApi.create).toHaveBeenCalled()); diff --git a/packages/web/src/__tests__/pages/InstallWizard.refNotAllowed.test.tsx b/packages/web/src/__tests__/pages/InstallWizard.refNotAllowed.test.tsx index fb9aea5c..9b7d387a 100644 --- a/packages/web/src/__tests__/pages/InstallWizard.refNotAllowed.test.tsx +++ b/packages/web/src/__tests__/pages/InstallWizard.refNotAllowed.test.tsx @@ -55,6 +55,12 @@ const catalogSources = { update: vi.fn(async () => ({ id: 'pofallon', name: 'pofallon', type: 'index-url' as const, url: 'https://example.test/catalog.json', trust: 'custom' as const, enabled: true })), }; +// Restore-on-install (spec 007): the wizard's new first step reads this +// route before a draft exists. No candidates in this fixture set. +const restoreCandidates = vi.fn(async (appId: string) => ({ + appId, lineages: [], defaultCandidateId: null, requiresExplicitChoice: false, +})); + vi.mock('../../utils/api-hybrid', () => ({ api: { drafts: draftsApi, @@ -63,19 +69,27 @@ vi.mock('../../utils/api-hybrid', () => ({ create: vi.fn(), subdomainAvailable: vi.fn(async (subdomain: string) => ({ subdomain, host: `${subdomain}.local.hola`, available: true })), }, + restoreCandidates: (appId: string) => restoreCandidates(appId), }, })); const { InstallWizard } = await import('../../pages/InstallWizard'); -function renderWizard(search = '?source=pofallon') { - return render( +/** Render, then advance past the new Restore Data step (spec 007) — no + * candidates in these fixtures (and auto-skipped outright for the + * install-by-ref case), so Next is immediately enabled there. Every + * assertion below predates this step and starts from "the draft now exists". */ +async function renderWizard(search = '?source=pofallon') { + const utils = render( } /> ); + await waitFor(() => expect(screen.getByRole('button', { name: /next/i })).not.toBeDisabled()); + fireEvent.click(screen.getByRole('button', { name: /next/i })); + return utils; } beforeEach(() => { @@ -88,7 +102,7 @@ afterEach(() => cleanup()); describe('InstallWizard REF_NOT_ALLOWED recovery', () => { it('offers the exact fix and applies it as an additional grant, then retries the install', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(screen.getByText('Registry not allowed')).toBeInTheDocument()); // The suggestion names the registry to be granted — not the raw ref. @@ -112,7 +126,7 @@ describe('InstallWizard REF_NOT_ALLOWED recovery', () => { it('does not offer to patch a source that has none to patch (install-by-ref)', async () => { // `/install/ref?ref=…` has no stored source record, so the only remedy is the // server-wide baseline — say so instead of dangling an unusable button. - renderWizard('?ref=ghcr.io/pofallon/hola-get2know-cms:0.1.13'); + await renderWizard('?ref=ghcr.io/pofallon/hola-get2know-cms:0.1.13'); await waitFor(() => expect(screen.getByText('Registry not allowed')).toBeInTheDocument()); expect(screen.queryByRole('button', { name: /allow .* for/i })).not.toBeInTheDocument(); @@ -126,7 +140,7 @@ describe('InstallWizard REF_NOT_ALLOWED recovery', () => { draftsApi.create.mockImplementation(async () => { throw Object.assign(new Error('REF_NOT_ALLOWED: blocked'), { code: 'REF_NOT_ALLOWED', statusCode: 403 }); }); - renderWizard(); + await renderWizard(); await waitFor(() => expect(screen.getByText('REF_NOT_ALLOWED: blocked')).toBeInTheDocument()); expect(screen.queryByRole('button', { name: /allow/i })).not.toBeInTheDocument(); @@ -134,7 +148,7 @@ describe('InstallWizard REF_NOT_ALLOWED recovery', () => { }); it('does not hammer the server: a failed draft is attempted once until the operator retries', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(screen.getByText('Registry not allowed')).toBeInTheDocument()); // Settle: the effect re-runs on every render, so a missing guard shows up here. diff --git a/packages/web/src/__tests__/pages/InstallWizard.restore.test.tsx b/packages/web/src/__tests__/pages/InstallWizard.restore.test.tsx new file mode 100644 index 00000000..dc2da3c0 --- /dev/null +++ b/packages/web/src/__tests__/pages/InstallWizard.restore.test.tsx @@ -0,0 +1,243 @@ +import React from 'react'; +import { render, screen, fireEvent, waitFor, cleanup } from '@testing-library/react'; +import { describe, it, expect, afterEach, beforeEach, vi } from 'vitest'; +import { MemoryRouter, Routes, Route } from 'react-router-dom'; +import type { CreateDraftRequest, CreateDraftResponse, Draft, ListRestoreCandidatesResponse } from '@hola/shared'; +import { globalCache } from '../../utils/cache'; + +// Restore-on-install (spec 007). Covers quickstart.md scenarios 44-48 (mode +// "W"): the restore step renders first, changing the choice re-creates the +// draft, carried values render as ordinary appEnv rows, the summary +// acknowledgement appears, and "no candidates" still lets Next proceed. +// Template: InstallWizard.channels.test.tsx. + +let draftCounter = 0; +const draftsByChoice = new Map(); + +const draftsApi = { + create: vi.fn(async (req: CreateDraftRequest): Promise => { + draftCounter += 1; + const draftId = `draft-${draftCounter}`; + const carried = req.restoreFrom?.candidateId === 'demo-source' && req.restoreFrom.carryEnv; + const appEnv = carried + ? [{ key: 'ADMIN_PASSWORD', value: 'carried-secret-value', isSecret: true, label: 'Admin password' }] + : [{ key: 'ADMIN_PASSWORD', value: '', isSecret: true, label: 'Admin password', generate: { kind: 'hex' as const, length: 4 } }]; + const draft: Draft = { draftId, appId: 'demo', version: '1.0.0', systemOverrides: {}, appEnv, ports: [] }; + draftsByChoice.set(draftId, draft); + return { draftId, app: { id: 'demo', name: 'Demo', icon: '📦' }, systemEnv: [], appEnv, defaults: { ports: [], volumes: [] } }; + }), + byId: vi.fn(async (id: string): Promise => draftsByChoice.get(id)!), + update: vi.fn(async (id: string, updates: Partial) => ({ ok: true as const, draft: { ...draftsByChoice.get(id)!, ...updates } })), + remove: vi.fn(async () => ({ ok: true as const })), + validate: vi.fn(async () => ({ ok: true, errors: [], warnings: [] })), + preflight: vi.fn(async () => ({ ok: true, checks: [] })), + finalize: vi.fn(async () => ({ spec: {}, checksum: 'x' })), +}; + +const create = vi.fn(async () => ({ deploymentId: 'dep1', releaseId: 'r1', jobId: 'j1' })); + +// One lineage, one candidate, WITH a carried environment record. +const DEFAULT_CANDIDATES_RESPONSE: ListRestoreCandidatesResponse = { + appId: 'demo', + lineages: [{ + lineageId: 'demo-source', + candidates: [{ + deploymentId: 'demo-source', + lineageId: 'demo-source', + app: 'demo', + name: 'Demo (original)', + subdomain: 'demo', + host: 'demo.local.hola', + appVersion: '1.0.0', + channel: 'stable', + carriesEnv: true, + capturedAt: '2026-01-01T00:00:00.000Z', + hasIdentityRecord: true, + skew: { kind: 'ok' }, + requiredAcknowledgements: [], + warnings: [], + }], + }], + defaultCandidateId: 'demo-source', + requiresExplicitChoice: false, +}; +// Mutable per test — reset to a deep copy of the default in `beforeEach`. +let restoreCandidatesResponse: ListRestoreCandidatesResponse = DEFAULT_CANDIDATES_RESPONSE; +const restoreCandidates = vi.fn(async (...__args: [string, (string | undefined)?, (string | undefined)?, (string | undefined)?]) => { + void __args; + return restoreCandidatesResponse; +}); + +vi.mock('../../utils/api-hybrid', () => ({ + api: { + drafts: draftsApi, + deployments: { + create: (data: unknown) => create(data), + subdomainAvailable: vi.fn(async (subdomain: string) => ({ subdomain, host: `${subdomain}.local.hola`, available: true })), + }, + restoreCandidates: (appId: string, version?: string, source?: string, channel?: string) => restoreCandidates(appId, version, source, channel), + }, +})); + +const { InstallWizard } = await import('../../pages/InstallWizard'); + +function renderWizard(query = '') { + return render( + + + } /> + Deployments} /> + + + ); +} + +/** Advance one step past Configuration or later, using the pre-advance draft + * save as the transition signal. */ +async function clickNext() { + const before = draftsApi.update.mock.calls.length; + fireEvent.click(screen.getByRole('button', { name: /next/i })); + await waitFor(() => expect(draftsApi.update.mock.calls.length).toBeGreaterThan(before)); +} + +/** Leave the restore step specifically — there's no draft yet to save, so the + * transition signal is the draft CREATE call it triggers instead. */ +async function leaveRestoreStep() { + const before = draftsApi.create.mock.calls.length; + fireEvent.click(screen.getByRole('button', { name: /next/i })); + await waitFor(() => expect(draftsApi.create.mock.calls.length).toBeGreaterThan(before)); +} + +beforeEach(() => { + globalCache.clear(); + draftCounter = 0; + draftsByChoice.clear(); + create.mockClear(); + draftsApi.create.mockClear(); + draftsApi.update.mockClear(); + draftsApi.remove.mockClear(); + restoreCandidates.mockClear(); + restoreCandidatesResponse = structuredClone(DEFAULT_CANDIDATES_RESPONSE); +}); + +afterEach(() => { + cleanup(); +}); + +describe('InstallWizard restore-on-install (spec 007)', () => { + // ---- scenario 44: the restore step renders at index 0, before Configuration ---- + it('scenario 44: renders the restore step first, and no draft exists yet', async () => { + renderWizard(); + + await waitFor(() => expect(screen.getByText(/Step 1 — Restore Data/)).toBeInTheDocument()); + // No draft created yet — the choice must be made first (research R1). + expect(draftsApi.create).not.toHaveBeenCalled(); + // The candidate is offered by name. + expect(await screen.findByText(/Demo \(original\)/)).toBeInTheDocument(); + }); + + // ---- scenario 48: with no candidates, the step says so and Next is enabled ---- + it('scenario 48: with no candidates, the step explains that and Next is enabled', async () => { + restoreCandidatesResponse = { appId: 'demo', lineages: [], defaultCandidateId: null, requiresExplicitChoice: false }; + renderWizard(); + + await waitFor(() => expect(screen.getByText(/No existing copies of demo were found/i)).toBeInTheDocument()); + expect(screen.getByRole('button', { name: /next/i })).not.toBeDisabled(); + }); + + // ---- scenario 46: carried values render as ordinary appEnv rows ---- + it('scenario 46: choosing a candidate carries its configuration into ordinary appEnv rows', async () => { + renderWizard(); + await waitFor(() => expect(screen.getByText(/Demo \(original\)/)).toBeInTheDocument()); + + // The candidate + carry-env are the defaults (FR-005 default selection, + // CLI/wizard default-on-when-available) — proceed straight to Configuration. + await leaveRestoreStep(); + + expect(draftsApi.create).toHaveBeenCalledWith( + expect.objectContaining({ restoreFrom: { candidateId: 'demo-source', carryEnv: true } }), + ); + + // Rendered through the SAME labeled input every other secret uses — no + // separate "restored value" widget. + const input = await screen.findByLabelText(/Admin password/i) as HTMLInputElement; + expect(input.value).toBe('carried-secret-value'); + }); + + // ---- scenario 47: the summary step's unconditional restore acknowledgement ---- + it('scenario 47: the summary step names data AND credentials, and that jobs/webhooks may fire', async () => { + renderWizard(); + await waitFor(() => expect(screen.getByText(/Demo \(original\)/)).toBeInTheDocument()); + await leaveRestoreStep(); // restore -> env, creates the draft + + await clickNext(); // env -> compose + await clickNext(); // compose -> files + await clickNext(); // files -> advanced + await clickNext(); // advanced -> validate + await clickNext(); // validate -> summary + + await waitFor(() => expect(screen.getByText('Summary & confirm')).toBeInTheDocument()); + expect(screen.getByText(/restores data/i)).toBeInTheDocument(); + expect(screen.getByText(/and credentials/i)).toBeInTheDocument(); + expect(screen.getByText(/jobs, ?\s*webhooks or integrations/i)).toBeInTheDocument(); + }); + + // ---- scenario 45: changing the choice deletes + re-creates the draft ---- + it('scenario 45: changing the restore choice after a draft exists deletes and re-creates it', async () => { + renderWizard(); + await waitFor(() => expect(screen.getByText(/Demo \(original\)/)).toBeInTheDocument()); + await leaveRestoreStep(); // commits the default choice, creates draft 1 + + expect(draftsApi.create).toHaveBeenCalledTimes(1); + const firstDraftId = (await draftsApi.create.mock.results[0]!.value).draftId; + + // Back to the restore step, deselect the candidate ("Start fresh"). + fireEvent.click(screen.getByRole('button', { name: /back/i })); + const freshRadio = await screen.findByRole('radio', { name: /Start fresh/i }); + fireEvent.click(freshRadio); + await leaveRestoreStep(); + + await waitFor(() => expect(draftsApi.remove).toHaveBeenCalledWith(firstDraftId)); + expect(draftsApi.create).toHaveBeenCalledTimes(2); + const secondCall = draftsApi.create.mock.calls[1]![0] as CreateDraftRequest; + expect(secondCall.restoreFrom).toBeUndefined(); + }); + + // ---- review: the step must judge skew against the version it will install ---- + it('reads candidates against the version/source/channel the draft will be created with', async () => { + renderWizard('?source=pofallon&channel=rc'); + await waitFor(() => expect(restoreCandidates).toHaveBeenCalled()); + + // Without a target version every candidate's skew comes back `unknown`, + // which would demand `restore-version-unknown` from every operator and + // hide RESTORE_SOURCE_NEWER / RESTORE_UPGRADE_PATH until create time. + // `source`/`channel` ride along so the route resolves the SAME concrete + // version `createDraft` will. + expect(restoreCandidates).toHaveBeenCalledWith('demo', 'latest', 'pofallon', 'rc'); + }); + + // ---- review: a refused re-create must not strand Retry on the bad choice ---- + it('falls back to the previous restore choice when re-creating the draft is refused', async () => { + renderWizard(); + await waitFor(() => expect(screen.getByText(/Demo \(original\)/)).toBeInTheDocument()); + await leaveRestoreStep(); // commits the default choice, creates draft 1 + + // Go back and switch to "Start fresh", but make the re-create fail — the + // old draft is already deleted by then, so `draftId` goes falsy and the + // wizard body unmounts behind the error panel. Retry re-runs the mount + // path, which reads the restore choice: it must be the one that worked. + fireEvent.click(screen.getByRole('button', { name: /back/i })); + fireEvent.click(await screen.findByRole('radio', { name: /Start fresh/i })); + draftsApi.create.mockRejectedValueOnce(new Error('RESTORE_CANDIDATE_GONE')); + fireEvent.click(screen.getByRole('button', { name: /next/i })); + await waitFor(() => expect(draftsApi.remove).toHaveBeenCalled()); + await waitFor(() => expect(draftsApi.create).toHaveBeenCalledTimes(2)); + + // Retry: the third create carries the choice that last worked, not the + // refused one, so the operator is not stuck on a permanently-failing retry. + fireEvent.click(await screen.findByRole('button', { name: /try again/i })); + await waitFor(() => expect(draftsApi.create).toHaveBeenCalledTimes(3)); + const retryCall = draftsApi.create.mock.calls[2]![0] as CreateDraftRequest; + expect(retryCall.restoreFrom).toEqual({ candidateId: 'demo-source', carryEnv: true }); + }); +}); diff --git a/packages/web/src/__tests__/pages/InstallWizard.secretWand.test.tsx b/packages/web/src/__tests__/pages/InstallWizard.secretWand.test.tsx index 441b37cc..3030d985 100644 --- a/packages/web/src/__tests__/pages/InstallWizard.secretWand.test.tsx +++ b/packages/web/src/__tests__/pages/InstallWizard.secretWand.test.tsx @@ -58,24 +58,37 @@ const draftsApi = { finalize: vi.fn(), }; +// Restore-on-install (spec 007): the wizard's new first step reads this +// route before a draft exists. No candidates in this fixture set. +const restoreCandidates = vi.fn(async (appId: string) => ({ + appId, lineages: [], defaultCandidateId: null, requiresExplicitChoice: false, +})); + vi.mock('../../utils/api-hybrid', () => ({ api: { drafts: draftsApi, deployments: { create: vi.fn() }, + restoreCandidates: (appId: string) => restoreCandidates(appId), }, })); // Imported after the mock so InstallWizard picks up the mocked api-hybrid. const { InstallWizard } = await import('../../pages/InstallWizard'); -function renderWizard() { - return render( +/** Render, then advance past the new Restore Data step (spec 007) — no + * candidates in these fixtures, so Next is immediately enabled there. Every + * assertion below predates this step and starts from "the draft now exists". */ +async function renderWizard() { + const utils = render( } /> ); + await waitFor(() => expect(screen.getByRole('button', { name: /next/i })).not.toBeDisabled()); + fireEvent.click(screen.getByRole('button', { name: /next/i })); + return utils; } beforeEach(() => { @@ -83,6 +96,7 @@ beforeEach(() => { draftsApi.create.mockClear(); draftsApi.byId.mockClear(); draftsApi.update.mockClear(); + restoreCandidates.mockClear(); }); afterEach(() => { @@ -100,7 +114,7 @@ function wandFor(labelPattern: RegExp): HTMLElement { describe('InstallWizard secret wand', () => { it('auto-fills a seeded secret with a `generate` recipe on draft load, no click required', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => { const generatedInput = screen.getByLabelText(/^Generated secret/) as HTMLInputElement; @@ -113,7 +127,7 @@ describe('InstallWizard secret wand', () => { }); it('does not auto-fill a seeded secret with no generate recipe', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(screen.getByLabelText(/^Generated secret/)).toBeInTheDocument()); const legacyInput = screen.getByLabelText(/^Legacy secret/) as HTMLInputElement; @@ -121,7 +135,7 @@ describe('InstallWizard secret wand', () => { }); it('uses generateSecretValue with the spec recipe when the wand is clicked again', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(screen.getByLabelText(/^Generated secret/)).toBeInTheDocument()); fireEvent.click(wandFor(/^Generated secret/)); @@ -133,7 +147,7 @@ describe('InstallWizard secret wand', () => { }); it('falls back to the legacy 32-byte-hex value for a seeded secret with no generate recipe', async () => { - renderWizard(); + await renderWizard(); await waitFor(() => expect(screen.getByLabelText(/^Legacy secret/)).toBeInTheDocument()); fireEvent.click(wandFor(/^Legacy secret/)); diff --git a/packages/web/src/pages/InstallWizard.tsx b/packages/web/src/pages/InstallWizard.tsx index a75c765b..f3aadf34 100644 --- a/packages/web/src/pages/InstallWizard.tsx +++ b/packages/web/src/pages/InstallWizard.tsx @@ -14,6 +14,9 @@ import type { AppProfileConfig, GetSubdomainAvailabilityResponse, RefNotAllowedDetails, + ListRestoreCandidatesResponse, + RestoreCandidate, + RestoreChoice, } from '@hola/shared'; import { providerGrantsFor } from '@hola/shared/contracts'; import { slugifySubdomain, STABLE_CHANNEL } from '@hola/shared'; @@ -26,7 +29,14 @@ import { useDraftFinalization } from '../hooks/useDraftFinalization'; import { usePrereleaseEnrolment } from '../hooks/usePrereleaseEnrolment'; import { api } from '../utils/api-hybrid'; +// Restore-on-install (spec 007, FR-038): forced to index 0 — Configuration +// (env, below) renders `appEnv`, and `appEnv` is seeded by the restore choice +// at draft creation (research R1), so the choice must be made before a draft +// exists at all. On the install-by-ref path (research R2 — no catalog index +// to judge a candidate's version against) the step auto-completes with no +// choice, so its Next/skip proceeds immediately without offering anything. const steps = [ + { id: 'restore', name: 'Restore Data', description: 'Optionally restore this install from an existing copy' }, { id: 'env', name: 'Configuration', description: 'Configure application settings and permissions' }, { id: 'compose', name: 'Compose Override', description: 'Upload custom Docker Compose configuration' }, { id: 'files', name: 'Additional Files', description: 'Upload configuration files and certificates' }, @@ -372,6 +382,62 @@ export const InstallWizard: React.FC = () => { // who hasn't enrolled in pre-release discovery (never on by default). const enrolled = usePrereleaseEnrolment(); + // Restore-on-install (spec 007). The candidates route is read directly off + // the route param (no draft needed — research R6), so it's available + // before the mount effect below even creates one. + const [restoreCandidatesData, setRestoreCandidatesData] = useState(null); + const [restoreCandidatesLoading, setRestoreCandidatesLoading] = useState(false); + const [restoreCandidatesError, setRestoreCandidatesError] = useState(null); + const [selectedCandidateId, setSelectedCandidateId] = useState(null); + const [restoreCarryEnv, setRestoreCarryEnv] = useState(true); + const [ackedRestoreCodes, setAckedRestoreCodes] = useState>(new Set()); + // Set once the operator has EITHER picked a candidate and confirmed, OR + // explicitly chosen to start fresh — this is what gates the draft-creation + // mount effect below, since `appEnv` can only be seeded once, at draft + // creation, from whatever the choice was (research R1). + const [restoreDecided, setRestoreDecided] = useState(false); + // The confirmed choice sent on draft creation. `undefined` means "start + // fresh" — a candidate existing is never itself consent to use it. + const [restoreChoice, setRestoreChoice] = useState(undefined); + const restoreChoiceRef = React.useRef(undefined); + React.useEffect(() => { restoreChoiceRef.current = restoreChoice; }, [restoreChoice]); + + const allRestoreCandidates: RestoreCandidate[] = (restoreCandidatesData?.lineages ?? []).flatMap(l => l.candidates); + const selectedRestoreCandidate = allRestoreCandidates.find(c => c.deploymentId === selectedCandidateId) ?? null; + // Install-by-ref has no catalog index to judge a candidate's version + // against (research R2) — the step auto-completes with nothing offered. + // Attempted-once guard. `restoreCandidatesData`/`…Loading` alone are NOT + // enough: on a failed read `data` stays null and `loading` flips back to + // false, and `loading` is itself a dependency — so the effect re-fires + // immediately and retries forever, hammering the server. Exactly the + // unbounded-retry shape the draft-creation effect below documents (its + // `creatingDraftRef` guard is deliberately left set after a failure). + const restoreCandidatesAttemptedRef = React.useRef(false); + React.useEffect(() => { + if (ociRef) { setRestoreDecided(true); return; } + if (!appId || restoreCandidatesData || restoreCandidatesLoading) return; + if (restoreCandidatesAttemptedRef.current) return; + restoreCandidatesAttemptedRef.current = true; + setRestoreCandidatesLoading(true); + // `'latest'` is the version the draft this step precedes will be created + // with (`createDraft({ appId, source, channel })` sends none, and the + // server resolves an absent version to `latest`) — the wizard has no + // pinned version to read here, since `version` is only known once a draft + // exists. Passing it is load-bearing, not cosmetic: WITHOUT a version the + // route has no target to compare against, every candidate's skew comes + // back `unknown`, and the step would demand `restore-version-unknown` + // from every operator while never surfacing RESTORE_SOURCE_NEWER or + // RESTORE_UPGRADE_PATH until the create fails (FR-029/FR-030/FR-032). + api.restoreCandidates(appId, 'latest', source, channel) + .then((resp) => { + setRestoreCandidatesData(resp); + // A single matching lineage supplies a default selection (FR-005). + if (resp.defaultCandidateId) setSelectedCandidateId(resp.defaultCandidateId); + }) + .catch((err) => setRestoreCandidatesError(err instanceof Error ? err.message : 'Failed to load restore candidates')) + .finally(() => setRestoreCandidatesLoading(false)); + }, [appId, ociRef, source, channel, restoreCandidatesData, restoreCandidatesLoading]); + // Real app metadata, resolved from the catalog when the draft is created. // Falls back to the route's appId for the brief window before the draft loads. const app = createDraftHook.data?.app ?? { id: appId ?? ociRef ?? '', name: appId ?? ociRef ?? 'app', icon: '📦' }; @@ -438,7 +504,7 @@ export const InstallWizard: React.FC = () => { const createAndSeedDraft = async (channelArg: string | undefined) => { const result = ociRef ? await createDraftHook.createDraft({ ociRef, credentialRef }) - : await createDraftHook.createDraft({ appId, source, channel: channelArg }); + : await createDraftHook.createDraft({ appId, source, channel: channelArg, restoreFrom: restoreChoiceRef.current }); // Auto-fill empty secrets that carry a manifest `generate` recipe — // these are machine tokens (runner registration keys, app secret @@ -481,7 +547,11 @@ export const InstallWizard: React.FC = () => { }; useEffect(() => { - if ((!appId && !ociRef) || createDraftHook.data || creatingDraftRef.current) return; + // Restore-on-install (spec 007, FR-038): the restore step comes first and + // must be DECIDED (a candidate confirmed, or explicitly skipped) before a + // draft is created at all — `appEnv` is seeded by the choice at creation + // time and can't be revised afterward except by re-creating the draft. + if ((!appId && !ociRef) || createDraftHook.data || creatingDraftRef.current || !restoreDecided) return; creatingDraftRef.current = true; createAndSeedDraft(channel).catch((err) => { @@ -496,7 +566,7 @@ export const InstallWizard: React.FC = () => { // as-is (it's a new reference each render, so the ref guard above — not this // list — is what keeps the draft created exactly once). // eslint-disable-next-line react-hooks/exhaustive-deps - }, [appId, ociRef, credentialRef, source, draftAttempt, createDraftHook]); + }, [appId, ociRef, credentialRef, source, draftAttempt, createDraftHook, restoreDecided]); // #428: the operator picked a different channel on the summary step. Drop // the current (now-wrong-version) draft, reset the per-draft state, and @@ -548,6 +618,64 @@ export const InstallWizard: React.FC = () => { } }; + // Restore-on-install (spec 007, FR-039): commit a restore choice. With no + // draft yet (the FIRST decision), just record it — the mount effect above + // creates the draft once `restoreDecided` is true. With a draft already + // existing (the operator went Back and changed the choice), delete + + // re-create it — `switchChannel`'s exact pattern, for the same reason: the + // choice can only take effect at draft CREATION (research R1), and + // re-creating resets consent the same way a channel change does. + const [restoreSwitching, setRestoreSwitching] = useState(false); + const [restoreSwitchError, setRestoreSwitchError] = useState(null); + const applyRestoreChoice = async (choice: RestoreChoice | undefined) => { + const oldDraftId = createDraftHook.data?.draftId; + if (!oldDraftId) { + restoreChoiceRef.current = choice; + setRestoreChoice(choice); + setRestoreDecided(true); + return; + } + // The choice that currently has a working draft, to fall back to if the + // re-create fails (see the catch below) — `switchChannel`'s `previousChannel`. + const previousChoice = restoreChoiceRef.current; + setRestoreSwitching(true); + setRestoreSwitchError(null); + try { + await api.drafts.remove(oldDraftId).catch(() => {}); + setEnvVars([]); + setSystemEnvVars([]); + setPorts([]); + setVolumes([]); + setSecurity(undefined); + setProvides(undefined); + setProfiles(undefined); + setSelectedProfiles(new Set()); + seededKeysRef.current = new Set(); + setTouchedKeys(new Set()); + setAckedGrants(new Set()); + setComposeOverride(''); + restoreChoiceRef.current = choice; + setRestoreChoice(choice); + await createAndSeedDraft(channel); + } catch (err) { + // Same trap `switchChannel` documents: the old draft is already gone and + // `useCreateDraft` clears `data` on failure, so `draftId` goes falsy and + // the whole wizard body — the restore step included — unmounts behind the + // DraftErrorPanel. Its Retry re-runs the mount path, which reads + // `restoreChoiceRef`: leaving it on the choice that just got REFUSED + // (RESTORE_SOURCE_NEWER, RESTORE_UPGRADE_PATH, a candidate that went + // away) strands the operator on a permanently-failing retry. Put it back + // to the last choice that produced a working draft. + restoreChoiceRef.current = previousChoice; + setRestoreChoice(previousChoice); + setSelectedCandidateId(previousChoice?.candidateId ?? null); + setAckedRestoreCodes(new Set(previousChoice?.acknowledge ?? [])); + setRestoreSwitchError(err instanceof Error ? err.message : 'Failed to change the restore choice'); + } finally { + setRestoreSwitching(false); + } + }; + /** Re-run draft creation after a failure the operator has just fixed. */ const retryDraft = React.useCallback(() => { creatingDraftRef.current = false; @@ -676,6 +804,24 @@ export const InstallWizard: React.FC = () => { const canProceed = () => { switch (currentStep) { case 0: { + // Restore Data. Nothing selected -> starting fresh, always fine once + // the candidates read has settled. A selected candidate additionally + // needs every acknowledgement it currently requires (FR-037a) — the + // same "informed consent gates Next" rule `grants`/`security` use. + // Also blocked mid-`applyRestoreChoice` (FR-039's delete-and-recreate + // in flight), same as the Channel switch above. + if (restoreSwitching) return false; + if (!selectedCandidateId) return !restoreCandidatesLoading; + // A REFUSED candidate is never acknowledgeable (FR-029/FR-030/FR-034): + // its refusal message is already on screen, and the server would throw + // the same refusal out of `drafts.create` a moment later. Block Next + // here so the operator picks another candidate (or Start fresh) + // instead of being bounced into a draft-creation error panel. + if (selectedRestoreCandidate?.skew.kind === 'refused') return false; + const required = selectedRestoreCandidate?.requiredAcknowledgements ?? []; + return !restoreCandidatesLoading && required.every(code => ackedRestoreCodes.has(code)); + } + case 1: { // Environment Variables. A completely empty row is an unused placeholder // (e.g. a default trailing row or one added via "Add variable") — it must // not block Next. Only rows the user actually started filling in are @@ -699,7 +845,7 @@ export const InstallWizard: React.FC = () => { const grantsAcked = providerGrantsFor(provides).every(g => ackedGrants.has(g.ref)); return !isLoading && requiredOk && noBlockingIssues && permissionsAcked && grantsAcked; } - case 4: // Validate & Preflight + case 5: // Validate & Preflight // Allow proceeding if not loading, and either checks haven't run yet OR both have passed return !isLoading && (!validationResult || (validationResult?.ok && preflightResult?.ok)); default: @@ -708,9 +854,26 @@ export const InstallWizard: React.FC = () => { }; const handleNext = async () => { + // Restore-on-install (spec 007): leaving the restore step commits the + // choice — first decision (no draft yet, the mount effect creates one) OR + // a CHANGE to an already-decided choice (FR-039: delete + re-create, + // `switchChannel`'s pattern) — `applyRestoreChoice` handles both. + let restoreChoiceChanged = false; + if (currentStep === 0) { + const nextChoice: RestoreChoice | undefined = selectedCandidateId + ? { candidateId: selectedCandidateId, carryEnv: restoreCarryEnv, ...(ackedRestoreCodes.size ? { acknowledge: [...ackedRestoreCodes] } : {}) } + : undefined; + if (!restoreDecided || JSON.stringify(nextChoice) !== JSON.stringify(restoreChoice)) { + restoreChoiceChanged = true; + await applyRestoreChoice(nextChoice); + } + } if (currentStep < steps.length - 1) { - // Update draft with current state before proceeding - if (draftId) { + // Update draft with current state before proceeding. Skipped right after + // a restore-choice change: the draft this closure captured was deleted + // and re-created by `applyRestoreChoice`, so `draftId`/`envVars` here are + // the pre-change values and the PATCH would target a draft that is gone. + if (draftId && !(currentStep === 0 && restoreChoiceChanged)) { await updateDraftData({ systemOverrides, appEnv: envVars, @@ -718,9 +881,9 @@ export const InstallWizard: React.FC = () => { composeOverride: composeOverride || undefined }); } - + // Run validation and preflight on validate step - if (currentStep === 4) { + if (currentStep === 5) { const isValid = await validateDraft(); if (isValid) { await runPreflight(); @@ -1056,7 +1219,99 @@ services: const renderStepContent = () => { switch (currentStep) { - case 0: { // Environment Variables + case 0: { // Restore Data (spec 007) + return ( +
+

+ Optionally restore this install from an existing copy of {app.name} on this host — its data, and its credentials, land before the app starts. +

+ {restoreCandidatesLoading &&
Loading restore candidates…
} + {restoreCandidatesError &&
{restoreCandidatesError}
} + {restoreSwitching &&
Applying the restore choice…
} + {restoreSwitchError &&
{restoreSwitchError}
} + {ociRef && ( +
+ Restore isn't available when installing directly from a package reference. +
+ )} + {!ociRef && !restoreCandidatesLoading && allRestoreCandidates.length === 0 && ( +
+ No existing copies of {app.name} were found on this host. This will be a fresh install. +
+ )} + {!ociRef && allRestoreCandidates.length > 0 && ( + <> + { setSelectedCandidateId(v === '__fresh__' ? null : v); setAckedRestoreCodes(new Set()); }} + options={[ + { value: '__fresh__', label: 'Start fresh', description: 'No data carried over' }, + ...allRestoreCandidates.map((c) => ({ + value: c.deploymentId, + label: `${c.name}${c.host ? ` — ${c.host}` : ''}`, + description: `${c.appVersion ? `v${c.appVersion}` : 'unknown version'} · configuration: ${c.carriesEnv ? 'carried' : 'not carried'}${c.capturedAt ? ` · captured ${new Date(c.capturedAt).toLocaleString()}` : ''}`, + })), + ]} + /> + {restoreCandidatesData?.requiresExplicitChoice && !selectedCandidateId && ( +
Two or more unrelated histories match — pick one explicitly (no default is offered).
+ )} + {selectedRestoreCandidate && ( +
+ + {selectedRestoreCandidate.skew.kind === 'refused' && ( +
{selectedRestoreCandidate.skew.message}
+ )} + {selectedRestoreCandidate.skew.kind === 'unknown' && ( + + )} + {(!restoreCarryEnv || !selectedRestoreCandidate.carriesEnv) && ( + + )} +
+ )} + + )} +
+ ); + } + case 1: { // Environment Variables // Seeded (manifest-declared) rows split into Basic (not `advanced`) // and Advanced; anything not seeded is a free-form Custom row (added // via "Add app variable", no spec — rendered with the original grid). @@ -1428,7 +1683,7 @@ services: ); } - case 1: // Compose Override + case 2: // Compose Override return (
@@ -1559,7 +1814,7 @@ services:
); - case 2: // Additional Files + case 3: // Additional Files return (
@@ -1580,7 +1835,7 @@ services:
); - case 3: // Advanced Options + case 4: // Advanced Options return (
@@ -1704,7 +1959,7 @@ services:
); - case 4: // Validate & Preflight + case 5: // Validate & Preflight return (
Validate & preflight
@@ -1772,7 +2027,7 @@ services:
); - case 5: // Summary & Confirm + case 6: // Summary & Confirm return (
Summary & confirm
@@ -1806,6 +2061,23 @@ services:
)} + {/* Restore-on-install (spec 007, FR-041): unconditional summary + acknowledgement whenever a restore is happening — naming + data AND credentials, and that jobs/webhooks/integrations + may fire the moment the app starts holding the restored + data (a cron job, a webhook subscription, a sync). */} + {restoreChoice && ( +
+
+ + + This install restores data and credentials from an existing copy. Any jobs, + webhooks or integrations the app runs may fire as soon as it starts. + +
+
+ )} +

Deployment name & address

)} - {/* Wizard Content - only show when draft is ready */} - {draftId && ( + {/* Wizard Content - only show when draft is ready. Restore-on-install + (spec 007, FR-038): the restore step is the one exception — it must + render BEFORE a draft exists, since the choice it collects is what + seeds the draft (research R1). */} + {(draftId || currentStep === 0) && ( <> {/* Progress Stepper */}
diff --git a/packages/web/src/utils/api-hybrid.ts b/packages/web/src/utils/api-hybrid.ts index af1de791..53a7312d 100644 --- a/packages/web/src/utils/api-hybrid.ts +++ b/packages/web/src/utils/api-hybrid.ts @@ -57,6 +57,8 @@ export const api = { // Registry credentials + install-by-ref (multi-catalog Slice 1) — SDK-only. registryCredentials: sdkAdapter.registryCredentials, installFromRef: sdkAdapter.installFromRef, + // Restore-on-install (spec 007) — SDK-only. + restoreCandidates: sdkAdapter.restoreCandidates, // Catalog sources (multi-catalog Slice 2) — SDK-only. catalogSources: sdkAdapter.catalogSources, diff --git a/packages/web/src/utils/sdk-adapter.ts b/packages/web/src/utils/sdk-adapter.ts index 02edf7cd..db29152e 100644 --- a/packages/web/src/utils/sdk-adapter.ts +++ b/packages/web/src/utils/sdk-adapter.ts @@ -37,7 +37,9 @@ import type { GetSettingsResponse, PatchSettingsRequest, PatchSettingsResponse, GetBackupSettingsResponse, PatchBackupSettingsRequest, PatchBackupSettingsResponse, // System types - GetSystemStatusResponse, GetUpdateCheckResponse + GetSystemStatusResponse, GetUpdateCheckResponse, + // Restore-on-install (spec 007) + ListRestoreCandidatesResponse } from '@hola/shared'; import { globalCache, CacheTTL } from './cache'; import { safeFetchEnhanced, createEnhancedError, type EnhancedError } from './error-enhanced'; @@ -390,6 +392,13 @@ export class SdkAdapter { preview: (url: string): Promise => this.sdk.catalogSources.preview(url), }; + // Restore-on-install (spec 007): candidates for one app, read fresh every + // call (no cache) — the wizard's restore step wants the CURRENT set, and a + // stale "eligible" candidate that quietly became busy/gone is exactly the + // failure this feature's job-time re-resolution otherwise catches late. + restoreCandidates = (appId: string, version?: string, source?: string, channel?: string): Promise => + this.sdk.restoreCandidates(appId, version, source, channel); + // Capability contract rollup (ADR 0004 Phase 4). Read-only and derived from the // installed set, so it rides the same cache as the rest and is dropped whenever // a deployment changes — installing a backup provider has to show up as coverage diff --git a/specs/007-restore-on-install/checklists/requirements.md b/specs/007-restore-on-install/checklists/requirements.md new file mode 100644 index 00000000..49f72084 --- /dev/null +++ b/specs/007-restore-on-install/checklists/requirements.md @@ -0,0 +1,71 @@ +# Specification Quality Checklist: Restore-on-Install from a Live Deployment + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: 2026-09-20 +**Feature**: [spec.md](../spec.md) + +## Content Quality + +- [x] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [x] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [x] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [x] No implementation details leak into specification + +## Notes + +**Deliberate deviation from "no implementation details", recorded rather than +silently taken** (matching the precedent set by specs 003–006 in this repo): + +- The **Assumptions** section names the *shape* of the environment record's + location (`///`) without naming the literal + directory. This is load-bearing: the prompt of record assumed the record lives + *inside* the app data root, and spec 006 shipped it *outside*. A reader who does + not know that will design a restore that reads configuration out of the captured + tree and find nothing there. Stating the shape is what prevents that; stating the + literal path would be detail the plan owns. +- The **Dependencies** section notes that the existing upgrade-path rules pass + through on a downgrade. This is not an implementation detail of *this* feature — + it is a correction to the prompt, which delegated "refuse a newer candidate" to a + check that does not produce it. FR-029 therefore states the rule independently. + +**Requirement coverage sanity check** (informal; `/speckit-analyze` is the formal gate): + +| Group | FRs | Stories | +|---|---|---| +| Candidate discovery | FR-001..006 (+ FR-004a) | US1, US5 | +| Entering the restore choice | FR-007..012 | US1, US2 | +| Executing the restore | FR-013..023 (+ FR-013a, FR-016a, FR-022a) | US1, US3, US4 | +| App restore declaration | FR-024..028 | US4 | +| Refusals and warnings | FR-029..037 (+ FR-037a) | US2, US3 | +| Install wizard | FR-038..042 | US1, US2 | +| Command line | FR-043..046 | US5 | +| Scope boundaries | FR-047..048 | (all) | + +53 functional requirements, 13 success criteria. All 16 items passed on the first +validation iteration and still pass after clarification (no state changes). + +**Post-clarification note.** The `a`-suffixed requirements were added by +`/speckit-clarify` and `/speckit-analyze` rather than renumbering, so every FR number cited elsewhere in +this spec's artifacts stays stable. FR-016 additionally carries an inline +correction to the prompt of record: the prompt's "locate the subtree" trap +describes a provider archive tool's absolute-path layout and does not apply to the +platform's own capture helper, which is root-relative. That trap belongs to +Sequence 6 and is recorded there rather than coded against here. diff --git a/specs/007-restore-on-install/contracts/api.md b/specs/007-restore-on-install/contracts/api.md new file mode 100644 index 00000000..0dc40faa --- /dev/null +++ b/specs/007-restore-on-install/contracts/api.md @@ -0,0 +1,176 @@ +# Contract: HTTP API surface + +**Feature**: `specs/007-restore-on-install` + +## None of this is a capability contract + +Stated plainly, because FR-047 forbids one and the vocabulary is adjacent. + +A **capability contract** in this platform (ADR 0004) is an entry in `CONTRACTS` +(`packages/shared/src/contracts.ts:146-199`) with a provider, acceptors, a +participation mode, and — for a privileged one — a grant the operator consents to. +The broker endpoints live under `/api/contracts/...` and are reached by provider +*apps* holding `hct_`-prefixed contract-scoped tokens. + +This feature adds **none of that**: + +- `CONTRACTS` gains no entry. `auth@1`, `backup@1`, `push@1` and + `container-logs@1` remain the complete set. +- No `/api/contracts/...` route is added or changed. +- No new grant kind. No app receives any new access to anything. +- No contract-scoped token is minted, and `contractCapability()` is untouched. + +What it adds is **one ordinary authenticated platform API read route** and fields +on two existing request bodies — the same class of surface as `channel` (#428) or +`profiles` (#162). The one restore-adjacent thing an *app* declares, +`accepts: ["restore@1"]`, is a participation marker read from the bundle manifest +(see [manifest.md](./manifest.md)); the server never brokers it between two +parties, which is what would make it a contract. + +--- + +## 1. `GET /api/apps/:appId/restore-candidates` (new) + +Lists deployments on this host that can serve as a restore source for `appId`. + +**Auth**: ordinary authenticated principal. A contract-scoped token must **not** +reach it — `authorizeRequest` already default-denies contract principals on any +route naming no capability (#471), so this needs no special handling, but it is +recorded because #477 was a regression in exactly that area. + +**Query**: `?version=` — optional. When supplied, each candidate's +`skew` and `requiredAcknowledgements` are computed against it. Omitted, `skew` is +reported as `unknown` for every candidate, because skew is meaningless without a +target. + +**Why a route and not a field on an existing response** (research R6): the restore +choice is the wizard's *first* step and is an input to draft creation, so a field +on the draft-create response would be circular — the client would need a draft to +learn what to put in the draft request. `--restore-list` must also work without +creating one. + +**200** + +```jsonc +{ + "appId": "mealie", + "lineages": [ + { + "lineageId": "mealie-3f2a9c11", + "candidates": [ + { + "deploymentId": "mealie-3f2a9c11", + "lineageId": "mealie-3f2a9c11", + "app": "mealie", + "name": "Recipes", + "subdomain": "recipes", + "host": "recipes.example.com", + "appVersion": "3.20.1", + "channel": "stable", + "carriesEnv": true, + "capturedAt": "2026-09-19T22:14:03.221Z", + "hasIdentityRecord": true, + "skew": { "kind": "ok" }, + "requiredAcknowledgements": [], + "warnings": [] + } + ] + } + ], + "defaultCandidateId": "mealie-3f2a9c11", + "requiresExplicitChoice": false +} +``` + +`defaultCandidateId` is `null` and `requiresExplicitChoice` is `true` whenever two +or more distinct lineages match (FR-036). Candidates are newest-first within a +lineage. An app with no candidates returns `lineages: []` — **200, not 404** +(FR-042): "nothing to restore from" is an answer, not an error, and the wizard +must render it without treating it as a failure. + +--- + +## 2. `POST /api/drafts` — `CreateDraftRequest` gains `restoreFrom` + +```ts +restoreFrom?: { + candidateId: string; + carryEnv: boolean; + acknowledge?: string[]; +}; +``` + +Accepted on the **catalog** path only. On the install-by-ref path +(`ociRef` supplied) it is **rejected**, never ignored — research R2. Ignoring it +would be the silent-empty-restore failure the whole spec exists to prevent. + +**Effect on success**: `appEnv` is seeded from the candidate's environment record +through `mergeUpgradeAppEnv`; `name` and `subdomain` default from the candidate +(FR-035). + +**409 `CONFLICT`** with `details.code` — same envelope as `PROVIDER_EXISTS` and +`ALREADY_INSTALLED`, so existing client handling generalises: + +| `details.code` | Cause | +|---|---| +| `RESTORE_NOT_SUPPORTED` | `restoreFrom` on the install-by-ref path | +| `RESTORE_NOT_ACCEPTED` | The target app declares no `restore@1` in `accepts` | +| `RESTORE_CANDIDATE_GONE` | No such candidate, or it is not a candidate for this app | +| `RESTORE_CANDIDATE_BUSY` | Candidate not in a settled state | +| `RESTORE_SOURCE_NEWER` | + `candidateVersion`, `targetVersion` | +| `RESTORE_UPGRADE_PATH` | + `suggestedVersion` | +| `RESTORE_ENV_REQUIRED` | + `missingKeys[]` | +| `RESTORE_ACK_REQUIRED` | + `required[]` | + +**Three codes are deliberately absent from this table.** +`RESTORE_TARGET_NOT_EMPTY`, `RESTORE_PAYLOAD_EMPTY` and `RESTORE_HOOK_FAILED` +(data-model.md §4) are reachable only inside the deploy job, after the create call +has returned. They surface on the deployment's error state and in the job log, not +in any HTTP response body. `RESTORE_CANDIDATE_GONE` and `RESTORE_CANDIDATE_BUSY` +appear in both places, because the job re-resolves the candidate (FR-013a). + +--- + +## 3. `POST /api/deployments` — re-validation, not a new field + +`CreateDeploymentFromDraftRequest` (`shared/src/index.ts:2257-2289`) is +**unchanged**. The restore choice arrives on the finalized manifest, not the +request. + +*(The type is `CreateDeploymentFromDraftRequest`. There is no +`CreateDeploymentRequest` in this codebase — the prompt of record named one.)* + +What changes is behaviour: `createFromDraft` re-validates the choice before +creating any state, because the candidate may have been deleted or started a +lifecycle action since the draft was made. Acknowledgement codes are enforced here +the way `grants` already are — computed from the manifest and the candidate, +refused when required and absent. Same failure mode, same place in the flow, for +the same reason (research R16). + +Returns the same 409 codes as §2. + +--- + +## 4. `GET /api/deployments/:id` — three additive fields + +`EnhancedDeploymentDetail` gains `lineageId?`, `restoreFrom?` and `restoredAt?` +(see [data-model.md](../data-model.md) §6). All optional; every record written +before this feature stays valid and reads them as `undefined`. + +`lineageId`'s absence-means-self rule is what makes this zero-migration: +`writeInstanceMarkers` becomes `deployment.lineageId ?? deployment.id`, which for +an older record yields exactly the value it has always written. + +--- + +## 5. Unchanged, listed so the boundary is checkable + +| Surface | Status | +|---|---| +| `/api/contracts/...` (broker) | untouched | +| `CONTRACTS` registry | no entry added | +| Grant kinds (`apps-data`, `container-logs`) | none added | +| `PatchDraftRequest` | still closed to its four fields | +| `POST /api/drafts/:id/finalize` | still takes no body | +| `JobType` | no new type — the restore rides the existing deploy job | +| `POST /api/backups/:id/restore` | **the pre-existing dead stub** (`server.ts:1663-1668`), deliberately untouched — research R19, tracked by #160 | diff --git a/specs/007-restore-on-install/contracts/cli.md b/specs/007-restore-on-install/contracts/cli.md new file mode 100644 index 00000000..a20922d5 --- /dev/null +++ b/specs/007-restore-on-install/contracts/cli.md @@ -0,0 +1,89 @@ +# Contract: CLI surface + +**Feature**: `specs/007-restore-on-install` · **Package**: `packages/cli` + +Not a capability contract — these are flags on an existing command. See +[api.md](./api.md) for why the distinction matters to FR-047. + +## Flags on `hola install` + +| Flag | Effect | +|---|---| +| `--restore-from ` | Restore from that deployment. | +| `--restore-from latest` | Restore from the newest candidate. Refuses when two or more lineages match (FR-036) — "latest" is ambiguous across unrelated histories. | +| `--no-restore` | Explicitly decline. Same behaviour as omitting every flag; exists so a script states intent. | +| `--restore-list` | List candidates and exit. Installs nothing. | +| `--carry-env` / `--no-carry-env` | Carry the candidate's configuration. Default **on** when the candidate has an environment record — carrying is what makes the restored data readable (spec US2), so the safe default is the one that preserves it. `--no-carry-env` requires `--ack restore-env-not-carried`. | +| `--ack ` | Supply an acknowledgement code. Repeatable or comma-separated, parsed exactly like `--grant` (`install.ts:82-93`). | + +## The default is the contract + +**With no restore flag, no restore happens** (FR-044). + +A candidate existing is not consent to use it. Silence must never overwrite an +operator's install decision with a guess — an unattended install that restored a +stale copy of production because a candidate happened to be present would be +unrecoverable by the time anyone noticed. + +This is why `--ack` exists rather than a `--yes`-style blanket. A scripted install +acknowledges each specific risk deliberately or fails closed (FR-046); it can +never satisfy an acknowledgement it did not name (SC-012). + +## `--restore-list` output + +``` +$ hola install mealie --restore-list +Restore candidates for mealie: + + mealie-3f2a9c11 Recipes recipes.example.com v3.20.1 env: yes 2026-09-19 22:14 + mealie-8b41d0e7 Recipes (old) old.example.com v3.18.0 env: no 2026-09-02 09:41 + ! configuration cannot be carried: POSTGRES_PASSWORD + requires --ack restore-env-not-carried + +2 candidates in 1 lineage. Default: mealie-3f2a9c11 +``` + +Reads the candidates route ([api.md](./api.md) §1) with no draft created — which +is why that route exists as a route (research R6). Every warning line names the +flag that would satisfy it, so the operator's next command is on screen. + +## Refusals + +Hints are built from structured `details`, **never** from the server's message. +`deploy-flow.ts:137-155` already establishes both the rule and the reason: the +server's message is deliberately surface-neutral and contains none of the CLI's +flag names, so a message-derived hint would be wrong in a way no server test +catches. + +``` +$ hola install mealie --restore-from mealie-8b41d0e7 +Error: this backup was taken on a newer version of mealie than the one +being installed. + + captured on v3.22.0 + installing v3.20.1 + +Install v3.22.0 instead, or pick a different candidate: + hola install mealie --version 3.22.0 --restore-from mealie-8b41d0e7 + hola install mealie --restore-list +``` + +Mapping from `details.code` to the hint, one row per code +([data-model.md](../data-model.md) §4): + +| `details.code` | Hint built from | +|---|---| +| `RESTORE_SOURCE_NEWER` | `candidateVersion`, `targetVersion` → suggest `--version ` | +| `RESTORE_UPGRADE_PATH` | `suggestedVersion` → install that, restore there, then promote | +| `RESTORE_ENV_REQUIRED` | `missingKeys[]` → name them; this app cannot restore without them | +| `RESTORE_ACK_REQUIRED` | `required[]` → the exact `--ack ` to add | +| `RESTORE_CANDIDATE_GONE` / `_BUSY` | suggest `--restore-list` to re-read the current set | +| `RESTORE_NOT_SUPPORTED` | install-by-ref cannot restore; use the catalog path | +| `RESTORE_NOT_ACCEPTED` | this app has not declared it can be restored; install fresh | + +## Unchanged + +`hola install`'s existing flags, `--grant` parsing, and the `ALREADY_INSTALLED` +branch at `deploy-flow.ts:145-155` all behave exactly as they do today. There is +no new command and no new subcommand — restore is an option on install, because +install is the only moment it is safe (spec §Executive Summary). diff --git a/specs/007-restore-on-install/contracts/manifest.md b/specs/007-restore-on-install/contracts/manifest.md new file mode 100644 index 00000000..dcb7e39b --- /dev/null +++ b/specs/007-restore-on-install/contracts/manifest.md @@ -0,0 +1,147 @@ +# Contract: the app-side `restore` declaration + +**Feature**: `specs/007-restore-on-install` · **Repo**: `try-hola/apps` (sibling PR) + +## This is not a capability contract + +`restore@1` in an app's `accepts` array names a **participation the app +declares**, read from the bundle `manifest.json`. It is not an entry in +`CONTRACTS` (`packages/shared/src/contracts.ts:146-199`), it brokers nothing +between two parties, and it carries no grant. The server reads the declaration and +acts on it directly, exactly as it already reads `backup`. FR-047 holds. + +Sequence 6 is what would introduce a real `restore@1` **contract** — a provider, +a staging grant, a polled queue. Nothing here presumes its shape. + +--- + +## Shape + +```jsonc +{ + "accepts": ["backup@1", "restore@1"], + + "restore": [ + { + "id": "default", + "discard": ["postgres"], + "hook": { + "service": "mealie-postgres", + "command": ["sh", "-c", + "psql -v ON_ERROR_STOP=1 -U mealie -d mealie -f /backups/mealie.sql"] + }, + "requiresEnv": false + } + ] +} +``` + +| Field | Type | Required | Meaning | +|---|---|---|---| +| `id` | `string` | yes | The **backup** participation id this restores. `default` for the legacy singular `backup` block, which `backupParticipations()` normalises to one participation of that name — the form every catalog app currently uses. | +| `discard` | `string[]` | no | Data-root-relative paths removed after the files land and **before any container starts**. | +| `hook` | `AppBackupHook` | no | `{ service, command }` — reused verbatim (`shared/src/index.ts:295-298`), so the schema references the existing `$defs/backupHook`. | +| `requiresEnv` | `boolean` | no (default `false`) | `true` turns the missing-environment-record **warning** into a **refusal**. | + +### The three states, and why the middle one already exists + +| Declaration | Server behaviour | +|---|---| +| no `restore@1` in `accepts` | Not restorable. Nobody considered it. | +| `restore@1`, **no** `restore` block | Plain file copy back. No discards, no hook. | +| `restore@1` + a `restore` block | Discards and/or hook apply. | + +The middle state is **not new**: 12 of the 17 apps that currently declare +`accepts: ["backup@1"]` carry no backup block at all. This feature gives that +existing shape a meaning — "a plain file copy back is all I need", true for every +SQLite and flat-file app — rather than inventing a state authors must adopt. + +--- + +## Why `discard` exists + +A file-level tar of a **live** `PGDATA` is read over minutes while the database +writes throughout: page 1 at T+0s, page 100000 at T+180s. That is not a snapshot, +it is a smear across time, and for Postgres a smear is corruption. +`tarGzipDir` concedes as much in its own comment — it suppresses tar's +"file changed as we read it" warning because crash-consistency is the whole of +what it promises. + +The `.sql` dump the app's `backup.preHook` already writes is the real payload, and +it is already captured: every hook-declaring app in the catalog mounts +`${HOLA_APP_DATA}/backups:/backups`, inside the data root. So the correct restore +is **discard the smeared `PGDATA`, let the container `initdb` a clean cluster, +load the dump into it** — which is exactly what `discard: ["postgres"]` plus a +one-line `psql` hook expresses. + +### Containment + +Every `discard` path is resolved through `resolveContainedDir` +(`packages/server/src/services/core/path-containment.ts:30`) — the same guard +push targets already use (`deployment.ts:2952`). A path that resolves outside the +target data root **refuses the restore**; it is not skipped with a warning. + +Resolution, not a `startsWith` test. That distinction is the subject of issue +#482 and of spec 006's review: a lexical prefix check is not a containment proof, +and `discard` is app-supplied data that names a directory for deletion — the +highest-consequence place in this feature to get it wrong. + +--- + +## JSON-schema addition (`schemas/manifest.schema.json`) + +```jsonc +"restore": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["id"], + "properties": { + "id": { "type": "string", "minLength": 1 }, + "discard": { "type": "array", "items": { "type": "string", "minLength": 1 } }, + "hook": { "$ref": "#/$defs/backupHook" }, + "requiresEnv": { "type": "boolean" } + } + } +} +``` + +Additive only. Every manifest valid today stays valid (FR-028). + +--- + +## Planned catalog changes + +All five hook-declaring apps use `pg_dump`, so each restore hook is one `psql` +line. Every one of their Postgres services **already declares a healthcheck**, so +`up -d --wait ` has a real readiness signal and this imposes no +migration on the catalog. + +| App | Hook service | `discard` | Hook | +|---|---|---|---| +| guacamole | `postgres` | PG data dir | `psql -v ON_ERROR_STOP=1 -U guacamole_user -d guacamole_db -f /backups/guacamole_db.sql` | +| immich | `immich-postgres` | PG data dir | `psql -v ON_ERROR_STOP=1 -U immich -d immich -f /backups/immich.sql` | +| mealie | `mealie-postgres` | `postgres` | `psql -v ON_ERROR_STOP=1 -U mealie -d mealie -f /backups/mealie.sql` | +| paperless-ngx | `db` | PG data dir | `psql -v ON_ERROR_STOP=1 -U paperless -d paperless -f /backups/paperless.sql` | +| postiz | `postiz-postgres` | PG data dir | `psql -v ON_ERROR_STOP=1 -U postiz-user -d postiz-db-local -f /backups/postiz.sql` | + +Only mealie's compose was read in full during planning (`discard: ["postgres"]`, +from `${HOLA_APP_DATA}/postgres:/var/lib/postgresql/data`). **Each remaining app's +`discard` path must be read from its own `compose.yaml` at implementation time**, +not assumed from mealie's — the mount point is per-app and getting it wrong either +discards nothing (restoring the smear) or discards the wrong directory. + +`-v ON_ERROR_STOP=1` is required on every hook. Without it `psql` reports success +after a failed statement, which under FR-021's fail-closed rule would hand a +partially loaded database to an app that then migrates it — the exact corruption +the fail-closed policy exists to prevent. + +The twelve remaining acceptor apps need **no change**: they already declare +`accepts` with no block, which is the plain-file-copy state. + +### Sequencing + +The platform half is useful without the catalog PR — plain file-copy restores work +for 12 of 17 apps on day one — so the two can land independently. The +database-backed five need both. diff --git a/specs/007-restore-on-install/data-model.md b/specs/007-restore-on-install/data-model.md new file mode 100644 index 00000000..fcb769e3 --- /dev/null +++ b/specs/007-restore-on-install/data-model.md @@ -0,0 +1,254 @@ +# Data Model: Restore-on-Install from a Live Deployment + +**Feature**: `specs/007-restore-on-install` · **Baseline**: `ca3d3f4` + +Every shape below is given with field, type, source expression, and nullability. +"Source expression" means *where the value actually comes from at runtime* — the +field that most often goes silently wrong is one whose source was never written +down (spec 006's `deployment.metadata.source` being the local precedent). + +--- + +## 1. `RestoreChoice` — what the operator decided + +Lives in `packages/shared/src/index.ts`. Created once at draft creation (R1), +carried on the draft → finalized manifest → deployment record, consumed once. + +| Field | Type | Source | Null? | Notes | +|---|---|---|---|---| +| `candidateId` | `string` | operator selection; the candidate's **deployment id** | no | Not a lineage id. Identifies one source exactly. | +| `carryEnv` | `boolean` | operator selection | no | Explicit, never defaulted from candidate state — declining is a decision (FR-033/4). | +| `acknowledge` | `string[]` | operator selection | yes (absent = `[]`) | Acknowledgement codes (§3). Modelled on `grants` (R16). | + +**Invariant.** `candidateId` must not equal the deployment being created — it +cannot, since the id is minted after the draft, but the guard is stated because +Sequence 6 may introduce ids that are not host deployments. + +--- + +## 2. `RestoreCandidate` — a source the operator can pick + +**Derived, never stored.** Computed on demand from deployments + identity records ++ catalog upgrade metadata. Returned by the candidates read route +([contracts/api.md](./contracts/api.md)). + +| Field | Type | Source expression | Null? | +|---|---|---|---| +| `deploymentId` | `string` | `deployment.id` | no | +| `lineageId` | `string` | `identity.lineageId ?? deployment.lineageId ?? deployment.id` | no | +| `app` | `string` | `identity.app ?? deployment.app` | no | +| `name` | `string` | `deployment.name` | no | +| `subdomain` | `string \| null` | `identity.subdomain ?? deployment.subdomain ?? null` | yes | +| `host` | `string \| null` | `identity.host ?? null` | yes | +| `appVersion` | `string \| null` | `identity.appVersion ?? deployment.version ?? null` | yes | +| `channel` | `string \| null` | `identity.channel ?? deployment.channel ?? null` | yes | +| `carriesEnv` | `boolean` | `exists(/.hola//env.json)` | no | +| `capturedAt` | `string \| null` | `identity.writtenAt ?? null` — ISO 8601 | yes | +| `hasIdentityRecord` | `boolean` | whether `.hola/instance.json` parsed | no | +| `skew` | `RestoreSkewVerdict` | §4 | no | +| `requiredAcknowledgements` | `string[]` | §3, derived | no | +| `warnings` | `RestoreWarning[]` | §5 | no | + +**Why the deployment record is only a fallback** (FR-003): for *this* slice both +sources are present and the record alone would do. Reading the identity record +first is what makes the same resolver work unchanged when Sequence 6 replaces "a +live deployment" with "an archive", where the record is the *only* source. A +candidate whose record is absent or unparseable is still offered, described from +the record alone, with `hasIdentityRecord: false` and `lineageId` degrading to the +deployment id. + +### Eligibility (FR-001, FR-004, FR-004a) + +A deployment is a candidate **iff all** hold: + +1. `deployment.app === ` +2. `deployment.id !== ` +3. `deployment.status` is settled — `running` or `stopped`; **not** in flight, + **not** `error` +4. `dirHasContents(appRootFor(deployment.id), [INSTALL_MARKERS_DIR])` is `true` + +Rule 4's ignore-list is load-bearing and is not a new invention: without it every +materialised install looks like it holds data, because `writeInstanceMarkers` +writes `.hola/instance.json` on every deploy. The identical call already guards +`capturePreUpgradeSnapshot` at `deployment.ts:2513`, where omitting it was found +to be a data-loss path during spec 006's review. + +### Ordering (FR-005, FR-036) + +Group by `lineageId`; within a lineage sort by `capturedAt` descending (a null +`capturedAt` sorts last). A single matching lineage supplies a default selection; +**two or more distinct lineages require an explicit pick** and no default is +offered. + +--- + +## 3. Acknowledgement codes + +A closed union. The server computes which are *required* for a given candidate; +the client supplies them in `RestoreChoice.acknowledge`; a required-and-absent +code fails the create exactly as a missing `grant` does (R16). + +| Code | Required when | What the operator is accepting | +|---|---|---| +| `restore-version-unknown` | `skew.kind === 'unknown'` | The version relationship could not be checked, so the app's own upgrade rules were not applied. | +| `restore-env-not-carried` | `carryEnv === false`, **or** `carriesEnv === false` | Platform-generated secrets will be minted fresh; data encrypted under the originals may be unreadable. | + +**Both conditions map to one code for `restore-env-not-carried`** deliberately: +the operator-facing risk is identical whether configuration *cannot* be carried or +they *chose* not to carry it, and a second code would invite a UI that treats one +as less serious. + +Codes are **not** required for a refusal — a refusal cannot be acknowledged away +(FR-029, FR-030, FR-034). Acknowledgement exists only for risks that are +proceedable. + +--- + +## 4. `RestoreSkewVerdict` — the version relationship + +``` +{ kind: 'ok' } +{ kind: 'unknown' } // acknowledgeable +{ kind: 'refused', code: RestoreRefusalCode, + message: string, suggestedVersion?: string } +``` + +Evaluated in this order (R15). Note that **only rows 3 and 4 come from +`checkUpgradePath`**; rows 1 and 2 are this feature's own rules, because +`checkUpgradePath` returns `ok` for both: + +| # | Condition | Verdict | +|---|---|---| +| 1 | `candidate.appVersion` absent, or target version absent, or no upgrade metadata | `unknown` → acknowledgeable | +| 2 | `isNewerVersion(candidate.appVersion, targetVersion)` | `refused`, `RESTORE_SOURCE_NEWER` | +| 3 | `checkUpgradePath(candidate.appVersion, targetVersion, meta)` returns `!ok` | `refused`, `RESTORE_UPGRADE_PATH`, carrying `suggestedVersion` | +| 4 | otherwise | `ok` | + +Row 2 must be evaluated **before** row 3. `checkUpgradePath` short-circuits on +`!isNewerVersion(to, from)` and would return `ok` for the newer-source case, +letting it through. + +### `RestoreRefusalCode` + +Closed union, carried in `details.code` so surfaces build guidance from structure +rather than prose (FR-037, and `deploy-flow.ts:137-155`'s established rule). + +| Code | Meaning | `details` also carries | +|---|---|---| +| `RESTORE_SOURCE_NEWER` | Candidate newer than the version being installed | `candidateVersion`, `targetVersion` | +| `RESTORE_UPGRADE_PATH` | The app's own rules guard this hop | `suggestedVersion` | +| `RESTORE_ENV_REQUIRED` | `restore.requiresEnv` and no environment record | `missingKeys[]` | +| `RESTORE_CANDIDATE_GONE` | Candidate deleted between choice and deploy | `candidateId` | +| `RESTORE_CANDIDATE_BUSY` | Candidate not in a settled state | `candidateId`, `status` | +| `RESTORE_TARGET_NOT_EMPTY` | Target data root already holds app data | `deploymentId` | +| `RESTORE_PAYLOAD_EMPTY` | Post-condition failed: nothing landed (FR-016) | `deploymentId` | +| `RESTORE_HOOK_FAILED` | A restore hook failed or its service never became healthy | `participationId`, `service` | +| `RESTORE_NOT_SUPPORTED` | `restoreFrom` supplied on the install-by-ref path | — | +| `RESTORE_NOT_ACCEPTED` | The target app declares no `restore@1` in `accepts` | `appId` | +| `RESTORE_ACK_REQUIRED` | A required acknowledgement code was absent | `required[]` | + +**Create-time vs job-time.** Seven of these can be returned synchronously from a +draft-create or deployment-create call, so a client sees them in a `409` body. +Three cannot: `RESTORE_TARGET_NOT_EMPTY`, `RESTORE_PAYLOAD_EMPTY` and +`RESTORE_HOOK_FAILED` are only reachable **inside the deploy job**, long after the +request returned. They surface on the deployment's error state and in the job log, +never in a create response — which is why they are absent from the error tables in +`contracts/api.md` and from the CLI's hint mapping in `contracts/cli.md`. The +union is one union; the delivery channel differs. `RESTORE_CANDIDATE_GONE` and +`RESTORE_CANDIDATE_BUSY` are the two that occur in **both** places, because the +job re-resolves the candidate (FR-013a). + +--- + +## 5. `RestoreWarning` — proceedable, named, non-fatal + +``` +{ code: 'env-not-carried', keys: string[] } // the isSecret && generate keys +{ code: 'host-divergence', from: string, to: string } +{ code: 'no-identity-record' } +``` + +`env-not-carried.keys` is **derived, not declared** (FR-033): every `AppEnvVar` +(`shared/src/index.ts:903-958`) where `isSecret === true` **and** `generate` is +present. Those are values the *platform* invented and will mint fresh; anything +the operator supplied is theirs to re-supply knowingly. Deriving beats a manifest +field that an app author must remember to maintain and that silently rots. + +--- + +## 6. Deployment record additions + +On `EnhancedDeploymentDetail` (`shared/src/index.ts:2017-2076`), persisted with +the record. All optional, so every existing record on disk stays valid with no +migration. + +| Field | Type | Source | Null? | Notes | +|---|---|---|---|---| +| `lineageId` | `string?` | `restoreFrom` ? candidate's `lineageId` : `deployment.id` | yes | **Reading it is the change spec 006 predicted.** `writeInstanceMarkers` becomes `deployment.lineageId ?? deployment.id` (`deployment.ts:3374`). The fallback is what makes this zero-migration: an older record reads `undefined` and yields exactly the value it always had. | +| `restoreFrom` | `RestoreChoice?` | the finalized manifest | yes | Persisted beside `channel` (`deployment.ts:1028`), **not** in the job payload (R3). | +| `restoredAt` | `string?` | set by the job on success — ISO 8601 | yes | Consumption marker. Its presence is what makes FR-012 enforceable: a restart/promote/rollback finds it set and skips. | + +**Why the record and not the payload** (R3): the deploy payload is +`{ releaseId, action: 'deploy' }` (`deployment.ts:1193`) — `deploymentId` is a +sibling `Job` field, never a payload key, so the prompt's assumed shape does not +exist. More substantively, a payload does not survive a job retry, and +"has this install already restored?" is a fact about the deployment's history. + +--- + +## 7. `AppRestoreDeclaration` — what an app says (manifest) + +Per backup participation, keyed by participation id. Reuses `AppBackupHook` +verbatim (`shared/src/index.ts:295-298`). Full schema in +[contracts/manifest.md](./contracts/manifest.md). + +| Field | Type | Null? | Notes | +|---|---|---|---| +| `id` | `string` | no | The **backup** participation id this restores. `default` for the legacy singular form, which `backupParticipations()` already normalises — and which is the form all five catalog apps use. | +| `discard` | `string[]?` | yes | Data-root-relative paths removed after extraction, before any container starts. Each resolved through `resolveContainedDir`; one that escapes refuses the restore (FR-017). | +| `hook` | `AppBackupHook?` | yes | `{ service, command }`. Run after discards, against the started-and-healthy service. | +| `requiresEnv` | `boolean?` | yes (default `false`) | `true` turns the no-environment-record **warning** into a **refusal** (FR-034). | + +### The three declaration states (FR-025) + +| Declaration | Meaning | +|---|---| +| No `accepts: ["restore@1"]` | Nobody has considered restoring this app. Not offered. | +| `accepts: ["restore@1"]`, no `restore` block | **"A plain file copy back is all I need."** Files land, nothing is discarded, no hook runs. | +| `accepts: ["restore@1"]` + a `restore` block | Discards and/or a hook apply. | + +The middle state **already exists in the catalog**: 12 of 17 acceptor apps +declare `accepts` with no backup block today. This feature gives that shape +meaning rather than introducing it. + +`restore@1` here names a *participation an app declares*, not a capability +contract the platform brokers. `CONTRACTS` (`contracts.ts:146-199`) gains no +entry — FR-047 holds. + +--- + +## 8. Lifecycle of a restore + +``` +draft create ──> RestoreChoice validated (candidate resolved, skew judged, + acknowledgements checked) ──> appEnv seeded from the candidate's + env record via mergeUpgradeAppEnv + +finalize ──────> restoreFrom rides OUTSIDE canonicalSpec, beside channel + +createFromDraft ─> re-validated (the candidate may have changed), acknowledgements + enforced as grants are; restoreFrom + lineageId persisted + +deploy job ────> if restoreFrom && !restoredAt: + assert target empty · re-resolve candidate · quiesce + capture + · extract · assert payload present · discard · rewrite marker + · write OIDC file · up --wait · run hooks + then set restoredAt + +later actions ─> restoredAt is set ⇒ skip. The restore is consumed exactly once. +``` + +**Two independent guards enforce FR-012**: the action must be the deployment's +first deploy, *and* `restoredAt` must be unset. Either alone would be enough in +the happy path; both together mean a retried job after a partial failure cannot +re-quiesce a live source. diff --git a/specs/007-restore-on-install/plan.md b/specs/007-restore-on-install/plan.md new file mode 100644 index 00000000..8fc79fd8 --- /dev/null +++ b/specs/007-restore-on-install/plan.md @@ -0,0 +1,255 @@ +# Implementation Plan: Restore-on-Install from a Live Deployment + +**Branch**: `007-restore-on-install` | **Date**: 2026-09-20 | **Spec**: [spec.md](./spec.md) + +**Input**: Feature specification from `/specs/007-restore-on-install/spec.md` + +**Baseline**: `ca3d3f4` (`main`, post-spec-006 `921c790`). Every anchor cited in +this plan and in [research.md](./research.md) was verified against that tree. + +## Summary + +During an app install the operator may pick an existing deployment of that app on +the same host as a **restore source**. The server quiesces the source with the +backup contract's existing pre-hooks, captures its data root, and lays that data +down into the new install **after images are pulled and before any container +starts** — the one moment in an app's life when the data root is empty and no +process holds it open. + +The design adds no capability contract, no grant, and no contract endpoint. The +server is both reader and writer, as it already is for a data-aware rollback. Its +whole surface is: a field on the draft-create request, a read route for +candidates, a `restore` block apps declare in their manifest, a widened +`composeUp`, one new step in the install wizard, and four CLI flags. + +The technical approach is almost entirely **reuse**: `capturePreUpgradeSnapshot`'s +hook-and-tar path, `runPreHooksFailClosed`'s policy, `restoreTarGzInto`, +`mergeUpgradeAppEnv`'s three-case rule, `resolveContainedDir`'s containment, +`checkUpgradePath`'s skew rules, `writeInstanceMarkers`, and `grants`' consent +shape. The genuinely new code is a candidate resolver, a restore sequence inside +the lifecycle job, and `composeUp`'s service-list/wait option. + +## Technical Context + +**Language/Version**: TypeScript on Bun (server, shared, cli), React + Vite (web). + +**Primary Dependencies**: no new runtime dependency. Existing: `docker compose` +CLI, GNU `tar` (via `snapshot-fs.ts`), `oras` (unchanged). + +**Storage**: the host filesystem — app data roots under `HOLA_APPS_BIND_ROOT` +(default `/srv/hola/apps`), platform state under the Hola data dir. No database +schema change; the deployment record is a JSON document that gains two optional +fields. + +**Testing**: `bun run test`. Server unit tests under +`packages/server/src/__tests__/deployments/`; the mode- and filesystem-sensitive +paths need the **real-filesystem harness** (`RealStorageService` + `mkdtemp` + +`HOLA_APPS_BIND_ROOT`), copied from `install-markers.test.ts:116-142` or +`snapshot.test.ts:99-125`. Web tests with vitest; CLI tests with vitest. +End-to-end verification on a disposable VM (`bin/vm-e2e-suite`, `vm-e2e` skill). + +**Target Platform**: Linux host running the Hola stack under Docker Compose. + +**Project Type**: monorepo (Bun workspaces) — server + web + shared + cli, with a +sibling change in the `try-hola/apps` catalog repository. + +**Performance Goals**: no latency target. The binding constraint is **disk**: a +restore's peak additional cost must be one compressed archive of the source's +data root (SC-013), not three simultaneous copies. See research R7. + +**Constraints**: +- The restore must run inside `runLifecycleJob` (Constitution III). +- A failed restore must fail the install; no partial-success start (FR-022). +- The no-restore path must be byte-for-byte unchanged (FR-023). +- `composeUp`'s existing 5-minute timeout is too short for a `--wait` on a + freshly-`initdb`'d Postgres and must be parameterised (research R12). + +**Scale/Scope**: one production package predominantly (`packages/server`), with +smaller changes in `shared`, `web`, `cli`, and a sibling catalog PR. An app data +root may be tens of gigabytes. A host typically has fewer than 30 deployments, so +candidate discovery is a small scan, not a query problem. + +## Constitution Check + +*Derived independently against `.specify/memory/constitution.md` v1.0.0. Not +pre-assessed by the requester.* + +| Principle | Verdict | Reasoning | +|---|---|---| +| **I. Traefik-Only Ingress** | **N/A** | No ingress, routing, port or image-pinning change. A restored install is routed exactly as any install; the compose validator is untouched. | +| **II. Remote Catalog as Single Source of Truth** | **PASS** | The `restore` block is read from the bundle `manifest.json`, never from `catalog.json` — the same place the `backup` block already comes from. No bundled catalog is introduced. `MockCatalogService` stays empty; candidate discovery reads *deployments*, which is host state, not catalog state. | +| **III. Async Deploy Lifecycle** | **PASS — and load-bearing** | The entire restore executes in `runLifecycleJob` (`deployment.ts:3780-3821`), between the cancellation check and `composeUp`. Create-time work is limited to *validation and refusal* — resolving the candidate, judging version skew, checking acknowledgements — which is the same thing `assertProviderAllowed` already does at `createFromDraft` before any state exists. Capturing tens of gigabytes and running `pg_dump` in a live container is precisely the slow, side-effectful work this principle keeps out of request handlers. | +| **IV. Real/Mock Service Pairs** | **PASS, with an explicit constraint** | No new service is introduced; the feature extends `DeploymentService`, `DockerService` and `DraftService`, all of which already have Real/Mock pairs in `simple-factory.ts`. **The constraint that must not be missed**: `composeUp`'s widened signature has to land in all three of the interface (`docker.ts:67`), the Real implementation (`:231-269`) **and** `MockDockerService` (`:722-725`). A Mock that accepts but ignores `services` would let every test pass while the real path starts the wrong containers. See *Known trap* below. | +| **V. Generic Cross-App Primitives** | **PASS — checked hard, see below** | | +| **VI. Auth Is Platform-Agnostic and Default-On** | **PASS — and protected** | The only auth-adjacent change is moving `writeOidcCredentialsFile` to after the restore (R10). Nothing Authentik-specific is added and `ProvisionerService`'s interface is untouched. The move *defends* this principle: leaving the write where it is means a restored install silently boots with SSO disabled, which is a default-on violation in practice even though no code says so. | +| **VII. Quality Gates Before Merge** | **PASS** | Branch + PR to `main`, no direct push. Full gate before the PR: `bun run typecheck && bun run lint && bun run typecheck && bun run test && bun run build` (typecheck twice, per the CLAUDE.md note about lint auto-fixes). Integration tests stay excluded from the default suite. Package versions stay in sync. | + +### Principle V, checked explicitly + +This is the principle most at risk in a feature about restoring *specific kinds of +datastore*, so it gets its own audit rather than a one-line verdict. The test is: +**does the server ever branch on which app it is restoring?** + +| Mechanism | App-specific knowledge lives… | Server behaviour | +|---|---|---| +| Which paths to discard | in the app's manifest `restore.discard` | Resolve each through `resolveContainedDir`, delete. Identical for every app. | +| How to reload the payload | in the app's manifest `restore.hook` (`{service, command}`) | Run the command in the named service. Identical for every app. | +| When the hook service is ready | in the app's own compose `healthcheck` | `docker compose up -d --wait `. Identical for every app. | +| Whether configuration is mandatory | in the app's manifest `restore.requiresEnv` | Refuse or warn. Identical for every app. | +| Which env keys to warn about | **derived**: `isSecret && generate` | Computed from `AppEnvVar`, not declared. No app can get it wrong. | + +There is no app name, image name, or datastore name anywhere in the server's +restore path. "Postgres needs its `PGDATA` discarded" is a sentence the *catalog* +says, five times, once per app that means it — never a sentence the server knows. + +The deliberate design pressure here is R12's rejection of a bespoke readiness +poll: knowing that "ready" means `pg_isready` for one app and something else for +another is exactly the per-app branching this principle forbids, so the feature +consumes the app's declared healthcheck instead. + +### Complexity Tracking + +**Empty — no violations to justify.** Recorded explicitly rather than omitted, so +a reader knows the gate was evaluated and not skipped. The two places this +feature could plausibly have earned an entry, and did not: + +- **A new service for restore orchestration** — not created. The work lives on + `RealDeploymentService` beside the snapshot machinery it reuses, which keeps one + owner for the app data root rather than two. +- **A second hook format for restore** — not created. `AppBackupHook` is reused + verbatim (R18), so the catalog schema references an existing `$defs/backupHook`. + +## Project Structure + +### Documentation (this feature) + +```text +specs/007-restore-on-install/ +├── plan.md # This file +├── spec.md # 53 FRs, 13 SCs, 5 clarifications +├── research.md # R1–R23: decisions, rationale, and four prompt corrections +├── data-model.md # Every record and request shape, field by field +├── quickstart.md # Numbered, individually-citable verification scenarios +├── contracts/ +│ ├── api.md # Candidates read route + draft/deployment request additions +│ ├── manifest.md # The app-side `restore` block and its JSON-schema shape +│ └── cli.md # The four flags and the acknowledgement flag +├── checklists/ +│ └── requirements.md # 16/16 +└── tasks.md # Phase 2 output — NOT created by /speckit-plan +``` + +### Source code + +```text +packages/shared/src/ +├── index.ts # RestoreChoice, RestoreCandidate, acknowledgement +│ # codes, AppRestoreDeclaration; additions to +│ # CreateDraftRequest, Draft, EnhancedDeploymentDetail +└── contracts.ts # UNCHANGED — CONTRACTS gains no entry (FR-047) + +packages/server/src/ +├── services/core/ +│ ├── deployment.ts # The bulk: candidate resolution, the restore +│ │ # sequence in runLifecycleJob, lineageId on the +│ │ # record, the writeOidcCredentialsFile move +│ ├── draft.ts # restoreFrom on createDraft (catalog path); +│ │ # refuse it on the install-by-ref path; +│ │ # carry it onto the finalized manifest +│ ├── docker.ts # composeUp: { services?, wait?, timeoutMs? } +│ │ # in the interface, Real AND Mock +│ ├── restore-candidates.ts # NEW: candidate discovery + eligibility + skew +│ ├── upgrade-env.ts # UNCHANGED — mergeUpgradeAppEnv reused as-is +│ ├── snapshot-fs.ts # UNCHANGED — tarGzipDir / restoreTarGzInto reused +│ └── path-containment.ts # UNCHANGED — resolveContainedDir reused +├── server.ts # The candidates read route +└── __tests__/deployments/ + └── restore-on-install.test.ts # NEW — real-filesystem harness + +packages/web/src/pages/ +└── InstallWizard.tsx # Restore step at index 0; re-create draft on change; + # acknowledgement checkboxes; summary warning + +packages/cli/src/ +├── index.ts # Flag registration +├── commands/install/install.ts # Parsing, mirroring parseGrants +└── lib/deploy-flow.ts # details-driven hints for restore refusals + +try-hola/apps (sibling PR, separate repo) +├── schemas/manifest.schema.json # The `restore` block, referencing $defs/backupHook +└── src/{guacamole,immich,mealie,paperless-ngx,postiz}/src/manifest.json +``` + +**Structure Decision**: the existing monorepo layout is used unchanged. One new +server module (`restore-candidates.ts`) is added rather than growing +`deployment.ts` further — candidate discovery is a pure, testable function of +(app id, deployments, identity records, catalog upgrade metadata) with no I/O +ordering concerns, and `deployment.ts` is already past 4,300 lines. The restore +*execution* stays in `deployment.ts` because it is inseparable from the lifecycle +job's ordering. + +## Design overview + +The implementation decomposes into six groups, in dependency order. + +**1 — Shared types.** `RestoreChoice`, `RestoreCandidate`, the acknowledgement +code union, and `AppRestoreDeclaration`; `restoreFrom` on `CreateDraftRequest`, +`Draft` and `FinalizedManifest`; `lineageId` + `restoreFrom` + `restoredAt` on +`EnhancedDeploymentDetail`. No change to `contracts.ts`. + +**2 — Candidate discovery** (`restore-candidates.ts`, new). Given an app id, list +eligible deployments (settled state, data root non-empty ignoring `.hola`), +describe each from its identity record with the deployment record as fallback, +group by lineage newest-first, and compute per-candidate skew verdicts and +required acknowledgement codes. Pure enough to unit-test without a filesystem for +the ranking and skew logic. + +**3 — Draft entry.** `createDraft` accepts `restoreFrom` on the catalog path, +reads the candidate's environment record from the apps-root sibling directory, +seeds `appEnv` via `mergeUpgradeAppEnv`, and defaults name/subdomain from the +candidate. The install-by-ref path refuses `restoreFrom`. `finalizeDraft` carries +it outside `canonicalSpec`, alongside `channel`. + +**4 — Persistence.** `createFromDraft` validates the choice again (the candidate +may have changed since draft creation), enforces acknowledgement codes the way +`grants` are enforced, and persists `restoreFrom` and `lineageId` onto the record. +`writeInstanceMarkers`' `lineageId` expression becomes +`deployment.lineageId ?? deployment.id`. + +**5 — Restore execution** (`runLifecycleJob`). The ten-step sequence in R8, +inserted between `:3813` and `:3815`, guarded so that an install with no restore +choice — or one whose restore has already been consumed — takes today's path +exactly. Requires `composeUp`'s widened signature. + +**6 — Surfaces.** The candidates read route; the wizard step; the CLI flags; the +catalog sibling PR; the four follow-up issues from R23. + +### Known trap, carried into tasks + +`MockDockerService.composeUp` (`docker.ts:722-725`) currently logs and returns +success. When its signature widens it must **record** the requested services and +wait flag so tests can assert on them. A Mock that accepts the new options and +ignores them produces a green suite over a broken restore — the single most +likely way this feature ships subtly wrong, and the reason Constitution IV exists. + +## Phase status + +- **Phase 0 — Research**: complete → [research.md](./research.md) (R1–R23, including + four corrections to the prompt of record: the job payload's shape, the absent + `CreateDeploymentRequest` type, the root-relative archive that voids the + "locate the subtree" trap, and `checkUpgradePath`'s inability to express two of + the four skew rules). +- **Phase 1 — Design & contracts**: complete → [data-model.md](./data-model.md), + [contracts/](./contracts/), [quickstart.md](./quickstart.md); `CLAUDE.md` plan + pointer updated. +- **Phase 2 — Tasks**: not started. `/speckit-tasks` produces `tasks.md`. + +### Post-design Constitution re-check + +Re-evaluated after the design artifacts were written. **No verdict changed.** The +design added one new server module and one new API route, neither of which touches +a principle: the module is internal and Real/Mock-neutral (a pure function over +already-fetched state), and the route is an ordinary authenticated platform read, +not a contract endpoint. Principle V's audit was performed *against the finished +`restore` block shape* in `contracts/manifest.md`, not against an intention, and +the block contains no server-interpreted app identity. diff --git a/specs/007-restore-on-install/quickstart.md b/specs/007-restore-on-install/quickstart.md new file mode 100644 index 00000000..d3f70898 --- /dev/null +++ b/specs/007-restore-on-install/quickstart.md @@ -0,0 +1,200 @@ +# Quickstart: verifying Restore-on-Install + +**Feature**: `specs/007-restore-on-install` · **Baseline**: `ca3d3f4` + +Every scenario below is numbered and individually citable, so a task in +`tasks.md` can say "implements scenario 14" and a reviewer can check it. The +**Mode** column says how each is verified: + +| Mode | Meaning | +|---|---| +| **U** | Server unit test, `bun --cwd packages/server test` | +| **U-fs** | Server unit test needing the **real-filesystem harness** — `RealStorageService` + `mkdtemp` + `HOLA_APPS_BIND_ROOT`. Copy `install-markers.test.ts:116-142` or `snapshot.test.ts:99-125`. | +| **W** | Web test, `cd packages/web && npx vitest run` | +| **C** | CLI test, `cd packages/cli && npx vitest run` | +| **VM** | Disposable VM — needs real Docker, real containers, a real database. `bin/vm-e2e-suite` / the `vm-e2e` skill. **Not** in the default suite. | + +> **Why so many U-fs.** `MockStorageService` discards file modes (issue #475) and +> `MockDockerService` starts no containers, so anything asserting on real files or +> real ordering has to use the real harness. This is the same constraint spec 006 +> hit; the harness is copy-paste, not new work. + +--- + +## 0. Prerequisites + +```bash +bun install +bun run typecheck && bun run lint && bun run test && bun run build +``` + +For VM scenarios, see `docs/MCP_VM_TESTING.md` and the `vm-e2e` skill. Use an app +with a database for anything exercising discards or hooks — **mealie** is the +cheapest (one Postgres, one app container, a declared healthcheck on both). + +--- + +## 1. Candidate discovery + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 1 | With one installed copy of an app holding data, the candidates route lists exactly that copy, described by name, address, version, `carriesEnv` and `capturedAt`. | U-fs | FR-001, FR-002, US1-AC1 | +| 2 | A candidate's description comes from its `.hola/instance.json`; with that file deleted it is **still listed**, described from the deployment record, with `hasIdentityRecord: false` and `lineageId` falling back to the deployment id. | U-fs | FR-003 | +| 3 | A deployment whose data root holds **only** `.hola/` is not listed. (This is the ignore-list rule — without it every install looks like it has data.) | U-fs | FR-004 | +| 4 | A deployment that is deploying, promoting or in `error` is not listed; the same deployment once `running` or `stopped` is. | U-fs | FR-004a | +| 5 | Three candidates across two lineages: grouped by lineage, newest-first within each, `defaultCandidateId: null` and `requiresExplicitChoice: true`. With all three in one lineage, the newest is the default. | U | FR-005, FR-036 | +| 6 | The candidates route answers with no backup provider installed and no grant consented anywhere on the host. | U-fs | FR-006, SC-001 | +| 7 | An app with no other copies returns `200` with `lineages: []` — not `404`. | U | FR-042 | + +--- + +## 2. Entering the choice + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 8 | `restoreFrom` on `POST /api/drafts` is accepted on the catalog path; the same body on the install-by-ref path is **refused** with `RESTORE_NOT_SUPPORTED`. | U | FR-007, FR-048 | +| 9 | `PatchDraftRequest` still rejects `restoreFrom`, and finalize still takes no body. | U | FR-007 | +| 10 | With `carryEnv: true`, the draft's `appEnv` carries the candidate's values; a key the new release declares that the candidate lacked **and** that has a `generate` recipe is freshly minted; a new key without one rides through to be surfaced by name. (The three-case rule.) Assert the carried values **equal** the source's, which is SC-003. | U-fs | FR-008, SC-003, US2-AC1, US2-AC2 | +| 11 | The finalized manifest carries `restoreFrom` **outside** `canonicalSpec`: two finalizes differing only in `restoreFrom` produce the **same** checksum. | U | FR-009 | +| 12 | After `createFromDraft`, the deployment record carries `restoreFrom` and `lineageId`. The deploy job payload is unchanged — still `{ releaseId, action }`. | U | FR-010 | +| 13 | A fresh install's `lineageId` equals its own id; a restored install's equals the candidate's. Confirm by reading `.hola/instance.json` after each. | U-fs | FR-011, SC-007 | +| 14 | A restored deployment sets `restoredAt`. A subsequent restart, promote and rollback each leave the data root untouched and quiesce nothing. | U-fs | FR-012 | + +--- + +## 3. Executing the restore + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 15 | The restore runs after `composePull` and before `composeUp`. Assert on call ordering through the Mock docker service. | U | FR-013 | +| 16 | A target data root that already holds app data aborts with `RESTORE_TARGET_NOT_EMPTY`. A root holding only `.hola/` proceeds — the marker is not app data. | U-fs | FR-014 | +| 16a | A candidate deleted, or moved out of a settled state, between the draft and the deploy job fails the install with `RESTORE_CANDIDATE_GONE` / `RESTORE_CANDIDATE_BUSY` — the job re-resolves rather than trusting the draft. | U-fs | FR-013a | +| 17 | The source's pre-hooks run before the capture, its post-hooks run after, and after the restore the source's data, status and address are unchanged and it is still running. | VM | FR-015, SC-006, US1-AC5 | +| 18 | After a restore the target data root holds the source's files. With the archive emptied, the install fails with `RESTORE_PAYLOAD_EMPTY` rather than reporting success. **No subtree search** — the post-condition is the mechanism (research R9). | U-fs | FR-016 | +| 19 | The staging archive lives under the **target** deployment, not the source, and is gone after both a successful and a failed restore. It never appears in the source's snapshot listing. | U-fs | FR-016a, SC-013 | +| 20 | Every `discard` path is removed before any container starts. A path resolving outside the data root (`../`, an absolute path, a symlink out) **refuses** the restore. | U-fs | FR-017, US4-AC1, US4-AC4 | +| 21 | After a restore, `.hola/instance.json` describes the **new** install — its `deploymentId`, `name` and `host` — while its `lineageId` is the source's. The source's record does not survive. | U-fs | FR-018, SC-007 | +| 22 | **The OIDC ordering trap.** With auth provisioned and a restore requested, `oidc.json` exists in the data root after the restore. Run the same install with the write left at its old position to confirm the test actually fails — a test that passes either way tests nothing. | U-fs | FR-019, SC-008 | +| 23 | `composeUp` with `{ services: ['db'], wait: true }` issues `up -d --wait db` and starts nothing else. `MockDockerService` **records** the services and wait flag. | U | FR-020 | +| 24 | A hook service that never becomes healthy fails the install with `RESTORE_HOOK_FAILED`; no other service is started. A restore hook exiting non-zero does the same. | VM | FR-021, US3-AC5, US4-AC3 | +| 25 | Every restore failure leaves a **failed install** — never a running app on an empty or half-restored root. | U-fs | FR-022, SC-004, US3-AC6 | +| 26 | A failed restore leaves the deployment in `error` with its data root intact, and that deployment is then absent from the candidates list. | U-fs | FR-022a | +| 27 | **The regression guard.** With no `restoreFrom`, the deploy job's calls and their order are identical to `main`'s — including `writeOidcCredentialsFile`'s original position. | U | FR-023, SC-009 | +| 28 | An end-to-end restore of a database-backed app: install mealie, add a recipe, install a second copy restoring from the first, and see the recipe in the second. Read back an item stored under a platform-generated secret to prove SC-003 end to end. | VM | US1-AC2, SC-002, SC-003 | + +--- + +## 4. The app's declaration + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 29 | A `restore` block keyed by participation id drives discards and the hook for that participation. An app with two participations restores both. | U-fs | FR-024, FR-027, SC-010 | +| 30 | `accepts: ["restore@1"]` with **no** block restores by plain file copy: files land, nothing is discarded, no hook runs. An app declaring nothing is not offered at all. | U-fs | FR-025, US4-AC5 | +| 31 | The restore hook uses `AppBackupHook`'s shape verbatim; no second hook type exists in `shared`. | U | FR-026 | +| 32 | Every current catalog manifest validates unchanged against the extended schema; a manifest with a `restore` block validates. | U | FR-028 | +| 33 | A real Postgres restore: discard the captured `PGDATA`, let `initdb` run clean, load the dump. The app serves the dumped data. | VM | FR-017, US4-AC1, US4-AC2 | + +--- + +## 5. Refusals and warnings + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 34 | A candidate **newer** than the target refuses with `RESTORE_SOURCE_NEWER`. **This is the rule `checkUpgradePath` cannot express** — assert directly that `checkUpgradePath(newer, older, meta)` returns `ok`, so the test documents why the rule is stated independently. | U | FR-029, US3-AC1 | +| 35 | A guarded older→target hop refuses with `RESTORE_UPGRADE_PATH` and surfaces `suggestedVersion`. | U | FR-030, US3-AC2, SC-005 | +| 36 | Equal versions, and older-with-a-clean-path, both proceed. | U | FR-031 | +| 37 | An unknown candidate version proceeds **only** with `restore-version-unknown`; without it, `RESTORE_ACK_REQUIRED` naming that code. | U | FR-032, US3-AC3 | +| 38 | With no environment record, the warning names exactly the keys that are `isSecret` **and** carry a `generate` recipe — not every secret, not every generated value. | U-fs | FR-033, US2-AC3 | +| 39 | With `requiresEnv: true` and no environment record, the restore is **refused** (`RESTORE_ENV_REQUIRED`), not warned. | U | FR-034, US3-AC4 | +| 40 | Name and subdomain default from the candidate; changing the subdomain produces a `host-divergence` warning naming both. | U | FR-035, US1-AC1 | +| 41 | Every refusal carries `details.code`; every version refusal that has a next step carries `suggestedVersion`. | U | FR-037, SC-005 | +| 42 | A required acknowledgement code that is absent fails the create with `RESTORE_ACK_REQUIRED` — the same shape as a missing `grant`. | U | FR-037a, FR-046 | +| 43 | Declining to carry configuration that **is** available still produces the named warning and still requires the acknowledgement. | U-fs | FR-033, US2-AC4 | + +--- + +## 6. Wizard + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 44 | The restore step renders at index 0, before Configuration. | W | FR-038 | +| 45 | Changing the restore choice deletes and re-creates the draft, resetting consent — the `switchChannel` pattern. | W | FR-039 | +| 46 | Carried values render as ordinary `appEnv` rows through the existing mask/reveal component, with no separate widget. | W | FR-040 | +| 47 | Any restore shows the acknowledgement on the summary step, naming data **and credentials** and the fact that jobs, webhooks and integrations may fire on start. | W | FR-041, US2-AC5 | +| 48 | With no candidates the step says so and Next is enabled. | W | FR-042 | + +--- + +## 7. CLI + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 49 | `--restore-from `, `--restore-from latest`, `--no-restore` and `--restore-list` each behave as specified; `latest` refuses across two lineages. | C | FR-043, FR-036, US5-AC2, US5-AC3 | +| 50 | An install with **no** restore flag performs no restore even when candidates exist. | C | FR-044, SC-012, US5-AC4 | +| 51 | Every refusal hint is built from `details`; with the message blanked and only `details` populated, the hint is still correct and still names the right flag. | C | FR-045, US5-AC5 | +| 52 | `--ack` parses repeated and comma-separated values exactly as `--grant` does. | C | FR-037a, FR-046 | +| 53 | **#429 closed**: one command produces a second independent copy of a running app holding its data, with the source untouched. | VM | SC-011, US5-AC1 | + +--- + +## 8. Scope boundary + +| # | Scenario | Mode | Covers | +|---|---|---|---| +| 54 | `CONTRACTS` is unchanged — still exactly `auth@1`, `backup@1`, `push@1`, `container-logs@1`. No new grant kind exists. No `/api/contracts/*` route changed. `JobType` gained nothing. | U | FR-047 | +| 55 | The pre-existing `POST /api/backups/:id/restore` stub and `RestoreBackupRequest` are untouched, and nothing in this feature imports them. | U | research R19 | +| 56 | A whole restore completes with no backup provider installed — `backrest` absent from the host entirely. | VM | FR-047, SC-001 | + +--- + +## 9. Gate + +```bash +bun run typecheck && bun run lint && bun run typecheck && bun run test && bun run build +``` + +Typecheck twice — CI has caught typecheck regressions that a lint auto-fix +introduced after the first run (CLAUDE.md `## Commands`). + +### Scope-boundary greps + +```bash +# No capability contract was added. +git diff main -- packages/shared/src/contracts.ts # must be empty + +# The dead restore stub was not touched. +git diff main -- packages/web/src/pages/Backups.tsx \ + packages/web/src/hooks/useBackupsApi.ts # must be empty +grep -rn "RestoreBackupRequest" packages/server/src # only the pre-existing stub + +# No per-app branching in the server's restore path (Constitution V). +grep -rniE "postgres|mealie|immich|gitea|paperless" \ + packages/server/src/services/core/restore-candidates.ts # must be empty +``` + +That last grep is the mechanical form of the Principle V audit in +[plan.md](./plan.md). If it ever matches, an app's name has leaked into the +platform and the design has regressed. + +--- + +## Coverage + +All 53 functional requirements and all 13 success criteria appear at least once +above. SC-003 (carried configuration matches the source's) is verified twice on +purpose — once as an assertion inside scenario 10, and once end to end in scenario +28 — because it is the criterion whose failure is silent: an app with mismatched +secrets starts cleanly and shows its data. Requirements verified only on the VM path — FR-015, FR-021 — are the two +that need real containers and a real database; both also have a U-fs scenario +covering their non-container half, so neither is untested in the default suite. + +| Group | Scenarios | +|---|---| +| Candidate discovery | 1–7 | +| Entering the choice | 8–14 | +| Executing the restore | 15–28 (incl. 16a) | +| App declaration | 29–33 | +| Refusals and warnings | 34–43 | +| Wizard | 44–48 | +| CLI | 49–53 | +| Scope boundary | 54–56 | diff --git a/specs/007-restore-on-install/research.md b/specs/007-restore-on-install/research.md new file mode 100644 index 00000000..f56f8c42 --- /dev/null +++ b/specs/007-restore-on-install/research.md @@ -0,0 +1,693 @@ +# Research: Restore-on-Install from a Live Deployment + +**Feature**: `specs/007-restore-on-install` · **Date**: 2026-09-20 · **Baseline**: `ca3d3f4` (post-spec-006) + +Every line number in this document was verified by a read-only sweep of the +working tree at `ca3d3f4`. The prompt of record was written before spec 006 +merged (`921c790`), so several of its anchors have moved — and three of its +claims are wrong in *shape*, not merely in position. Those are called out +individually below, because a plan that silently inherits them produces code that +guards against conditions this codebase cannot produce. + +--- + +## R1 — Where the restore choice enters: `CreateDraftRequest`, and nowhere else + +**Decision.** `CreateDraftRequest` gains `restoreFrom?: RestoreChoice`. It is not +patchable and not supplied at finalize. + +**Rationale.** This is forced by three existing properties, not chosen: + +| Property | Where | Consequence | +|---|---|---| +| `PatchDraftRequest` is closed to four fields | `shared/src/index.ts:1339` | A patch route for the restore choice would mean widening a deliberately narrow type | +| `updateDraft` re-hardens `appEnv` against the stored spec on every patch | `draft.ts:694-696` | Env seeded by a *patch* would be reverted by the next patch | +| `finalizeDraft` freezes `appEnv` into the checksummed `canonicalSpec` | `draft.ts:886-908` | A post-finalize env overlay breaks the property that the checksum describes what is deployed | + +The restore choice determines `appEnv`. `appEnv` is only ever legitimately +established at draft creation. Therefore the restore choice is only ever +legitimately established at draft creation. + +**Alternatives considered.** (a) A patch field — rejected: `updateDraft`'s +re-hardening would silently undo it, which is the worst kind of bug, one that +works in a unit test and fails in the wizard. (b) A finalize body — rejected: +finalize takes no body today and adding one to carry a value that must be +reflected in the checksummed spec is a contradiction. (c) A separate +"restore plan" resource joined at create-deployment time — rejected as a second +lifecycle for one boolean-plus-id, with its own orphan-cleanup problem. + +--- + +## R2 — Only the catalog draft path accepts a restore choice; install-by-ref refuses it + +**Decision.** `restoreFrom` is honoured on the catalog path. On the +install-by-ref path it is **rejected with an error**, never ignored. + +**Rationale.** `resolvePlatformTokens` seeds `appEnv` at **two** call sites: +`draft.ts:473` (catalog) and `draft.ts:606` (install-by-ref). The prompt named +only the first. Honouring the choice on one path and silently dropping it on the +other is precisely the silent-empty-restore failure the spec exists to prevent +(spec §"The failure this exists to prevent"): the operator asks for their data, +the install succeeds, and the data is not there. + +Install-by-ref is excluded rather than supported because a candidate's version +skew is judged against catalog upgrade metadata (`checkUpgradePath`, R15), and +the install-by-ref path deliberately has no catalog index to consult — `draft.ts` +already fails closed there for channel resolution (`channel: STABLE_CHANNEL`, +`channelPublished: false`, `:634-635`), for the same reason. Rather than invent a +weaker version check for one path, this feature declines the path. + +**Alternatives considered.** Supporting install-by-ref with the version check +skipped — rejected: it would make the *least* inspectable install path the *only* +one that restores data without a skew check. Silently ignoring `restoreFrom` +there — rejected outright; fail closed. + +--- + +## R3 — The choice reaches the deploy job on the deployment record, not in the job payload + +**Decision.** Persist the restore choice on the deployment record (spec FR-010). +Add no new job-payload key. + +**This corrects the prompt.** The prompt states the lifecycle payload is +`{ releaseId, action, deploymentId }`. It is not: + +```ts +// deployment.ts:1186-1197 (maybeStartJob) +deploymentId, // :1192 — a sibling Job field +payload: { releaseId, action: 'deploy' } // :1193 — two keys, not three +``` + +`deploymentId` has never been a payload key. Extra payload keys do exist — the +rollback path sets `restoreData` and `targetReleaseId` at `:1541-1549` — so the +prompt's *instinct* (that a payload key was available) was reachable, just not by +the route it described. + +**Rationale for the record over a payload key**, which is the substantive +decision: + +1. **A payload does not survive a retry.** The record does. If the deploy job is + re-run, a payload-borne choice is either lost or replayed depending on how the + job was re-created — neither is a defensible answer to "should this install + restore?". +2. **FR-012 is a fact about the deployment, not about one job.** "The restore + applies to the first deploy only" is history. History belongs on the record + that persists, next to the `restoredAt` marker that records consumption. +3. **It matches `channel`'s route exactly** (`deployment.ts:1028`), which is the + precedent the prompt correctly identified even while misdescribing the payload. + +Rollback's `restoreData` is a genuinely different case and stays a payload key: +it is a property of *that rollback invocation*, chosen at the moment the operator +clicks rollback, and is meaningless outside it. + +--- + +## R4 — `lineageId` becomes a persisted deployment-record field + +**Decision.** Add `lineageId` to the deployment record. Set it to the deployment's +own id on a fresh install and to the candidate's lineage on a restore. Change +`writeInstanceMarkers` to read it. + +**Rationale.** The shipped code already specifies this change, in the file that +will perform it: + +```ts +// deployment.ts:3366-3374, inside writeInstanceMarkers +// Derived, not persisted ...: it always equals `deploymentId` today, so it +// needs no storage of its own — Sequence 5 (restore-on-install) is +// what forces a `lineageId` onto the deployment record, at which +// point this expression becomes `deployment.lineageId ?? deployment.id`. +lineageId: deployment.id, +``` + +The `?? deployment.id` fallback is what makes this a zero-migration change: every +deployment record written before this feature has no `lineageId`, reads as +`undefined`, and falls back to exactly the value it has always had. **No backfill +is required, and no identity record already on disk is wrong.** Spec 006 wrote +the field into every record precisely so that captures taken before this feature +would already carry it. + +--- + +## R5 — Candidate discovery: deployments plus their identity records, restricted to settled states + +**Decision.** A candidate is an existing deployment of the same app, other than +the one being created, that (a) is in a settled state — running or stopped —, and +(b) has a data root holding app data. Describe it from its identity record, with +the live deployment record as the per-field fallback. + +**Rationale.** + +- **"Holds app data"** is already a solved question with a subtle answer: + `dirHasContents(appRoot, [INSTALL_MARKERS_DIR])` (`snapshot-fs.ts`, used at + `deployment.ts:2513`). The marker directory must be excluded or *every* + materialised install looks like it has data — spec 006's own review caught this + as a data-loss path. Candidate discovery reuses the identical call rather than + re-deriving the rule. +- **Settled states only** (spec FR-004a). A deployment mid-deploy, mid-promote or + mid-rollback is being written by the platform itself; its data root may be + mid-replacement by `restoreTarGzInto`, which `rm -rf`s the destination before + extracting. An `error`-state deployment may hold a partially restored tree + (FR-022a deliberately leaves it in place). Capturing either produces a corrupt + source silently. +- **Identity record first, deployment record as fallback** (FR-003). For *this* + slice both are present and the deployment record would suffice — but `lineageId` + exists only in the identity record for installs that predate R4, and reading the + record here is what makes the same code path work unchanged when Sequence 6 + replaces "a live deployment" with "an archive". + +**Alternatives considered.** Deriving candidates purely from deployment records — +rejected: loses `lineageId` for pre-R4 installs and builds a discovery path +Sequence 6 would have to rewrite. Offering error-state deployments with a warning +— rejected: a partial tree is not a thing to warn about, it is a thing to exclude. + +--- + +## R6 — Candidates are served by a new platform API read route + +**Decision.** A new read route, keyed by app, that does not require a draft. + +**Rationale.** FR-047's prohibition is on *capability-contract* endpoints — the +`/api/contracts/...` broker surface governed by ADR 0004. It is not a prohibition +on ordinary platform API routes, and reading it as one would leave the feature +with no way to show the operator anything. + +The route is forced rather than chosen: + +- **A field on the draft-creation response is circular.** The restore choice is + the wizard's *first* step (FR-038) and is an input to draft creation (R1). The + client would need a draft in order to learn what to put in the draft request. +- **`--restore-list` must work without creating a draft at all** (FR-043). + Creating and discarding a draft to answer a read-only question is a side effect + in a listing command. + +**Alternatives considered.** Extending the catalog app-detail response — rejected: +candidates are a fact about *this host's deployments*, not about the catalog, and +`RealCatalogService` fetches a remote index (Constitution II). Putting it on the +deployments list with a filter — rejected: the answer needs per-candidate derived +fields (carriesEnv, version skew, warnings) that do not belong on a generic list. + +--- + +## R7 — Capture staging: under the target, deleted in `finally`, with no separate extraction + +**Decision.** Tar the source's data root to a staging file under the **target** +deployment's own directory, distinct from the pre-upgrade snapshot store, and +delete it in a `finally` regardless of outcome. Extract straight into the target +data root — there is no intermediate extracted copy. + +**Rationale.** The disk arithmetic is the decision. An app data root can be tens +of gigabytes, and a naive stage-then-move design holds three copies at once (the +archive, the extracted staging tree, the final data root). It does not have to: + +```ts +// snapshot-fs.ts +tarGzipDir: tar -czf ... -C . // contents, ROOT-RELATIVE +restoreTarGzInto: rm -rf destDir; mkdir -p destDir; tar -xzf -C +``` + +Because the archive is root-relative and the extractor targets a directory, the +payload can be extracted **directly into the target data root**. Peak additional +cost is one compressed archive (SC-013). + +**Under the target, not the source**, for three reasons: it is deleted with the +target on uninstall if anything ever leaks; it does not enter the *source's* +`pruneSnapshots` retention, where it would either be pruned mid-restore or +displace a real pre-upgrade snapshot; and a failed restore's staging file is +found next to the failed install an operator is inspecting, not next to a healthy +unrelated app. + +**Note the ordering hazard this creates and R8 resolves**: `restoreTarGzInto` +`rm -rf`s its destination. The destination is the app data root, which +`materializeCompose` has already populated with `.hola/instance.json`. That is +survivable *only* because FR-018 rewrites the marker afterwards — and is exactly +why FR-019 must move the OIDC file write to after the restore. + +--- + +## R8 — Placement in `runLifecycleJob`, and the exact order + +**Decision.** Insert into the `deploy / start / rollback` branch +(`deployment.ts:3780-3821`), **after** the cancellation check at `:3813` and +**before** `composeUp` at `:3815`. + +Current branch, verified: + +| Line | Call | +|---|---| +| 3789-3793 | rollback `restoreData` branch (`composeDown`, `restoreAppDataSnapshot`) | +| 3796 | `provisionAuth` | +| 3797 | `materializeCompose` (creates the app root, writes `.hola/instance.json`) | +| **3800** | `writeOidcCredentialsFile` ← **moves** | +| 3805 | `resolveRegistryAuth` | +| 3807 | `composePull` | +| 3813 | `ctx.isCancelled()` | +| **←** | **restore inserts here** | +| 3815 | `composeUp` | +| 3818 | `completeAuthWiring` | + +Ordering within the restore, each step justified: + +1. **Assert the target root holds no app data** (FR-014) — `dirHasContents(appRoot, + [INSTALL_MARKERS_DIR])`, the same ignore-list as R5. Refuse if it does. +2. **Re-resolve the candidate and re-check it is settled** — it may have been + deleted or started a lifecycle action since the draft was created (spec Edge + Cases). +3. **Quiesce and capture the source** (R14, FR-015). +4. **Extract into the target root** (R7). This `rm -rf`s the root, destroying the + marker written at `:3797`. +5. **Assert the payload is present** (FR-016, R9). +6. **Apply `discard` paths** (R13, FR-017). +7. **Rewrite `.hola/instance.json`** (FR-018) — restores the marker step 4 + destroyed, *and* corrects it to describe the new install rather than the source. +8. **Write the OIDC credentials file** (FR-019) — after step 4, not before. +9. **Start only the hook services and wait** (FR-020, R12). +10. **Run restore hooks fail-closed** (FR-021). + +Then `composeUp` at `:3815` starts everything else unchanged. + +**Why after the pull rather than before it.** Pulling first means a restore is not +attempted at all for an install that cannot get its images — the cheaper failure +happens first, and the expensive capture of a live source is not wasted. It also +puts the restore after the cancellation check, so a cancelled install has not +quiesced somebody else's database. + +**Why inside the job and not at create time.** Constitution III. The capture runs +`pg_dump` in a live container and tars tens of gigabytes; it is exactly the +"slow, side-effectful, failure-prone" work the principle exists to keep out of +request handlers. + +--- + +## R9 — FR-016 is a post-condition, not a subtree search + +**Decision.** Assert after extraction that the target data root holds app data +(ignoring the marker directory). Do **not** search for a nested payload directory. + +**This corrects the prompt.** The prompt lists as trap (c): + +> The subtree must be located, not assumed. A restore recreates the absolute path +> structure under the target, so the payload is at ``, +> not at ``. + +That is a true and important statement **about a provider's archive tool** — +restic, borg and tar-with-absolute-paths all reproduce the source's absolute path +under the restore target. It is not true of this codebase's own helpers, which +use `-C .` on the way in and `-C ` on the way out (R7) and are +therefore root-relative in both directions. + +Writing a subtree search into this slice would be code defending against a layout +this slice cannot produce — untestable except by constructing an archive the +system never creates, and a maintenance liability that reads as a mystery to the +next person. + +Stating the requirement as the **outcome** ("the payload must actually be there") +rather than the mechanism gives the same protection against the real failure — +an empty-looking restore that reports success — and holds unchanged when +Sequence 6 introduces an archive shape where the subtree question is live. The +trap is recorded here so Sequence 6 inherits it rather than rediscovering it. + +--- + +## R10 — `writeOidcCredentialsFile` moves to after the restore + +**Decision.** Move the call from `:3800` to inside the restore sequence, after +extraction (R8 step 8). When no restore is requested, it must run exactly where +it runs today. + +**Rationale.** `writeOidcCredentialsFile` (`:3673-3700`) writes the provisioned +OIDC credentials into the app's data root, so a bundle sidecar can render the +app's SSO config before first boot. `restoreTarGzInto` `rm -rf`s that data root. +Written before the restore, the file is destroyed by it, the sidecar finds +nothing, and the app boots without SSO — reporting no error at any layer. Immich's +`immich-oidc-init` bolt-on is the concrete instance; the shape is general. + +**Implementation constraint.** The no-restore path must not change (FR-023). +Moving the call unconditionally would reorder it relative to `composePull` for +every install on the host, which is a behaviour change outside this feature's +remit. The call site therefore becomes conditional on whether a restore is being +performed. + +--- + +## R11 — The instance marker is rewritten after the restore + +**Decision.** Call the existing marker writer again, after extraction and +discards, before any container starts. + +**Rationale.** Two independent reasons, either sufficient: + +1. **The restore destroys it.** Step 4 `rm -rf`s the data root including + `.hola/`. Without a rewrite the new install has no identity record at all. +2. **The restored tree carries the source's record.** The archive contains the + *source's* `.hola/instance.json` — its `deploymentId`, its `name`, its `host`. + Extracted verbatim, the new install claims to be the old one. Every capture + taken afterwards inherits the lie, and a restore chain compounds it. + +The rewrite is not new code: `writeInstanceMarkers(deployment, appRoot, host)` +already exists (`:3322`) and already computes every field correctly — including +`lineageId`, which after R4 reads the carried lineage rather than the new id, +which is exactly the behaviour SC-007 asserts. + +--- + +## R12 — `composeUp` gains a service list and a wait; the timeout is parameterised + +**Decision.** Widen `composeUp` to accept `{ services?: string[]; wait?: boolean; +timeoutMs?: number }`. Implement in the interface (`docker.ts:67`), Real +(`:231-269`) and Mock (`:722-725`). Rely on the service's own healthcheck. + +**This is genuinely new work.** There is **no `--wait` anywhere in `docker.ts` +today** — verified by search. The current Real implementation is: + +``` +docker compose -f -p up -d // :247 +``` + +with a 5-minute `execAsync` timeout at `:248`, and `profiles` travelling via the +`COMPOSE_PROFILES` environment variable (`withComposeProfiles`, `:192-195`) rather +than a CLI flag. The prompt's `{ services?: string[]; wait?: boolean }` is correct +in intent; it is not a small change. + +**The timeout must be parameterised, and this is not optional.** `--wait` blocks +until every named service is healthy. A Postgres restore starts a *freshly +initdb'd* cluster (R13) — on a slow disk, with a large `shared_buffers`, or under +`pgautoupgrade`, that can approach or exceed five minutes. Inheriting the existing +5-minute cap would turn a slow-but-correct restore into a failed install, which +under FR-022 means the operator loses the whole install. The restore's wait +therefore passes its own, larger timeout. + +**Why the app's healthcheck rather than a poll.** A bespoke readiness poll would +need per-app knowledge of what "ready" means for that datastore — the exact +per-app branching Constitution V forbids. Compose's `--wait` consumes the +healthcheck the app already declares. The catalog survey confirms this costs +nothing: **all five** hook-declaring apps already declare a healthcheck on their +Postgres service. An app whose hook service declares none is a declaration bug, +surfaced as a refusal (FR-020) rather than papered over with a timer. + +**Mock behaviour matters** (Constitution IV). `MockDockerService.composeUp` must +accept and record the new options and continue returning success. A mock that +ignores `services` would let every test pass while the real path started the wrong +containers. + +--- + +## R13 — `discard` paths, and why they exist + +**Decision.** `discard` is a list of data-root-relative paths removed after +extraction and before any container starts. Every path is resolved through +`resolveContainedDir` (`path-containment.ts:30`), exactly as push targets already +are (`deployment.ts:2952`). + +**Rationale — why discarding is *required*, not a convenience.** A file-level tar +of a **live** `PGDATA` is read over a window of minutes. Page 1 is read at +T+0s and page 100000 at T+180s, with the database writing throughout. The result +is not a snapshot; it is a smear across time, which for Postgres is corruption. +`tarGzipDir` says as much in its own comment — it suppresses tar's +"file changed as we read it" warning because crash-consistency is all it promises. + +The `.sql` dump written by the app's `preHook` is the real payload, and the +catalog survey confirms it is captured: **all five** hook apps mount +`${HOLA_APP_DATA}/backups:/backups`, inside the data root. So the correct restore +is: discard the smeared `PGDATA`, let the container `initdb` a clean cluster, and +load the dump into it. + +**Containment.** A discard path is app-supplied data that names a filesystem path +for deletion. `resolveContainedDir` resolves it and confines it to the data root; +a path that resolves outside is refused (FR-017). This is the same guard push +targets use, and the reason spec 006's review and issue #482 both landed on +resolution rather than a `startsWith` test: lexical prefix checks are not +containment proofs. + +--- + +## R14 — Quiescing and restore hooks both reuse the existing fail-closed policy + +**Decision.** Capture-side quiescing reuses `runPreHooksFailClosed` +(`:2684-2701`) and `runPostHooks` (`:2709-2724`) with `BackupParticipant` +(`:112-117`) unchanged. Restore-side hooks reuse the same policy. + +**Rationale.** The policy is already correct and already reviewed: hooks run in +declaration order; a failure propagates; cleanup runs the `postHook` of every +**started** participation only. That started-only rule is not incidental — it is +the same rule the contract broker's prepare/finalize applies, and it exists so a +cleanup never runs against a participation whose pre-hook never ran. + +For restore hooks, fail-closed is the whole point (FR-021). A half-loaded database +handed to an app that then runs its own migrations is a corruption the operator +will not detect until much later. + +**Alternatives considered.** A best-effort restore hook with a warning — rejected +against spec §"The failure this exists to prevent": a warning in a job log is not +a thing an operator reads before trusting their data. + +--- + +## R15 — Version skew: three rules, only one of which `checkUpgradePath` can express + +**Decision.** Judge each candidate with three rules, in order: + +| Relationship (candidate → target) | Outcome | Source of the rule | +|---|---|---| +| Candidate version **unknown** | Allow **only** with an acknowledgement code | This feature (R16) | +| Candidate **newer** than target | **Refuse** always | **This feature** | +| Candidate older, path guarded | **Refuse**, surface `suggestedVersion` | `checkUpgradePath` | +| Candidate equal, or older with a clean path | Allow | `checkUpgradePath` | + +**This corrects the prompt.** The prompt delegates all four rows to +`checkUpgradePath`. It cannot carry two of them: + +```ts +// shared/src/index.ts:423-460 +if (!meta || !fromVersion || !toVersion) return { ok: true }; // unknown ⇒ OK +if (!isNewerVersion(toVersion, fromVersion)) return { ok: true }; // downgrade ⇒ OK +``` + +A candidate newer than the target is, in `checkUpgradePath`'s terms, a +*downgrade* — and downgrades pass through deliberately, because the function was +written to guard **promotes**, where rollback is a legitimate operation the +caller owns. Restoring newer data into an older release is not a rollback: there +is no release to roll back to, and the data may use a schema the older binary +cannot read. The rule must therefore be stated independently, and the spec's +FR-029 does. + +The same is true of the unknown-version row: `checkUpgradePath` returns `ok` when +either version or the metadata is missing. The prompt noticed this ("since +`checkUpgradePath` returns ok for unknown versions") and reached the right +conclusion — an explicit acknowledgement — without noticing that the newer-than +row has the identical problem. + +`checkUpgradePath` is reused verbatim for the two rows it *does* express. Its only +two production call sites today are `deployment.ts:1104` and `:3178`, both on the +promote path; this adds a third, on a different axis, with the candidate's version +as `from` and the install's version as `to`. + +--- + +## R16 — Acknowledgements are codes, modelled on `grants` + +**Decision.** The restore choice carries `acknowledge?: string[]`. The server +computes which codes the chosen candidate requires and refuses the create when +one is absent. + +**Rationale.** The platform already has exactly this pattern, one type away: + +```ts +// shared/src/index.ts:2257-2289, CreateDeploymentFromDraftRequest +// Capability contract grants the operator consents to (ADR 0004 §4) ... The +// server intersects this with what the manifest actually declares in `provides` +// ... and REJECTS the install when a declared grant is missing: an app whose job +// is acting on other apps' data, installed without the access to do it, fails +// silently at the worst possible moment. Client-supplied — the wizard's consent +// checkboxes, the CLI's `--grant`. +grants?: string[]; +``` + +The reasoning transfers without modification: a risk the operator did not +acknowledge must fail the install loudly, at create time, not silently at run +time. Reusing the shape means the wizard's checkbox and the CLI's flag are the +same mechanics operators already know, and FR-046's non-interactive equivalent is +buildable rather than merely mandated. + +*(Note the correct type name: there is no `CreateDeploymentRequest` in this +codebase. It is `CreateDeploymentFromDraftRequest`.)* + +Two codes are defined (see data-model.md): one for an unverifiable version +relationship, one for configuration that cannot be carried. + +--- + +## R17 — Carrying configuration: the three-case merge, reading from the sibling record + +**Decision.** Seed the draft's `appEnv` from the candidate's environment record +using `mergeUpgradeAppEnv` (`upgrade-env.ts:35-44`) unchanged. + +Its three cases are exactly the rule this feature needs: + +```ts +if (hasOwnProperty(carriedAppEnv, entry.key)) return { ...entry, value: carried }; // carried wins +if (entry.generate && !entry.value) return { ...entry, value: generateSecretValue(entry.generate) }; // mint +return entry; // ride through +``` + +Its only production call site today is `server.ts:1295` (the upgrade path); this +adds a second. + +**Where the record is read from — and the correction this forces.** The prompt +says the environment record is read from "the candidate's `.hola/env.json`", +inside the data root. Spec 006 shipped it **outside** the data root: + +```ts +// deployment.ts:2474-2475 +private envRecordDirFor(deploymentId: string): string { + return `${this.appsBindRoot()}/${INSTALL_ENV_ROOT_DIR}/${deploymentId}`; +} +``` + +For this slice that is convenient rather than limiting — the source is a live +deployment on the same host, so the server reads the record directly from the +apps root. But it means a **captured data root alone does not carry +configuration**, which is a fact Sequence 6 must confront and this document +records so it is not rediscovered late. + +**Deriving severity rather than asking the catalog** (FR-033). When the record is +absent, the keys that matter are computable: an `AppEnvVar` (`:903-958`) with +`isSecret: true` **and** a `generate` recipe is a value the *platform* invented, +which restored data may depend on and which will be minted fresh. Naming those +keys is strictly better than a manifest field an app author would have to +remember to set, and it cannot go stale. + +--- + +## R18 — The manifest `restore` block reuses `AppBackupHook` verbatim + +**Decision.** A per-participation `restore` block keyed by backup participation +id, reusing `AppBackupHook` (`:295-298`, `{ service, command }`) for the hook. + +**Rationale.** A second hook format would be a second thing to validate, document +and get wrong, for no expressive gain — a restore hook is a command run in a named +service, which is what `AppBackupHook` already is. The catalog schema already +carries `$defs/backupHook`, so the schema change references an existing definition. + +**Keying by participation id** follows spec 004's cardinality model: an app with +two stateful services declares two backup participations, and each needs its own +restore. `backupParticipations()` already normalises the legacy singular block to +one participation named `default`, so the five catalog apps that use the singular +form (all of them — the survey found no plural declarations in the catalog) map +cleanly without a migration. + +**`accepts: ["restore@1"]` with no block** is the meaningful third state. The +survey shows **12 of 17** acceptor apps already declare `accepts` with no backup +block — so this state exists in the catalog today and this feature gives it +meaning rather than introducing it. + +*(`restore@1` here names a participation the app declares, not a capability +contract the platform brokers. `CONTRACTS` (`contracts.ts:146-199`) gains no +entry — see R19 and FR-047.)* + +--- + +## R19 — The pre-existing dead `restore` surface is not touched + +**Decision.** Leave it entirely alone. File a follow-up issue noting the adjacency. + +**What exists.** A complete restore API stub, unrelated to this feature: + +| Thing | Where | State | +|---|---|---| +| `POST /api/backups/:id/restore` | `server.ts:1663-1668` | Parses the body, **discards it**, returns `{ jobId: crypto.randomUUID() }` — no job is ever created | +| `RestoreBackupRequest` / `RestoreBackupResponse` | `shared/src/index.ts:1720-1721` | Declared, used only by the stub | +| `JobType` includes `'restore'` | `shared/src/index.ts:587` | Never produced by any real job | +| `Backups.tsx`, `useBackupsApi.ts`, `BackupCoverage.tsx` | `packages/web` | A full UI talking to the fake endpoint | + +This is scaffolding from a pre-`backup@1` design, superseded by the hook and +snapshot machinery. Open issue **#160** ("Implement real Backups and Notifications +(currently empty stubs)") already tracks it. + +**Why not touch it.** There is no actual collision: this feature adds a different +route (candidates, keyed by app), different types (`RestoreChoice`, +`RestoreCandidate`), and **no new job type at all** — it rides the existing deploy +job. Deleting the stub would remove an operator-visible page, which is a product +decision outside this feature's scope boundary and squarely inside #160's. + +**Why record it anyway.** The vocabulary is adjacent enough that an implementer +reaching for "the restore request type" could plausibly find `RestoreBackupRequest` +and wire this feature into a dead endpoint. Naming it here is cheaper than +discovering it in review. + +--- + +## R20 — A failed restore leaves everything in place + +**Decision.** The deployment stays in `error` state and its data root is left +untouched. Nothing is deleted automatically. Such a deployment is excluded from +future candidacy (R5). + +**Rationale.** This is what a failed deploy already does — the `catch` in +`runLifecycleJob` sets `status = 'error'` and persists. Matching it means no new +failure semantics to learn. More importantly, automatic cleanup would delete the +single most useful artifact for diagnosing the failure, and deleting an operator's +data root unprompted is a destructive action the platform does not take on its own. +The existing uninstall path already removes both the data root and the sibling env +record (`removeAppData`, `:4271-4304`) when the operator decides. + +--- + +## R21 — Wizard: a first step that re-creates the draft + +**Decision.** Insert a restore step at index 0 of `steps` +(`InstallWizard.tsx:29-36`), before Configuration. Changing the choice deletes and +re-creates the draft. + +**Rationale.** Forced by R1: Configuration renders `appEnv`, and `appEnv` is +seeded by the restore choice at draft creation. A restore step *after* +Configuration would display env the choice is about to invalidate. + +Re-creating the draft is the established pattern, not a workaround — `switchChannel` +(`:507-549`) already deletes the draft and re-creates it on a channel change, +deliberately resetting consent. The same reasoning applies with more force here: +a restore changes which secrets the install will use. + +Carried values render as ordinary `appEnv` rows through the same masking the +wizard already applies to every minted secret (minting at `:449-454`; the +mask/reveal UI at `:1187-1372`). They are deliberately **not** a privileged +visual class — an operator reviewing configuration should not have to learn a +second widget to see a carried key. + +--- + +## R22 — CLI: four flags, mirroring `--grant`, defaulting to no restore + +**Decision.** `--restore-from `, `--restore-from latest`, `--no-restore`, +`--restore-list`, plus an acknowledgement flag (R16). Parsing mirrors +`parseGrants` (`install.ts:82-93`). + +**The default is the decision.** With no flag, no restore happens (FR-044). A +candidate existing is not consent to use it; silence must never overwrite an +operator's install decision with a guess. `--no-restore` exists to make the +intent explicit in a script, not because it changes the default. + +Refusal hints are built from structured `details`, never from message text — +`deploy-flow.ts:137-155` already establishes this, with the comment explaining +why: the server's message is deliberately surface-neutral and contains none of +the CLI's flags. + +--- + +## R23 — Deferred work (issues, never inline TODOs) + +Repository hard rule. Each of these becomes a filed issue during implementation: + +1. **The dead `/api/backups/:id/restore` stub and its UI** (R19) — note the + adjacency to this feature's vocabulary; points at #160. +2. **`install-markers.test.ts:704` carries a stale line-number comment** — it + cites `~:3410`/`~:3414` for the restore-before-materialize ordering; the real + lines are `:3793`/`:3797`. The ordering claim is still true. A trivial fix, + but it is evidence that line-number comments rot, so the issue should propose + the comment cite symbols instead. +3. **Sequence 6 inherits the subtree trap** (R9) — record that a provider archive + reproduces absolute paths and that FR-016's post-condition assertion is what + generalises, so Sequence 6's author does not have to re-derive it. +4. **`composeUp`'s 5-minute default timeout** (R12) — now that one caller + parameterises it, the other call sites' inherited default deserves a + deliberate second look rather than remaining an accident. diff --git a/specs/007-restore-on-install/spec.md b/specs/007-restore-on-install/spec.md new file mode 100644 index 00000000..8c157130 --- /dev/null +++ b/specs/007-restore-on-install/spec.md @@ -0,0 +1,732 @@ +# Feature Specification: Restore-on-Install from a Live Deployment + +**Feature Branch**: `007-restore-on-install` + +**Created**: 2026-09-20 + +**Status**: Draft + +**Input**: User description: "Restore-on-install from a live deployment — the whole install-side machinery, with no backup provider involved. During an app install, detect existing captures of that app and offer to restore one immediately. The restore source in this slice is another LIVE deployment on the same host, captured with the backup broker's pre-hooks run first. At install time the data root is empty and no container is running, so a restore never overwrites anything, never races a live process, and lands inside a job that already owns the ordering. Zero provider work, zero new grants, zero new trust boundary. Closes #429." + +**Source issues**: closes [`try-hola/hola#429`](https://github.com/try-hola/hola/issues/429) (Clone-with-data rehearsal: `hola install --channel --from `). **Prompt of record**: Notion Spec Prompts row "Restore-on-install from a live deployment — the whole install-side machinery, with no backup provider involved" (Sequence 5, Status Ready, fetched 2026-09-20) — [page](https://app.notion.com/p/3e1acfdc54e8817fb3a4ef062b9b7576). + +**Depends on**: Sequence 4 / spec [006-install-identity](../006-install-identity/spec.md), shipped on `main` as `921c790`. Every app data root now carries an install identity record, and every install carries an environment record. This feature is that feature's first reader. + +**Blocks / excluded**: Sequence 6 (`restore@1` — the provider half: a staging grant, a snapshot index, a provider-polled restore queue). Restoring an app that **no longer exists on this host** is Sequence 6's job and is explicitly out of scope here. + +## Executive Summary + +Hola can take a backup. It cannot put one back. + +`backup@1` quiesces every accepting app and lets a provider copy its data root; +spec 006 made each of those copies self-describing. What no operator can do today +is the other direction — take a copy of an app's data and end up with a working +app running on it. The only restore path that exists is an internal one: +`rollback` can put back a snapshot **the platform itself took**, of **the same +deployment**, taken **minutes earlier**. That is a safety net for an upgrade, not +a restore route. + +This feature builds the restore route, and it builds it at **install time**. + +That timing is not a UX convenience, it is the entire reason the design is small. +Restoring into a *running* app is a hard problem: the data root is full, a +database process holds files open, and a half-applied restore leaves an app in a +state no one can reason about. At install time every one of those problems is +absent by construction — `createFromDraft` mints a brand-new deployment id, the +data root does not exist until the deploy job creates it, and no container has +ever started. A restore at install time therefore **cannot overwrite anything, +cannot race a live process, and needs no stop/start choreography**. It slots into +a lifecycle job that already owns the ordering. + +**This slice deliberately contains no backup provider.** The restore source is +another *live deployment on the same host*: the server quiesces it with the +backup broker's existing pre-hooks, tars its data root, and lays that down into +the new install. Every one of those parts is already in production — the +pre-upgrade snapshot capture, the fail-closed pre-hook runner, and a restore +helper that already accepts an arbitrary target deployment id. So this feature +adds **zero provider work, zero new grants, and zero new trust boundary**: the +server is both the reader and the writer, exactly as it is for a rollback. + +The payoff is sequencing. Every genuinely subtle part of restore lives on the +install side — where in the job the files land, what must be rewritten +afterwards, what refuses and when, how carried configuration reaches the wizard. +Shipping those against a source that needs no contract negotiation means Sequence +6 arrives with only one new job: *put the files in a staging directory*. It faces +machinery that has been in production for a release. + +And it closes #429 outright. "Install a copy of this app with its data, so I can +rehearse an upgrade" is the same mechanism pointed at a different intent. + +### The failure this exists to prevent + +An operator who restores data and gets back a running-but-**empty** app is the +worst outcome in this feature, worse than a loud failure, because it is silent. +They see a healthy container at the right address, assume the restore worked, and +discover weeks later that it did not. Two requirements follow from that and +govern the whole design: + +1. **A restore that cannot succeed refuses before any container starts.** There + is no partial-success state worth starting an app in. +2. **A restore that carries data must carry the configuration that data was + written under**, or say loudly which values it could not carry. Data encrypted + with a key the platform generated is unreadable beside a freshly generated + one, and nothing in the running app reports that as an error. + +## Scope + +### In scope + +- Discovering restore **candidates** for an app being installed: other + deployments of the same app that exist on this host and carry a data root. +- A **restore choice on the draft**, seeding the new install's app environment + from the candidate's recorded environment. +- Executing the restore **inside the deploy job**, between image pull and + container start, in a fixed order. +- An app-declared **restore participation** in the catalog: per-participation + restore hooks, paths to discard before start, and whether carried configuration + is mandatory. +- **Refusals and warnings**: version skew, missing carried configuration, address + divergence, ambiguous candidates. +- **Install wizard** step and **CLI** flags to make the choice. +- Closing #429 (clone-with-data) as a consequence of the above. + +### Out of scope (deliberately) + +- Any **backup provider** involvement: no new contract, no new grant, no new + contract endpoint, no staging grant, no snapshot index, no provider-polled + restore queue. All of that is Sequence 6. +- Restoring an app that **no longer exists** on this host. The candidate in this + slice is a live deployment; there is no archive to read from. +- **Scheduled or repeated** restores, restore of a *running* app, and any restore + path outside install. +- Cross-host restore, or restoring an app onto a **different app**. +- A manifest field describing whether an app makes outbound calls. No existing + declaration captures it and #429 reached the same conclusion; the acknowledgement + in this feature is unconditional instead. + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - An operator installs an app and gets their data back (Priority: P1) + +An operator is reinstalling an app — they are moving it, they broke it, or they +are rebuilding the host. They start an ordinary install. Before they configure +anything, the installer tells them it found an existing copy of this app on the +host and offers to restore from it. They accept, finish the install, and the app +comes up already holding its data. + +**Why this priority**: This is the feature. Everything else in the spec either +makes this safe or makes it reachable from another surface. Without it there is +no restore route in the product at all. + +**Independent Test**: Install an app, put recognisable data in it, install a +second copy choosing the first as the restore source, and confirm the second copy +serves that data on first boot. Fully testable with two installs of one app and +no backup provider anywhere. + +**Acceptance Scenarios**: + +1. **Given** one installed, running copy of an app that holds data, **When** the + operator begins installing that same app, **Then** the installer presents that + copy as a restore candidate, identified by the name the operator gave it, its + address, its app version, and when it was captured. +2. **Given** the operator selects that candidate, **When** the install completes, + **Then** the new install's data root holds the candidate's data and the app + serves it without any further operator action. +3. **Given** the operator selects no candidate, **When** the install completes, + **Then** the install behaves exactly as an install does today — an empty data + root, first-run state, no restore step in the record. +4. **Given** an app being installed for which no other copy exists on the host, + **When** the operator begins the install, **Then** the installer states plainly + that there is nothing to restore from and does not block progress. +5. **Given** a candidate is selected, **When** the restore runs, **Then** the + source deployment is quiesced first and is left running and unchanged + afterwards — a restore reads the source, it never alters it. + +--- + +### User Story 2 - The restored app can read the data it was given (Priority: P1) + +The operator restores an app whose stored data is encrypted or signed with a +value the platform generated at its original install — an encryption key, a +secret key, stored third-party tokens. The restored install comes up using +**those** values, not freshly minted ones, so the data is actually readable. + +**Why this priority**: Equal-first with US1 because US1 without it is the silent +failure this feature exists to prevent. An app that starts cleanly, shows its +records and cannot decrypt a single stored credential is indistinguishable from +success until the operator needs one. + +**Independent Test**: Restore an app that stores something encrypted under a +generated key, then read that stored item back through the app. Confirm the +install's configuration values match the source's, and that the wizard showed +them as ordinary configuration the operator could review. + +**Acceptance Scenarios**: + +1. **Given** a candidate whose environment record exists, **When** the operator + chooses to carry configuration forward, **Then** the new install's + configuration values are the candidate's values, and they are visible in the + install's configuration step as ordinary (masked) fields. +2. **Given** a carried configuration set, **When** the app's current release + declares a configuration key the candidate never had, **Then** that key is + freshly generated if the app knows how to generate it, and otherwise surfaced + to the operator by name rather than silently defaulted. +3. **Given** a candidate whose environment record cannot be found, **When** the + operator selects it, **Then** the installer states that configuration cannot be + carried, names every generated secret the app will therefore mint fresh, and + requires the operator to acknowledge that before proceeding. +4. **Given** the operator chooses **not** to carry configuration from a candidate + that has it, **When** the install proceeds, **Then** fresh values are minted and + the same named warning is shown — declining is allowed, doing it unaware is not. +5. **Given** a restore has been chosen, **When** the operator reaches the final + confirmation, **Then** they are told in plain words that the app will resume + with the data **and credentials** it had when the source was captured, and that + scheduled jobs, webhooks and integrations may fire against real systems as soon + as it starts. + +--- + +### User Story 3 - A restore that cannot be right refuses before the app starts (Priority: P1) + +The operator picks a candidate that cannot be safely restored into this install — +it was captured on a newer version of the app, it needs a version hop the app's +own upgrade rules forbid, its configuration is missing and the app declares it +mandatory, or a restore hook fails partway. In every case the install stops with +a specific reason and the app never starts. + +**Why this priority**: Also P1. The absence of this is the failure mode that makes +US1 dangerous rather than useful. A wrong restore that proceeds produces an app +that looks fine and is not. + +**Independent Test**: Drive each refusal condition against a real install and +confirm (a) the install fails, (b) the failure names the specific reason, and (c) +no app container ever started. + +**Acceptance Scenarios**: + +1. **Given** a candidate captured at a version **newer** than the version being + installed, **When** the operator selects it, **Then** the restore is refused + with a reason naming both versions — data written by a newer release is not + something an older release can be trusted to read. +2. **Given** a candidate captured at an older version whose path to the target + version is guarded by the app's own upgrade rules, **When** the operator selects + it, **Then** the restore is refused and the operator is told which version to + install and restore into first, before promoting to the one they wanted. +3. **Given** a candidate whose recorded version is unknown, **When** the operator + selects it, **Then** the restore proceeds only behind an explicit acknowledgement + that the version relationship could not be checked. +4. **Given** an app that declares carried configuration mandatory, **When** the + selected candidate has no environment record, **Then** the restore is refused + outright rather than warned about. +5. **Given** a selected candidate and an app that declares restore hooks, **When** + any restore hook fails, **Then** the whole install fails, the failure names the + hook, and the app's own services are never started. +6. **Given** any restore failure at all, **When** the operator looks at the result, + **Then** they see a failed install — never a successful install holding an + empty or half-restored data root. +7. **Given** a restore was requested, **When** the deploy job begins the restore, + **Then** it first verifies the target data root is empty of app data and aborts + if it is not — a restore never writes over existing data, in any circumstance. + +--- + +### User Story 4 - An app says how it wants to be restored (Priority: P2) + +A catalog app declares what a restore into it means: which captured paths are +worthless or actively harmful and must be discarded before anything starts, which +command reloads its real payload, and whether it can be restored at all without +its original configuration. Apps that need none of this say so explicitly. + +**Why this priority**: P2 because the platform half can ship and be exercised with +plain file-copy apps first. But it is what makes restore *correct* for the +database-backed apps that most need it, so it is not optional for long. + +**Independent Test**: Restore a database-backed app whose declaration discards the +captured database directory and loads a dump, and confirm the resulting app holds +the dumped data rather than the smeared directory. Separately confirm an app +declaring participation with no detail restores by plain file copy. + +**Acceptance Scenarios**: + +1. **Given** an app whose declaration lists paths to discard, **When** a restore + lands its files, **Then** those paths are removed before any of the app's + containers start, and the app starts against the remaining files. +2. **Given** an app whose declaration names a restore hook, **When** the files are + in place and the discards applied, **Then** the hook's service is started on its + own and the hook is run against it before any other service starts. +3. **Given** a declared restore hook whose service does not become healthy, **When** + the restore runs, **Then** the install fails rather than running the hook against + a service that is not ready. +4. **Given** a discard path that points outside the app's own data root, **When** + the restore runs, **Then** it is refused — a declaration can never reach outside + the data root it is restoring into. +5. **Given** an app that declares it participates in restore but supplies no + detail, **When** it is restored, **Then** its files are copied back as-is and no + hook runs — a meaningfully different state from an app that declares nothing. + +--- + +### User Story 5 - Clone an app with its data, from the command line (Priority: P2) + +An operator wants a second copy of a running app holding the same data — to +rehearse an upgrade against real data, to test a configuration change, or to keep +a spare. They run one install command naming the existing deployment as the +source and get a second, independent copy. + +**Why this priority**: P2 because it is the same machinery as US1 reached from a +different surface. It is listed separately because it is the outcome #429 asks +for, and because a non-interactive surface has its own hard rule about defaults. + +**Independent Test**: From the CLI, install a second copy of a running app naming +the first as the restore source, and confirm the second is independently +addressable and holds the first's data while the first keeps running untouched. + +**Acceptance Scenarios**: + +1. **Given** a running deployment of an app, **When** the operator installs that + app naming that deployment as the restore source, **Then** a second independent + deployment is created holding the source's data, and the source is unchanged + and still running. +2. **Given** several candidates for an app, **When** the operator asks to list + them, **Then** each is listed with enough detail to choose — its identifier, + name, address, version, whether it carries configuration, and any warning that + would apply. +3. **Given** several candidates, **When** the operator asks for the most recent + one rather than naming an identifier, **Then** the most recently captured + candidate is used. +4. **Given** an install run with no restore flag at all, **When** it completes, + **Then** **no restore happened** — silence never means "guess". A candidate + existing is not consent to use it. +5. **Given** any restore refusal, **When** it reaches the command line, **Then** + the operator is shown a specific, actionable hint derived from structured + failure detail, not a re-printed server message. + +--- + +### Edge Cases + +- **Several copies of the same app, from different histories.** Candidates that + share a lineage are one story; candidates from unrelated installs are not. The + installer groups by lineage, offers the most recent within a lineage as the + default, and requires an explicit pick when two unrelated lineages both match. +- **A candidate whose data root is empty.** An app installed and never used has + nothing to restore. It is not offered as a candidate. +- **A candidate that is the app being installed's own future self.** A restore + candidate is always another, existing deployment; an install can never name + itself. +- **A candidate that is stopped rather than running.** Still a valid source: its + files are at rest, which is the easiest possible capture. Its quiescing hooks + simply have nothing to run against, which must not be an error. +- **A candidate that is deleted between choosing it and the deploy job running.** + The job re-resolves the candidate and fails the install with a clear reason + rather than restoring a stale or partial copy. +- **A candidate that starts its own lifecycle action after being chosen.** The same + re-resolution applies: the job re-checks that the source is settled and fails the + install rather than capturing a data root the platform is mid-way through + rewriting. +- **A previous restore of this app that failed.** Its data root holds a partial + tree, so it is never offered as a source, even though the deployment and its + files are deliberately left in place for the operator to inspect. +- **The app's address changes.** Absolute URLs baked into restored data point at + the old address. The installer defaults the new install's name and address from + the candidate's and warns when the operator changes them anyway. +- **A restore into an app with SSO provisioned.** The provisioned credentials file + the platform writes into the data root must survive the restore. A restore that + replaces the whole data root after that file is written silently destroys it, + and the app boots without SSO. +- **A restored tree carrying the source install's identity record.** Left in + place, the new install claims to be the old one, and every later capture + compounds the error. +- **An app whose restore hook service has no health signal.** There is no way to + know when it is ready. The app must declare one; the restore refuses rather than + guessing with a timer. +- **A capture taken before install identity shipped.** It has no environment + record. This is the ordinary "cannot carry configuration" path, not an error. +- **A restore requested on an action that is not a first install** (a restart, a + promote, a rollback). The restore runs once, at the install that requested it, + and never again on any later action for that deployment. + +## Clarifications + +### Session 2026-09-20 + +- **Q: Where is the capture staged during a restore, and when is it cleaned up? + An app data root can be tens of gigabytes.** → **A:** Under the **target** + deployment's own directory, in a dedicated restore-staging area separate from + the pre-upgrade snapshot store, and deleted in a `finally` whether the restore + succeeds or fails. It is staged under the target rather than the source so it + disappears with the install that requested it and never enters the source's + snapshot retention. **There is no separate extraction step**: the platform's own + capture helper archives the data root's *contents* relative to the root, and its + restore helper extracts straight into the destination, so the peak cost is one + compressed archive plus the extracted data root — not the three simultaneous + copies a stage-then-move design would need. + +- **Q: How does the candidate list reach the install wizard and the CLI — + a new read route, or a field on an existing response?** → **A:** A **new + ordinary platform API read route**, keyed by app. The scope ban in FR-047 is on + *capability-contract* endpoints, not on platform API routes. A field on the + draft-creation response is circular: the restore choice is made **before** the + draft exists and determines how the draft is created, so the client would need a + draft in order to learn what to put in the draft request. The CLI's + list-candidates flag likewise must work without creating a draft at all. The + route is therefore forced, not chosen. + +- **Q: After a restore fails mid-way, what happens to the target deployment + record and its partially written data root?** → **A:** Both are **left in + place**, the deployment in its error state, exactly as a failed deploy behaves + today. The platform does not delete an operator's data root on its own; the + evidence of the failure is the most useful thing present, and the existing + uninstall path already removes it when the operator decides to. The safety rule + this forces is stated as a requirement: such a deployment MUST NOT itself be + offered as a restore candidate, because its data root holds a partial tree. + +- **Q: May a deployment with a lifecycle job in flight be used as a restore + source?** → **A:** **No — refuse, fail-closed.** A source that is mid-deploy, + mid-promote or mid-rollback is being written by the platform itself and its data + root may be mid-replacement. Only a deployment in a settled state (running or + stopped) may be a source; anything in flight, or in an error state, is excluded + from candidacy rather than captured optimistically. + +- **Q: How are the "unknown version" and "configuration could not be carried" + acknowledgements represented, so the non-interactive equivalent FR-046 demands + is actually buildable?** → **A:** As an **explicit list of acknowledgement + codes on the restore choice**, mirroring the existing consent array the platform + already uses for privileged capability grants at install. The server computes + which acknowledgements the chosen candidate requires and refuses the create when + a required code is absent — the same shape, and the same failure, as a missing + grant consent. The wizard's checkboxes and the CLI's flags both supply codes, so + a scripted install acknowledges deliberately or fails closed, and neither surface + needs to parse prose. + +## Requirements *(mandatory)* + +### Functional Requirements + +#### Candidate discovery + +- **FR-001**: The system MUST be able to list, for a given app, the restore + candidates available on this host, through a dedicated read route that does + **not** require a draft to exist. A candidate is an existing deployment of the + same app, other than the one being created, whose data root exists and holds app + data. +- **FR-002**: Each candidate MUST be described with: its deployment identifier, + its lineage identifier, the operator-given name, the address it was served at, + the app version its data was written under, whether its configuration can be + carried, and the time the description was last written. +- **FR-003**: The system MUST source a candidate's description from the install + identity record in its data root, falling back to the live deployment record for + any field the record does not carry. A candidate whose identity record is absent + or unreadable MUST still be offered, described from the deployment record alone, + and marked as carrying no lineage. +- **FR-004**: A deployment whose data root is empty of app data MUST NOT be offered + as a candidate. +- **FR-004a**: Only a deployment in a **settled** state — running or stopped — MUST + be offered as a candidate. A deployment with a lifecycle action in flight, or in + an error state, MUST be excluded: the platform itself may be mid-write to its + data root, and an install whose own restore failed holds a partial tree. +- **FR-005**: Candidates MUST be grouped by lineage and ordered most-recent-first + within a lineage. +- **FR-006**: Candidate discovery MUST NOT require any backup provider, contract + grant, or contract endpoint. + +#### Entering the restore choice + +- **FR-007**: The restore choice MUST enter the install at **draft creation**, + carrying the chosen candidate's identifier and whether to carry its + configuration forward. It MUST NOT be settable by patching an existing draft, and + it MUST NOT be supplied at finalize. +- **FR-008**: When a restore choice names a candidate and configuration is to be + carried, the draft's app environment MUST be seeded from the candidate's recorded + environment at draft creation, using the platform's existing three-case rule: a + carried value wins; a key the app newly declares with a generation recipe is + freshly minted; any other new key rides through to be surfaced to the operator + by name. +- **FR-009**: The restore choice MUST ride onto the finalized release description + **outside** the checksummed canonical specification, and MUST NOT alter that + checksum's meaning. +- **FR-010**: The restore choice MUST be persisted on the deployment record at + creation, because the deploy job's payload carries only the release, the action + and the deployment identifier, and reads everything else from the record. +- **FR-011**: A deployment record MUST carry a lineage identifier. For an install + with no restore it is the deployment's own identifier; for a restore it is the + candidate's lineage identifier, carried forward. The install identity record MUST + read the persisted lineage in preference to deriving it. +- **FR-012**: The restore choice MUST apply to the **first** deploy of that + deployment only. Any later action on the same deployment (restart, promote, + rollback) MUST NOT re-run the restore. + +#### Executing the restore + +- **FR-013**: The restore MUST execute inside the deploy job, **after** images are + pulled and **before** any of the app's containers are started. +- **FR-013a**: Before capturing anything, the restore MUST **re-resolve the chosen + candidate** and re-check that it still exists and is still in a settled state. + A candidate deleted, or one that started a lifecycle action, between the draft + being created and the deploy job running MUST fail the install with a specific + reason rather than being captured optimistically. +- **FR-014**: Before writing anything, the restore MUST verify the target data root + contains no app data, ignoring the platform's own reserved marker directory, and + MUST abort the install if it does. +- **FR-015**: The restore MUST quiesce the source deployment using the same + fail-closed pre-hook policy the platform already uses before a pre-upgrade + capture, capture its data root, and leave the source running and unmodified. +- **FR-016**: The restore MUST verify the payload it landed is the app's data-root + contents, and MUST fail rather than proceed if the target root is empty or holds + only the platform's own marker directory after the restore. It MUST NOT infer + success from the capture or extraction step reporting no error. + + > *Correction to the prompt of record, made during clarification.* The prompt + > required the restore to "locate the subtree, because a restore recreates the + > absolute path structure under the target, so the payload is at + > ``". That is true of a provider's archive tool, which + > stores absolute paths — and therefore a real trap for **Sequence 6**. It is not + > true of the platform's own capture helper, which archives a directory's + > *contents* relative to that directory and extracts them straight into the + > destination. Writing a subtree search into this slice would be code guarding + > against a layout this slice cannot produce. The requirement is therefore stated + > as the outcome — *the payload must actually be there* — which holds under either + > archive shape and survives Sequence 6 unchanged. + +- **FR-016a**: The capture MUST be staged under the **target** deployment's own + directory, in a staging area distinct from the pre-upgrade snapshot store, and + MUST be deleted whether the restore succeeds or fails. It MUST NOT be written + under the source deployment, and MUST NOT participate in snapshot retention. +- **FR-017**: After the files land, the restore MUST remove every path the app's + restore declaration marks for discard, before any container starts. Each discard + path MUST be resolved and confined to the target data root; a path that resolves + outside it MUST be refused. +- **FR-018**: After the files land and discards are applied, the restore MUST + rewrite the install identity record so it describes the **new** install. The + source install's record MUST NOT survive into the restored tree. +- **FR-019**: The provisioned SSO credentials file MUST be written **after** the + restore has laid down its files, not before. Writing it before a whole-data-root + restore destroys it, and the app boots without SSO with no error reported. +- **FR-020**: When the app declares restore hooks, the system MUST be able to start + a **named subset** of the app's services and wait for them to become healthy, + before starting the rest. It MUST rely on the service's own declared health + signal rather than a fixed wait. +- **FR-021**: Restore hooks MUST run fail-closed: a hook that fails, or a hook + service that never becomes healthy, MUST fail the install. A partially loaded + data set MUST never be handed to an app that will then start against it. +- **FR-022**: A failed restore MUST fail the install. The system MUST NOT fall back + to starting the app with an empty or partially restored data root. +- **FR-022a**: A failed restore MUST leave its deployment record and data root in + place, in the error state, exactly as a failed deploy does today. The system MUST + NOT delete either automatically; removing them stays the operator's decision, + through the existing uninstall path. +- **FR-023**: When no restore was requested, the deploy job's behaviour MUST be + byte-for-byte what it is today. + +#### The app's restore declaration + +- **FR-024**: An app MUST be able to declare, per backup participation, how it is + restored: paths to discard, a restore hook, and whether carried configuration is + mandatory. +- **FR-025**: An app declaring that it accepts restore **without** supplying any + detail MUST be treated as "a plain file copy back is sufficient" — a state + meaningfully distinct from an app that declares nothing at all. +- **FR-026**: The restore declaration MUST reuse the existing hook shape the backup + declaration already uses; it MUST NOT introduce a second, parallel hook format. +- **FR-027**: A restore hook MUST be keyed to a backup participation, so an app + with two stateful services restores each correctly. +- **FR-028**: The catalog schema MUST accept the restore declaration, and existing + app declarations MUST remain valid unchanged. + +#### Refusals and warnings + +- **FR-029**: A candidate captured at a version **newer** than the version being + installed MUST be refused. +- **FR-030**: A candidate captured at an older version whose upgrade path to the + target is guarded by the app's own upgrade rules MUST be refused, and the refusal + MUST name the version the operator should install and restore into first. +- **FR-031**: A candidate captured at the same version, or at an older version with + a clear upgrade path, MUST be allowed. +- **FR-032**: A candidate whose version is unknown MUST be allowed only behind an + explicit operator acknowledgement, because an unknown version cannot be checked + against the app's upgrade rules. +- **FR-033**: When configuration cannot be carried, the system MUST derive the + severity itself rather than asking the app: every configuration key the app + declares that is both a secret **and** has a generation recipe is a + platform-invented value the restored data may depend on. The system MUST warn, + naming those keys. +- **FR-034**: When configuration cannot be carried and the app's restore + declaration marks carried configuration mandatory, the system MUST refuse rather + than warn. +- **FR-035**: The new install's name and address MUST default from the candidate's + recorded address. When the operator sets a different address, the system MUST warn + that absolute addresses stored inside the restored data will not be rewritten. +- **FR-036**: When candidates from two or more distinct lineages match, the system + MUST require an explicit choice rather than defaulting. +- **FR-037**: Every restore-side refusal MUST carry a structured, machine-readable + reason code alongside its human message, so non-interactive surfaces can build + their own guidance without parsing prose. +- **FR-037a**: Every acknowledgement this feature requires MUST be represented as a + **code supplied on the restore choice**, in the same shape the platform already + uses for privileged grant consent at install. The system MUST compute which + acknowledgement codes the chosen candidate requires and MUST refuse the create + when a required code is absent — the same failure as a missing grant consent. + +#### Install wizard + +- **FR-038**: The install wizard MUST present the restore choice as its **first** + step, before configuration — because configuration renders the app environment + and the app environment is seeded by the restore choice. +- **FR-039**: Changing the restore choice MUST re-create the draft, following the + existing pattern used when the operator changes release channel mid-install. +- **FR-040**: Carried configuration values MUST reach the wizard as ordinary + configuration fields, masked by the same component that masks every other + generated secret. They MUST NOT be presented as a separate or privileged class. +- **FR-041**: When a restore is chosen, the final confirmation step MUST show an + unconditional acknowledgement stating that the app will resume with the data and + credentials it had when the source was captured, and that scheduled jobs, + webhooks and integrations may fire against real systems as soon as it starts. +- **FR-042**: When no candidate exists for the app, the wizard MUST say so and MUST + NOT obstruct the install. + +#### Command line + +- **FR-043**: The CLI MUST let an operator name a restore source by identifier, + ask for the most recent candidate, explicitly decline a restore, and list the + candidates for an app. +- **FR-044**: With no restore flag given, the CLI MUST perform **no restore**. + Silence MUST NOT be interpreted as consent to restore. +- **FR-045**: The CLI MUST build its guidance for a restore refusal from the + structured reason code and detail, never from the server's message text. +- **FR-046**: Every acknowledgement the wizard requires (unknown version, uncarried + configuration) MUST have an explicit non-interactive equivalent that supplies the + same acknowledgement code, so a scripted install either acknowledges deliberately + or fails closed. Neither surface may infer an acknowledgement from silence. + +#### Scope boundaries + +- **FR-047**: This feature MUST NOT define a new capability contract, a new grant, + or a new contract endpoint, and MUST NOT require any backup provider to be + installed. +- **FR-048**: This feature MUST NOT restore from anything other than a live + deployment present on this host. + +### Key Entities + +- **Restore candidate**: An existing deployment of the same app on this host that + holds data and can serve as a restore source. Described by deployment identifier, + lineage identifier, operator-given name, address, app version, whether + configuration can be carried, and when its description was written. Derived — it + is not stored anywhere; it is computed from deployments and their identity + records on demand. +- **Restore choice**: The operator's decision, made once at draft creation: which + candidate, whether to carry its configuration, and which acknowledgement codes + they are supplying. Rides the draft through finalize, lands on the deployment + record, and is consumed exactly once by the first deploy. +- **Acknowledgement code**: A named risk the operator must accept for a specific + restore to be allowed — an unverifiable version relationship, or configuration + that cannot be carried. Computed by the system from the candidate, supplied by + the operator, and refused when required and absent. Modelled on the existing + privileged-grant consent, not invented anew. +- **Lineage identifier**: The identity of an *app instance* across reinstalls, + distinct from a deployment identifier which identifies one install. A fresh + install is its own lineage; a restored install inherits the source's. Written + only by the platform; never operator-visible as a concept to set. +- **Restore declaration**: What an app says about being restored into — discard + paths, a restore hook per backup participation, and whether carried configuration + is mandatory. Lives in the app's manifest alongside the backup declaration. +- **Restore refusal**: A structured reason a restore cannot proceed, carrying a + machine-readable code, a human message, and where applicable the version the + operator should use instead. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: An operator can restore an app's data during its install with **no + backup provider installed on the host** and **no new grant consented**. +- **SC-002**: A restored app serves the source's data on first boot, with zero + manual steps between the install completing and the data being visible. +- **SC-003**: 100% of restores that carry configuration produce an install whose + configuration values match the source's — so data encrypted under a + platform-generated value remains readable. +- **SC-004**: 100% of restore failures result in a **failed install**. No restore + failure produces a running app with an empty or partially restored data root. +- **SC-005**: Every refusal names its specific cause, and every version-skew + refusal that has an actionable next step names that version. +- **SC-006**: A restore never modifies the source deployment: after a restore, the + source's data, status and address are unchanged and it is still running if it was + running before. +- **SC-007**: A restored install's identity record describes the **new** install, + and its lineage identifier equals the source's — verifiable by reading the + record after the install. +- **SC-008**: A restored install with SSO provisioned boots with its SSO + configuration intact. +- **SC-009**: An install run with no restore flag and no restore choice produces a + deployment indistinguishable from one produced before this feature shipped. +- **SC-010**: An app with two stateful services can restore both correctly from one + capture. +- **SC-011**: Issue #429 is closed: an operator can create a second copy of a + running app holding its data in a single command. +- **SC-012**: A scripted, non-interactive install can never perform a restore it + was not explicitly told to perform, and can never satisfy an acknowledgement it + did not explicitly supply. +- **SC-013**: A restore's peak additional disk cost is one compressed archive of + the source's data root, released whether the restore succeeds or fails — no + restore leaves staged bytes behind. + +## Assumptions + +- **Restore source is a live deployment.** The candidate is another deployment on + the same host, discoverable through the platform's own records. No archive + format, catalogue of snapshots, or external storage is read. +- **The capture mechanism is the one that already exists.** Quiescing the source + and capturing its data root reuses the platform's pre-upgrade capture path and + its fail-closed pre-hook policy verbatim; no second capture mechanism is built. +- **Install identity has shipped.** Spec 006 is on `main`, so every app data root + carries an identity record, and every install has an environment record. Copies + installed before it will gain both on their next deploy, so candidates predating + it are handled as the "no carried configuration" path rather than as an error. +- **The environment record lives beside the data root, not inside it.** Spec 006's + final placement puts it under a reserved sibling directory at the apps root + (`///`), deliberately outside the mount an + app's own containers receive. For *this* feature that is convenient rather than + limiting: the source is on the same host, so the server reads the record + directly. It does mean a captured data root alone does not carry configuration — + a fact Sequence 6 must confront, and this spec records it rather than assuming + the prompt's original placement. +- **Lineage becomes a persisted field.** Spec 006 derives lineage as "the + deployment's own id" and states in the shipped code that this feature is what + forces it onto the deployment record. This spec does that. +- **The restore hook contract is the backup hook contract.** The shape an app uses + to declare a quiescing hook is reused verbatim for a restore hook; no second hook + format is introduced. +- **Health signals come from the app.** Waiting for a restore hook's service uses + the service's own declared health check. An app whose hook service declares none + is a declaration bug, surfaced as a refusal rather than worked around with a + timer. +- **Address rewriting is not attempted.** Absolute addresses stored inside restored + data are warned about, never rewritten — rewriting them would require per-app + knowledge of where they are stored, which violates the platform's generic-primitive + rule. +- **Two live copies of one app are permitted.** Installing a second copy of an app + already relies on the existing same-app install rules; this feature does not + change them, and a clone-with-data install obeys whatever those rules already say. +- **No new operator concept is introduced for lineage.** Operators pick a candidate + by its name and address. Lineage is how the system groups and orders candidates, + not a thing the operator names or manages. + +## Dependencies + +- **Spec 006 (install identity)** — shipped on `main` (`921c790`). Supplies the + identity record this feature reads, the environment record it carries forward, + and the lineage field it promotes to the deployment record. +- **The backup broker's pre-hook machinery** — in production since the `backup@1` + work. Reused unchanged to quiesce the source. +- **The pre-upgrade capture and restore helpers** — in production since the + data-aware rollback work. The restore helper already accepts an arbitrary target + deployment, which is what makes a cross-deployment restore possible without new + file-handling code. +- **The app upgrade-path rules** — reused to judge version skew per candidate. + Note that these rules deliberately pass through on a downgrade, so "the candidate + is newer than the target" is a rule **this** feature must state on its own; it is + not something the existing check reports. +- **A sibling change in the catalog repository (`try-hola/apps`)** — the restore + declaration on acceptor apps and the schema that validates it. The platform half + is useful without it (plain file-copy restores work), so the two can land + independently, but the database-backed apps need both. diff --git a/specs/007-restore-on-install/tasks.md b/specs/007-restore-on-install/tasks.md new file mode 100644 index 00000000..3e4a6421 --- /dev/null +++ b/specs/007-restore-on-install/tasks.md @@ -0,0 +1,319 @@ +--- + +description: "Task list for spec 007 — Restore-on-Install from a Live Deployment" +--- + +# Tasks: Restore-on-Install from a Live Deployment + +**Input**: Design documents from `/specs/007-restore-on-install/` + +**Prerequisites**: [plan.md](./plan.md), [spec.md](./spec.md), [research.md](./research.md), [data-model.md](./data-model.md), [contracts/](./contracts/), [quickstart.md](./quickstart.md) + +**Tests**: Included. The spec's FRs demand test-backed verification and +`quickstart.md` supplies 57 numbered scenarios; test task descriptions cite those +numbers so a reviewer can check coverage mechanically. + +**Closes**: `try-hola/hola#429` + +## Format: `[ID] [P?] [Story] Description` + +- **[P]**: Can run in parallel (different files, no dependency on an incomplete task) +- **[Story]**: `[US1]`–`[US5]`, matching spec.md's user stories +- Every task names an exact file path + +## Line numbers + +**Use `research.md` and the verified anchors, never the original prompt's line +numbers** — they predate spec 006's merge (`921c790`) and several are wrong in +shape, not just position. The key ones: + +| Thing | Line | +|---|---| +| `runLifecycleJob` deploy/start/rollback branch | `deployment.ts:3780-3821` | +| `writeOidcCredentialsFile` call (to be moved) | `deployment.ts:3800` | +| `composePull` / `isCancelled` / `composeUp` | `:3807` / `:3813` / `:3815` | +| deployment record literal, `channel,` | `:1008-1062`, `:1028` | +| `writeInstanceMarkers`, `lineageId` expression | `:3322-3402`, `:3374` | +| `capturePreUpgradeSnapshot` (Real) | `:2492-2578` | +| `runPreHooksFailClosed` / `runPostHooks` | `:2684-2701` / `:2709-2724` | +| draft appEnv seed: catalog / install-by-ref | `draft.ts:473` / `:606` | +| `canonicalSpec` / checksum / outside-spec comment | `draft.ts:886-906` / `:908` / `:932-943` | +| `composeUp` iface / Real / Mock | `docker.ts:67` / `:231-269` / `:722-725` | + +--- + +## Phase 1: Setup + +**Purpose**: There is no project initialization to do. One confirmation task only. + +- [X] T001 Confirm the working branch is `007-restore-on-install` off `main` (`ca3d3f4` or later) and the tree is clean, via `git rev-parse --abbrev-ref HEAD && git status --porcelain` + +--- + +## Phase 2: Foundational (BLOCKING — no user story can start until this is done) + +**Purpose**: The shared spine every story rides on. + +> **Read this before starting Phase 3.** US1 (restore the data) and US2 (carry the +> configuration) share **one** draft-seed call site and **one** restore sequence. +> They are separate user stories because they fail separately and deliver value +> separately — but they are **not** separate code paths. Phase 2 therefore carries +> the shared spine (types, the widened `composeUp`, the candidate module skeleton, +> `lineageId` on the record), and each story phase adds only its own behaviour and +> its own tests on top. An implementer who builds a second restore path for US2 +> has misread this; there is exactly one. + +- [X] T002 [P] Add `RestoreChoice`, `RestoreCandidate`, `RestoreSkewVerdict`, `RestoreRefusalCode`, `RestoreWarning` and `AppRestoreDeclaration` to `packages/shared/src/index.ts` per data-model.md §1–§5, §7 +- [X] T003 [P] Add `restoreFrom?: RestoreChoice` to `CreateDraftRequest` (`packages/shared/src/index.ts:1299-1316`) and to `Draft` (`:1220-1297`); leave `PatchDraftRequest` (`:1339`) closed to its existing four fields +- [X] T004 [P] Add `restoreFrom?: RestoreChoice` to `FinalizedManifest` in `packages/server/src/services/core/draft.ts:56-124` +- [X] T005 [P] Add `lineageId?: string`, `restoreFrom?: RestoreChoice` and `restoredAt?: string` to `EnhancedDeploymentDetail` (`packages/shared/src/index.ts:2017-2076`), all optional so every record already on disk stays valid with no migration +- [X] T006 Widen `composeUp` to `{ services?: string[]; wait?: boolean; timeoutMs?: number }` in the interface at `packages/server/src/services/core/docker.ts:67` and the Real implementation at `:231-269`, emitting `docker compose … up -d [--wait] [services…]`; keep `profiles` travelling via `COMPOSE_PROFILES` (`withComposeProfiles`, `:192-195`), NOT a CLI flag; `timeoutMs` overrides the 5-minute `execAsync` cap at `:248` because a `--wait` on a freshly-`initdb`'d Postgres can exceed it (research R12) +- [X] T007 Update `MockDockerService.composeUp` at `packages/server/src/services/core/docker.ts:722-725` to accept the new options and **record** the requested services and wait flag on the instance so tests can assert on them — plan.md's "Known trap": a Mock that accepts and ignores `services` yields a green suite over a restore that starts the wrong containers (Constitution IV) +- [X] T008 Create `packages/server/src/services/core/restore-candidates.ts` with the candidate resolver's shape: eligibility predicate, description merge (identity record first, deployment record fallback), lineage grouping/ordering, skew verdict, required-acknowledgement derivation. Pure functions over already-fetched state — no I/O, no app names (Constitution V) +- [X] T009 Persist `lineageId` on the deployment record in `createFromDraft` (`packages/server/src/services/core/deployment.ts`, record literal `:1008-1062`, beside `channel,` at `:1028`): the candidate's lineage on a restore, else the deployment's own id +- [X] T010 Change `writeInstanceMarkers`' `lineageId` expression at `packages/server/src/services/core/deployment.ts:3374` from `deployment.id` to `deployment.lineageId ?? deployment.id`, and update the comment at `:3366-3373` — which explicitly predicted this change — to say it has now happened and to name spec 007. The `??` fallback is what makes this zero-migration: an older record reads `undefined` and yields exactly the value it always wrote + +**Checkpoint**: `bun --cwd packages/server test` green; `bun run typecheck` green. + +--- + +## Phase 3: User Story 1 — An operator installs an app and gets their data back (P1) + +**Goal**: Pick a candidate during install; the app comes up holding its data. + +**Independent test**: Install an app, put recognisable data in it, install a second +copy restoring from the first, confirm the second serves that data on first boot. + +### Candidate discovery + route + +- [X] T011 [US1] Implement candidate eligibility in `packages/server/src/services/core/restore-candidates.ts` per data-model.md §2: same app, not self, settled state (`running`/`stopped` — not in flight, not `error`), and `dirHasContents(appRoot, [INSTALL_MARKERS_DIR])`. The ignore-list is load-bearing — without it every materialised install looks like it holds data, which is the data-loss shape spec 006's review caught at `deployment.ts:2513` +- [X] T012 [US1] Implement candidate description in `restore-candidates.ts`: read `.hola/instance.json`, fall back per-field to the deployment record, set `hasIdentityRecord`, degrade `lineageId` to the deployment id when the record is absent (FR-003) +- [X] T013 [US1] Implement lineage grouping and newest-first ordering in `restore-candidates.ts`, with `defaultCandidateId: null` + `requiresExplicitChoice: true` when two or more lineages match (FR-005, FR-036) +- [X] T014 [US1] Add `GET /api/apps/:appId/restore-candidates` to `packages/server/src/server.ts` per contracts/api.md §1, with the optional `?version=` query; return `200` with `lineages: []` when there are none — never `404` (FR-042) + +### Draft entry + persistence + +- [X] T015 [US1] Accept `restoreFrom` on the catalog draft path in `packages/server/src/services/core/draft.ts` at the `resolvePlatformTokens` seed site (`:473`), resolving and validating the candidate before seeding +- [X] T016 [US1] **Refuse** `restoreFrom` on the install-by-ref path (`draft.ts:606`) with `RESTORE_NOT_SUPPORTED` — never ignore it. Honouring it on one seed path and dropping it on the other is exactly the silent-empty-restore failure the spec exists to prevent (research R2) +- [X] T017 [US1] Carry `restoreFrom` onto the finalized manifest **outside** `canonicalSpec` in `draft.ts:932-943`, beside `channel`, and extend that comment to cover it so the checksum stays a pure function of the deployable spec (FR-009) +- [X] T018 [US1] Re-validate the restore choice in `createFromDraft` (`deployment.ts:915`) before any state is created — the candidate may have been deleted or started a lifecycle action since the draft — and persist `restoreFrom` onto the record (FR-010) + +### The restore sequence + +- [X] T019 [US1] Implement the restore sequence in `runLifecycleJob`'s deploy/start/rollback branch (`deployment.ts:3780-3821`), inserted after the cancellation check at `:3813` and before `composeUp` at `:3815`, in exactly the ten-step order of research R8: assert target empty → re-resolve candidate → quiesce + capture → extract → assert payload present → discard → rewrite marker → write OIDC file → `up --wait` hook services → run restore hooks +- [X] T020 [US1] Guard the whole sequence on `restoreFrom && !restoredAt` **and** the action being the deployment's first deploy, so a restart/promote/rollback never re-runs it and a retried job cannot re-quiesce a live source (FR-012, data-model.md §8) +- [X] T021 [US1] Implement capture staging in `deployment.ts`: tar the source's data root to a staging file under the **target** deployment's own directory, distinct from the snapshot store, deleted in a `finally` on success and failure alike. Extract straight into the target data root — the archive is root-relative (`tar -C .`), so there is no intermediate extracted copy and peak cost is one archive (research R7, FR-016a, SC-013) +- [X] T022 [US1] **Move** the `writeOidcCredentialsFile` call from `deployment.ts:3800` into the restore sequence, after extraction — conditionally, so that an install with no restore keeps the call exactly where it is today (FR-019, FR-023, research R10) +- [X] T023 [US1] Rewrite `.hola/instance.json` after extraction and discards by calling the existing `writeInstanceMarkers` (`:3322`), which already computes every field correctly. Two independent reasons it is required: extraction `rm -rf`s the root and destroys the marker, and the restored tree carries the **source's** record (FR-018, research R11) +- [X] T024 [US1] Set `restoredAt` on the deployment record on success (data-model.md §6) +- [X] T025 [US1] Fail the install on any restore failure, with no fallback to starting the app on an empty or partial root (FR-022) + +### Tests (US1) + +- [X] T026 [P] [US1] `packages/server/src/__tests__/deployments/restore-on-install.test.ts` — set up the **real-filesystem harness** (`RealStorageService` + `mkdtemp` + `HOLA_APPS_BIND_ROOT`), copied from `install-markers.test.ts:116-142` or `snapshot.test.ts:99-125`. Required because `MockStorageService` discards file modes (#475) and `MockDockerService` starts no containers +- [X] T027 [P] [US1] Tests for quickstart **scenarios 1–4** (candidate listing, identity-record fallback, marker-only root excluded, unsettled/error states excluded) in `restore-on-install.test.ts` — U-fs +- [X] T028 [P] [US1] Tests for quickstart **scenarios 5, 6, 7** (lineage grouping/ordering/explicit-pick; works with no provider installed; empty list is `200` not `404`) in `restore-on-install.test.ts` — **mixed modes**: 5 and 7 are pure unit, but **6 is U-fs** and needs the real-filesystem harness +- [X] T029 [P] [US1] Tests for quickstart **scenarios 8, 9** (catalog path accepts, install-by-ref refuses with `RESTORE_NOT_SUPPORTED`; patch and finalize still closed) in `restore-on-install.test.ts` +- [X] T030 [P] [US1] Test for quickstart **scenario 11** (two finalizes differing only in `restoreFrom` produce the **same** checksum) in `restore-on-install.test.ts` +- [X] T031 [P] [US1] Tests for quickstart **scenarios 12, 13, 14** (record carries `restoreFrom`+`lineageId` and the payload is unchanged; fresh vs restored `lineageId`; `restoredAt` makes later actions skip) — U-fs +- [X] T032 [P] [US1] Tests for quickstart **scenarios 15, 16** (restore runs between `composePull` and `composeUp`; non-empty target aborts with `RESTORE_TARGET_NOT_EMPTY` while a marker-only root proceeds) — U-fs +- [X] T033 [US1] Test for quickstart **scenario 18** — **highest value**. Proves FR-016 is a post-condition assertion and not a subtree search: with the archive emptied the install must fail `RESTORE_PAYLOAD_EMPTY` rather than report success. The prompt's "locate the subtree at ``" describes a provider archive tool and cannot occur with this codebase's root-relative helpers (research R9). U-fs — real-filesystem harness +- [X] T034 [P] [US1] Test for quickstart **scenario 19** (staging lives under the target, is gone after success and failure, never appears in the source's snapshot listing) — U-fs +- [X] T035 [US1] Test for quickstart **scenario 22** — **highest value**. The OIDC ordering trap. It MUST be written so that it **fails** if the `writeOidcCredentialsFile` call is left at `:3800`; a test that passes either way tests nothing. Verify by temporarily reverting T022 and watching it go red. U-fs — real-filesystem harness +- [X] T036 [US1] Test for quickstart **scenario 23** — asserts `MockDockerService` **records** `services` and `wait` and that `{services:['db'],wait:true}` starts nothing else. This is plan.md's "Known trap" (Constitution IV) +- [X] T037 [US1] Test for quickstart **scenario 27** — **highest value**. The regression guard: with no `restoreFrom`, the deploy job's calls and their order are identical to `main`'s, including `writeOidcCredentialsFile`'s original position (FR-023, SC-009) +- [X] T038 [P] [US1] Tests for quickstart **scenarios 21, 25** (restored marker describes the new install while `lineageId` is the source's; every failure leaves a failed install) — U-fs + +**Checkpoint**: US1 independently testable — a restore works end to end for a +plain-file-copy app, with no configuration carried and no app declaration. + +--- + +## Phase 4: User Story 2 — The restored app can read the data it was given (P1) + +**Goal**: Carried configuration means data encrypted under a generated value stays readable. + +**Independent test**: Restore an app storing something encrypted under a generated +key and read that item back through the app. + +> **Shares US1's spine.** The seeding happens at the same `draft.ts` call site +> T015 touched, and the restore sequence is unchanged. This phase adds the env +> read, the merge, the derived warning and the acknowledgement gate — not a second +> code path. + +- [X] T039 [US2] Read the candidate's environment record from `/.hola//env.json` in `draft.ts` — **outside** the app data root, which is where spec 006 actually shipped it (`envRecordDirFor`, `deployment.ts:2474-2475`), not inside it as the prompt of record assumed (research R17) +- [X] T040 [US2] Seed `appEnv` through the existing `mergeUpgradeAppEnv` (`packages/server/src/services/core/upgrade-env.ts:35-44`) unchanged — carried value wins, a new key with a `generate` recipe is minted, anything else rides through. This adds a second production call site beside `server.ts:1295` +- [X] T041 [US2] Derive the uncarried-configuration warning in `restore-candidates.ts`: exactly the `AppEnvVar` entries (`shared/src/index.ts:903-958`) where `isSecret === true` **and** `generate` is present. Derived, never declared — a manifest field would rot and an app author could get it wrong (FR-033) +- [X] T042 [US2] Implement the two acknowledgement codes (`restore-version-unknown`, `restore-env-not-carried`) in `restore-candidates.ts` + `createFromDraft`, computed server-side and enforced exactly as `grants` already are on `CreateDeploymentFromDraftRequest` (`shared/src/index.ts:2257-2289`) — refused when required and absent (FR-037a, research R16) +- [X] T043 [US2] Default the new install's `name` and `subdomain` from the candidate, and emit a `host-divergence` warning when the operator changes the subdomain (FR-035) + +### Tests (US2) + +- [X] T044 [P] [US2] Test for quickstart **scenario 10** (the three-case merge: carried wins, generate-recipe key minted, other new key rides through) — U-fs +- [X] T045 [P] [US2] Tests for quickstart **scenarios 38, 43** (the warning names exactly the `isSecret && generate` keys — not every secret; declining available configuration still warns and still requires the acknowledgement) — U-fs +- [X] T046 [P] [US2] Tests for quickstart **scenarios 40, 42** (name/subdomain default and divergence warning; a required-and-absent acknowledgement fails with `RESTORE_ACK_REQUIRED`) + +**Checkpoint**: US2 independently testable — a restored install's configuration +matches its source's, and every uncarried key is named. + +--- + +## Phase 5: User Story 3 — A restore that cannot be right refuses before the app starts (P1) + +**Goal**: Every unsafe restore fails loudly, before any container runs. + +**Independent test**: Drive each refusal and confirm the install fails, the reason +is specific, and no app container started. + +- [X] T047 [US3] Implement the skew verdict in `restore-candidates.ts` in the order of data-model.md §4, evaluating **source-newer before** `checkUpgradePath`. `checkUpgradePath` (`shared/src/index.ts:423-460`) short-circuits on `!isNewerVersion(to, from)` and returns `ok` for a newer source, so evaluating it first would let the case through (research R15) +- [X] T048 [US3] Implement `RESTORE_SOURCE_NEWER` (refuse always) and `RESTORE_UPGRADE_PATH` (refuse, surfacing `suggestedVersion`), reusing `checkUpgradePath` verbatim for the two rows it can express — a third production call site beside `deployment.ts:1104` and `:3178` (FR-029, FR-030) +- [X] T049 [US3] Implement the unknown-version path: proceed only with `restore-version-unknown`, else `RESTORE_ACK_REQUIRED` naming it (FR-032) +- [X] T050 [US3] Implement `RESTORE_ENV_REQUIRED`: an app whose `restore` block sets `requiresEnv: true` **refuses** when there is no environment record, rather than warning (FR-034) +- [X] T051 [US3] Implement `RESTORE_CANDIDATE_GONE` and `RESTORE_CANDIDATE_BUSY` on re-resolution inside the job (research R8 step 2), so a candidate deleted or made busy between choice and deploy fails the install rather than being captured optimistically. Covers quickstart **scenario 16a** — U-fs, real-filesystem harness (FR-013a) +- [X] T052 [US3] Run restore hooks fail-closed via the existing `runPreHooksFailClosed` policy (`deployment.ts:2684-2701`) with started-only cleanup (`runPostHooks`, `:2709-2724`), failing the install with `RESTORE_HOOK_FAILED` on a hook failure or a hook service that never becomes healthy (FR-021) +- [X] T053 [US3] Ensure a failed restore leaves the deployment in `error` with its data root intact — no automatic deletion — and that such a deployment is thereafter excluded from candidacy (FR-022a, research R20) +- [X] T054 [US3] Ensure every refusal carries `details.code` (and `suggestedVersion` where applicable) in the same `CONFLICT` envelope as `PROVIDER_EXISTS`/`ALREADY_INSTALLED`, so clients build guidance from structure (FR-037) + +### Tests (US3) + +- [X] T055 [US3] Test for quickstart **scenario 34** — **highest value**. Must assert *directly* that `checkUpgradePath(newer, older, meta)` returns `ok`, so the test itself documents why FR-029 is stated independently rather than delegated +- [X] T056 [P] [US3] Tests for quickstart **scenarios 35, 36, 37** (guarded hop refuses with `suggestedVersion`; equal and clean-path proceed; unknown version gated on its acknowledgement) +- [X] T057 [P] [US3] Tests for quickstart **scenarios 39, 41** (`requiresEnv` refuses; every refusal carries `details.code`) +- [X] T058 [P] [US3] Test for quickstart **scenario 26** (failed restore stays in `error`, keeps its data root, and drops out of the candidate list) — U-fs, real-filesystem harness + +**Checkpoint**: US3 independently testable — every refusal path verified without +a container ever starting. + +--- + +## Phase 6: User Story 4 — An app says how it wants to be restored (P2) + +**Goal**: Discards and hooks make restore correct for database-backed apps. + +**Independent test**: Restore an app whose declaration discards the captured +database directory and loads a dump; confirm the result holds the dumped data. + +- [X] T059 [US4] Implement `AppRestoreDeclaration` reading in `deployment.ts`, keyed by **backup** participation id, reusing `AppBackupHook` (`shared/src/index.ts:295-298`) verbatim — no second hook format (FR-026, FR-027) +- [X] T060 [US4] Implement the three declaration states: no `restore@1` ⇒ not offered; `restore@1` with no block ⇒ plain file copy, no discards, no hook; `restore@1` + block ⇒ apply it. The middle state already exists on 12 of 17 catalog acceptors, so this gives an existing shape meaning rather than requiring adoption (FR-025) +- [X] T061 [US4] Apply `discard` paths after extraction and before any container starts, resolving each through `resolveContainedDir` (`packages/server/src/services/core/path-containment.ts:30`) exactly as push targets do (`deployment.ts:2952`); a path resolving outside the data root **refuses** the restore rather than being skipped (FR-017) +- [X] T062 [US4] Start only the hook services with `{ services, wait: true, timeoutMs }` and run each participation's restore hook against them, relying on the app's own declared `healthcheck` rather than any bespoke readiness poll — per-app readiness knowledge in the server is exactly what Constitution V forbids (FR-020) +- [X] T063 [P] [US4] Prepare the catalog change in `/workspaces/apps` — `schemas/manifest.schema.json` gains the `restore` array referencing the existing `$defs/backupHook`, and the five hook apps (guacamole, immich, mealie, paperless-ngx, postiz) gain a `restore` block per contracts/manifest.md. **Read each app's own `compose.yaml` for its `discard` path — do not assume mealie's `postgres`**; the mount point is per-app and a wrong path either discards nothing or discards the wrong directory. Every hook needs `-v ON_ERROR_STOP=1`. **STOP after preparing the diff and report it — do NOT open a PR on `try-hola/apps`** (repository hard rule: no PR on a repo without the user's explicit per-instance permission) + +### Tests (US4) + +- [X] T064 [P] [US4] Tests for quickstart **scenarios 20, 29** (discards applied before any container starts; escaping path refuses; two participations both restore) — U-fs +- [X] T065 [P] [US4] Tests for quickstart **scenarios 30, 31, 32** (no-block plain copy vs. undeclared app; hook shape is `AppBackupHook` with no second type in `shared`; every current catalog manifest still validates and a `restore` block validates) — **mixed modes**: 31 and 32 are pure unit, but **30 is U-fs** and needs the real-filesystem harness + +**Checkpoint**: US4 independently testable at the unit level; scenario 33 (a real +Postgres restore) is covered by the manual VM task. + +--- + +## Phase 7: User Story 5 — Clone an app with its data, from the command line (P2) + +**Goal**: One command produces a second copy holding the source's data. Closes #429. + +**Independent test**: From the CLI, install a second copy naming the first as the +source; confirm it is independently addressable and holds the data while the +source keeps running. + +- [X] T066 [P] [US5] Register `--restore-from `, `--no-restore`, `--restore-list`, `--carry-env`/`--no-carry-env` and `--ack ` in `packages/cli/src/index.ts` beside the existing `--grant` registration at `:163` +- [X] T067 [US5] Parse them in `packages/cli/src/commands/install/install.ts`, mirroring `parseGrants` (`:82-93`) for the repeatable/comma-separated `--ack` +- [X] T068 [US5] Implement `--restore-list` against the candidates route with **no draft created**, rendering per contracts/cli.md and naming on each warning line the flag that would satisfy it +- [X] T069 [US5] Make the non-interactive default **no restore** (FR-044): with no restore flag, no restore happens even when candidates exist. A candidate existing is not consent, and silence must never overwrite an operator's install decision with a guess +- [X] T070 [US5] Refuse `--restore-from latest` when two or more lineages match — "latest" is ambiguous across unrelated histories (FR-036) +- [X] T071 [US5] Build every restore refusal hint from `details` in `packages/cli/src/lib/deploy-flow.ts`, never from the server's message, per the rule and reasoning already at `:137-155`; one branch per `RESTORE_*` code (FR-045) + +### Tests (US5) + +- [X] T072 [P] [US5] Tests for quickstart **scenarios 49, 50** (each flag behaves as specified, `latest` refuses across lineages; no flag ⇒ no restore even with candidates present) in `packages/cli` +- [X] T073 [P] [US5] Tests for quickstart **scenarios 51, 52** (hint correct from `details` alone with the message blanked; `--ack` parses repeated and comma-separated exactly as `--grant`) in `packages/cli` + +**Checkpoint**: US5 independently testable; #429's outcome reachable in one command. + +--- + +## Phase 8: Polish & Cross-Cutting + +### Wizard + +- [X] T074 Add the restore step at index 0 of `steps` in `packages/web/src/pages/InstallWizard.tsx:29-36`, before Configuration — forced, because Configuration renders `appEnv` and `appEnv` is seeded by the restore choice (FR-038) +- [X] T075 Re-create the draft when the restore choice changes, following `switchChannel`'s existing delete-and-recreate pattern at `InstallWizard.tsx:507-549`; this is the established pattern, not a workaround (FR-039) +- [X] T076 Render carried values as ordinary `appEnv` rows through the existing mask/reveal component (`:1187-1372`), with no separate or privileged widget (FR-040) +- [X] T077 Add acknowledgement checkboxes and the unconditional summary-step acknowledgement naming data **and credentials** and that jobs, webhooks and integrations may fire on start (FR-041) +- [X] T078 [P] Web tests for quickstart **scenarios 44–48** in `packages/web/src/__tests__/pages/` + +### Manual verification + +- [ ] T079 **MANUAL — NOT part of the default suite.** Run quickstart **scenarios 17, 24, 28, 33, 53, 56** on a disposable VM (`bin/vm-e2e-suite` / the `vm-e2e` skill): source untouched after capture; failing hook aborts the install; end-to-end mealie restore; real Postgres discard-and-load; #429 clone-with-data; a whole restore with no backup provider installed. These need real containers and a real database. Record the outcome in the PR body + +### Docs + +- [X] T080 [P] Document restore-on-install in `docs/OPERATIONS.md` — what it does, when it refuses, and that a restore carries credentials as well as data +- [X] T081 [P] Add a bullet to the `CLAUDE.md` architecture notes covering restore-on-install, in the style of the existing capability-contract and release-channel bullets + +### Follow-up issues (no inline TODOs — repository hard rule) + +- [X] T082 [P] `gh issue create` — the dead `POST /api/backups/:id/restore` stub (`server.ts:1663-1668`) fabricates a jobId and creates no job; `RestoreBackupRequest`/`Response` and `JobType`'s `'restore'` are unused; `Backups.tsx`/`useBackupsApi.ts`/`BackupCoverage.tsx` talk to it. Note the vocabulary adjacency to spec 007 and point at #160. Record the issue number here: `#484` +- [X] T083 [P] `gh issue create` — `install-markers.test.ts:704` cites `~:3410`/`~:3414` for the restore-before-materialize ordering; the real lines are `:3793`/`:3797`. The ordering claim still holds. Propose the comment cite symbols rather than line numbers, since this is direct evidence that line-number comments rot. Record: `#485` +- [X] T084 [P] `gh issue create` — Sequence 6 inherits the absolute-path subtree trap: a provider archive tool (restic/borg) reproduces the source's absolute path under the restore target, so its payload is NOT at the staging root. Spec 007's FR-016 post-condition assertion is what generalises. Record: `#486` +- [X] T085 [P] `gh issue create` — `composeUp`'s 5-minute `execAsync` default (`docker.ts:248`) deserves a deliberate look now that one caller parameterises it; the other call sites inherit it by accident rather than by decision. Record: `#487` + +### Scope boundary + gate + +- [X] T086 Verify the scope boundary with quickstart §9's greps: `git diff main -- packages/shared/src/contracts.ts` empty (no capability contract added, FR-047); `git diff main -- packages/web/src/pages/Backups.tsx packages/web/src/hooks/useBackupsApi.ts` empty (dead stub untouched); `grep -rniE "postgres|mealie|immich|gitea|paperless" packages/server/src/services/core/restore-candidates.ts` empty (no app name leaked into the platform — the mechanical form of plan.md's Principle V audit) +- [X] T087 Test for quickstart **scenarios 54, 55** — `CONTRACTS` still exactly `auth@1`/`backup@1`/`push@1`/`container-logs@1`, no new grant kind, no `/api/contracts/*` change, `JobType` unchanged; nothing in this feature imports `RestoreBackupRequest` +- [X] T088 Final gate: `bun run typecheck && bun run lint && bun run typecheck && bun run test && bun run build`. Typecheck twice — CI has caught regressions a lint auto-fix introduced after the first run + +--- + +## Dependencies + +``` +Phase 1 (T001) + └─> Phase 2 (T002-T010) BLOCKING — the shared spine + ├─> Phase 3 US1 (T011-T038) P1 ← MVP + │ └─> Phase 4 US2 (T039-T046) P1 shares US1's seed site + sequence + │ └─> Phase 5 US3 (T047-T058) P1 guards US1's sequence + │ └─> Phase 6 US4 (T059-T065) P2 discards/hooks need US3's fail-closed + ├─> Phase 7 US5 (T066-T073) P2 needs T014's route + T015's draft field only + └─> Phase 8 (T074-T088) +``` + +**Story independence.** US1 is a complete MVP alone. US2, US3 and US4 each extend +US1's single restore sequence rather than adding their own — this is stated three +times on purpose, in Phase 2's note, in Phase 4's note, and here. US5 is the only +story reachable without the restore sequence being finished, since `--restore-list` +needs just the candidates route. + +**Within-phase parallelism.** `[P]` tasks touch different files. The largest +parallel batches: T002–T005 (four independent type additions), and the test tasks +inside each story phase once that story's implementation lands. T006 and T007 are +**not** parallel with each other — same file, and T007 depends on T006's signature. + +## Implementation strategy + +1. **MVP** = Phase 1 + Phase 2 + Phase 3 (US1). A working restore for the 12 + plain-file-copy catalog apps, with no configuration carried. +2. **Make it safe** = Phase 5 (US3). Land before Phase 4 if time is short: an + unsafe restore that proceeds is worse than one that cannot carry configuration. +3. **Make it correct for databases** = Phase 4 + Phase 6. +4. **Reach** = Phase 7 + Phase 8. + +## Task count + +| Phase | Tasks | +|---|---| +| 1 Setup | 1 | +| 2 Foundational | 9 | +| 3 US1 (P1) | 28 | +| 4 US2 (P1) | 8 | +| 5 US3 (P1) | 12 | +| 6 US4 (P2) | 7 | +| 7 US5 (P2) | 8 | +| 8 Polish | 15 | +| **Total** | **88** | + +All 57 quickstart scenarios are cited by a test task. The six VM-mode scenarios +(17, 24, 28, 33, 53, 56) are consolidated into T079 and are explicitly **not** +part of the default suite. From f368e690dd086629be0529621197f3226539d7af Mon Sep 17 00:00:00 2001 From: Paul O'Fallon Date: Sun, 20 Sep 2026 20:38:40 +0000 Subject: [PATCH 2/5] test(deploy): cite the restore-before-materialize ordering by symbol, not line MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The comment at install-markers.test.ts read "~:3410 (restore) before ~:3414 (materialize)". A single intervening feature merge (spec 006, 921c790) moved both call sites by roughly 380 lines while the ordering claim itself stayed true, so the citation was wrong within one release of being written. Cite the symbols instead — `restoreAppDataSnapshot` runs before `materializeCompose` in `runLifecycleJob`'s deploy/start/rollback branch — and record why, because this is the second time in two specs that stale line-number citations have cost verification work: spec 007's planning had to re-verify roughly thirty anchors against the current tree because its prompt's numbers predated the same merge, several of them wrong in shape rather than merely in position. A citation that rots faster than the invariant it documents is worse than no citation: the next reader has to verify it before trusting the sentence built around it. Closes #485 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh --- .../deployments/install-markers.test.ts | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/packages/server/src/__tests__/deployments/install-markers.test.ts b/packages/server/src/__tests__/deployments/install-markers.test.ts index 4626ff21..fce4e55d 100644 --- a/packages/server/src/__tests__/deployments/install-markers.test.ts +++ b/packages/server/src/__tests__/deployments/install-markers.test.ts @@ -700,11 +700,19 @@ describe('Install identity markers (spec 006)', () => { // Roll back to v1 WITH restoreData: true. The lifecycle job wipes and // replaces the whole data root from the v1 snapshot (which carries v1's // OLD record, with the OLD writtenAt) BEFORE materializeCompose reruns - // for the release actually being brought up (v1) — see deployment.ts - // ~:3410 (restore) before ~:3414 (materialize). If a refactor ever hoists - // the marker write earlier in the lifecycle job, the freshly-written - // record would be wiped by the restore and this test would see the STALE - // writtenAt from the snapshot survive, rather than a fresh one. + // for the release actually being brought up (v1) — in `runLifecycleJob`'s + // deploy/start/rollback branch, `restoreAppDataSnapshot` runs before + // `materializeCompose`. If a refactor ever hoists the marker write earlier + // in the lifecycle job, the freshly-written record would be wiped by the + // restore and this test would see the STALE writtenAt from the snapshot + // survive, rather than a fresh one. + // + // Cited by symbol, not line number, deliberately (#485). This comment + // previously read "~:3410 (restore) before ~:3414 (materialize)"; a single + // intervening feature merge (spec 006, 921c790) moved both by ~380 lines + // while the ordering claim itself stayed true. A citation that rots faster + // than the invariant it documents is worse than no citation, because the + // next reader has to verify it before trusting the sentence around it. const rolledBack = await deployments.rollback(created.deploymentId, { targetReleaseId: v1ReleaseId, restoreData: true, From 9bd4470781d0c8892dcd2c484ca48ec3796eaba6 Mon Sep 17 00:00:00 2001 From: Paul O'Fallon Date: Sun, 20 Sep 2026 20:49:17 +0000 Subject: [PATCH 3/5] fix(restore): make two correct restore refusals legible (#489, #490) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both defects are the same shape: restore-on-install refuses for the right reason and tells the operator nothing they can act on. Neither refusal changes — FR-022 forbids starting an app on a data root the platform cannot vouch for, and an address is an operator decision — only what the refusal says, and the structured `details` it carries (FR-037). #489 — a restore that fails AFTER extraction left the install permanently unstartable behind a generic `RESTORE_TARGET_NOT_EMPTY`, which reads as "somebody else's data is in the way" when in fact the data is this install's own half-landed payload. `restoreStartedAt` is now persisted (and flushed, so it survives a hard kill mid-extraction) at the last read-only moment of the sequence, and `classifyNonEmptyTarget` uses it to split the refusal into `RESTORE_INCOMPLETE` — naming the state and the recovery, uninstall + reinstall or clear the data root — and the unchanged generic code. The field is a diagnostic only: the job-entry gate is untouched, so a retry still enters the restore sequence and still refuses. Two alternatives from the issue are rejected in code comments so they are not re-proposed: persisting an attempted-marker so a retry takes the no-restore path, and re-running only the post-extraction steps. Both can start an app on a half-extracted tree — the silent-empty-app outcome the spec's Executive Summary names as the worst in this feature. #490 — FR-035 defaults a restored install's address to the candidate's own, but a candidate is by definition a deployment that still exists on this host and still owns that route, so the default could never succeed; it fell through to `routingService.validateRule`'s bare host `CONFLICT`, which carries no `details.code` and never mentions restore. `resolveRestoreNameDefaults` now refuses with `RESTORE_ADDRESS_REQUIRED`, naming the candidate and the address it holds, and only when the caller supplied no name — an operator who names the install has made the address decision and gets the routing layer's own conflict. It compares against the candidate's LIVE subdomain (its deployment record), not its identity record's, which can be stale. Deriving a distinct/suffixed slug is rejected in a comment: that would put data full of the old address's absolute URLs at a new address nobody chose. Also: a CLI hint branch for `RESTORE_ADDRESS_REQUIRED` built from `details`; `RESTORE_INCOMPLETE` gets none, because job-time codes reach `hola install` as a failed job, not a `HolaApiError` — which is why its recovery is spelled out in the server's message. data-model.md §4, contracts/api.md and contracts/cli.md updated to agree on the code list, with the create-time vs job-time split now nine/four. Closes #489 Closes #490 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh --- packages/cli/src/__tests__/restore.test.ts | 18 +++ packages/cli/src/lib/deploy-flow.ts | 11 ++ .../deployments/restore-on-install.test.ts | 133 +++++++++++++++++- .../server/src/services/core/deployment.ts | 52 ++++++- .../src/services/core/restore-candidates.ts | 104 +++++++++++++- packages/shared/src/index.ts | 22 +++ specs/007-restore-on-install/contracts/api.md | 28 +++- specs/007-restore-on-install/contracts/cli.md | 12 ++ specs/007-restore-on-install/data-model.md | 54 +++++-- specs/007-restore-on-install/quickstart.md | 2 + 10 files changed, 404 insertions(+), 32 deletions(-) diff --git a/packages/cli/src/__tests__/restore.test.ts b/packages/cli/src/__tests__/restore.test.ts index 55191e9c..3fafcce7 100644 --- a/packages/cli/src/__tests__/restore.test.ts +++ b/packages/cli/src/__tests__/restore.test.ts @@ -139,6 +139,9 @@ describe('restore-on-install CLI (spec 007)', () => { { code: 'RESTORE_CANDIDATE_GONE', details: {}, expect: '--restore-list' }, { code: 'RESTORE_CANDIDATE_BUSY', details: {}, expect: '--restore-list' }, { code: 'RESTORE_NOT_SUPPORTED', details: {}, expect: 'install-by-ref' }, + // #490: the hint names the candidate and the address it still holds, and + // asks for --name — it never invents a suffixed address as the answer. + { code: 'RESTORE_ADDRESS_REQUIRED', details: { candidateId: 'mealie-aaa', candidateName: 'Recipes', subdomain: 'recipes' }, expect: '--name' }, ]; for (const c of cases) { @@ -152,4 +155,19 @@ describe('restore-on-install CLI (spec 007)', () => { errSpy.mockRestore(); } }); + + // ---- #490: the RESTORE_ADDRESS_REQUIRED hint names what is in the way ---- + it('the RESTORE_ADDRESS_REQUIRED hint names the candidate and the address it holds', () => { + const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + reportDeployError( + new HolaApiError('', 409, { + details: { code: 'RESTORE_ADDRESS_REQUIRED', candidateId: 'mealie-aaa', candidateName: 'Recipes', subdomain: 'recipes' }, + }), + ); + const hint = String(errSpy.mock.calls.find(call => String(call[0]).startsWith('Hint:'))?.[0] ?? ''); + expect(hint).toContain('Recipes'); + expect(hint).toContain('recipes'); + expect(hint).toContain('--name'); + errSpy.mockRestore(); + }); }); diff --git a/packages/cli/src/lib/deploy-flow.ts b/packages/cli/src/lib/deploy-flow.ts index 131802d7..cf081272 100644 --- a/packages/cli/src/lib/deploy-flow.ts +++ b/packages/cli/src/lib/deploy-flow.ts @@ -151,6 +151,8 @@ export function reportDeployError(err: unknown): undefined { missingKeys?: string[]; required?: string[]; candidateId?: string; + candidateName?: string; + subdomain?: string; } | undefined) : undefined; if (details?.code === 'ALREADY_INSTALLED' && details.existing) { @@ -191,6 +193,15 @@ export function reportDeployError(err: unknown): undefined { console.error('Hint: install-by-ref cannot restore — use the catalog install path instead.'); } else if (details?.code === 'RESTORE_NOT_ACCEPTED') { console.error('Hint: this app has not declared that it can be restored; install it fresh and move data in yourself.'); + } else if (details?.code === 'RESTORE_ADDRESS_REQUIRED') { + // #490: the restore would have defaulted to an address the candidate still + // holds. The address is the operator's call, so name what is in the way and + // ask for one — never invent a suffixed slug on their behalf. + console.error( + `Hint: ${details.candidateName ? `'${details.candidateName}'` : 'the restore source'} still uses ` + + `${details.subdomain ? `'${details.subdomain}'` : 'that address'}. Give the restored copy its own address ` + + `with --name (e.g. --name ${details.subdomain ? `${details.subdomain}-restored` : 'myapp-restored'}).`, + ); } // #246: a single-instance app already installed (older/untyped error shape, // e.g. a pre-spec-005 server) — the message already names the diff --git a/packages/server/src/__tests__/deployments/restore-on-install.test.ts b/packages/server/src/__tests__/deployments/restore-on-install.test.ts index 6b8a51c1..e061e668 100644 --- a/packages/server/src/__tests__/deployments/restore-on-install.test.ts +++ b/packages/server/src/__tests__/deployments/restore-on-install.test.ts @@ -53,6 +53,7 @@ import { checkCandidateStillEligible, judgeRestoreChoice, resolveRestoreNameDefaults, + classifyNonEmptyTarget, type CandidateSource, } from '../../services/core/restore-candidates'; @@ -732,6 +733,59 @@ describe('Restore-on-install (spec 007) — real filesystem harness', () => { expect(await dirHasContents(emptyRoot, ['.hola'])).toBe(false); }); + test('scenario 16b: a restore that fails AFTER extraction refuses the retry with RESTORE_INCOMPLETE, never a silent start (#489)', async () => { + // Pure half: the discriminator is `restoreStartedAt`, nothing else. Both + // rows REFUSE — the difference is which state they name and which recovery + // they hand the operator. + expect(classifyNonEmptyTarget({ deploymentId: 'd1', deploymentName: 'Target', restoreStartedAt: undefined })) + .toMatchObject({ code: 'RESTORE_TARGET_NOT_EMPTY', details: { deploymentId: 'd1' } }); + const incomplete = classifyNonEmptyTarget({ + deploymentId: 'd1', deploymentName: 'Target', restoreStartedAt: '2026-01-01T00:00:00.000Z', + }); + expect(incomplete).toMatchObject({ + code: 'RESTORE_INCOMPLETE', + details: { deploymentId: 'd1', restoreStartedAt: '2026-01-01T00:00:00.000Z' }, + }); + // The message has to carry the recovery: this code is job-time only, so it + // never reaches a client as `details` it could build a hint from. + expect(incomplete.message).toMatch(/uninstall and reinstall/i); + expect(incomplete.message).toMatch(/clear its data root/i); + + // End-to-end half: an escaping discard path fails the restore AFTER + // extraction (step 6), which is exactly the #489 shape — the payload is on + // disk, `restoredAt` was never set. + const system = makeSystem(); + const source = await install(system, { name: 'source' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + acceptsConfig = ['restore@1', 'backup@1']; + restoreConfig = [{ id: 'default', discard: ['../escape'] }]; + + const failed = await install(system, { + name: 'half-restored', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + expect(failed.job?.status).toBe('failed'); + // The payload really did land, and really was left in place (FR-022a). + expect(await readExtraData(failed.deploymentId, 'note.txt')).toBe('hello'); + const afterFailure = await system.deployments.getDeployment(failed.deploymentId); + expect(afterFailure.restoredAt).toBeUndefined(); + + // Retrying the deploy re-enters the restore sequence (the gate is + // unchanged) and refuses at step 1 — now naming the half-landed state and + // the way out, instead of the generic "already holds app data". + const { jobId } = await system.deployments.executeAction(failed.deploymentId, { action: 'start' }); + const retried = await waitForJob(system.jobs, jobId!); + expect(retried.status).toBe('failed'); + expect(retried.error).toMatch(/already wrote data here/i); + expect(retried.error).toMatch(/uninstall and reinstall/i); + // Still a REFUSAL, not a proceed: nothing started, the data is untouched, + // and the restore was never marked consumed. + const afterRetry = await system.deployments.getDeployment(failed.deploymentId); + expect(afterRetry.status).toBe('error'); + expect(afterRetry.restoredAt).toBeUndefined(); + expect(await readExtraData(failed.deploymentId, 'note.txt')).toBe('hello'); + }); + // ========================================================================= // Restore hook service names are APP-SUPPLIED (review, spec 007 target A) // ========================================================================= @@ -1026,25 +1080,31 @@ describe('Restore-on-install (spec 007) — real filesystem harness', () => { test('scenario 40: name/subdomain default from the candidate (pure); a divergent choice warns host-divergence (end-to-end)', async () => { // The pure half: defaulting itself, isolated from the routing/collision - // machinery a real install exercises (which a SECOND live copy of the - // SAME app at the SAME address would correctly refuse — that's Traefik - // routing working as intended, not this rule failing). + // machinery a real install exercises. `candidateLiveSubdomain: null` is + // the case where the default is actually reachable — the candidate no + // longer routes under the address its record remembers. const defaults = resolveRestoreNameDefaults({ requestedName: undefined, + candidateId: 'demoapp-aaaaaaaa', candidateName: 'Recipes', candidateSubdomain: 'recipes', + candidateLiveSubdomain: null, appId: 'demoapp', deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), }); - expect(defaults).toEqual({ name: 'Recipes', subdomain: 'recipes', warnings: [] }); + expect(defaults).toEqual({ ok: true, name: 'Recipes', subdomain: 'recipes', warnings: [] }); const diverging = resolveRestoreNameDefaults({ requestedName: 'a-totally-different-name', + candidateId: 'demoapp-aaaaaaaa', candidateName: 'Recipes', candidateSubdomain: 'recipes', + candidateLiveSubdomain: 'recipes', appId: 'demoapp', deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), }); + expect(diverging.ok).toBe(true); + if (!diverging.ok) throw new Error('unreachable'); expect(diverging.subdomain).toBe('a-totally-different-name'); expect(diverging.warnings).toEqual([{ code: 'host-divergence', from: 'recipes', to: 'a-totally-different-name' }]); @@ -1065,6 +1125,71 @@ describe('Restore-on-install (spec 007) — real filesystem harness', () => { await waitForJob(system.jobs, diverged.jobId!); }); + test('scenario 40a: with no name, the FR-035 default onto the live candidate\'s own address refuses RESTORE_ADDRESS_REQUIRED (#490)', async () => { + // Pure half: the default resolves onto the address the candidate STILL + // routes under, so it is refused rather than returned — no suffixed slug + // is invented, because an address is an operator decision. + const refused = resolveRestoreNameDefaults({ + requestedName: undefined, + candidateId: 'demoapp-aaaaaaaa', + candidateName: 'Recipes', + candidateSubdomain: 'recipes', + candidateLiveSubdomain: 'recipes', + appId: 'demoapp', + deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), + }); + expect(refused).toMatchObject({ + ok: false, + code: 'RESTORE_ADDRESS_REQUIRED', + details: { candidateId: 'demoapp-aaaaaaaa', candidateName: 'Recipes', subdomain: 'recipes' }, + }); + + // It fires ONLY on the default. An operator who names the install has made + // the address decision; a collision there is the routing layer's to report. + const named = resolveRestoreNameDefaults({ + requestedName: 'recipes-restored', + candidateId: 'demoapp-aaaaaaaa', + candidateName: 'Recipes', + candidateSubdomain: 'recipes', + candidateLiveSubdomain: 'recipes', + appId: 'demoapp', + deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), + }); + expect(named.ok).toBe(true); + + // A candidate whose recorded subdomain differs from the one it routes + // under today: the recorded address is free, so the default still stands. + const stale = resolveRestoreNameDefaults({ + requestedName: undefined, + candidateId: 'demoapp-aaaaaaaa', + candidateName: 'Recipes', + candidateSubdomain: 'old-recipes', + candidateLiveSubdomain: 'recipes', + appId: 'demoapp', + deriveSubdomain: (name, appId) => slugifySubdomain(name || appId), + }); + expect(stale).toMatchObject({ ok: true, subdomain: 'old-recipes' }); + + // End-to-end half: the SDK/API shape an omitted `name` actually produces — + // a structured CONFLICT naming the candidate, not the routing layer's bare + // "Host '...' is already in use" (which carries no details.code at all). + const system = makeSystem(); + const source = await install(system, { name: 'source-recipes' }); + await writeExtraData(source.deploymentId, 'note.txt', 'hello'); + const { draftId } = await system.drafts.createDraft({ + appId: APP_ID, version: '1.0.0', + restoreFrom: { candidateId: source.deploymentId, carryEnv: false, acknowledge: ['restore-env-not-carried'] }, + }); + await system.drafts.updateDraft(draftId, { composeOverride: COMPOSE_WITH_DATA }); + await system.drafts.finalizeDraft(draftId); + await expect( + system.deployments.createFromDraft({ draftId, options: { autoStart: false }, allowMultiple: true }), + ).rejects.toMatchObject({ + code: 'CONFLICT', + details: { code: 'RESTORE_ADDRESS_REQUIRED', candidateId: source.deploymentId, subdomain: 'source-recipes' }, + }); + }); + test('scenario 42: a required-and-absent acknowledgement fails the create with RESTORE_ACK_REQUIRED', async () => { const system = makeSystem(); const source = await install(system, { name: 'source' }); diff --git a/packages/server/src/services/core/deployment.ts b/packages/server/src/services/core/deployment.ts index cef7dc10..b182133b 100644 --- a/packages/server/src/services/core/deployment.ts +++ b/packages/server/src/services/core/deployment.ts @@ -59,6 +59,7 @@ import { deriveEnvNotCarriedKeys, judgeRestoreChoice, resolveRestoreNameDefaults, + classifyNonEmptyTarget, type CandidateSource, type RestoreIdentitySnapshot, } from './restore-candidates'; @@ -1020,6 +1021,11 @@ abstract class InMemoryDeploymentService implements DeploymentService { // candidate when the operator supplied no explicit name. let restoreCandidateName: string | undefined; let restoreCandidateSubdomain: string | null | undefined; + // The label the candidate ACTUALLY routes under right now — read from its + // deployment record, not its identity record (#490). The identity record's + // `subdomain` is a snapshot and can be stale; only the live one tells us + // whether FR-035's default would land on an occupied address. + let restoreCandidateLiveSubdomain: string | null | undefined; const restoreFrom = artifacts?.manifest.restoreFrom; if (restoreFrom) { const source = await this.getRestoreSource(restoreFrom.candidateId); @@ -1041,6 +1047,7 @@ abstract class InMemoryDeploymentService implements DeploymentService { restoreLineageId = judged.candidate.lineageId; restoreCandidateName = judged.candidate.name; restoreCandidateSubdomain = judged.candidate.subdomain; + restoreCandidateLiveSubdomain = source?.deployment.subdomain ?? null; } // Release channel this deployment follows (#428): copied from the @@ -1100,15 +1107,27 @@ abstract class InMemoryDeploymentService implements DeploymentService { // different slug) diverges from the candidate's. `resolveRestoreNameDefaults` // is the pure form of this rule (restore-candidates.ts) — unaffected // installs skip it entirely and keep today's plain `deriveSubdomain` call. + // When the default would land on the address the candidate itself still + // routes under, this REFUSES with `RESTORE_ADDRESS_REQUIRED` (#490) + // rather than handing `onBeforeCreate` an address it is certain to + // reject with a bare routing conflict that never mentions restore. const restoreNameDefaults = restoreFrom ? resolveRestoreNameDefaults({ requestedName: request.name, + candidateId: restoreFrom.candidateId, candidateName: restoreCandidateName ?? app, candidateSubdomain: restoreCandidateSubdomain ?? null, + candidateLiveSubdomain: restoreCandidateLiveSubdomain ?? null, appId: app, deriveSubdomain, }) : undefined; + if (restoreNameDefaults && !restoreNameDefaults.ok) { + throw new ConflictError(restoreNameDefaults.message, { + code: restoreNameDefaults.code, + ...restoreNameDefaults.details, + }); + } const subdomain = restoreNameDefaults?.subdomain ?? deriveSubdomain(request.name, app); const restoreWarnings: RestoreWarning[] = restoreNameDefaults?.warnings ?? []; @@ -3881,9 +3900,11 @@ export class RealDeploymentService extends InMemoryDeploymentService { * (research R8). Ten steps, each independently justified in R8 / * data-model.md §8: * - * 1. assert the target root holds no app data (FR-014) + * 1. assert the target root holds no app data (FR-014) — and, if it does, + * say WHICH non-empty case it is (`classifyNonEmptyTarget`, #489) * 2. re-resolve the candidate — it may have changed since the draft (FR-013a) * 3. quiesce + capture the source (FR-015, R14) + * 3a. persist `restoreStartedAt` — the last read-only moment (#489) * 4. extract into the target root (R7 — root-relative, no intermediate copy) * 5. assert the payload actually landed (FR-016, R9 — a post-condition, not a subtree search) * 6. apply `discard` paths (FR-017, R13) @@ -3917,11 +3938,19 @@ export class RealDeploymentService extends InMemoryDeploymentService { // Step 1 (FR-014): the ignore-list is the same rule candidate eligibility // uses (research R5) — the `.hola` marker the platform itself just wrote // via `materializeCompose` must not count as "already holds app data". + // + // The refusal is not one refusal (#489): `classifyNonEmptyTarget` splits it + // into "something else put data here" (`RESTORE_TARGET_NOT_EMPTY`) and + // "this install's own restore half-landed and failed" (`RESTORE_INCOMPLETE`), + // which have different recoveries. Both still REFUSE — see that function for + // the two retry-and-proceed designs that were considered and rejected. if (await dirHasContents(targetAppRoot, [INSTALL_MARKERS_DIR])) { - throw new ConflictError( - `Cannot restore into '${deployment.name}': its data root already holds app data.`, - { code: 'RESTORE_TARGET_NOT_EMPTY', deploymentId: deployment.id }, - ); + const refusal = classifyNonEmptyTarget({ + deploymentId: deployment.id, + deploymentName: deployment.name, + restoreStartedAt: deployment.restoreStartedAt, + }); + throw new ConflictError(refusal.message, { code: refusal.code, ...refusal.details }); } // Step 2 (FR-013a): re-resolve rather than trust the draft-time choice — @@ -3997,6 +4026,19 @@ export class RealDeploymentService extends InMemoryDeploymentService { await this.runPostHooks(participants); } + // Record that this install's restore is about to WRITE, and flush it to + // disk before the write happens (#489). Everything above this line is + // read-only with respect to the target root; everything below it may + // leave the root holding a partial payload. Persisting here rather than + // relying on the job's own success/catch persist is what keeps the fact + // true across a hard server kill mid-extraction. It is a DIAGNOSTIC, not + // a gate: `willRestore` is unchanged, so a retry still enters this + // sequence and still refuses at step 1 — only now it can say which + // refusal it is (`classifyNonEmptyTarget`). + deployment.restoreStartedAt = new Date().toISOString(); + this.deployments.set(deployment.id, deployment); + await this.persistDeployment(deployment); + // Step 4 (R7): extract straight into the target root. Root-relative on // both sides (`tar -C .` in, `-C ` out) — no intermediate // extracted copy, so peak additional disk cost is one compressed diff --git a/packages/server/src/services/core/restore-candidates.ts b/packages/server/src/services/core/restore-candidates.ts index 52263a1b..565c4e16 100644 --- a/packages/server/src/services/core/restore-candidates.ts +++ b/packages/server/src/services/core/restore-candidates.ts @@ -303,6 +303,10 @@ export interface RestoreNameDefaults { warnings: RestoreWarning[]; } +export type RestoreNameResolution = + | ({ ok: true } & RestoreNameDefaults) + | { ok: false; code: RestoreRefusalCode; message: string; details: Record }; + /** * FR-035: default a restored install's name/subdomain from the candidate * when the operator supplied no explicit name, and warn when an explicit @@ -311,23 +315,117 @@ export interface RestoreNameDefaults { * own `deriveSubdomain` is injected rather than imported, so this stays a * function of its arguments (Constitution III/V) and is directly testable * without the routing/collision machinery a real install exercises. + * + * **The address the default lands on can be occupied (#490).** A restore + * candidate is by definition a deployment that still exists on this host, and + * a deployment owns its route whether it is running or stopped — so FR-035's + * "default from the candidate's recorded address" resolves, whenever the + * candidate still routes under that address, onto an address that is taken. + * `candidateLiveSubdomain` is the label the candidate ACTUALLY routes under + * right now (its deployment record's, not its identity record's, which can be + * stale); when the resolved default equals it, this refuses with + * `RESTORE_ADDRESS_REQUIRED` instead of returning an address the create is + * guaranteed to reject a few lines later with a bare routing `CONFLICT` that + * names no candidate and mentions no restore (FR-037). + * + * The refusal fires ONLY when the operator supplied nothing. An operator who + * names the install has made the address decision, collision included, and + * gets the routing layer's own conflict — which names the owning deployment. + * + * REJECTED alternative, deliberately not implemented: deriving a distinct + * (e.g. suffixed) slug so the default always succeeds. That silently creates + * an install at a NEW address holding data full of the OLD address's absolute + * URLs — manufacturing exactly the divergence the `host-divergence` warning + * exists to WARN about, without the operator ever choosing it. An address is + * an operator decision; when the default cannot be honoured, the operator has + * to make it. */ export function resolveRestoreNameDefaults(input: { requestedName: string | undefined; + candidateId: string; candidateName: string; candidateSubdomain: string | null; + /** The label the candidate deployment routes under today, or `null` if it routes nowhere. */ + candidateLiveSubdomain: string | null; appId: string; deriveSubdomain: (name: string | undefined, appId: string) => string; -}): RestoreNameDefaults { +}): RestoreNameResolution { const name = input.requestedName || input.candidateName; const subdomain = !input.requestedName && input.candidateSubdomain ? input.candidateSubdomain - : input.deriveSubdomain(input.requestedName || input.candidateName, input.appId); + : input.deriveSubdomain(name, input.appId); + + if (!input.requestedName && input.candidateLiveSubdomain && subdomain === input.candidateLiveSubdomain) { + return { + ok: false, + code: 'RESTORE_ADDRESS_REQUIRED', + message: + `Restoring from '${input.candidateName}' (${input.candidateId}) would default this install to ` + + `'${subdomain}', the address that candidate still uses. Give the new install its own name or ` + + `subdomain — addresses stored inside the restored data still point at '${subdomain}'.`, + details: { + candidateId: input.candidateId, + candidateName: input.candidateName, + subdomain, + }, + }; + } + const warnings: RestoreWarning[] = []; if (input.candidateSubdomain && input.candidateSubdomain !== subdomain) { warnings.push({ code: 'host-divergence', from: input.candidateSubdomain, to: subdomain }); } - return { name, subdomain, warnings }; + return { ok: true, name, subdomain, warnings }; +} + +/** + * Which "the target data root is not empty" refusal a restore is looking at + * (#489, FR-014). Both refuse — FR-022 forbids starting an app on a data root + * the platform cannot vouch for, and that is the right answer in both cases — + * but they describe different states and have different recoveries, so they + * are different codes. + * + * `restoreStartedAt` is the discriminator: it is persisted immediately before + * the extraction that wipes and rewrites the root, so its presence means THIS + * install's own restore already wrote here and then failed somewhere after. + * Absent, the data predates this install's restore entirely. + * + * Two REJECTED alternatives, deliberately not implemented — do not re-propose + * them: + * + * - Persisting an attempted-marker (or `restoredAt`) before extraction so a + * RETRY takes the no-restore path and brings the app up on whatever landed. + * A genuinely half-extracted tree would then start: the silent-empty-app + * failure FR-022 exists to forbid, and the worst outcome named in the + * spec's Executive Summary. `restoreStartedAt` here is a diagnostic only — + * it changes the WORDS of the refusal, never the fact of it. + * - Letting a retry re-run the post-extraction steps only (discards, marker + * rewrite, hooks). Nothing can distinguish "extraction completed and a + * discard failed" from "extraction died halfway", so re-running discards + * and hooks risks operating on a partial tree — the same failure by a + * longer route. + */ +export function classifyNonEmptyTarget(input: { + deploymentId: string; + deploymentName: string; + restoreStartedAt: string | undefined; +}): { code: RestoreRefusalCode; message: string; details: Record } { + if (input.restoreStartedAt) { + return { + code: 'RESTORE_INCOMPLETE', + message: + `Cannot restore into '${input.deploymentName}': its own restore already wrote data here at ` + + `${input.restoreStartedAt} and then failed, so the data root holds a partially restored copy. ` + + `That data is still on disk and is NOT safe to start against. Uninstall and reinstall this app ` + + `to retry the restore, or clear its data root first if you want to keep this install.`, + details: { deploymentId: input.deploymentId, restoreStartedAt: input.restoreStartedAt }, + }; + } + return { + code: 'RESTORE_TARGET_NOT_EMPTY', + message: `Cannot restore into '${input.deploymentName}': its data root already holds app data.`, + details: { deploymentId: input.deploymentId }, + }; } export interface RestoreValidationInput { diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index 66f4cb5f..21ad1c2f 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -399,8 +399,21 @@ export type RestoreRefusalCode = | 'RESTORE_CANDIDATE_GONE' | 'RESTORE_CANDIDATE_BUSY' | 'RESTORE_TARGET_NOT_EMPTY' + // Distinct from RESTORE_TARGET_NOT_EMPTY on purpose (#489). That one means + // "something ELSE put data in this root" — an operator pre-seeded it, or an + // uninstall left it behind — and the data is not ours. This one means "THIS + // install's own restore already wrote here and then failed": the data IS the + // half-landed payload, and the recovery differs (uninstall + reinstall, or + // clear the data root, then retry). One code for both would hand every + // surface a recovery that is wrong for half the cases it fires on. + | 'RESTORE_INCOMPLETE' | 'RESTORE_PAYLOAD_EMPTY' | 'RESTORE_HOOK_FAILED' + // FR-035's address default resolved onto an address the chosen candidate + // still owns (#490). Refused here, naming the candidate, rather than left to + // fall through to the routing layer's bare host conflict, which says nothing + // about restore and gives a non-interactive caller nothing to act on. + | 'RESTORE_ADDRESS_REQUIRED' | 'RESTORE_NOT_SUPPORTED' // Distinct from RESTORE_NOT_SUPPORTED on purpose: that one means "this // INSTALL PATH cannot restore" (install-by-ref, no catalog index), and its @@ -2221,6 +2234,15 @@ export type EnhancedDeploymentDetail = DeploymentDetail & { // The restore choice that was applied to this install's first deploy, carried // from the finalized manifest. Absent when this install was not restored. restoreFrom?: RestoreChoice; + // ISO time the restore began WRITING — persisted immediately before the + // extraction that `rm -rf`s the target data root (#489). It is purely a + // diagnostic: it never changes which path a later job takes (the gate stays + // `restoreFrom && !restoredAt && !previousReleaseId`), it only lets the + // refusal that a retry hits say WHICH non-empty-target case it is — this + // install's own half-landed payload (`RESTORE_INCOMPLETE`) rather than data + // something else put there (`RESTORE_TARGET_NOT_EMPTY`). Server-side only: + // not projected onto `DeploymentDetail`. + restoreStartedAt?: string; // ISO time the restore completed successfully. Its presence is the // consumption marker: a restart/promote/rollback finds it set and skips the // restore sequence — a restore applies to the first deploy only. diff --git a/specs/007-restore-on-install/contracts/api.md b/specs/007-restore-on-install/contracts/api.md index 0dc40faa..f6424acb 100644 --- a/specs/007-restore-on-install/contracts/api.md +++ b/specs/007-restore-on-install/contracts/api.md @@ -121,13 +121,27 @@ through `mergeUpgradeAppEnv`; `name` and `subdomain` default from the candidate | `RESTORE_UPGRADE_PATH` | + `suggestedVersion` | | `RESTORE_ENV_REQUIRED` | + `missingKeys[]` | | `RESTORE_ACK_REQUIRED` | + `required[]` | - -**Three codes are deliberately absent from this table.** -`RESTORE_TARGET_NOT_EMPTY`, `RESTORE_PAYLOAD_EMPTY` and `RESTORE_HOOK_FAILED` -(data-model.md §4) are reachable only inside the deploy job, after the create call -has returned. They surface on the deployment's error state and in the job log, not -in any HTTP response body. `RESTORE_CANDIDATE_GONE` and `RESTORE_CANDIDATE_BUSY` -appear in both places, because the job re-resolves the candidate (FR-013a). +| `RESTORE_ADDRESS_REQUIRED` | FR-035's default would land on the address the candidate still routes under; + `candidateId`, `candidateName`, `subdomain` | + +**`RESTORE_ADDRESS_REQUIRED` (#490).** With no `name` in the request, FR-035 +defaults the new install's address to the candidate's own — and the candidate, +being an existing deployment on this host, still owns it. That default is +refused here, with the candidate named, rather than allowed to reach +`routingService.validateRule` and come back as a bare host `CONFLICT` carrying +no `details.code` and no mention of restore (FR-037). It fires **only** when the +caller supplied no `name`: an operator who names the install has made the +address decision and gets the routing layer's own conflict, which names the +owning deployment. The server never derives a distinct or suffixed slug on the +operator's behalf — that would put the restored data, full of the old address's +absolute URLs, at a new address nobody chose. + +**Four codes are deliberately absent from this table.** +`RESTORE_TARGET_NOT_EMPTY`, `RESTORE_INCOMPLETE`, `RESTORE_PAYLOAD_EMPTY` and +`RESTORE_HOOK_FAILED` (data-model.md §4) are reachable only inside the deploy +job, after the create call has returned. They surface on the deployment's error +state and in the job log, not in any HTTP response body. +`RESTORE_CANDIDATE_GONE` and `RESTORE_CANDIDATE_BUSY` appear in both places, +because the job re-resolves the candidate (FR-013a). --- diff --git a/specs/007-restore-on-install/contracts/cli.md b/specs/007-restore-on-install/contracts/cli.md index a20922d5..384a39fc 100644 --- a/specs/007-restore-on-install/contracts/cli.md +++ b/specs/007-restore-on-install/contracts/cli.md @@ -80,6 +80,18 @@ Mapping from `details.code` to the hint, one row per code | `RESTORE_CANDIDATE_GONE` / `_BUSY` | suggest `--restore-list` to re-read the current set | | `RESTORE_NOT_SUPPORTED` | install-by-ref cannot restore; use the catalog path | | `RESTORE_NOT_ACCEPTED` | this app has not declared it can be restored; install fresh | +| `RESTORE_ADDRESS_REQUIRED` | `candidateName`, `subdomain` → name what still holds the address, ask for `--name ` | + +The table lists every code a **create call** can return (api.md §2). The four +job-time codes — `RESTORE_TARGET_NOT_EMPTY`, `RESTORE_INCOMPLETE`, +`RESTORE_PAYLOAD_EMPTY`, `RESTORE_HOOK_FAILED` (data-model.md §4) — have no row +and no hint branch: they arrive after the create returned, as a **failed job**, +which `hola install` reports by exit code and streamed log, never as a +`HolaApiError` carrying `details`. A hint branch for one of them would be +unreachable, and building it from the job's message text instead is exactly the +prose-sniffing this mapping exists to avoid. Their recovery therefore lives in +the server's own refusal message — which is why `RESTORE_INCOMPLETE` spells out +"uninstall and reinstall, or clear the data root" in the message itself (#489). ## Unchanged diff --git a/specs/007-restore-on-install/data-model.md b/specs/007-restore-on-install/data-model.md index fcb769e3..dd9beb7d 100644 --- a/specs/007-restore-on-install/data-model.md +++ b/specs/007-restore-on-install/data-model.md @@ -140,23 +140,44 @@ rather than prose (FR-037, and `deploy-flow.ts:137-155`'s established rule). | `RESTORE_ENV_REQUIRED` | `restore.requiresEnv` and no environment record | `missingKeys[]` | | `RESTORE_CANDIDATE_GONE` | Candidate deleted between choice and deploy | `candidateId` | | `RESTORE_CANDIDATE_BUSY` | Candidate not in a settled state | `candidateId`, `status` | -| `RESTORE_TARGET_NOT_EMPTY` | Target data root already holds app data | `deploymentId` | +| `RESTORE_TARGET_NOT_EMPTY` | Target data root already holds app data **something else put there** | `deploymentId` | +| `RESTORE_INCOMPLETE` | This install's own restore wrote into the root and then failed (#489) | `deploymentId`, `restoreStartedAt` | | `RESTORE_PAYLOAD_EMPTY` | Post-condition failed: nothing landed (FR-016) | `deploymentId` | | `RESTORE_HOOK_FAILED` | A restore hook failed or its service never became healthy | `participationId`, `service` | | `RESTORE_NOT_SUPPORTED` | `restoreFrom` supplied on the install-by-ref path | — | | `RESTORE_NOT_ACCEPTED` | The target app declares no `restore@1` in `accepts` | `appId` | | `RESTORE_ACK_REQUIRED` | A required acknowledgement code was absent | `required[]` | - -**Create-time vs job-time.** Seven of these can be returned synchronously from a +| `RESTORE_ADDRESS_REQUIRED` | FR-035's default would land on the address the candidate still routes under (#490) | `candidateId`, `candidateName`, `subdomain` | + +**`RESTORE_TARGET_NOT_EMPTY` vs `RESTORE_INCOMPLETE` (#489).** Both are FR-014 +refusing to write into a data root that already holds app data, and both stay +refusals — FR-022 forbids starting an app on a tree the platform cannot vouch +for. They differ in what the data *is* and therefore in the recovery, so they +are different codes. The discriminator is `deployment.restoreStartedAt`, an ISO +timestamp persisted immediately **before** the extraction that wipes and +rewrites the root: present, this install's own restore half-landed here +(`RESTORE_INCOMPLETE`, recovery = uninstall + reinstall, or clear the data +root); absent, the data predates this install's restore (`RESTORE_TARGET_NOT_EMPTY`). +`restoreStartedAt` is a **diagnostic only** — the job-entry gate stays +`restoreFrom && !restoredAt && !previousReleaseId`, so a retry still enters the +restore sequence and still refuses. Two designs that would instead let a retry +proceed were considered and rejected: persisting an attempted-marker so the +retry takes the no-restore path, and re-running only the post-extraction steps. +Both can start an app on a half-extracted tree, which is the silent-empty-app +outcome the Executive Summary names as the worst in this feature. + +**Create-time vs job-time.** Nine of these can be returned synchronously from a draft-create or deployment-create call, so a client sees them in a `409` body. -Three cannot: `RESTORE_TARGET_NOT_EMPTY`, `RESTORE_PAYLOAD_EMPTY` and -`RESTORE_HOOK_FAILED` are only reachable **inside the deploy job**, long after the -request returned. They surface on the deployment's error state and in the job log, -never in a create response — which is why they are absent from the error tables in -`contracts/api.md` and from the CLI's hint mapping in `contracts/cli.md`. The -union is one union; the delivery channel differs. `RESTORE_CANDIDATE_GONE` and -`RESTORE_CANDIDATE_BUSY` are the two that occur in **both** places, because the -job re-resolves the candidate (FR-013a). +Four cannot: `RESTORE_TARGET_NOT_EMPTY`, `RESTORE_INCOMPLETE`, +`RESTORE_PAYLOAD_EMPTY` and `RESTORE_HOOK_FAILED` are only reachable **inside +the deploy job**, long after the request returned. They surface on the +deployment's error state and in the job log, never in a create response — which +is why they are absent from the error tables in `contracts/api.md` and from the +CLI's hint mapping in `contracts/cli.md`. The union is one union; the delivery +channel differs. `RESTORE_CANDIDATE_GONE` and `RESTORE_CANDIDATE_BUSY` are the +two that occur in **both** places, because the job re-resolves the candidate +(FR-013a). `RESTORE_ADDRESS_REQUIRED` is create-time only: it is raised while +the deployment's address is being resolved, before any state is created. --- @@ -186,6 +207,7 @@ migration. |---|---|---|---|---| | `lineageId` | `string?` | `restoreFrom` ? candidate's `lineageId` : `deployment.id` | yes | **Reading it is the change spec 006 predicted.** `writeInstanceMarkers` becomes `deployment.lineageId ?? deployment.id` (`deployment.ts:3374`). The fallback is what makes this zero-migration: an older record reads `undefined` and yields exactly the value it always had. | | `restoreFrom` | `RestoreChoice?` | the finalized manifest | yes | Persisted beside `channel` (`deployment.ts:1028`), **not** in the job payload (R3). | +| `restoreStartedAt` | `string?` | set **and flushed** by the job immediately before extraction — ISO 8601 | yes | Diagnostic only (#489), never a gate. Persisted before the first write so it survives a hard kill mid-extraction; read only by `classifyNonEmptyTarget` to tell `RESTORE_INCOMPLETE` from `RESTORE_TARGET_NOT_EMPTY`. Not projected onto `DeploymentDetail`. | | `restoredAt` | `string?` | set by the job on success — ISO 8601 | yes | Consumption marker. Its presence is what makes FR-012 enforceable: a restart/promote/rollback finds it set and skips. | **Why the record and not the payload** (R3): the deploy payload is @@ -241,11 +263,17 @@ createFromDraft ─> re-validated (the candidate may have changed), acknowledgem deploy job ────> if restoreFrom && !restoredAt: assert target empty · re-resolve candidate · quiesce + capture - · extract · assert payload present · discard · rewrite marker - · write OIDC file · up --wait · run hooks + · persist restoreStartedAt · extract · assert payload present + · discard · rewrite marker · write OIDC file + · up --wait · run hooks then set restoredAt later actions ─> restoredAt is set ⇒ skip. The restore is consumed exactly once. + +retry after a ─> restoredAt still unset ⇒ the sequence is re-entered and the +failed restore "assert target empty" step refuses again — now as + RESTORE_INCOMPLETE, because restoreStartedAt is set, naming + the recovery instead of a bare "already holds app data". ``` **Two independent guards enforce FR-012**: the action must be the deployment's diff --git a/specs/007-restore-on-install/quickstart.md b/specs/007-restore-on-install/quickstart.md index d3f70898..4a2e272e 100644 --- a/specs/007-restore-on-install/quickstart.md +++ b/specs/007-restore-on-install/quickstart.md @@ -68,6 +68,7 @@ cheapest (one Postgres, one app container, a declared healthcheck on both). |---|---|---|---| | 15 | The restore runs after `composePull` and before `composeUp`. Assert on call ordering through the Mock docker service. | U | FR-013 | | 16 | A target data root that already holds app data aborts with `RESTORE_TARGET_NOT_EMPTY`. A root holding only `.hola/` proceeds — the marker is not app data. | U-fs | FR-014 | +| 16b | A restore that fails **after** extraction leaves `restoreStartedAt` persisted; a later `start` re-enters the sequence and refuses with `RESTORE_INCOMPLETE` — naming the half-landed state and the recovery — rather than the generic `RESTORE_TARGET_NOT_EMPTY`. It still refuses: the app is never started on a partial tree (#489). | U-fs | FR-014, FR-022, FR-037 | | 16a | A candidate deleted, or moved out of a settled state, between the draft and the deploy job fails the install with `RESTORE_CANDIDATE_GONE` / `RESTORE_CANDIDATE_BUSY` — the job re-resolves rather than trusting the draft. | U-fs | FR-013a | | 17 | The source's pre-hooks run before the capture, its post-hooks run after, and after the restore the source's data, status and address are unchanged and it is still running. | VM | FR-015, SC-006, US1-AC5 | | 18 | After a restore the target data root holds the source's files. With the archive emptied, the install fails with `RESTORE_PAYLOAD_EMPTY` rather than reporting success. **No subtree search** — the post-condition is the mechanism (research R9). | U-fs | FR-016 | @@ -107,6 +108,7 @@ cheapest (one Postgres, one app container, a declared healthcheck on both). | 38 | With no environment record, the warning names exactly the keys that are `isSecret` **and** carry a `generate` recipe — not every secret, not every generated value. | U-fs | FR-033, US2-AC3 | | 39 | With `requiresEnv: true` and no environment record, the restore is **refused** (`RESTORE_ENV_REQUIRED`), not warned. | U | FR-034, US3-AC4 | | 40 | Name and subdomain default from the candidate; changing the subdomain produces a `host-divergence` warning naming both. | U | FR-035, US1-AC1 | +| 40a | With no `name` supplied, the FR-035 default lands on the address the candidate still routes under and is refused with `RESTORE_ADDRESS_REQUIRED` naming the candidate — never a bare routing `CONFLICT`, and never an auto-suffixed slug the operator did not choose (#490). | U | FR-035, FR-037 | | 41 | Every refusal carries `details.code`; every version refusal that has a next step carries `suggestedVersion`. | U | FR-037, SC-005 | | 42 | A required acknowledgement code that is absent fails the create with `RESTORE_ACK_REQUIRED` — the same shape as a missing `grant`. | U | FR-037a, FR-046 | | 43 | Declining to carry configuration that **is** available still produces the named warning and still requires the acknowledgement. | U-fs | FR-033, US2-AC4 | From 05f64ef2db3acb0aa28984da578c25a73863b4cd Mon Sep 17 00:00:00 2001 From: Paul O'Fallon Date: Sun, 20 Sep 2026 20:49:07 +0000 Subject: [PATCH 4/5] fix(deploy): state composeUp's timeout at every call site, not by omission (#487) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec 007 turned composeUp's hardcoded 300000ms into a parameter with a default, which left every other call site on a five-minute cap nobody had ever chosen for it. Audited all three sites in deployment.ts: - the restore-hook `--wait` (research R12) already passes 15 minutes — unchanged; - the deploy/start/rollback bring-up and the restart recreate now pass COMPOSE_UP_TIMEOUT_MS (15 minutes) explicitly, with the reason at the call site. Five minutes is too thin for what these actually do. `up -d` is not just container creation: Compose blocks on every `depends_on: service_healthy` gate, so a first install of a large multi-service stack waits out an `initdb` plus each dependency's healthcheck — the same cost R12 measured for the restore hook, which is why this reuses that number rather than inventing a third. A rollback that just replaced the data directory and a restore-on-install whose database just received a large dump start equally cold. The restart path is additionally the one composeUp with no composePull in front of it, so a recreate that must fetch a pruned image was running under a cap six times tighter than the one composePull was given for exactly that reason. The ceiling only delays reporting a wedged `up`; too low SIGKILLs a slow-but-correct install. composeUp's own 300000ms stays as a fallback for a caller with no opinion and is commented as such. Closes #487 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh --- .../__tests__/deployments/lifecycle.test.ts | 26 ++++++++++ .../server/src/services/core/deployment.ts | 49 ++++++++++++++++++- packages/server/src/services/core/docker.ts | 21 +++++--- 3 files changed, 87 insertions(+), 9 deletions(-) diff --git a/packages/server/src/__tests__/deployments/lifecycle.test.ts b/packages/server/src/__tests__/deployments/lifecycle.test.ts index 7918d76c..97f62435 100644 --- a/packages/server/src/__tests__/deployments/lifecycle.test.ts +++ b/packages/server/src/__tests__/deployments/lifecycle.test.ts @@ -309,6 +309,32 @@ describe('Deployment lifecycle (real orchestration wiring)', () => { expect(upProfiles).toEqual([['metrics']]); }); + test('every composeUp call site states its own timeout rather than inheriting the fallback (#487)', async () => { + // `composeUp`'s 300000ms is a FALLBACK for a caller with no opinion, not a + // ceiling anyone chose for a deploy or a restart. Both lifecycle call sites + // must therefore pass one explicitly: `up -d` blocks on each + // `depends_on: service_healthy` gate, so a first install of a large stack + // (or a restart whose recreate has to fetch a pruned image) can legitimately + // outlast five minutes, and a silent SIGKILL there fails a correct install. + const docker = new MockDockerService(); + const { jobs, drafts, deployments } = makeSystem(docker); + + const created = await deployments.createFromDraft({ draftId: await finalizedDraft(drafts), name: 'gitea' }); + await waitForJob(jobs, created.jobId!); + + const restart = await deployments.executeAction(created.deploymentId, { action: 'restart' }); + expect((await waitForJob(jobs, restart.jobId!)).status).toBe('completed'); + + // One `up` for the install, one for the restart — and neither omitted the + // timeout. Asserting "defined" AND the value keeps the guard honest if the + // constant is ever retuned: the point is that a number was chosen here. + expect(docker.composeUpCalls).toHaveLength(2); + for (const call of docker.composeUpCalls) { + expect(call.timeoutMs).toBeDefined(); + expect(call.timeoutMs).toBe(900_000); + } + }); + test('lifecycle logs are streamed to the deployment log target', async () => { const sys = makeSystem(); const lines: string[] = []; diff --git a/packages/server/src/services/core/deployment.ts b/packages/server/src/services/core/deployment.ts index b182133b..b44b67fa 100644 --- a/packages/server/src/services/core/deployment.ts +++ b/packages/server/src/services/core/deployment.ts @@ -148,6 +148,32 @@ const DEFAULT_APPS_BIND_ROOT = '/srv/hola/apps'; */ const RESTORE_HOOK_WAIT_TIMEOUT_MS = 900_000; // 15 minutes +/** + * `composeUp`'s `execFile` ceiling for a full bring-up (#487). Stated at every + * call site rather than inherited, because `composeUp`'s own 300000ms default + * was never a per-call-site decision — it was whatever the one implementation + * happened to hardcode before spec 007 made it a parameter. + * + * Five minutes is too thin for the bring-up this codebase actually issues. + * `up -d` does not just create containers: Compose blocks on every + * `depends_on: { condition: service_healthy }` gate before starting the + * dependent service, so a first install of a large multi-service stack (Postiz + * is the known worst case) waits out an `initdb` plus each dependency's own + * healthcheck — the SAME cost research R12 measured for the restore hook's + * `--wait`, which is why this deliberately agrees with that number rather than + * inventing a third one. A rollback that just replaced the data directory, and + * a restore-on-install whose database was handed a large dump moments earlier, + * start just as cold. + * + * This is a ceiling, not a wait: a healthy stack returns as soon as Compose + * does. The asymmetry decides the number — too low SIGKILLs a slow-but-correct + * install (the failure `composePull`'s own 30-minute ceiling was raised to + * stop), while too high only delays reporting a genuinely wedged `up`, and the + * job queue already tolerates exactly that for the 30-minute `composePull` a + * deploy runs immediately beforehand. + */ +const COMPOSE_UP_TIMEOUT_MS = 900_000; // 15 minutes + /** * Reserved locations for this feature's platform-authored JSON records (spec * 006). There are TWO, at two different levels under the apps bind root, and @@ -4230,7 +4256,17 @@ export class RealDeploymentService extends InMemoryDeploymentService { // init container is deliberately left exited rather than re-run. const before = await this.composeStateById(composeDir, projectName); const registryAuth = await this.resolveRegistryAuth(deployment); - const res = await this.dockerService.composeUp(composeDir, projectName, registryAuth, deployment.selectedProfiles); + // Timeout stated, not inherited (#487). A restart recreates whatever + // the freshly materialized compose changed and clears the same + // `depends_on: service_healthy` gates a deploy does, only against warm + // data. It is also the ONE `composeUp` with no `composePull` in front + // of it — the registryAuth above exists precisely so a recreate that + // must fetch a missing image (a pruned host) still authenticates, and + // that pull would otherwise run under a cap six times tighter than the + // one `composePull` was given for the very same reason. + const res = await this.dockerService.composeUp(composeDir, projectName, registryAuth, deployment.selectedProfiles, { + timeoutMs: COMPOSE_UP_TIMEOUT_MS, + }); output = res.output; if (!res.success) throw new Error(res.output); const restarted = await this.restartUntouchedServices(composeDir, projectName, before, deployment.selectedProfiles, logBoth); @@ -4292,7 +4328,16 @@ export class RealDeploymentService extends InMemoryDeploymentService { await this.performRestoreOnInstall(deployment, composeDir, projectName, registryAuth, provisioned, logBoth); } - const res = await this.dockerService.composeUp(composeDir, projectName, registryAuth, deployment.selectedProfiles); + // Timeout stated, not inherited (#487). This is the first install of a + // large multi-service stack: the images are already local (the pull + // above saw to that), so what remains is `initdb` and every + // `depends_on: service_healthy` gate in the app's compose — the + // expensive part, and the part five minutes does not reliably cover. + // A rollback that just restored a data snapshot, and a restore that + // just extracted one, reach this line equally cold. + const res = await this.dockerService.composeUp(composeDir, projectName, registryAuth, deployment.selectedProfiles, { + timeoutMs: COMPOSE_UP_TIMEOUT_MS, + }); output = res.output; if (!res.success) throw new Error(res.output); if (provisioned) await this.completeAuthWiring(deployment, provisioned, projectName, logBoth); diff --git a/packages/server/src/services/core/docker.ts b/packages/server/src/services/core/docker.ts index 88b24eae..fb647586 100644 --- a/packages/server/src/services/core/docker.ts +++ b/packages/server/src/services/core/docker.ts @@ -69,8 +69,12 @@ export interface DockerService { * the project, today's behaviour). `options.wait` adds `--wait`, blocking until * each named service's own `healthcheck` reports healthy — used by the restore * sequence to start only a hook's service before running its hook (spec 007, - * FR-020). `options.timeoutMs` overrides the default 5-minute `execAsync` cap, - * because a `--wait` on a freshly-`initdb`'d Postgres can exceed it (research R12). + * FR-020). `options.timeoutMs` is the `execFile` ceiling; the 5-minute fallback + * below applies only to a caller with no opinion, and every `deployment.ts` call + * site now states its own (#487) — `up -d` blocks on each + * `depends_on: service_healthy` gate, so a first install, a cold rollback, and a + * `--wait` on a freshly-`initdb`'d Postgres (research R12) can all legitimately + * exceed five minutes. */ composeUp( projectPath: string, @@ -262,10 +266,13 @@ export class RealDockerService implements DockerService, HealthCheckable { // Images are pre-pulled by composePull, so `up` only starts local images. // `--wait` (when requested) blocks until every NAMED service reports // healthy via its own declared healthcheck — no bespoke readiness poll - // (Constitution V). `options.timeoutMs` overrides the default 5-minute - // cap: a `--wait` against a freshly-`initdb`'d Postgres can exceed it - // (research R12). A scoped DOCKER_CONFIG is passed as a fallback so a - // recreate that needs to pull still authenticates. + // (Constitution V). `options.timeoutMs` is the caller's own ceiling: a + // `--wait` against a freshly-`initdb`'d Postgres can exceed five minutes + // (research R12), and so can a plain `up -d` that has to clear a large + // stack's `depends_on: service_healthy` gates. The 300000ms below is a + // FALLBACK for a caller that expressed no preference, not a considered + // ceiling for any operation (#487). A scoped DOCKER_CONFIG is passed as + // a fallback so a recreate that needs to pull still authenticates. // // Built as an argv array and run through `execFile` (NO shell), the same // rule `composeExec` already states: `options.services` carries @@ -276,7 +283,7 @@ export class RealDockerService implements DockerService, HealthCheckable { const args = ['compose', '-f', composeFile, '-p', projectName, 'up', '-d']; if (options?.wait) args.push('--wait'); if (options?.services?.length) args.push(...options.services); - const timeout = options?.timeoutMs ?? 300000; // 5 minute default + const timeout = options?.timeoutMs ?? 300000; // 5 minute fallback (see above) const { stdout, stderr } = await execFileAsync('docker', args, { cwd: projectPath, timeout, env }); const output = [stdout, stderr].filter(Boolean).join('\n'); From 73d6ce979ea9a40862e14a0db2eef456512ef673 Mon Sep 17 00:00:00 2001 From: Paul O'Fallon Date: Sun, 20 Sep 2026 20:49:21 +0000 Subject: [PATCH 5/5] fix(cli): stop --carry-env swallowing the next positional (#488) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `hola install --carry-env gitea --restore-from latest` aborted with sade's "Insufficient arguments!": mri classifies a flag as boolean only by `typeof opts.default[key]`, and `--carry-env` is registered without a default, so mri treated it as value-taking and ate `gitea`. It cannot simply take a `false` default — `carryEnv` is tri-state (contracts/cli.md): `undefined` means "default to the candidate's own `carriesEnv`", `true`/`false` are explicit operator choices, and a `false` default collapses the first into the second. Normalising after the fact, the way `streamOpts` handles `--no-stream`, cannot help either: the token is consumed during parsing, before any handler sees an opts bag. So classify it at the parse layer instead. `parseOpts()` names `carry-env` in mri's `boolean` list (sade forwards the object verbatim, overriding only `alias`/`default`), which makes it boolean WITHOUT giving it a default — and mri's boolean branch pushes the token it looked at back onto `_`, so the positional is handed back rather than lost. Tri-state is preserved exactly: omitted → undefined, `--carry-env` → true, `--no-carry-env` and `--carry-env=false` → false. No documented behaviour changed, so contracts/cli.md is untouched. Tests parse the real flag shapes through sade rather than asserting on a hand-built opts bag, since the bug lives in parsing: the issue's exact repro, both flag positions, the `=`-form, the omitted case, and a guard that index.ts actually hands `parseOpts()` to `prog.parse`. Closes #488 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh --- packages/cli/src/__tests__/opts.test.ts | 94 ++++++++++++++++++++++++- packages/cli/src/index.ts | 11 ++- packages/cli/src/lib/opts.ts | 32 +++++++++ 3 files changed, 134 insertions(+), 3 deletions(-) diff --git a/packages/cli/src/__tests__/opts.test.ts b/packages/cli/src/__tests__/opts.test.ts index 1afb02ff..7af3c0ad 100644 --- a/packages/cli/src/__tests__/opts.test.ts +++ b/packages/cli/src/__tests__/opts.test.ts @@ -1,6 +1,9 @@ import { describe, it, expect } from 'vitest'; +import { readFileSync } from 'fs'; +import { join } from 'path'; +import sade from 'sade'; -import { camelKeys, streamOpts } from '../lib/opts'; +import { camelKeys, parseOpts, streamOpts } from '../lib/opts'; describe('camelKeys', () => { it('camelCases kebab-case flag keys (the sade/mri multi-word fix)', () => { @@ -39,3 +42,92 @@ describe('streamOpts (--no-stream normalization)', () => { expect(streamOpts({ noStream: true }).noStream).toBe(true); }); }); + +/** + * `--carry-env` is tri-state, so it is the one flag that cannot be registered + * with a `false` default — and that default is what would otherwise put its + * name in mri's boolean list. Without it mri treated the flag as value-taking + * and swallowed the next positional (#488). These parse the real flag shapes + * through sade + `parseOpts()` rather than asserting on a hand-built opts bag, + * because the bug lives in parsing, not in normalization. + */ +describe('parseOpts (--carry-env tri-state, #488)', () => { + /** Mirrors `install`'s restore-flag registration in src/index.ts. */ + const installProg = () => + sade('hola') + .command('install ') + .option('--restore-from', 'Restore from an existing deployment of this app') + .option('--no-restore', 'Explicitly install with no restore', false) + .option('--restore-list', 'List restore candidates and exit', false) + // No default — the tri-state flag under test. + .option('--carry-env', 'Carry the restore candidate\'s configuration') + .option('--no-carry-env', 'Do not carry the restore candidate\'s configuration', false) + .option('--ack', 'Acknowledge a restore risk by code') + .action(() => {}); + + /** Parse an `hola …` argv the way index.ts does, without running a handler. */ + function parseInstall(argv: string[]) { + const parsed = installProg().parse(['node', 'hola', ...argv], { ...parseOpts(), lazy: true }) as + | { args: unknown[] } + | undefined; + if (!parsed) throw new Error('sade refused the argv'); // e.g. "Insufficient arguments!" + const args = parsed.args; + return { + appId: args[0] as string, + opts: camelKeys(args[args.length - 1] as Record), + }; + } + + it('does not swallow the positional after a bare --carry-env (the issue repro)', () => { + // `hola install --carry-env gitea --restore-from latest` used to abort with + // sade's "Insufficient arguments!" because mri assigned carryEnv: 'gitea'. + const { appId, opts } = parseInstall(['install', '--carry-env', 'gitea', '--restore-from', 'latest']); + expect(appId).toBe('gitea'); + expect(opts.carryEnv).toBe(true); + expect(opts.restoreFrom).toBe('latest'); + }); + + it('still reads --carry-env when it follows the positional', () => { + const { appId, opts } = parseInstall(['install', 'gitea', '--carry-env', '--restore-from', 'latest']); + expect(appId).toBe('gitea'); + expect(opts.carryEnv).toBe(true); + }); + + it('reads --no-carry-env as an explicit false, in either position', () => { + expect(parseInstall(['install', '--no-carry-env', 'gitea']).opts.carryEnv).toBe(false); + const trailing = parseInstall(['install', 'gitea', '--no-carry-env', '--ack', 'restore-env-not-carried']); + expect(trailing.appId).toBe('gitea'); + expect(trailing.opts.carryEnv).toBe(false); + expect(trailing.opts.ack).toBe('restore-env-not-carried'); + }); + + it('leaves carryEnv undefined when neither flag is given — NOT false', () => { + // The whole constraint: `undefined` means "default to the candidate's own + // carriesEnv" (contracts/cli.md). Collapsing it to `false` would silently + // stop carrying configuration on every restore that did not ask. + const { appId, opts } = parseInstall(['install', 'gitea', '--restore-from', 'latest']); + expect(appId).toBe('gitea'); + expect(opts.carryEnv).toBeUndefined(); + }); + + it('reads an explicit --carry-env=false as false, not as truthy "false"', () => { + expect(parseInstall(['install', 'gitea', '--carry-env=false']).opts.carryEnv).toBe(false); + expect(parseInstall(['install', 'gitea', '--carry-env=true']).opts.carryEnv).toBe(true); + }); + + it('returns a fresh object each call (sade and mri both mutate what they get)', () => { + const a = parseOpts(); + const b = parseOpts(); + expect(a).not.toBe(b); + expect(a.boolean).not.toBe(b.boolean); + }); + + it('src/index.ts actually hands parseOpts() to prog.parse, for every flag it names', () => { + // The mirror above only proves the mechanism; this proves the real CLI uses it. + const src = readFileSync(join(__dirname, '../index.ts'), 'utf8'); + expect(src).toMatch(/prog\.parse\(process\.argv,\s*parseOpts\(\)\)/); + for (const name of parseOpts().boolean) { + expect(src).toContain(`.option('--${name}'`); + } + }); +}); diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index ea5bb67f..67c555f2 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -2,7 +2,7 @@ import sade from 'sade'; import { CLI_VERSION } from './version'; -import { camelKeys, streamOpts } from './lib/opts'; +import { camelKeys, parseOpts, streamOpts } from './lib/opts'; // Lazy command loaders to avoid unnecessary dependencies until invoked const load = async (p: Promise): Promise => p; @@ -164,6 +164,10 @@ prog .option('--restore-from', 'Restore from an existing deployment of this app: a deployment id, or "latest" (refuses across two+ unrelated lineages)') .option('--no-restore', 'Explicitly install with no restore (the default when no restore flag is given at all)', false) .option('--restore-list', 'List restore candidates for this app and exit — installs nothing', false) + // Deliberately DEFAULTLESS, unlike every other boolean flag here: `carryEnv` + // is tri-state and a `false` default would collapse "not asked" into "no" + // (#488). The `parseOpts()` handed to `prog.parse` at the bottom of this file + // is what stops mri treating it as value-taking; don't add a default here. .option('--carry-env', 'Carry the restore candidate\'s configuration (default: on when it has an environment record)') .option('--no-carry-env', 'Do not carry the restore candidate\'s configuration (needs --ack restore-env-not-carried)', false) .option('--ack', 'Acknowledge a restore risk by code, e.g. restore-env-not-carried (repeatable, or comma-separated)') @@ -349,5 +353,8 @@ if (process.argv.length <= 2) { console.log('Hola CLI. New here? Set up a server: hola bootstrap --host user@vm'); console.log('Installed? Try: hola catalog · hola install · hola deployments'); } else { - prog.parse(process.argv); + // `parseOpts()` classifies the tri-state `--carry-env` as a boolean without + // giving it a default, which a `.option(…, false)` registration cannot do + // (#488). See lib/opts.ts. + prog.parse(process.argv, parseOpts()); } diff --git a/packages/cli/src/lib/opts.ts b/packages/cli/src/lib/opts.ts index f09bbd88..c77fe74b 100644 --- a/packages/cli/src/lib/opts.ts +++ b/packages/cli/src/lib/opts.ts @@ -10,6 +10,38 @@ export const camelKeys = >(opts: T): T => { return out as T; }; +/** + * Extra mri options for `prog.parse` — sade forwards this object to mri + * verbatim, overriding only its `alias` and `default` keys, so `boolean` is + * ours. Returns a FRESH object per call because both sade and mri mutate what + * they are handed. + * + * It exists for one flag. `--carry-env` is the CLI's only TRI-STATE option + * (spec 007, contracts/cli.md): `undefined` means "default to the candidate's + * own `carriesEnv`", `true`/`false` are explicit operator choices. Every other + * boolean flag declares a `false` default, and that default is the *only* + * thing that lands a name in mri's boolean list — mri classifies by + * `typeof opts.default[key]`. `--carry-env` cannot take a `false` default + * without collapsing `undefined` into `false`, so it was classified as + * value-taking and greedily ate the following token: + * `hola install --carry-env gitea --restore-from latest` consumed `gitea` and + * aborted with sade's "Insufficient arguments!" (#488). + * + * Naming it here classifies it as a boolean WITHOUT giving it a default, which + * is the whole point — mri's boolean branch also pushes the token it looked at + * back onto `_`, so the swallowed positional is handed back rather than lost: + * + * (omitted) → undefined (mri's defaults pass writes sade's own `undefined`) + * --carry-env → true + * --carry-env gitea → true, and `gitea` returns to the positionals + * --no-carry-env → false (mri's `--no-` branch) + * --carry-env=false → false (mri reads the literal 'false'/'true') + * + * Normalizing after the fact — the `streamOpts` shape below — cannot fix this: + * the token is consumed during parsing, before any handler sees an opts bag. + */ +export const parseOpts = (): { boolean: string[] } => ({ boolean: ['carry-env'] }); + /** * camelKeys + normalize the `--no-stream` and `--no-generate-secrets` flags. * sade/mri routes `--no-stream` to `{ stream: false }` (and `--no-generate-secrets`