feat(deploy): restore-on-install from a live deployment (spec 007) - #491
Conversation
During an app install the operator may name an existing deployment of the same app on this host as a restore source. The server quiesces it with the backup contract's existing pre-hooks, captures its data root, and lays that data into the new install AFTER images are pulled and BEFORE any container starts — the one moment in an app's life when the data root is empty and no process holds it open, so a restore cannot overwrite anything, cannot race a live process, and needs no stop/start choreography. This slice deliberately contains no backup provider: the source is a live deployment on the same host, so the server is both reader and writer exactly as it already is for a data-aware rollback. No capability contract, no grant, no contract endpoint, no new job type. Sequence 6 (`restore@1`, the provider half) is the consumer of this groundwork. The design is almost entirely reuse — `capturePreUpgradeSnapshot`'s hook-and-tar path, `runPreHooksFailClosed`'s fail-closed policy, `restoreTarGzInto`, `mergeUpgradeAppEnv`'s three-case rule, `resolveContainedDir`'s containment, `checkUpgradePath`'s skew rules, `writeInstanceMarkers`, and `grants`' consent shape. Genuinely new: a candidate resolver, the restore sequence in the lifecycle job, and `composeUp`'s service-list/wait option. Two failures govern the whole design. A restore that cannot succeed refuses before any container starts (there is no partial-success state worth starting an app in), and a restore that carries data carries the configuration that data was written under — or names loudly every generated secret it could not carry, since data encrypted under a key the platform generated is unreadable beside a freshly generated one and nothing in the running app reports that as an error. Also promotes `lineageId` onto the deployment record, which spec 006 predicted in the shipped code it now changes. The `?? deployment.id` fallback makes this zero-migration: an older record reads undefined and yields the value it always wrote, and captures taken since spec 006 already carry the field. Review caught and fixed three defects worth naming. A shell-injection sink: `composeUp` interpolated manifest-supplied service names into a string run through `exec`, in the same file whose backup path builds argv explicitly "so provisioned values can't be injected" — now argv via `execFileAsync`, plus a Compose-service-name pattern at coercion. A `discard: ["."]` could wipe the restored payload after the post-condition passed, because `resolveContainedDir` treats the root as contained — now also requires `isStrictlyInside`. And the wizard judged no version skew at all, demanding an acknowledgement from every operator on every restore. Four of the prompt's own claims were wrong against the post-spec-006 tree and are corrected in the artifacts with evidence: the job payload's shape, the absent `CreateDeploymentRequest` type, a "locate the subtree" trap that describes a provider archive tool rather than this codebase's root-relative helpers, and `checkUpgradePath`'s inability to express "refuse a newer source" (it returns ok for anything that is not a forward upgrade). 53 FRs, 13 SCs, 57 verification scenarios, 88 tasks. Server 1086 -> 1130, web 360 -> 367, CLI 267 -> 274. Three fixes are mutation-proved: the injection probe, the FR-014 end-to-end refusal, and the wizard rollback. The try-hola/apps catalog change (schema + 5 manifests) is prepared but NOT submitted — it targets a repo requiring explicit per-instance permission. Closes #429 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh
… not line The comment at install-markers.test.ts read "~:3410 (restore) before ~:3414 (materialize)". A single intervening feature merge (spec 006, 921c790) moved both call sites by roughly 380 lines while the ordering claim itself stayed true, so the citation was wrong within one release of being written. Cite the symbols instead — `restoreAppDataSnapshot` runs before `materializeCompose` in `runLifecycleJob`'s deploy/start/rollback branch — and record why, because this is the second time in two specs that stale line-number citations have cost verification work: spec 007's planning had to re-verify roughly thirty anchors against the current tree because its prompt's numbers predated the same merge, several of them wrong in shape rather than merely in position. A citation that rots faster than the invariant it documents is worse than no citation: the next reader has to verify it before trusting the sentence built around it. Closes #485 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh
Both defects are the same shape: restore-on-install refuses for the right reason and tells the operator nothing they can act on. Neither refusal changes — FR-022 forbids starting an app on a data root the platform cannot vouch for, and an address is an operator decision — only what the refusal says, and the structured `details` it carries (FR-037). #489 — a restore that fails AFTER extraction left the install permanently unstartable behind a generic `RESTORE_TARGET_NOT_EMPTY`, which reads as "somebody else's data is in the way" when in fact the data is this install's own half-landed payload. `restoreStartedAt` is now persisted (and flushed, so it survives a hard kill mid-extraction) at the last read-only moment of the sequence, and `classifyNonEmptyTarget` uses it to split the refusal into `RESTORE_INCOMPLETE` — naming the state and the recovery, uninstall + reinstall or clear the data root — and the unchanged generic code. The field is a diagnostic only: the job-entry gate is untouched, so a retry still enters the restore sequence and still refuses. Two alternatives from the issue are rejected in code comments so they are not re-proposed: persisting an attempted-marker so a retry takes the no-restore path, and re-running only the post-extraction steps. Both can start an app on a half-extracted tree — the silent-empty-app outcome the spec's Executive Summary names as the worst in this feature. #490 — FR-035 defaults a restored install's address to the candidate's own, but a candidate is by definition a deployment that still exists on this host and still owns that route, so the default could never succeed; it fell through to `routingService.validateRule`'s bare host `CONFLICT`, which carries no `details.code` and never mentions restore. `resolveRestoreNameDefaults` now refuses with `RESTORE_ADDRESS_REQUIRED`, naming the candidate and the address it holds, and only when the caller supplied no name — an operator who names the install has made the address decision and gets the routing layer's own conflict. It compares against the candidate's LIVE subdomain (its deployment record), not its identity record's, which can be stale. Deriving a distinct/suffixed slug is rejected in a comment: that would put data full of the old address's absolute URLs at a new address nobody chose. Also: a CLI hint branch for `RESTORE_ADDRESS_REQUIRED` built from `details`; `RESTORE_INCOMPLETE` gets none, because job-time codes reach `hola install` as a failed job, not a `HolaApiError` — which is why its recovery is spelled out in the server's message. data-model.md §4, contracts/api.md and contracts/cli.md updated to agree on the code list, with the create-time vs job-time split now nine/four. Closes #489 Closes #490 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh
…ssion (#487) Spec 007 turned composeUp's hardcoded 300000ms into a parameter with a default, which left every other call site on a five-minute cap nobody had ever chosen for it. Audited all three sites in deployment.ts: - the restore-hook `--wait` (research R12) already passes 15 minutes — unchanged; - the deploy/start/rollback bring-up and the restart recreate now pass COMPOSE_UP_TIMEOUT_MS (15 minutes) explicitly, with the reason at the call site. Five minutes is too thin for what these actually do. `up -d` is not just container creation: Compose blocks on every `depends_on: service_healthy` gate, so a first install of a large multi-service stack waits out an `initdb` plus each dependency's healthcheck — the same cost R12 measured for the restore hook, which is why this reuses that number rather than inventing a third. A rollback that just replaced the data directory and a restore-on-install whose database just received a large dump start equally cold. The restart path is additionally the one composeUp with no composePull in front of it, so a recreate that must fetch a pruned image was running under a cap six times tighter than the one composePull was given for exactly that reason. The ceiling only delays reporting a wedged `up`; too low SIGKILLs a slow-but-correct install. composeUp's own 300000ms stays as a fallback for a caller with no opinion and is commented as such. Closes #487 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh
`hola install --carry-env gitea --restore-from latest` aborted with sade's "Insufficient arguments!": mri classifies a flag as boolean only by `typeof opts.default[key]`, and `--carry-env` is registered without a default, so mri treated it as value-taking and ate `gitea`. It cannot simply take a `false` default — `carryEnv` is tri-state (contracts/cli.md): `undefined` means "default to the candidate's own `carriesEnv`", `true`/`false` are explicit operator choices, and a `false` default collapses the first into the second. Normalising after the fact, the way `streamOpts` handles `--no-stream`, cannot help either: the token is consumed during parsing, before any handler sees an opts bag. So classify it at the parse layer instead. `parseOpts()` names `carry-env` in mri's `boolean` list (sade forwards the object verbatim, overriding only `alias`/`default`), which makes it boolean WITHOUT giving it a default — and mri's boolean branch pushes the token it looked at back onto `_`, so the positional is handed back rather than lost. Tri-state is preserved exactly: omitted → undefined, `--carry-env` → true, `--no-carry-env` and `--carry-env=false` → false. No documented behaviour changed, so contracts/cli.md is untouched. Tests parse the real flag shapes through sade rather than asserting on a hand-built opts bag, since the bug lives in parsing: the issue's exact repro, both flag positions, the `=`-form, the omitted case, and a guard that index.ts actually hands `parseOpts()` to `prog.parse`. Closes #488 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh
Issue sweep folded in (5 commits, one CI run)The sweep normally lands as its own PR off
The judgement call on #489 and #490Both describe a restore that refuses correctly but illegibly. The refusal is right in each case and stays; what was broken is that the operator learned nothing actionable. Two of #489's three suggested directions — persist an attempted-marker before extraction, or re-run only the post-extraction steps — would let a half-extracted tree start the app. Nothing can distinguish "extraction finished, a discard failed" from "extraction died halfway", so both trade a safe-but-terminal state for a possibly-silently-corrupt running one. That is exactly the trade FR-022 forbids. Rejected, with the reasoning written into the code so it is not re-proposed. #490's auto-suffixed-slug option was rejected for a parallel reason: it would stand up an app at a new address holding data full of the old address's absolute URLs — manufacturing the divergence FR-035 exists to warn about, without the operator choosing it. One correction to the brief, caught by the implementerMy specification for #489's trigger ( It is persisted immediately before extraction, which superficially resembles the rejected attempted-marker, so to be explicit: it is a diagnostic, never a gate. Verified after integration — the job-entry condition is still Gate, re-run once on the integrated branch
Still outstanding, unchanged
One out-of-scope finding worth its own issue later: |
Closes #429.
During an app install the operator may name an existing deployment of the same app on this host as a restore source. The server quiesces it with the backup contract's existing pre-hooks, captures its data root, and lays that data into the new install after images are pulled and before any container starts.
That timing is the whole reason the design is small. Restoring into a running app is hard — the data root is full, a database holds files open, a half-applied restore leaves a state nobody can reason about. At install time every one of those problems is absent by construction:
createFromDraftmints a fresh id, the data root does not exist until the job creates it, and no container has ever started.Why there is no backup provider in this
The source is a live deployment on the same host, so the server is both reader and writer — exactly as it already is for a data-aware rollback. No capability contract, no grant, no contract endpoint, no new
JobType.packages/shared/src/contracts.tsis untouched.Every subtle part of restore lives on the install side. Shipping those against a source that needs no contract negotiation means Sequence 6 (
restore@1, the provider half) arrives with one new job — put the files in a staging directory — against machinery that has been in production for a release.The two failures that govern the design
Design summary
Almost entirely reuse:
capturePreUpgradeSnapshot's hook-and-tar path,runPreHooksFailClosed's fail-closed policy,restoreTarGzInto,mergeUpgradeAppEnv's three-case rule,resolveContainedDir's containment,checkUpgradePath's skew rules,writeInstanceMarkers, andgrants' consent shape. Genuinely new: a candidate resolver, the restore sequence in the lifecycle job, andcomposeUp's service-list/wait option.The ten-step sequence in
runLifecycleJob, between the cancellation check andcomposeUp:Three of those steps exist because of traps that are invisible until they bite:
writeOidcCredentialsFilemoves to after extraction. Left where it was, a whole-root restore destroys the file it just wrote and the app boots with SSO silently disabled.lineageIdis promoted onto the deployment record — the change spec 006 predicted, in the code it now changes. The?? deployment.idfallback makes it zero-migration.Review found three defects worth naming
composeUpinterpolated manifest-supplied service names into a string run throughexec— in the same file whose backup path builds argv explicitly "so provisioned values can't be injected". Now argv viaexecFileAsync, plus a Compose-service-name pattern at coercion. New, not pre-existing; mutation-proved exploitable before the fix.discard: ["."]wipes the payloadresolveContainedDirtreats the root itself as contained, so a manifest couldrm -rfthe restored tree after the post-condition passed and start the app empty reporting success. Now also requiresisStrictlyInside.skew: unknown— demanding an acknowledgement from every operator on every restore, and never surfacing a real version refusal until the draft create failed.Four prompt claims corrected against the post-spec-006 tree
Recorded in the artifacts with their evidence, so they aren't re-inherited:
{ releaseId, action }—deploymentIdis a siblingJobfield, never a payload key;CreateDeploymentRequesttype; it isCreateDeploymentFromDraftRequest;<staging><candidate.path>" trap describes a provider archive tool — this codebase's helpers are root-relative, so FR-016 is a post-condition assertion rather than a subtree search (the trap is recorded for Sequence 6 in Sequence 6 (restore@1 provider) inherits the absolute-path subtree trap FR-016 generalises #486);checkUpgradePathcannot express "refuse a newer source" — it returnsokfor anything that is not a forward upgrade, so that rule is stated independently and evaluated first.Test evidence
bun run typecheck && bun run lint && bun run typecheck && bun run test && bun run build— green (typecheck twice, per the CLAUDE.md note).Three fixes are mutation-proved, not merely covered: the injection probe, the FR-014 end-to-end refusal, and the wizard rollback. The OIDC ordering test was separately verified to fail when the write is returned to its old position — a test that passes either way tests nothing.
Spec artifacts: 53 FRs, 13 SCs, 57 numbered verification scenarios, 88 tasks, 100% coverage both directions, Constitution Check clean with Complexity Tracking legitimately empty.
Gates that still apply before merge
try-hola/appscatalog change is prepared but NOT submitted — schema + 5 manifests, all 17 catalog manifests validated withajv. It targets a repo needing explicit per-instance permission. The platform half is useful without it: 12 of 17 acceptor apps restore by plain file copy today.Follow-up issues filed
#484 (dead
/api/backups/:id/restorestub, points at #160) · #485 (stale line-number comment) · #486 (Sequence 6 inherits the subtree trap) · #487 (composeUp's 5-minute default) · #488 (--carry-envarg parsing) · #489 (a restore failing after extraction leaves the install unstartable) · #490 (FR-035's subdomain default cannot succeed against a live candidate)🤖 Generated with Claude Code
https://claude.ai/code/session_01Vck5KSX2CLxhohx14nb5Sh