Repository navigation
Deliver retained-data protection before stateless portal cleanup #4819
Description
🤖 Generated by the Agentic Engineer
Description
The assessed stateless portal can be removed separately from its database, but the protection preflight in #4814 is not yet satisfied. The retained claims are controller-owned, so changing a rendered application manifest alone does not establish protection on the existing storage.
Steps to Reproduce
- Read the retained claims and reciprocal Bound volume bindings using the production read-only identity.
- Compare their effective protection with the retirement procedure and the accepted storage-retention model.
- The required protection is absent; cleanup must remain held.
Expected Behavior
Deliver a reviewed, bounded protection step before stateless cleanup. The existing claims, backing storage, database configuration, credentials and running instances must remain intact. Follow the Retain-at-storage-layer direction in #3369 and docs/adr/storage-retention.md; do not create new standing prune opt-outs as a substitute.
Acceptance Criteria
- Join each retained claim to its actual Bound volume by namespace, name, UID and reciprocal binding; incomplete or changed evidence stops before mutation.
- Protect the existing backing data through the reviewed storage-retention mechanism, changing only Delete to Retain and preserving storage handles, claim/volume UIDs and every other specification field. Coordinate the bounded slice with Set every existing volume to Retain and report the ones that are not #4441; it does not complete that broader migration or its standing check.
- Reconcile the retirement precondition with the actual storage backstop and captured-identity admission fence, with explicit adversarial controls. No check is removed merely to make execution possible.
- Any additional temporary ownership-handoff metadata has a named removal obligation under Retire the unused Backstage scaffold #4266, no standing global opt-out, and a separately reviewed rationale.
- Use a protected, confirmed, first-attempt main workflow for required writes, with UID/resourceVersion preconditions, source/deployment binding and separated readbacks; do not perform local production mutations or widen read-only RBAC.
- Reproduce missing protection and replacement/binding races before the fix; prove only the intended protection fields change. Read back database availability and retained bindings after deployment/execution.
- Before activating the durable retirement fence, verify that normal database/controller reconciliation continues through authenticated, explicitly field-scoped exceptions. Prove certificate/metadata updates with positive fixtures and reject foreign identities, replacement/deletion, ownership changes and protection removal with adversarial fixtures. Do not automatically clear the journal to permit them.
- Do not delete a database, namespace, claim, volume, backup or credential. Independent backup acceptance and eventual data retirement remain separate work.
Additional Context
Fresh read-only preflight on 13 October found both retained claims Bound but without the procedure's required protection, and their bound volumes still using Delete. No protection or retirement write was performed. This is a concrete execution dependency, not a reason to restore the unused portal, fabricate a missing credential, silence Coroot, or call retirement complete.
Part of #4266. Related to #4814, #3369 and #4441.
Acceptance verified: 2026-10-20. The storage-only prerequisite and its terminal operation reconciliation are complete. The protected storage run 38738313022 verified preservation; the protected first-attempt completion 39161262289 removed only the terminal journal after separated preservation checks and verified Flux recovery. The DELETE-probe safety repair merged in #5040. Fresh read-only acceptance confirms the retained storage remains protected and the database healthy. #4266 remains open for independent backup and retained-data acceptance and any remaining retirement obligations. No new workflow or production mutation was performed for this acceptance.
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Effort
Start date
Projects
- Status
Activity
test: distinguish native retirement deadlines from other failures
devantler commentedon Oct 13, 2026
The retirement source repair is now on #4814 at 3430236. Its native controller proof, retained-writer identity/profile controls and 93 evaluated admission cases pass. Production execution remains blocked here; no storage or retirement mutation has run.
The bounded implementation sequence is:
Regression cases must distinguish missing protection, mixed/partial policies, replacement between read and patch, wrong claimRef, unrelated spec changes, failed second readback, and unhealthy retained instances. Storage protection is not data retirement and does not complete #4441 or the parent #4266.
This is refinement of the existing prerequisite, not a new implementation start while downstream WIP is over its ceiling.
chore: agent-claim #4819 nonce=5fef4f9d20bef99a5aa982ebb81920e1
devantler commentedon Oct 13, 2026
Reviewed-source implementation plan
The smallest coherent repair is a storage-only protection step, followed by separately confirmed stateless retirement. A claim annotation is not evidence that its backing volume retains data.
Ordering matters: protection must precede installation of the retirement journal. That fence deliberately rejects changes to the captured volume specification, including a reclaim-policy change. The protection step must therefore refuse an existing journal; it must never clear or adopt one automatically.
Regression coverage needs the actual submitted patches, missing/unknown protection, replacement/rebinding, partial completion, ambiguous transport, stale source/deployment and positive/adversarial controller reconciliation. Include the new command/workflow in CI and update the lifecycle ADR in the same delivery.
This is a bounded prerequisite for #4266, not completion of #4441 or removal of global prune protections. No production write or retirement dispatch was made during this analysis.
devantler commentedon Oct 13, 2026
The source prerequisite #4814 is now merged and its protected production deployment passed. A fresh read-only platform-health check still fails on the retired portal release and backend; the retained database remains outside the cleanup scope.
I am taking this narrow prerequisite as confirmed incident work (Service class: Expedite), with a recorded WIP exception while ordinary In Progress occupancy is 16 against its ceiling of 12. No existing card is moved backwards and no global limit is changed. The work is limited to physical retained-storage protection, its fail-closed cleanup precondition, tests, and protected execution/readback. General storage migration and independent backup acceptance remain separate.
Implementation starts from current main, with issue ownership, an isolated working copy, test-first regressions, current-head non-Copilot review, CI and the merge queue. No local production mutation or retirement has been performed.
chore: agent-claim #4819 nonce=d887d83286cdf8471e3b5cf85a030776
72 remaining items
chore: agent-claim #4819 nonce=050ee6534b28b2d541ec36b3d687852a
devantler commentedon Oct 17, 2026
The maintainer has now authorized Backstage retirement directly in this chat: “I authorize backstage retirement”. This run owns the operation under the shared coordination claim.
I will use that grant for one first-attempt dispatch of the reviewed
retire-backstage-stateless.yamlworkflow from currentmain, after joining the accepted production deployment with a fresh readback. The database, claims, volumes, backups, credentials and namespace remain retained. The workflow repeats all source, inventory, identity, admission and preservation checks before writing.The prior production deployment completed successfully and merged as current main
97a1afbbc81881ac63db58c883ab62f4f947df89. The single authorized retirement was dispatched as run 39070923390, attempt 1, on that exact source and its accepted artifact. This authorization is now consumed; no rerun or second dispatch is authorized by it. A refusal or a possibly submitted native deletion will be inspected as the existing operation; neither permits a replay or clearing its journal.chore: agent-claim #4819 nonce=9e70d59efe10f08e7f47bb9f5cd880b1
chore: agent-claim #4819 nonce=b5c1069984b23e2977c456882435328d
chore: agent-claim #4819 nonce=a89b28519c6d42de6c9aa79ff7b95134
chore: agent-claim #4819 nonce=83e6b73461169b1800f34638ab84e7c2
chore: agent-claim #4819 nonce=2781fb287cc1a602c5f5377127488267
chore: agent-claim #4819 nonce=60e78e0596a2ebe9d0cbc242f82a3c22
devantler commentedon Oct 17, 2026
The single authorized retirement attempt 39070923390 ran at reviewed source
97a1afbbc81881ac63db58c883ab62f4f947df89, attempt 1. It reportedPORTAL-RETIREMENT=HOLD_ADMISSION; it is not a validated successful workflow result.Independent readback found the assessed stateless footprint absent and production health passing. Both retained database instances remained ready; the two claims, reciprocal bound-volume identities and Retain policies matched the independent baseline. The durable operation journal remains present. No replay, journal clearance or data-retirement operation was performed.
The discrepancy exposed a source defect: admission DELETE probes supplied dry-run only in the query while sending a body. Kubernetes takes DELETE options from a nonempty body, so a query-only probe can submit real deletion. The old marker therefore cannot establish that no lifecycle was started. The source repair and regression are in draft #5040; the earlier transport test checked the query but missed this body contract.
This authorization is consumed. Keep the existing operation and retained-data protections in place. The issue remains open for the safety repair and complete outcome reconciliation; neither the green health check nor this bounded readback establishes backup acceptance or data-retirement readiness.
chore: agent-claim #4819 nonce=2ae3040f757537e6fe4027fade7767fd
chore: agent-claim #4819 nonce=ed96e7fdc6960d6d489d77aba806d913
devantler commentedon Oct 20, 2026
The retained-storage prerequisite is accepted. I independently verified the complete job census and executed safety controls for protected storage run 38738313022, plus terminal completion 39161262289, attempt 1. The latter actually reported
PORTAL-RETIREMENT=COMPLETED journal=REMOVED preservationSamples=2 flux=READY dataRetirement=UNASSESSED. The DELETE-probe repair in #5040 is merged, and all nine native child issues are completed.Two fresh scoped read-only samples agree: the assessed stateless workload and terminal journal are absent, retained storage has reciprocal Bound identities and Retain protection, the database is healthy, and Flux has reconciled its current source. Exact resource evidence stays private.
All acceptance criteria for this storage-protection slice are satisfied. Closing #4819 does not complete #4266: independent backup and retained-data acceptance and the parent's remaining retirement obligations are still outstanding. This run dispatched no workflow and made no production mutation.
chore: agent-claim #4819 nonce=dec12e1904d17afd8007ac93d0147ead