Skip to content

Retained portal database loses its monitoring dependency #4856

Closed

Description

@devantler

🤖 Generated by the Agentic Engineer

Expected Behavior

A database retained during portal retirement remains observable until its separately reviewed data cleanup completes. The retired application and its access stay removed.

Actual Behavior

Retirement removed the declarative monitoring dependency along with the portal. The retained database still requests monitoring, so the agent cannot collect its database health. The temporary database-only network policy also omits the monitoring path.

Steps to Replicate

  1. Render the production apps after portal retirement.
  2. Compare the retained database monitoring requirements with the rendered resources and network policy.
  3. Observe missing monitoring resources while the database remains retained.

Acceptance criteria

  • Restore only the dedicated database-monitoring dependency and narrowly scoped scrape path; do not redeploy the portal or change application/database storage.
  • Prove the production render includes the dependency, with no periodic credential regeneration.
  • Test allowed monitoring and rejected unrelated traffic, retaining control and replication coverage.
  • Keep native cleanup preservation complete for the restored resources.
  • After reviewed CI and merge, verify monitoring and database health through read-only production evidence. Keep this issue open until runtime verification.

Related to #4266. Priority rationale: live observability breakage; restore visibility of retained data without reopening the retired portal. End-to-end effort includes regression tests, current-head review, deployment and live readback.

Activity

  1. added theissue type on Oct 13, 2026
  2. self-assigned this
    on Oct 13, 2026
  3. added a commit that references this issue on Oct 13, 2026
  4. moved this to 🏃🏻‍♂️ In Progress in 🌊 Project Boardon Oct 13, 2026
  5. added 2 commits that reference this issue on Oct 13, 2026
  6. devantler commented on Oct 13, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    The regression-tested repair is published as draft #4859, with the validation prerequisite separately isolated in #4858 and recorded as this issue's blocking dependency. All local source and native admission checks pass, but neither PR has merged or deployed. This issue remains open for the accepted deployment, database-monitoring reconciliation and live Coroot readback.

  7. added a commit that references this issue on Oct 14, 2026
  8. devantler commented on Oct 14, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Post-merge verification: repair #4859 merged at 54a8d855c08615a26af96da8997f20da40144678 and its normal production deploy completed successfully. The subsequent #4860 deployment also passed and its published artifact is the revision currently applied by Flux.

    Read-only runtime checks confirm the retained database has both instances ready, continuous archiving is healthy, and the restored monitoring dependency is synced. The current production declaration still includes the dedicated monitoring dependency and its narrow network path. This establishes deployment and database readiness; an actual Coroot database-health readback remains unverified, so #4856 stays open. Portal retirement and protected storage acceptance remain separate #4819 work.

  9. devantler commented on Oct 14, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Runtime acceptance is now verified. The exact #4859 source head 8ae5df0 merged as 54a8d85. Normal merge-group CI 38489448119 completed successfully, including production deployment, queue-membership proof and required checks.

    Fresh OIDC reads show the restored ExternalSecret Ready/SecretSynced with CreatedOnce refresh, the retained Cluster declaring the pg_monitor role and dedicated monitoring Secret, and only the scoped monitoring ingress on the database port. Both database pods are Running/Ready with zero restarts; all five Flux roots are Ready at their current generations and one live artifact revision.

    Two separated authenticated Coroot PostgreSQL samples now show both database instances up, primary/replica roles, live queries and latency measurements, and healthy availability, latency, replication lag and connections. The first sample reports zero replica lag; the second independently reports the lag check healthy. The full-project integration-required counter is now zero. This is actual monitoring, not inferred from manifest or Secret readiness.

    The acceptance criteria for restored retained-database monitoring are complete. No portal workload was restored and no volume was patched or deleted. Physical protection and stateless cleanup remain open #4819 work; this closes neither general data-retirement nor whole-platform clean-state acceptance.

  10. moved this from 🏃🏻‍♂️ In Progress to ✅ Done in 🌊 Project Boardon Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

None yet

Effort

None yet

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions