Skip to content

Storage health hides archive failures and recovered write failures #1279

Description

@frahlg

On v3.5.3-beta.1 (8650c58), /api/health checks the live writer's current error and recent rejected ticks, but does not expose a failed hourly archive/retention job. A later successful commit also clears last_error, leaving no counter or timestamp for recovered commit failures. A ready learned model is likewise not proof that observation and model persistence currently work.

Expose the current maintenance phase/result, last success, last failure and failure counts under history storage. Keep recovered failures as diagnostic history while clearing the current degraded state after successful recovery. Make archive failures affect overall health. Provide explicit forecast learning health for the simple UI in #1278; do not derive Healthy from model readiness alone or invent learning start dates.

Tests must cover a failed maintenance run followed by a successful one, a recovered live commit, waiting for fresh measurements, pending model/observation persistence and queue overflow. Related: #1275, #1276, #1277 and #1149.

Activity

  1. frahlg commented on Sep 24, 2026

    @frahlg
    MemberAuthor

    This is in the code that v0.132.0-beta.1 is running.

    /api/health includes history_storage.maintenance with phase, last success, last failure and failure count. A failed maintenance run marks health degraded. The next successful run clears the current error and returns health to ok, and keeps last_failure_error and the failure count. A recovered history commit clears last_error and keeps commit_failures and last_failure_error.

    TestHealthShowsFailedArchiveAndRecovery and TestHealthShowsRejectedPrimaryHistory passed. On homelab-rpi after the native switch the writer shows two recovered SQLITE_BUSY commits and later successful writes, which is this diagnostic, not a hidden failure.

    Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions