fix(health): the Health 68 liveness oracle counts a run that compared nothing as alive

[stranske]

Why

The liveness oracle added by #3179 / PR #3189 counts a Health 68 run that compared nothing as
evidence that Health 68 is alive. That is the same failure mode #3179 was filed to end — 23 days of
silence reading as health — reconstructed in the mechanism built to detect it.

scripts/check_durable_tracker_liveness.py:302 declares:

EXECUTABLE_CONCLUSIONS = frozenset({"success", "failure", "cancelled", "timed_out"})

and _latest_executable_run returns the newest run whose conclusion is in that set. But the
workflow_run debounce added to .github/workflows/health-68-consumer-sync-drift.yml skips the
comparison steps while the job still concludes success. Read live on 2026-08-24:

run 32723431817  2026-08-24T11:44:52Z workflow_run  conclusion=success
  Debounce workflow_run fan-out        => success
  Setup Python                         => skipped
  Install dependencies                 => skipped
  Compare consumer repos to templates  => skipped     <-- nothing was measured
  Upload drift report                  => skipped

Sampled across five recent runs, only two actually compared:

32723538883 11:46 workflow_dispatch  compare-step=success
32722651700 11:35 workflow_run       compare-step=skipped
32721899468 11:26 workflow_run       compare-step=success
32719152874 10:54 workflow_run       compare-step=skipped
32717090456 10:30 workflow_run       compare-step=skipped

So a repository in which Health 68 executes the comparison never again — only debouncing — is
reported healthy by the checker, and no FYI comment is posted on issue #2210. The blocking quantity
("hours since the last run") and the drainable quantity ("hours since the last actual comparison") are
different numbers, and only the first is measured or reported.

The debounce compounds it. The jq selector in
.github/workflows/health-68-consumer-sync-drift.yml picks the newest prior run whose conclusion is
success, failure or timed_out — which includes its own no-op runs — so the 30-minute clock
resets on nothing happening. It is not latched shut (a job-level skipped conclusion breaks the
chain, and a comparison did run at 11:26), but the interval it enforces drifts with trigger volume
rather than tracking the last comparison.

Related, and cheap to fix in the same change: #3179's Implementation Notes specified an optional live
probe that "fails against live state today; that standing failure is the acceptance signal for the
fix", skipping only when no token is present. What shipped in
tests/workflows/test_health_68_liveness.py:52 also requires RUN_LIVE_HEALTH_68_PROBE=1, so the
probe never runs in CI even where a token exists — confirmed as one of the four skips in a full
tests/workflows run.

Full trace: Code/Audits/Workflows/2026-08-24-implementation-verification.md.

Scope

Make "did Health 68 compare?" an observable, asserted quantity:
.github/workflows/health-68-consumer-sync-drift.yml (emit an outcome the API can read),
scripts/check_durable_tracker_liveness.py (read it instead of the job conclusion), and the two test
files that gate them.

Non-Goals

• Do NOT remove the debounce. Roughly 180 full 14-repo comparisons per day is the cost fix(health): restore Health 68 execution and assert durable-tracker liveness #3179 debounced
  away on purpose.
• Do NOT remove the daily schedule: trigger from .github/workflows/health-68-consumer-sync-drift.yml.
• Do NOT add a second liveness auditor. scripts/check_durable_tracker_liveness.py is the tool;
  extend it.
• Do NOT add any human approval, review, label, or queue step. The sweep stays FYI-only and must never
  open or close a gate.
• Do NOT make the checker's failure block a merge or fail the Health 71 job; continue-on-error on
  that step is deliberate.
• Do NOT reclassify a debounced run as a failure. It is a correct no-op; it is simply not evidence of
  execution.
• A scaffolded stub, a TODO, a renamed file, or a test that passes without exercising the change
  does NOT count as done and is a failure of this issue.

Tasks

• Add a step-level marker to .github/workflows/health-68-consumer-sync-drift.yml that records whether the comparison ran, for example a job output compared=true|false set beside the existing drift_failed output.
• Change _latest_executable_run in scripts/check_durable_tracker_liveness.py to accept a per-tracker require_step name and select the newest run whose named step conclusion is not skipped, falling back to the job conclusion when no require_step is configured.
• Add require_step: Compare consumer repos to templates to the health-68-consumer-sync-drift.yml entry in config/durable_tracker_liveness.yml.
• Report both numbers in the checker output and in the tracker comment body written by scripts/check_durable_tracker_liveness.py: hours since the newest run, and hours since the newest run that actually compared.
• Change the debounce selector in .github/workflows/health-68-consumer-sync-drift.yml so it measures the age of the last run that COMPARED, not the last run that concluded successfully.
• Remove the RUN_LIVE_HEALTH_68_PROBE term from the skipif in tests/workflows/test_health_68_liveness.py so the live probe skips only on a missing GH_TOKEN or GITHUB_TOKEN, with the reason naming the missing variable.
• Add test_liveness_ignores_runs_whose_comparison_step_was_skipped to tests/workflows/test_durable_tracker_liveness.py using the monkeypatch pattern already in that file.

Acceptance Criteria

• Named test gate: python3 -m pytest tests/workflows/test_durable_tracker_liveness.py::test_liveness_ignores_runs_whose_comparison_step_was_skipped -q exits 0. It must feed the checker two runs — one success with the comparison step skipped, one older success with it success — and assert the reported age is measured from the OLDER run.
• Second named gate: python3 -m pytest tests/workflows/test_health_68_liveness.py::test_consumer_drift_detector_debounces_workflow_run -q exits 0 and asserts the debounce selector reads a comparison marker rather than the bare run conclusion.
• Deliberate break: delete the require_step line from config/durable_tracker_liveness.yml; python3 -m pytest tests/workflows/test_durable_tracker_liveness.py -q must FAIL naming the new test. Restore it; it must pass. Quote both transcripts as raw pytest output in the PR body.
• Second deliberate break: re-add RUN_LIVE_HEALTH_68_PROBE to the skipif in tests/workflows/test_health_68_liveness.py; a test asserting the skip condition names only the token variables must FAIL, then pass on revert. Quote both transcripts as raw pytest output.
• python3 scripts/check_durable_tracker_liveness.py --json prints, for health-68-consumer-sync-drift.yml, both an age since the newest run and an age since the newest comparing run.
• python3 -m pytest tests/workflows -q exits 0, and the count of skipped tests does not increase.
• actionlint .github/workflows/health-68-consumer-sync-drift.yml reports no findings.
• No approval step, review gate, label handoff, or queue is introduced anywhere in the diff.

[agents-workflows-bot]

🏷️ Label Suggestions

Based on the issue content, these labels might be relevant:

• feature (90% confidence)
• enhancement (90% confidence)
• documentation (90% confidence)
• docs (90% confidence)
• bug (91% confidence)
• workflows (95% confidence)
• workflow:source-dependabot (95% confidence)
• verify:create-new-pr (95% confidence)
• tracker:durable (95% confidence)
• test (95% confidence)
• sync-failure (95% confidence)
• sync (95% confidence)
• repo-review-meta-audit (95% confidence)
• repo-review-approved (95% confidence)
• python (95% confidence)
• phase-1 (95% confidence)
• needs-autofix-review (95% confidence)
• integration-test (95% confidence)
• integration-sync-failure (95% confidence)
• integration-sync (95% confidence)
• integration-failure (95% confidence)
• health:repo (95% confidence)
• health:coverage (95% confidence)
• github-actions (95% confidence)
• dependencies (95% confidence)
• cross-repo (95% confidence)
• consumer-sync (95% confidence)
• config (95% confidence)
• ci-failure (95% confidence)
• ci (95% confidence)
• campaign:sync-dependabot (95% confidence)
• agents:max-runs:1 (95% confidence)
• agents:auto-pilot-failed (95% confidence)

Auto-applied: agents:allow-change

How to use these suggestions

• Click the label name in the sidebar to add it
• Or use the GitHub CLI: gh issue edit 3243 --add-label "label-name"

---

Auto-generated by label matcher

[stranske]

Closer sequencing correction: PR #3247 is merged and has now been labeled verify:compare. This issue is reopened until the merged PR has a durable verifier/provider-comparison PASS and no remaining disposition debt; the closer will close it after that evidence is recorded.

[stranske]

Closer verifier disposition evidence (cursor 2026-08-25T04:43Z): merged #3247 core acceptance gates pass on current main despite provider split (OpenAI CONCERNS / Anthropic PASS). The CONCERNS cite missing deliberate-break transcripts in the merged PR body and bundled scope creep; local re-run of the two named gates on current main succeeds:

$ python3 -m pytest tests/workflows/test_durable_tracker_liveness.py::test_liveness_ignores_runs_whose_comparison_step_was_skipped -q
.                                                                        [100%]
1 passed in 2.22s

$ python3 -m pytest tests/workflows/test_health_68_liveness.py::test_consumer_drift_detector_debounces_workflow_run -q
.                                                                        [100%]
1 passed in 2.34s

Issue remains open until durable verifier PASS or evidence-backed disposition of the scope-creep concern; the Health 68 comparison/debounce fix itself is implemented and gated green on main.