fix(scheduler): require a strictly later period, not merely a different one - #46
Conversation
…nt one is_due compared period keys with !=, so a run stamped in a FUTURE period made the workflow due on every tick until that period arrived. Because last_run_started_at returns the newest row by started_at, the future-dated run stays "the last run" no matter how many runs tick adds after it -- so the retry was unbounded, not a one-off. That is precisely the storm the module docstring says is impossible by construction, and the line above the comparison already stated the intended rule: "due only when now falls in a later calendar period than the last run." Compare with ">". All three cadence keys are ordered tuples -- (y, m, d), (isoyear, isoweek), (y, m) -- so the ordering is well defined, and past and same-period behaviour is unchanged. A future-dated run is not exotic: clock skew between the scheduler host and the store, an imported store, or a backfill all produce one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XjLWb6igee2tVTG7wg93Fx
…od is spent Gate review: replacing "!=" with ">" stopped the storm but started silent starvation. A row stamped ten years ahead -- which ADR 002 designs for, since timestamps may be supplied for imports and backfills -- made the workflow due on no tick at all: 3,465 consecutive days skipped, printing the same line as a healthy skip, with no ledger event and exit 0. A storm announces itself in 300 ledger entries and 300 charges; starvation is invisible until someone notices the digest stopped arriving. Both failures come from the same place: is_due was asked about the NEWEST run, while docs/governance.md states the rule over the set -- "a daily workflow is due unless SOME run already started today ... any run in the period counts". A run stamped in another period, past or future, simply is not a run in this one, and collapsing the history to one row cannot express that. Comparing the newest row's period for inequality fires on every tick; comparing it for order fires on none. So take the rule literally: is_due now receives the start times and asks whether any of them falls in now's period, staying pure as the module promises; runs_started_this_period supplies them, bounded in SQL to the period's first instant so a long history costs nothing. Both existing behaviours are preserved (ran earlier today -> not due; ran yesterday -> due). Measured end-to-end with a run stamped 10 years ahead, 300 ticks in one day: main runs 300 times, the ">" version 0, this 1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XjLWb6igee2tVTG7wg93Fx
Gate verdict — PASS after one amendment roundThe review confirmed the arithmetic was airtight and then found something better: the fix as first written traded an unbounded storm for unbounded silent starvation. The findingWith a run stamped ten years ahead — a condition ADR 002 explicitly designs for, since "timestamps may be supplied for imports and the demo seeder" — the A storm announces itself — 300 ledger entries, 300 provider charges, an alert. Starvation is invisible in the console, the ledger and the exit code until someone notices a digest stopped arriving. Worse, the first fix put the code in direct contradiction with
With only a tomorrow-stamped row, no run started today, so governance says DUE. The Root cause — both failures are the same mistake
FixTake the documented rule literally. Both existing behaviours are preserved: ran earlier today → not due; ran yesterday → due. Measured, end-to-end through the real CLI300
Ruled out by the review
CI green. Left open for human review. |
Found by an autonomous
bughuntiteration.Bug
scheduler.is_duecompares calendar-period keys with!=:Inequality is true when
nowis in a later period than the last run — and equally true when it is in an earlier one. So a run stamped in a future period makes the workflow due on every tick:The line directly above the comparison already states the intended rule:
Why it is unbounded, not a one-off
last_run_started_atreturns the newest row bystarted_at:A future-dated run therefore stays "the last run" no matter how many runs
tickappends after it. Every subsequent tick sees a different period key and fires again — until real time catches up with that stamp.That is exactly what the module docstring says cannot happen:
End-to-end, with a run stamped tomorrow already in the store, three consecutive
flightdeck tickinvocations each print:A cron calling
tickevery minute would run the workflow every minute, spending real budget against a real provider, for as long as the skew lasts.A future-dated run is not exotic: clock skew between the scheduler host and the store, an imported store, or a backfill all produce one.
Fix
All three cadence keys are ordered tuples —
(y, m, d),(isoyear, isoweek),(y, m)— so>is well defined. Past and same-period behaviour is unchanged; the eight existing due-logic assertions pass untouched.Test
tests/test_scheduler.py:test_a_future_dated_run_does_not_make_the_period_due_foreverFalsefor a future stamptest_future_dated_run_stays_the_newest_row_so_the_storm_would_be_unboundedlast_run_started_ateven after a later run is added — the reason the retry has no boundtest_tick_does_not_storm_on_a_future_dated_runtickinvocations add zero runsAll three verified to fail without the fix (
git stashon the source alone); the third fails with"daily-digest: ran"on every tick.Validation
All three CI jobs green locally:
python -m pytest --cov=flightdeck --cov-fail-under=85→ 261 passed, coverage 94.48%ruff check src tests→ All checks passed!flightdeck demo+flightdeck audit verify→ ledger verified, 2,434 entries, chain intact🤖 Generated with Claude Code
https://claude.ai/code/session_01XjLWb6igee2tVTG7wg93Fx