Skip to content

fix(crons): Compute next_checkin_latest in UTC during DST fold - #128740

Open
spawnia wants to merge 2 commits into
getsentry:masterfrom
spawnia:fix-cron-margin-during-dst-fold
Open

spawnia wants to merge 2 commits into
getsentry:masterfrom
spawnia:fix-cron-margin-during-dst-fold

Conversation

@spawnia

@spawnia spawnia commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Crontab monitors in a DST timezone report every check-in as missed during the repeated hour after DST ends.
get_next_expected_checkin_latest now adds the check-in margin in UTC instead of local time.

Local datetime arithmetic resets the fold, so the margin deadline lands an hour early

get_next_expected_checkin returns the next slot in the monitor's timezone.
In the repeated hour that datetime carries fold=1.
Adding a timedelta to it is wall-clock arithmetic and returns fold=0, the first pass of that hour.

With * * * * *, Europe/Berlin and a 5 minute margin, a check-in at 2024-10-27 01:00:03 UTC gives:

UTC
next_checkin 01:01
next_checkin_latest before 00:06
next_checkin_latest after 01:06

check_missed sees next_checkin_latest already in the past and creates a missed check-in for the slot at once.
The real check-in arrives seconds later as ok, but with failure_issue_threshold: 1 every slot from 01:00 to 01:54 UTC opened a monitor failure issue.
Slots whose deadline falls after the repeated hour, from 02:55 local time on, are unaffected.

Observed on a self-hosted instance on 2026-10-25

Every minutely and five-minutely monitor in Europe/Berlin showed a missed check-in exactly at each expected time from 01:00 to 01:54 UTC, next to an ok check-in for the same slot about 3 seconds later.
No phantom misses appeared before or after that window.

Spring forward is unaffected: a skipped local time resolves to the correct UTC instant.

Related: #66868 fixed DST handling in the schedule itself

Replacing croniter with cronsim made get_next_schedule and get_prev_schedule correct across DST.
This bug sits after the schedule, in the margin addition.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

🤖 Generated with Claude Code

Adding the check-in margin to the next expected check-in in the monitor's
timezone uses wall-clock arithmetic, which resets the datetime's fold.
During the repeated hour after DST ends, next_checkin_latest then lands
an hour early, so check_missed marks every slot missed although the
check-in arrives on time.

🤖 Generated with Claude Code
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 25, 2026
@spawnia
spawnia marked this pull request as ready for review October 25, 2026 09:51
@spawnia
spawnia requested a review from a team as a code owner October 25, 2026 09:51
Python returns False for an inter-zone == when one operand is an
ambiguous local time with fold=1 (PEP 495).

🤖 Generated with Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant