fix(backfill): anchor nightly backfill to a fixed wall clock via cron#196
Merged
Conversation
The nightly backfill used setInterval(24h) registered in onModuleInit, so its fire time was boot + N*24h — anchored to whatever time the process last restarted. Every redeploy silently moved the window (most recently to ~03:35 UTC), making the schedule unpredictable and the verification window a moving target. Switch to @nestjs/schedule's @Cron (ScheduleModule is already wired up and used by MaintainerPopulateService). Default '10 0 * * *' in America/Chicago = 12:10am local, stable across redeploys, with timeZone handling the CST/CDT shift so it stays at local midnight year-round. Overridable via NIGHTLY_BACKFILL_CRON / NIGHTLY_BACKFILL_TZ; the :10 offset preserves the prior 00:10 stagger. Behavior preserved: still no run-at-startup, NIGHTLY_BACKFILL_ENABLED still disables it, and the static per-repo jobId still dedupes overlapping ticks. Replaces the removed NIGHTLY_BACKFILL_INTERVAL_MS knob in .env.example.
The cron expression and timezone won't realistically change, so drop the NIGHTLY_BACKFILL_CRON / NIGHTLY_BACKFILL_TZ env indirection and make them plain constants. Condense the surrounding comments.
PR #195 dropped DEFAULT_BACKFILL_DAYS 40->10 but .env.example kept the stale 40.
entrius
approved these changes
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The nightly backfill scheduled itself with
setInterval(24h)registered inonModuleInit, so its fire time was boot + N×24h — anchored to whatever time the process last restarted, not a real clock. Every redeploy silently shifted the window. After PR #195's deploy it landed at ~03:35 UTC instead of the historical 00:10 UTC, purely because the container happened to reboot at 03:35. That makes the schedule unpredictable and the rate-limit verification window a moving target (the working notes had to keep re-deriving it fromdocker inspect ... StartedAt).Fix
Switch to
@nestjs/schedule's@Cron, which evaluates against the wall clock and is stable across redeploys. The infra is already in place —ScheduleModule.forRoot()is registered inapp.module.tsandMaintainerPopulateServicealready uses@Cron.'10 0 * * *'inAmerica/Chicago→ 12:10am local, year-round (thetimeZoneoption handles the CST/CDT shift, so it doesn't wander an hour twice a year).:10offset preserves the prior00:10off-the-top-of-the-hour stagger.Behavior preserved
NIGHTLY_BACKFILL_ENABLED=falsestill disables it (guarded in bothonModuleInitlogging and the job body).jobId: backfill-<repo>-nightlystill dedupes a tick that lands while the prior night's run is still draining — so dropping the boot-anchored timer loses nothing.NIGHTLY_BACKFILL_INTERVAL_MSknob from.env.example. The cron/timezone are plain constants (they won't realistically change), not env vars.Test
tsc --noEmit,eslint,prettier --check: clean.npm test(das): 21/21 pass.Deploy note
After this deploys, the nightly fires at 00:10 America/Chicago going forward, regardless of reboot time — no more re-deriving the window from container boot.