fix(backfill): make nightly backfill incremental and defer on GraphQL rate limits

[anderdc]

Problem

The nightly backfill (RepoBackfillScheduleService, daily) re-fetched every PR in a 40-day window from scratch every night — enqueuing a PR_METADATA + PR_FILES (content) job per PR. For high-PR-volume GitHub accounts this exhausted the per-installation GraphQL budget (5k points/hr) and flooded logs.

Evidence (prod ghm-das, 7-day buffer): 100% of GraphQL RATE_LIMIT log lines fall in the post-enqueue window (00:10–02:xx UTC); the other ~22h/day are clean. Driven entirely by two installations (the two largest accounts). Rate-limit + Halving lines were ~16% of all log volume, amplified ~5× per failed batch by the 50→25→12→6→5 retry ladder. Not webhook- or reconcile-driven.

Changes

Incremental backfill — the same single nightly backfill, now only re-fetching what changed:

• Skip the expensive PR_FILES content fetch when content is already stored (scoring_data_stored) and head+base SHA are unchanged (file diff is fully determined by head+base SHA).
• Skip PR_METADATA when GitHub's PR updatedAt is unchanged (new nullable pull_requests.updated_at column).
• Both gates fail safe: new PR / missing content / moved SHA / null timestamp → re-fetch.

Rate-limit deferral instead of failure:

• A GraphQL 200 carrying RATE_LIMIT/SECONDARY_RATE_LIMIT/graphql_rate_limit is now surfaced as a typed GitHubRateLimitError (these bypass HTTP-status handling since GraphQL returns 200).
• The processor catches it → job.moveToDelayed(now + retryAfterMs, token) + DelayedError, re-queuing without consuming a retry attempt and freeing the worker slot. No more halving an already-exhausted budget in a tight loop.

Tuning & observability:

• Default backfill window 40 → 10 days.
• Greppable [backfill-summary] (per-repo enqueued-vs-skipped tallies) and [rate-limit-defer] log lines.

Tests & CI:

• Detection/gating logic extracted to dependency-free modules (isGraphQLRateLimit, needsContentRefresh, needsMetadataRefresh).
• 21 unit tests (jest) covering detection, gating branches, and the processor deferral.
• New Test CI workflow mirroring the existing Build/Lint jobs.

Deploy notes

1. Apply the schema column to prod before deploy (additive, nullable, safe to run early):

   ALTER TABLE pull_requests ADD COLUMN IF NOT EXISTS updated_at TIMESTAMPTZ;

2. First night after deploy does one full pass (every updated_at starts null → gates fail safe); steady state (night 2+) re-fetches only changed PRs.

Verification (post-deploy)

After ≥2 nights, the 00:10–02:xx UTC graphql_rate_limit flood for the two affected installations should collapse toward zero, and [backfill-summary] should show high *_skipped counts for the firehose repos.

Notes

• Worker concurrency stays at 5: the rate-limit deferral self-heals secondary-limit hits (it backs off using the retry-after/reset headers), so lowering concurrency would be redundant prevention with a small throughput cost on real-time webhook jobs.
• Follow-up (not in this PR): page the backfill by updatedAt-desc with early-stop so paging cost becomes independent of window size — would let us run a large safety-net window cheaply. ~100 LOC, needs new per-repo last_backfilled_at state.