fix(espn): fetch a window's edge months whole, cap chunk requests per process

[ChuckBuilds]

Problem

On ledpi (soccer-scoreboard 2.39.2, 8 leagues, core main), every start or reload logged ~90 NameResolutionErrors for site.api.espn.com within a minute (89 from soccer) plus update() timed out after 30.0s. Over 6 hours soccer made 3,812 requests (807 throttled by the fetch service's token bucket, 410 retries, 198 s of rate-limit wait). DNS works otherwise; the burst overwhelms the Pi's resolver.

Root cause

ESPN answers dates=YYYYMMDD-YYYYMMDD with 400, so espn_dates re-asks a range as whole months plus one request per day for partial edge months. A scoreboard's default window is a fortnight either side of today: 29 days across two partial months, so 29 day requests per league per fetch. Soccer fetches that window from several managers at once (the plugin half is in the plugins PR), and each fetch_espn_date_chunks() call had its own pool of six threads, so ~40-50 requests were in flight. Every one past a session's connection pool opened a new connection, and each new connection meant a fresh getaddrinfo (Python has no DNS cache). On top of that, every window spent one doomed range request on the first fetch after start, because the "ranges are rejected" memo starts empty: 11 at once.

Fix (core: src/common/espn_dates.py)

1. Ask for an edge month whole and trim it. When the window covers ESPN_MONTH_COVER_MIN_DAYS (7) or more days of a partial month, that month is one YYYYMM request. The answer is trimmed to the window's days using each event's US Eastern start date, which is the day ESPN's dates=YYYYMMDD means. I checked this against the live API on 2026-10-03: for 5 soccer leagues over Sep, Mar and Nov 2026 (Nov includes the end of DST), 417 of 417 events came back from the day query named by their Eastern date. A month query also matched the sum of its day queries exactly (eng.1 30/30, UCL 18/18, MLS 74/74, ger.1 27/27). A 29-day window now costs 2 requests, down from 29.
   • Short windows stay as day requests, so a live poll's 1-2 days never downloads a whole month.
   • If a trimmed month comes back at the 500-event cap, only the window's days are re-asked.
   • The capped payload is dropped in the worker.
   • An event with no readable date is kept: a game is never dropped on a guess.
   • Without tz data, nothing is trimmed and the edge days stay day requests.
   • New public helper: espn_request_chunks(). espn_date_chunks() is unchanged.
2. One process-wide cap on chunk requests. A BoundedSemaphore(ESPN_CHUNK_WORKERS) is held around every chunk request, so the limit is 6 in flight across all windows, not 6 per window.
3. Start as if a range was just rejected. _ranges_rejected_until starts at now + RANGE_RETRY_SECONDS. The range is still retried 6 h in, so the workaround still retires itself if ESPN reverts.

Trade-off: a whole month is about 2-3x the bytes of the half that a fortnight window holds (measured, decoded JSON: eng.1 0.34 → 0.71 MB, MLS 0.71 → 1.63 MB, college football 3.58 → 8.87 MB). In exchange, requests drop from 29 to 2, and the requests are what break the resolver. Trimming happens in the worker, so only window events are held.

Measurements (live ESPN, Windows desktop, 8 leagues = ledpi's config, cold cache, alternating order B C P F F P C B B F)

The harness builds SoccerScoreboardPlugin, runs one update(), then waits for the background service to go quiet. It counts HTTPAdapter.send and socket.getaddrinfo calls, with peak concurrency for each.

variant	requests / start	peak in flight	DNS lookups	update()
B: core main + soccer 2.39.2	439 / 448 / 468	49 / 45 / 42	76 / 73 / 75	5.8-7.8 s
C: this PR + soccer 2.39.2	46 / 47	13 / 13	30 / 29	1.9-2.0 s
P: core main + soccer 2.39.3	260 / 261	47 / 40	56 / 52	2.6-2.7 s
F: this PR + soccer 2.39.3	39 / 39 / 39	15 / 16 / 15	29 / 27 / 29	1.6-2.0 s

Simulated hourly refresh (clock advanced 1 h, update() again, 3 cycles): B 451-478 requests/h, F 20-24/h.

Tests

• test/test_espn_dates.py, new cases:
  • edge-month planning: the threshold, short windows, whole months, no tz data, exact day coverage;
  • Eastern trimming at both window edges and across the DST change, keeping undated events;
  • a capped trimmed month re-asks only the window's days;
  • concurrent windows share one budget (mutation-checked: it fails with the semaphore widened);
  • a fresh process skips the range request.
• test/test_fetch_service.py: the caller-identity chunk count now uses espn_request_chunks.
• Full suite against an origin/main baseline worktree: the sorted FAILED/ERROR lists are identical (62 failed / 6 errors, the known Windows set). Passing went from 8296 to 8310.
• mypy ratchet (scripts/check_types.py, mypy 1.20.2): same 2 pre-existing fetch_service.py unused-ignore notes as on main; espn_dates.py is clean.
• Scoreboard plugin suites (soccer, football, baseball, hockey, basketball) against this branch and against main: identical failure lists (10 pre-existing Windows failures).

Companion

ChuckBuilds/ledmatrix-plugins#615 (soccer-scoreboard 2.39.3) removes the plugin's duplicate window fetches and shares the connection pool. Each PR helps on its own; together, a start drops from ~450 requests to 39.

Not merging; for review.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 55 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: d3f61c98-c371-4d69-bc45-b327be8ccceb

📥 Commits

Reviewing files that changed from the base of the PR and between 2236ff3 and 8da7fc0.

📒 Files selected for processing (5)

• CHANGELOG.md
• src/common/README.md
• src/common/espn_dates.py
• test/test_espn_dates.py
• test/test_fetch_service.py

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codacy-production]

Not up to standards ⛔

🔴 Issues 1 medium

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category	Results
Security	1 medium

View in Codacy

🟢 Metrics 33 complexity

Metric	Results
Complexity	33

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}