feat(fetch): one ESPN scoreboard cache key and a max-age response cache (fetch service stage 2)

[ChuckBuilds]

Stage 2 of the shared fetch service (#702): one cache key per ESPN scoreboard, a short response cache that honours ESPN's Cache-Control: max-age, and counters for both. Every change to an existing signature is additive. No new core module, so the plugins' check_min_core_version table needs no follow-up for this PR.

1. One cache key per ESPN scoreboard

Before this PR, three consumers cached the same endpoint under three different names:

Consumer	Key before
odds-ticker	scoreboard_data_{sport}_{league}_{date}
APIHelper.fetch_espn_scoreboard	espn_{sport}_{league}_{date}
the scoreboards	{sport_key}_schedule_{window} and {sport_key}_scoreboard_current

New in src/common/espn_dates.py:

• espn_scoreboard_cache_key(sport, league, dates=None) builds espn_scoreboard_{sport}_{league}_{dates}.
  • sport and league are ESPN's path segments (football/college-football), not a plugin's sport_key, so every plugin names a league the same way.
  • dates is the dates= value: YYYY, YYYYMM, YYYYMMDD, YYYYMMDD-YYYYMMDD, a date, or a (start, end) pair. None is the undated current scoreboard.
  • Anything else raises ValueError rather than inventing a key. espn_scoreboard_cache_key_for_url(url, dates) and espn_scoreboard_url() go with it.
• get_espn_scoreboard(session, sport, league, dates, *, cache_manager, max_age, legacy_keys, ...) is the cache-through read.
  • It returns a copy at most max_age seconds old, whoever wrote it.
  • On a miss it fetches with fetch_espn_scoreboard (limit=500, ranges split) and caches the result under the canonical key.
  • max_age=0 always fetches, and still leaves its copy for other readers.
  • read_espn_scoreboard_cache() and store_espn_scoreboard_cache() are the two halves. The read takes an accept(data, age) predicate, which odds-ticker uses to hold a payload with a live game to its live interval.
• Never older than the reader's TTL. CacheManager lets a writer's stored ttl override the reader's max_age, and its memory tier times an entry from when it was loaded, not when it was written. A key shared by readers with different TTLs can rely on neither.
  • The read therefore checks the record's own timestamp.
  • The shared entry is written with no ttl, so each reader applies its own limit.
  • A test against a real CacheManager shows the hazard: a plain get(max_age=30) returns a 60 s old entry stored with ttl=3600, and the new read does not.
• Old keys are passed as legacy_keys and read after the canonical key for one release, so an upgrade serves the copies already on disk instead of refetching every league at once.

The core helpers that cache a scoreboard now use the canonical key:

• APIHelper.fetch_espn_scoreboard uses it by default. The old default key is a fallback; an explicit cache_key behaves as before.
• SportsFetchMixin._schedule_cache_key(datestring) / _cached_schedule(key, legacy_keys) are for the scoreboards' schedule windows.
  • _cached_schedule is the same read as before (same default max age, a stored ttl still wins), so only where a window is cached changes, not for how long.
  • The canonical key carries the window's dates, so it moves on a day as the window slides. A miss deletes the previous day's copy of the same window, so a league keeps one window file instead of a week of them.
• _fetch_season_directly(cache_key=None) caches under the canonical key.

2. Response cache (Cache-Control: max-age)

ESPN sends no validators, only max-age. Measured from this machine on 2026-10-02: most scoreboard answers say 1-9 s, some 56-75 s, and Premier League days 169-496 s. It looks like the CDN's remaining freshness.

• What is kept. FetchService keeps a finished 200 for its max-age minus Age. An identical GET inside that window is answered without a request.
  • "Identical" uses the validator store's key: URL, query, effective headers, and the session too when it has cookies or auth.
  • Timeout and retry policy don't matter here: a finished 200 is the same answer either way.
• Never older than the caller accepts.
  • A caller passes cache_max_age=<its TTL> to fetch_get or fetch_espn_scoreboard; 0 always asks the network.
  • A caller that passes nothing gets at most 30 s (fetch_service.response_cache.default_max_age).
  • BaseOddsManager.get_odds passes its update interval, APIHelper.get its cache_ttl, and get_espn_scoreboard its max_age.
• Never kept: no-store, no-cache, private, Vary: *, Set-Cookie, non-200 responses and streams.
• Bounds: 64 entries, 6 MB, 2 MB per entry (a college-football Saturday is ~1 MB; months, at 5-7 MB, are never kept), and 10 minutes at most. Expired entries are dropped on every insert.
• Off switch: fetch_service.response_cache.enabled: false. Any failure in the cache falls back to a plain request.

3. Counters

Three new counters in GET /api/v3/plugins/fetch-stats, per plugin, per host and in total:

• memo_hits: answered by the response cache;
• cache_hits: scoreboard fetches answered by a shared cache entry;
• legacy_cache_hits: reads served from a pre-stage-2 key. It should fall to zero within a day of an upgrade, which says when the fallback can go.

response_cache reports the cache's current size. A plugin's hosts counts now include both kinds of hit.

Plugins: branch pushed, no PR

ChuckBuilds/ledmatrix-plugins branch claude/espn-canonical-scoreboard-keys (2 commits on 82dbb0df). It waits for the core release that ships this PR. It floors on 3.9.0, assuming the release after 3.8.0 is 3.9.0; adjust versions[0] and compatible_versions if it ships under another number.

• odds-ticker 1.8.0.
  • Per-day scoreboards use read_/store_espn_scoreboard_cache under the canonical key, with scoreboard_data_* as the fallback.
  • The fetch goes through fetch_espn_scoreboard (limit=500, the identifying session, counted) instead of a bare requests.get.
  • A day holding a live game is held to the live interval by the entry's own age.
• baseball 1.55.0, basketball 1.42.0, football 3.19.0, hockey 1.43.0. The nine schedule-window managers use _schedule_cache_key / _cached_schedule, with {sport_key}_schedule_{window} as the fallback.
• afl 1.36.0, nrl 1.35.0, soccer 2.40.0. Their window keys move to the canonical key for the URL, with the old key as the fallback.
• Live poll.
  • baseball, football and hockey still ask ESPN on every live poll, but now leave the answer under the shared key (get_espn_scoreboard(max_age=0)).
  • basketball and soccer move their 30 s live cache from {sport_key}_scoreboard_current (whatever the dates) to the shared key for the dates actually asked.
  • So odds-ticker's entry for today and the scoreboards' live poll read each other's copy.
• Release housekeeping. Version bumps with new versions[] entries, CHANGELOGs and plugins.json.
• Checks. These pass against this branch's core: check_version_bump (with the 8 ids, and --all), check_manifest_version_fields (with the ids), test_espn_dates_copies, check_sports_helpers_parity, check_module_collisions, check_sports_drift, check_min_core_version and its test, check_core_api_signatures, check_manifests_ascii and test_odds_fetch_scope.
• Plugin tests. Each touched plugin's test scripts, run against this core, fail only where plugins main fails against core main: afl test_afl_plugin (a requests import shadow), baseball test_odds_placement, and football's interactive test_football_plugin.
• Still on their own keys. ncaam/ncaaw basketball's undated current scoreboard (still limit=1000 through a raw session.get) and afl/nrl's live 30 s cache. Odds-ticker doesn't overlap with either.

No core-module follow-up is needed: this PR adds no src/ module, so check_min_core_version's MODULE_FIRST_VERSION table is unaffected.

Live A/B against ESPN

ab_live.py is the plugins' two call patterns, copied from plugins main (before) and the branch (after). Consumer A is a scoreboard's live poll: college football in the football style (no cache, then max_age=0) and the Premier League in the soccer style (30 s cache). Consumer B is odds-ticker's day read (300 s). The day is 2026-10-03, 54 college-football games.

• 3 rounds per run, the consumer order alternating per round (A,B / B,A / A,B).
• Each run is a fresh process with a real CacheManager on a temp dir.
• Every HTTP request that left the process is counted at requests.Session.send.
• Runs alternate, new first in the cold pair: after, before, after, before, before, after.

run	requests sent	fetch-service requests	cache_hits	seconds	distinct payloads per league
after	4	4	8	0.39	1, 1
before	6	4	0	0.90	1, 1
after	4	4	8	0.32	1, 1
before	6	4	0	0.82	1, 1
before	6	4	0	0.79	1, 1
after	4	4	8	0.32	1, 1

• Where the saving comes from. Before, odds-ticker fetched each league once on its own, through a bare requests.get the fetch service never saw (6 sent, 4 counted). After, it is served from the scoreboard's copy, and every request is counted.
• Both consumers saw the same events in every run (54 for college football; the Premier League had none that day).
• The response cache did not fire (memo_hits 0): ESPN said max-age=1 for that college-football day. Its own effect is pinned by the unit tests.
• What this does not show is a rig's rate. That depends on how often the two consumers' polls overlap and which leagues a rig shows; the soak below measures it.

Tests

• test/test_espn_scoreboard_cache.py (new, 50):
  • Key: one spelling across callers (get_espn_scoreboard, APIHelper, the mixin's _schedule_cache_key and _fetch_season_directly, URL-derived); every dates form; refusals.
  • Old-key fallback: canonical key first; a stale old key is a miss; the old key is counted as legacy.
  • Age: an injected clock (now=); never older than the reader's TTL whatever ttl the writer stored, also against a real CacheManager, on disk and after loading into memory; the canonical entry stores no ttl; the accept predicate.
  • Fetching: a miss fetches limit=500 and caches; a hit sends nothing; max_age=0; ranges in chunks; a failure caches nothing; the response cache is bounded by the caller's TTL.
  • Sharing: two consumers, one fetch (counted per plugin); concurrent misses merge.
  • Windows: _cached_schedule's legacy read and the previous-day window retirement.
• test/test_fetch_service.py (+36):
  • Response cache: max-age reuse, expiry, Age, the caller's TTL, the 30 s default, 0, cache_max_age never reaching the session, nine kinds of response that must not be kept, streams, representation keys, cookie sessions, size bounds and LRU, expiry on insert, the off switch, merging plus caching.
  • Bounded callers: odds never older than their interval; APIHelper.get bounded by its cache_ttl.
  • Counters: per plugin and per host, legacy, avoided_request=False, every snapshot field.
• test/test_api_helper.py: the two tests that pinned the old default key now pin the canonical one. Added: an explicit key works as before, and the old key is read after an upgrade.
• Mutation check: 13 mutations, each caught by a test. They break the caller-TTL bound, the default limit, Age, no-store/no-cache/private, storing on the merge path, the record-timestamp check, the legacy flag, get_espn_scoreboard's TTL passed to the cache, legacy keys, window retirement, the mixin's legacy keys, the odds interval and APIHelper's TTL.
• Full suite against an origin/main baseline worktree (5aa7a63), on Windows: the sorted FAILED/ERROR test IDs are identical.
  • Baseline: 61 failed, 6 errors, 7867 passed, 214 skipped.
  • Branch: 61 failed, 6 errors, 7955 passed, 219 skipped.
  • The 5 extra skips are test_sports_stage3_parity cases auto-generated for the new sports_fetch members. They skip because no plugin bundles a copy.
  • After rebasing onto fix(ipc): a plugin reload no longer freezes the panel during Vegas #723 (IPC and Vegas, no overlap), the fetch, ESPN, sports, APIHelper, odds and background suites were re-run: 358 passed.
• scripts/check_types.py (mypy 1.20.2): clean on all 93 modules, including fetch_service, espn_dates, sports_fetch, api_helper and base_odds_manager.

Soak needed (not run; no rig was touched)

On hdpi or ledpi, with the plugins branch installed on this core, alternating with the current main + plugins main, 45 min per phase:

• /api/v3/plugins/fetch-stats every 15 min: requests per host, cache_hits, memo_hits and legacy_cache_hits. legacy_cache_hits should be non-zero only in the first phase after the upgrade.
• Journal: no stale live scores (the live game's clock should move every live interval), no update() timed out at boot beyond what main shows, and no refetch storm on the first boot after the upgrade.
• Disk: one espn_scoreboard_*_<window> file per league, not a growing set. Also the extra live-poll writes from baseball, football and hockey: one day's scoreboard per live poll, the same rate basketball and soccer already write.
• Frame timing: scripts/frame_soak.py --preview for no regression from the cache writes.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: f27e94a8-bd75-44ae-8684-04336dd0f374

📥 Commits

Reviewing files that changed from the base of the PR and between a7b3f33 and 2aca6c4.

📒 Files selected for processing (11)

• CHANGELOG.md
• docs/PLUGIN_API_REFERENCE.md
• docs/REST_API_REFERENCE.md
• src/base_odds_manager.py
• src/common/api_helper.py
• src/common/espn_dates.py
• src/common/fetch_service.py
• src/common/sports_fetch.py
• test/test_api_helper.py
• test/test_espn_scoreboard_cache.py
• test/test_fetch_service.py

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codacy-production]

Not up to standards ⛔

🔴 Issues 1 medium

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category	Results
Security	1 medium

View in Codacy

🟢 Metrics 142 complexity

Metric	Results
Complexity	142

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}