Skip to content

perf(fetch): cache ESPN scoreboard windows without the parts nothing reads - #749

Open
ChuckBuilds wants to merge 1 commit into
mainfrom
perf/slim-espn-windows
Open

ChuckBuilds wants to merge 1 commit into
mainfrom
perf/slim-espn-windows

Conversation

@ChuckBuilds

Copy link
Copy Markdown
Owner

Summary

The sports scoreboards cache their Recent/Upcoming window (14 days back, 7 ahead) as the raw ESPN response, and while it's fresh it stays parsed in the memory tier. Most of it is never drawn. BackgroundDataService now drops those parts of an ESPN /scoreboard response before caching and delivering it (src/common/espn_payload.py):

removed where
leaders per-competitor and per-competition stat leaders
probables, status.featuredAthletes athlete cards with headshots and season stats
team.links, event.links web and app URLs
headlines, highlights article and video blurbs
geoBroadcasts regional broadcast details

Everything else is kept, including odds, records, linescores, situation, statistics, notes, broadcasts and venue. Only those exact keys are ever removed. submit_fetch_request(..., slim_payload=False) caches a response whole.

On the four hdpi windows (MLB, college football, NHL, NFL) this takes the JSON from 10.6 MB to 3.0 MB and the parsed objects from ~40 MB to ~12 MB. Parsing an expired window, which some scoreboard paths do on the render thread, gets 3–4x cheaper.

Type of change

  • Bug fix (performance / memory)

Test plan

  • Render equivalence, against ledmatrix-plugins main (f568fd3) and this branch rebased on 07abd87: I ran scripts/check_plugin.py for baseball (MLB), football (NFL, and college football with ncaa_fb enabled) and hockey (NHL) at every size, seeded with the real hdpi windows, once raw and once slimmed. Records, odds and rankings were switched on. 88 PNGs, all byte-identical. They show real games (e.g. PHI–ATL Final 2–6, Stable #3 Notre Dame vs UNC).
  • Nothing reads the removed keys. I grepped core and every plugin on ledmatrix-plugins main. The only hits are leaders in f1-scoreboard and nfl-stat-leaders, which read different ESPN APIs (not /scoreboard, not via this service), and "highlights" in baseball-scoreboard, which is a config value, not a payload key.
  • Ran on a real Raspberry Pi with hardware: hdpi (Pi 4, 512x64), 75-min windows from a fresh restart, main then this branch. RSS was 30–40 MB lower at every 10-minute mark (306–326 vs 337–385 MB). Late frames were 4.7 vs 4.1 per 1,000, with no regression.
  • Ran the test suite (pytest): new test_espn_payload.py checks that exactly the listed keys go and everything else is unchanged, that odd shapes pass through, that the URL check rejects look-alike hosts, that the service caches and delivers the same slimmed object, the opt-out, and that other URLs are untouched. 145 related tests pass.

Notes for reviewer

🤖 Generated with Claude Code

…reads

The sports scoreboards cache their Recent/Upcoming window as the raw ESPN
response, and it stays parsed in the memory tier while fresh. Measured on
hdpi, most of it is never drawn: per-team stat leaders, athlete cards
(featuredAthletes, probables), team and event links, headlines, video
highlights and geo broadcasts. None of those keys is read by core or by any
plugin in ledmatrix-plugins.

BackgroundDataService now drops them from an ESPN /scoreboard response
before caching and delivering it (src/common/espn_payload.py), keeping
everything else. On the five hdpi windows that is 10.6MB -> 3.0MB of JSON
and ~40MB -> ~12MB of parsed objects, and parsing an expired window gets
3-4x cheaper. submit_fetch_request(slim_payload=False) caches a response
whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkfgXMqqwn2w4NN7LRzhxy
@coderabbitai

coderabbitai Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 46 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 29b029f1-8335-434f-8097-5b541d011ba3
📥 Commits

Reviewing files that changed from the base of the PR and between 07abd87 and 00456eb.

📒 Files selected for processing (3)
  • src/background_data_service.py
  • src/common/espn_payload.py
  • test/test_espn_payload.py
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codacy-production

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 high

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
ErrorProne 1 high

View in Codacy

🟢 Metrics 20 complexity

Metric Results
Complexity 20

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant