Skip to content

fix(soccer-scoreboard): fetch a league's schedule window once per miss (2.39.3) - #615

Merged
ChuckBuilds merged 3 commits into
mainfrom
claude/fix-soccer-fetch-burst
Oct 4, 2026
Merged

ChuckBuilds merged 3 commits into
mainfrom
claude/fix-soccer-fetch-burst

Conversation

@ChuckBuilds

Copy link
Copy Markdown
Owner

Problem

On ledpi (soccer-scoreboard 2.39.2, 8 leagues, core main), every start or reload logged ~90 NameResolutionErrors for site.api.espn.com within a minute, plus update() timed out after 30.0s. Over 6 hours soccer made 3,812 requests: 807 throttled, 410 retries, 198 s of rate-limit wait.

Root cause (plugin half)

On a cache miss, BaseSoccerManager._fetch_soccer_api_data() did two things with the same window:

  • submitted it to the core's background service;
  • fetched it again on the spot, as "partial data" for immediate display (_get_weeks_data()). Since the horizon fix, that "partial" window is the same window.

The recent and upcoming managers share the cache key, but they come due in the same update(), so both missed the cache and both did both. ESPN rejects date ranges, which makes each window 29 day requests (core half: ChuckBuilds/LEDMatrix#751). That is up to 4 x 29 requests per league at every start and every hourly refresh.

Each of the 24 managers also had its own requests.Session pool, so connections (and DNS lookups) were never reused across leagues.

Fix (soccer-scoreboard 2.39.3)

  • A miss fetches the window once, on the calling thread, with _fetch_season_directly (which caches it). This costs what the stand-in fetch already cost. The background submission was pure duplication, because the stand-in blocked on the same window anyway.
  • A per-cache-key lock: the league's other manager waits for that fetch and reads the cache.
  • Managers call core's share_connection_pool() with the retry policy they already mount, so all 24 use one pool. The import is guarded with try/except ImportError (src.common.fetch_service is None in the min-core table), so older cores keep per-manager pools.
  • Version 2.39.3, a new versions[] entry, CHANGELOG, and update_registry.py. ledmatrix_min_version stays 3.8.0.

Measurements (live ESPN, 8 leagues = ledpi's config, cold cache, alternating order)

variant requests / start peak in flight DNS lookups hourly refresh
core main + 2.39.2 439-485 42-50 73-76 451-478
core main + 2.39.3 259-261 40-47 52-56 248-251
LEDMatrix#751 + 2.39.2 46-47 13 29-30 -
LEDMatrix#751 + 2.39.3 39 15-16 27-29 20-24

The harness builds SoccerScoreboardPlugin, runs update(), and waits for the background service to go quiet, counting HTTPAdapter.send and getaddrinfo calls. For the hourly figure, the clock is advanced an hour before each further update().

Tests

  • New test_schedule_fetched_once.py. It checks four things:

    • a miss fetches once, with no stand-in fetch and no background submission;
    • a cache hit fetches nothing;
    • concurrent recent and upcoming managers share one fetch;
    • different leagues are not serialised, and the managers share the core's pool.

    Against the 2.39.2 code it fails 6 checks.

  • run_plugin_tests.py for soccer, football, baseball, hockey and basketball against core main and against #751 gives identical failure lists (10 pre-existing Windows failures); this branch adds one pass.

  • These all pass with LEDMATRIX_CORE set: check_version_bump.py soccer-scoreboard, check_manifest_version_fields.py soccer-scoreboard, check_min_core_version, check_sports_drift, check_sports_helpers_parity, check_module_collisions, test_espn_dates_copies and check_manifests_ascii.

Overlap

The unmerged claude/espn-canonical-scoreboard-keys (waiting for core 3.9.0) changes the same cache-key and cache-read lines in _fetch_soccer_api_data: canonical key plus _cached_schedule(cache_key, (legacy_key,)). It also bumps the same manifest and CHANGELOG. Whichever lands second needs a small rebase:

  • keep this PR's lock and single fetch;
  • use that branch's key and _cached_schedule read inside _cached_window;
  • re-bump the version.

Not merging; for review.

🤖 Generated with Claude Code

…s (2.39.3)

Every start on ledpi (8 leagues) logged ~90 NameResolutionErrors for
site.api.espn.com within a minute, and six hours made 3,812 requests.
On a cache miss _fetch_soccer_api_data submitted the window to the
core's background service AND fetched the same window on the spot as
stand-in data, in both the recent and the upcoming manager; with ESPN
rejecting date ranges each window is 29 day requests.

A miss now fetches the window once on the calling thread
(_fetch_season_directly, which caches it) under a lock per cache key,
so the league's other manager waits and reads the cache. Managers
share the core's connection pool where the core has one, so
connections and DNS lookups are reused across leagues.

Live ESPN, 8 leagues, core main: ~440 -> ~260 requests per start,
~470 -> ~250 per hourly refresh; with LEDMatrix #751 39 and ~20.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a8f53053-4a47-4e1a-b7f7-b6675d2437a6
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 28 complexity

Metric Results
Complexity 28

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@ChuckBuilds
ChuckBuilds merged commit 47bc85c into main Oct 4, 2026
5 checks passed
@ChuckBuilds
ChuckBuilds deleted the claude/fix-soccer-fetch-burst branch October 4, 2026 19:52
ChuckBuilds added a commit that referenced this pull request Oct 4, 2026
…2.39.4) (#618)

A per-league mode in switch mode (soccer_esp.1_recent, soccer_usa.1_upcoming, ...) now consults only the named league's manager; when it has nothing to show the mode returns False and the core's empty-mode handling moves on. Merged after #615 (2.39.3 stays as the versions[] and CHANGELOG entry below 2.39.4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant