Skip to content

fix(cache): cache keys too long to be a filename; memory hits judged by the record's own age - #738

Merged
ChuckBuilds merged 2 commits into
mainfrom
claude/ledmatrix-bug-fixes-02ac41
Oct 4, 2026
Merged

ChuckBuilds merged 2 commits into
mainfrom
claude/ledmatrix-bug-fixes-02ac41

Conversation

@ChuckBuilds

Copy link
Copy Markdown
Owner

Summary

Two cache bugs, found from hdpi's journal and a code review.

1. A cache key too long to be a filename was never cached

  • Seen on hdpi: the calendar plugin's key joins every calendar ID the user picked: calendar_events_<6 calendar ids>, over 300 bytes. ext4 caps a filename at 255 bytes.
  • Result: every hourly write failed with [Errno 36] File name too long, at all three write stages (temp file, direct write, home-dir fallback).
  • Misleading log: the final warning said (permission denied) whatever the error was, so it looked like a cache-directory ownership problem.
  • Fix: DiskCache.get_cache_path keeps any key up to 200 UTF-8 bytes as its filename, exactly as before, so existing files keep their names. A longer key becomes its first 183 bytes, cut on a character boundary, plus a 16-hex hash of the whole key.
  • The shortened stem is itself short, so the web UI's cache list (which names keys by filename) still deletes the right file.
  • The give-up warning now names the real error.

2. The memory tier served data older than the reader asked for

  • Cause: a record loaded from disk was timed in memory from the load, not the write. So get(key, max_age=300) could return data close to 600 s old after a restart, after the hourly memory sweep, or in the other process. A stored ttl was stretched the same way. feat(fetch): one ESPN scoreboard cache key and a max-age response cache (fetch service stage 2) #728's _fresh_cached already worked around this for the scoreboard; every other caller was exposed.
  • Fix: get_cached_data / load_cache now also check a memory hit against the record's own timestamp, with DiskCache.get's rule that a stored ttl wins.
  • A stale copy falls through to disk, which returns the other process's newer write if there is one.
  • Records without a timestamp keep the memory tier's own clock.

Tests

  • New: test/test_cache_long_keys.py (7) and test/test_cache_memory_tier_age.py (5).
  • Mutation check: reverting either fix makes 5 and 2 of them fail.
  • Related suites: 547 cache/fetch/odds/health tests pass.

Validation on hardware

  • ledpi (ext4): a script with hdpi's key shape. The old module dropped the key; the new one wrote a 205-byte filename and read it back.
  • hdpi: read-only for this session. Its journal is where the bug showed: 15 failed calendar writes in 7 days.

Part of a bug sweep

This is one of 10 independent fix PRs from one sweep, all based on main ef69201.

  • Merge order: any. 45 pairwise test merges gave 0 conflicts, and each PR also merges cleanly with fix(ipc): ticks carry the volatile timestamps, so current-status stays known over the socket #737.
  • CHANGELOG: each PR adds its bullet at a different place in Unreleased → Fixes, so squash-merging them one after another needs no conflict fixing.
  • Full suite (Windows), all 10 merged together vs plain main: the same 62 failures and 6 errors on both. These are the known Windows path and file-locking tests. 191 more tests pass.
    • One extra failure in that run, test_backup_manager.py::test_create_backup_contents (os.replace → WinError 5 on a temp zip), was a Windows file-lock flake. It passes on rerun, and nothing here touches create_backup.
  • ledpi: all 10 together ran on ledpi (Pi 4) on top of main, with a clean start and no errors or render stalls in the journal. ledpi is back on plain main.

🤖 Generated with Claude Code

ChuckBuilds and others added 2 commits October 3, 2026 20:48
The calendar plugin's cache key joins every calendar id the user picked.
On hdpi it passed 300 bytes; ext4 caps a filename at 255, so every write
(the temp file, the direct-write fallback and the home-directory fallback)
failed with ENAMETOOLONG, once an hour, and the final warning said
"(permission denied)" whatever the error was.

DiskCache.get_cache_path keeps a key of up to 200 UTF-8 bytes as its
filename, exactly as before, and turns a longer one into its first 183
bytes (cut on a character boundary) plus a 16-hex-digit hash of the whole
key. The temp file adds 15 bytes, so the longest name is 215. The
shortened stem is itself short, so the web UI's cache list, which names a
key by its filename, deletes the same file. The give-up warning now names
the real error.

Validated on ledpi's ext4: the old module drops the hdpi-shaped key, the
new one writes a 205-byte filename and reads it back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A record loaded from disk went into the memory tier timed from the load,
so get(key, max_age=300) could return data close to 600 s old: after a
restart, after the memory sweep, or in a second process. A stored ttl was
stretched the same way. #728's _fresh_cached works around it for the
scoreboard; every other caller was exposed.

get_cached_data and load_cache now also check a memory hit against the
record's embedded timestamp, with DiskCache.get's rule that a stored ttl
wins over max_age. A stale copy is dropped and the read falls through to
disk, which returns the other process's newer write if there is one.
Records without a timestamp keep the memory tier's own clock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 29 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 05f5c808-132f-44f2-8b81-777c9bd7f378
📥 Commits

Reviewing files that changed from the base of the PR and between ef69201 and d53a898.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • src/cache/disk_cache.py
  • src/cache_manager.py
  • test/test_cache_long_keys.py
  • test/test_cache_memory_tier_age.py
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 11 complexity

Metric Results
Complexity 11

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@ChuckBuilds
ChuckBuilds merged commit a6e9e3e into main Oct 4, 2026
15 checks passed
@ChuckBuilds
ChuckBuilds deleted the claude/ledmatrix-bug-fixes-02ac41 branch October 4, 2026 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant