Skip to content

disk: bound resident LRU index entries with WithMaxEntries (WS2) - #32

Draft
shreyas-blacksmith wants to merge 1 commit into
bounded-zstd-transcodingfrom
lru-entry-bound
Draft

disk: bound resident LRU index entries with WithMaxEntries (WS2)#32
shreyas-blacksmith wants to merge 1 commit into
bounded-zstd-transcodingfrom
lru-entry-bound

Conversation

@shreyas-blacksmith

@shreyas-blacksmith shreyas-blacksmith commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The disk cache's byte budget bounds cached bytes but not entry count: zero-byte blobs charge nothing and tiny blobs at most one 4 KiB block, while each resident entry costs a measured ~268 B of index metadata (key string, entry struct, list node, map slot). A byte-full 10 GiB cache can hold ~2.6M entries, i.e. ~670 MiB of request-independent resident memory.
  • Adds an opt-in WithMaxEntries(n) option: inserting a new key past the cap evicts from the LRU tail, exactly like the byte budget but counting entries. The startup scan enforces it too (oldest trimmed during load, like a restart with a smaller max size). Default (n <= 0) is bit-identical to upstream.
  • Adds bazel_remote_disk_cache_max_entries_evictions_total to separate count-cap evictions from byte-cap evictions, and an opt-in measurement test (TestLRUEntryMetadataCost) documenting the per-entry constant.

Part of the embedder's memory-envelope program (WS2); stacked on the bounded zstd transcoding branch (#27).

Test plan

  • go test ./cache/... ./server/... green
  • New unit tests: tail eviction order, zero-byte entries bounded, overwrite does not evict, disabled by default
  • New integration tests: disk.New + WithMaxEntries under puts; startup trim of an over-cap cache directory

Made with Cursor


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled. (Staging)


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

The byte budget does not bound entry count: zero-byte blobs charge
nothing and tiny blobs at most one 4 KiB block, while each resident
entry costs a measured ~268 B of index metadata (key string, entry
struct, list node, map slot). A byte-full 10 GiB cache can therefore
hold ~2.6M entries (~670 MiB of metadata). WithMaxEntries adds an
opt-in entry-count cap with the same eviction semantics as the byte
budget, enforced on Add and during the startup scan, plus a counter
separating count-cap evictions from byte-cap evictions.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant