Repository navigation
perf: O(1) expected random pick for RRCache eviction and random_key - #115
Merged
Merged
Conversation
RRCache.evict() and random_key() chose an entry with table.iter().nth(random), which walks the table: O(n) per eviction. With a full cache of 100k entries an insert cost ~23 µs. Pick a random bucket index and keep the first occupied one instead. Conditioned on hitting an occupied bucket, every entry is equally likely, so the distribution stays uniform. hashbrown only grows by doubling, so a full cache stays roughly a quarter occupied or more even after eviction churn, which means a few probes on average. A sparser table (pre-sized via `capacity`, thinned by removals, or holding few heavy entries under `getsizeof`) falls back to the old uniform O(n) pick after 32 misses. Tests cover uniformity in a full and in a sparse table, draining a sparse table with popitem(), and keeping the cache full under eviction.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
RRCachepicks the entry to evict (and the key forrandom_key()) withtable.iter().nth(random).RawIterdoesn't overridenth, so this walks the table: every eviction is O(n). With a full cache, the cost of an insert grows linearly withmaxsize, and RRCache becomes slower than cachetools' pure-PythonRRCachefrom ~10k entries.Fix
Pick a random bucket index and keep the first occupied one (
RawTable::checked_bucket), up to 32 tries, then fall back to the previousiter().nth(random).capacity, thinned by removals, or a few heavy entries undergetsizeof) still work: they fall back to the old O(n) pick.Results
Get-or-set loop on a pre-filled cache, key space 4x
maxsize(miss-heavy). ns/op, lower is better, best of 3 interleaved runs.mainand this branch built the same way (maturin develop --release), CPython 3.13, macOS arm64.maxsizerandom_key(): mainTests
New tests in
TestRRCache:random_key()is uniform in a full table and in a sparse one (the fallback path),popitem()drains a sparse table, and eviction keeps the cache full. The statistical checks use a ±30% tolerance, about 10 standard deviations, so they don't flake.Full suite in the CI configuration (debug,
--features use-small-offset,HYPOTHESIS_PROFILE=slow,-n auto): 1152 passed on 3.13, 1151 passed on free-threaded 3.14t. ThePython Testworkflow passes on the fork across the whole matrix (3.10–3.15, 3.14t/3.15t, PyPy, ubuntu/macos/windows).Independent of #114 (the
HashTablequestion): this keeps the vendoredRawTable.