Repository navigation
perf: mix Python hashes so hashbrown control tags are well distributed - #111
Conversation
The vendored hashbrown takes its 7-bit SIMD control tag from the top bits of the hash. CPython hashes are often low-entropy there: hash(n) == n for small ints, and many floats hash to small values. All such keys got tag 0, so every group probe matched every occupied slot and fell back to Python __eq__ calls. Fold-multiply the hash once in pyobject_hash (as foldhash/wyhash do). The mixed value is only used inside the tables; it is never exposed or pickled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KRuJihprbupFprYtRYAs92
|
Thanks a lot ❤️🔥 I’ll review and merge both of your PRs ASAP. (#112) |
Entries expired at t≈3s and the sweeper woke via Event.wait(3) at the same moment. Event.wait() may return a few ms early (Windows timer resolution is ~15.6 ms), so the sweep could run just before expiry; the next one was at t≈6s, after the assert at 3.5s. Seen as `assert 3 == 0` on windows-latest. Use ttl=1 with sweep_interval=1 and assert at 2.5s, so at least one full sweep runs after expiry. Also makes the test a second faster. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KRuJihprbupFprYtRYAs92
|
About Test failure on windows: It looks like a pre-existing timing race: entries expire at t≈3s and the sweeper wakes via Event.wait(3) at the same moment; on Windows the wait can return a few ms early (~15.6 ms timer resolution), so the sweep runs before expiry and the next one is at t≈6s, after the assert at 3.5s. Making the test robust (e.g. sweep_interval=1 and sleep(4.5)) |
|
I m trying to fix the problem with test on wndows |
Equal numbers (1, 1.0, True) share one entry, keys with equal Python hashes (-1, -2) stay distinct, fully colliding keys work, and low-entropy keys (small/negative/huge ints, floats) round-trip and delete correctly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Yes I noticed that. I was fixing it myself when you also fixed it 😂❤️ |
|
Done
|
|
The impact of this change on the |
The vendored hashbrown takes its 7-bit SIMD control tag from the top bits of the hash. CPython hashes are often low-entropy there: hash(n) == n for small ints, and many floats hash to small values. All such keys got tag 0, so every group probe matched every occupied slot and fell back to Python eq calls.
Fold-multiply the hash once in pyobject_hash (as foldhash/wyhash do). The mixed value is only used inside the tables; it is never exposed or pickled.
update: Fix for a flaky test
TestTTLCachePolicy::test_sweep_interval failed on windows-latest with assert 3 == 0. It is a pre-existing timing race unrelated to this change: entries expired at t≈3s while the sweeper woke via Event.wait(3) at the same moment, and on Windows the wait can return a few ms early, so the next sweep only ran at t≈6s, after the assert at 3.5s. The test now uses ttl=1, sweep_interval=1 and asserts at 2.5s.