Repository navigation
v4: cache hits reuse the deserialized result for provably-shareable types - #3349
Merged
Merged
Conversation
…e types An Entry memoizes ONE parsed result with the handler that produced it (first writer wins, handler-identity keyed, carried across slab compaction); the hit funnel serves the memo before touching the reader. Core auto-enrolls strings and no-reference value types (IsReferenceOrContainsReferences, conservative down-level); richer rules opt in through the new public IRespResultReuse<TResult> on the handler - the HybridCache integration seam.
…zeOf, and the core sizes what it enrolls The charge is paired with the Memo object itself - set wins charge, whoever takes the memo (removal, replacement, slab move via ownership transfer, with a tombstone closing the slot) credits exactly what was charged - so MaxBytes cannot drift under the discard/late-set race. Strings are header+UTF-16, boxed no-ref structs are header+Unsafe.SizeOf; handler-opted results volunteer an estimate, 0 meaning uncounted.
This was referenced Nov 12, 2026
* Per-key removal notification for the client-side cache
A context enrolled via RespDatabaseContext.WithRemovalHandler is told - with
the key and a reason - when a key it read stops being servable from the
client-side cache, so layers above (HybridCache tag memos, backplane
adapters) can keep derived state coherent without polling.
Observers hang on the key-table NODE, the one object every death already
touches: invalidation fires promptly while the corpse is still resident,
flush walks the nodes it already walks, and slab compaction is not a
removal. Enrollment is one-shot - armed by a serve, consumed by the first
removal at or after it, re-armed by reading again - which makes exactly-once
structural (an Interlocked take, as the result memo does) and bounds
dangling handlers. Arm-then-recheck closes the serve/death race, and a value
handed out already dead (a fill that raced an invalidation, a stale-grace
serve) is announced immediately rather than silently.
Dispatch is the library's existing model: never on the caller's thread -
the socket reader, the table's locks, the evictor - but queued to the
thread pool like pub/sub and connection events, with exceptions swallowed
per handler. No key copy: Node.Key is already immortal.
Reasons separate value-changed (Invalidated, LocalWrite, Replaced) from
cache hygiene (Evicted, Expired) and the bulk case (Flushed); a
correctness-only consumer treats them all as drop. A fill whose tracking
prefix was dropped enrolls nothing: no invalidation is coming for it to
hear.
* Per-caller cache behaviour is one policy, not a wither per knob
WithMaxCacheAge and WithRemovalHandler are gone; the context carries a
whole CachePolicy (WithCachePolicy, on both public context types), and
CachePolicy - now a record, so callers derive with 'policy with {...}' -
gains RemovalHandler. The surface stops growing contexts-times-knobs:
a new read-time knob is one init property, and the first ship had
already dropped a cell of that matrix (no WithRemovalHandler on
RespServerContext).
A per-read policy replaces the default wholesale and narrows, never
widens: the default TimeToLive stays a ceiling, reclamation follows the
default policy alone, and a per-read grace window only functions when
the default policy enabled stamping. RefreshAfter is now genuinely
per-caller. A RemovalHandler on the DefaultPolicy is connection-wide
enrollment - every read arms it, including background refresh
completions - which is what a backplane-style consumer wants, and has a
new test.
The policy's tick conversions are memoised at construction (the old
MaxCacheAgeService did this; now the default-policy hit path gets it
too), which surfaced a static-ordering trap: Default must be declared
after the field its construction reads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cache hits currently re-parse the retained RESP payload on every serve. For HybridCache-style consumers (and plain
string/scalar traffic), the deserialized result is often provably safe to hand back as the same instance — this PR memoizes it on the cache entry.Design
Each cache entry gains one memo slot: an immutable
(parser, value, bytes)atom, CAS'd in on a first-wins basis. There is no map and no lookup: a fetch is a singleReferenceEqualson the parser reference, so a different parser instance for the same entry simply re-parses (never a wrong result), and the slot never churns.The decision happens post-parse in the send funnel — no flag threads through
RespReaderor the context.The core auto-enrolls only provably-shareable results:
string, and (where the runtime can detect it,#if NET) structs with no interior references viaRuntimeHelpers.IsReferenceOrContainsReferences<T>(); down-level falls back to primitives and enums only.Richer rules are opt-in via a new public capability side-interface alongside
IRespHandler<TResult>:This is how an external layer (e.g. HybridCache) teaches its deserializing handler to share results under its own reuse rules.
Budget accounting
A memoized value charges
MaxByteswhen its size is known: the core sizes its own enrollments (24 + 2·lenfor strings,16 + sizeof(T)for boxed no-ref structs), and opted-in handlers volunteer viaSizeOf(0 = uncounted). The charge is paired with the memo atom itself: whoever takes the memo (eviction, replacement, slab compaction — which transfers ownership rather than copying) credits back exactly what was debited, and taking leaves a tombstone so a late setter racing a discard loses the CAS and charges nothing. The budget cannot drift, lock-free.Scope cuts (documented in code)
Tests
11 new tests in
RespCacheResultReuseTestscover instance-reuse for strings and no-ref structs, non-enrollment of ref-bearing structs (RedisValue), handler opt-in/refusal per value, parser-instance guarding, memo death with the entry on local writes, and exact charge/credit of both core-sized and volunteered sizes. The 270-test cache/tracking sweep and all-TFM library build are clean.