Repository navigation
Per-key removal notification for the client-side cache - #3350
Merged
mgravell merged 2 commits intoNov 12, 2026
Merged
Conversation
A context enrolled via RespDatabaseContext.WithRemovalHandler is told - with the key and a reason - when a key it read stops being servable from the client-side cache, so layers above (HybridCache tag memos, backplane adapters) can keep derived state coherent without polling. Observers hang on the key-table NODE, the one object every death already touches: invalidation fires promptly while the corpse is still resident, flush walks the nodes it already walks, and slab compaction is not a removal. Enrollment is one-shot - armed by a serve, consumed by the first removal at or after it, re-armed by reading again - which makes exactly-once structural (an Interlocked take, as the result memo does) and bounds dangling handlers. Arm-then-recheck closes the serve/death race, and a value handed out already dead (a fill that raced an invalidation, a stale-grace serve) is announced immediately rather than silently. Dispatch is the library's existing model: never on the caller's thread - the socket reader, the table's locks, the evictor - but queued to the thread pool like pub/sub and connection events, with exceptions swallowed per handler. No key copy: Node.Key is already immortal. Reasons separate value-changed (Invalidated, LocalWrite, Replaced) from cache hygiene (Evicted, Expired) and the bulk case (Flushed); a correctness-only consumer treats them all as drop. A fill whose tracking prefix was dropped enrolls nothing: no invalidation is coming for it to hear.
WithMaxCacheAge and WithRemovalHandler are gone; the context carries a
whole CachePolicy (WithCachePolicy, on both public context types), and
CachePolicy - now a record, so callers derive with 'policy with {...}' -
gains RemovalHandler. The surface stops growing contexts-times-knobs:
a new read-time knob is one init property, and the first ship had
already dropped a cell of that matrix (no WithRemovalHandler on
RespServerContext).
A per-read policy replaces the default wholesale and narrows, never
widens: the default TimeToLive stays a ceiling, reclamation follows the
default policy alone, and a per-read grace window only functions when
the default policy enabled stamping. RefreshAfter is now genuinely
per-caller. A RemovalHandler on the DefaultPolicy is connection-wide
enrollment - every read arms it, including background refresh
completions - which is what a backplane-style consumer wants, and has a
new test.
The policy's tick conversions are memoised at construction (the old
MaxCacheAgeService did this; now the default-policy hit path gets it
too), which surfaced a static-ordering trap: Default must be declared
after the field its construction reads.
Member
Author
|
Reworked per review discussion: the per-knob withers are gone. |
mgravell
added a commit
that referenced
this pull request
Nov 12, 2026
…ypes (#3349) * spike: cache hits reuse the deserialized result for provably-shareable types An Entry memoizes ONE parsed result with the handler that produced it (first writer wins, handler-identity keyed, carried across slab compaction); the hit funnel serves the memo before touching the reader. Core auto-enrolls strings and no-reference value types (IsReferenceOrContainsReferences, conservative down-level); richer rules opt in through the new public IRespResultReuse<TResult> on the handler - the HybridCache integration seam. * A memoized result charges the cache budget: IRespResultReuse gains SizeOf, and the core sizes what it enrolls The charge is paired with the Memo object itself - set wins charge, whoever takes the memo (removal, replacement, slab move via ownership transfer, with a tombstone closing the slot) credits exactly what was charged - so MaxBytes cannot drift under the discard/late-set race. Strings are header+UTF-16, boxed no-ref structs are header+Unsafe.SizeOf; handler-opted results volunteer an estimate, 0 meaning uncounted. * Per-key removal notification for the client-side cache (#3350) * Per-key removal notification for the client-side cache A context enrolled via RespDatabaseContext.WithRemovalHandler is told - with the key and a reason - when a key it read stops being servable from the client-side cache, so layers above (HybridCache tag memos, backplane adapters) can keep derived state coherent without polling. Observers hang on the key-table NODE, the one object every death already touches: invalidation fires promptly while the corpse is still resident, flush walks the nodes it already walks, and slab compaction is not a removal. Enrollment is one-shot - armed by a serve, consumed by the first removal at or after it, re-armed by reading again - which makes exactly-once structural (an Interlocked take, as the result memo does) and bounds dangling handlers. Arm-then-recheck closes the serve/death race, and a value handed out already dead (a fill that raced an invalidation, a stale-grace serve) is announced immediately rather than silently. Dispatch is the library's existing model: never on the caller's thread - the socket reader, the table's locks, the evictor - but queued to the thread pool like pub/sub and connection events, with exceptions swallowed per handler. No key copy: Node.Key is already immortal. Reasons separate value-changed (Invalidated, LocalWrite, Replaced) from cache hygiene (Evicted, Expired) and the bulk case (Flushed); a correctness-only consumer treats them all as drop. A fill whose tracking prefix was dropped enrolls nothing: no invalidation is coming for it to hear. * Per-caller cache behaviour is one policy, not a wither per knob WithMaxCacheAge and WithRemovalHandler are gone; the context carries a whole CachePolicy (WithCachePolicy, on both public context types), and CachePolicy - now a record, so callers derive with 'policy with {...}' - gains RemovalHandler. The surface stops growing contexts-times-knobs: a new read-time knob is one init property, and the first ship had already dropped a cell of that matrix (no WithRemovalHandler on RespServerContext). A per-read policy replaces the default wholesale and narrows, never widens: the default TimeToLive stays a ceiling, reclamation follows the default policy alone, and a per-read grace window only functions when the default policy enabled stamping. RefreshAfter is now genuinely per-caller. A RemovalHandler on the DefaultPolicy is connection-wide enrollment - every read arms it, including background refresh completions - which is what a backplane-style consumer wants, and has a new test. The policy's tick conversions are memoised at construction (the old MaxCacheAgeService did this; now the default-policy hit path gets it too), which surfaced a static-ordering trap: Default must be declared after the field its construction reads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #3349 (
marc/v4-cache-result-reuse); only the last commit is new here.What
A caching layer built on top of the client-side cache (StackExchange.Redis.HybridCache's tag-timestamp memo is the concrete consumer; a FusionCache backplane adapter is a prospective second) keeps derived structures coherent with cached keys, and needs to be told when a key stops being current. This adds that: a per-context removal observer.
CacheKeyRemovedHandler—void (ReadOnlySpan<byte> key, CacheKeyRemovalReason reason); the span is the key exactly as stored (prefixes included), valid only for the callback.CacheKeyRemovalReason—Invalidated(server push),LocalWrite(our own write),Replaced(a fresh fill displaced a live value, e.g. refresh-ahead),Expired(aged out, fired at physical reclamation),Evicted(budget),Flushed(flush/reconnect/dispose).Enrollment model: one-shot, armed by the read
You hear about the keys you read, nothing else. Each read through the context arms one notification for that key; the first removal at-or-after the read consumes it; reading again re-arms. That makes exactly-once structural (
Interlocked.Exchangetake), needs no unenrollment API, and bounds dangling handlers — a consumer that stops reading a key stops hearing about it after at most one trailing event, and dropping a notification on the floor is always safe.Observers hang on the key-table node, not the entry: invalidation touches the node while entry release is lazy, nodes are immortal and carried by reference across growth, and
Node.Keyis already an immortalbyte[]— so slab compaction is invisible (not a death, enrollment survives) and dispatch needs no key copy.The contract holds under the races:
Invalidated.Dispatch
Never on the killing thread.
NotifyRemovedtakes the observer set and queues oneThreadPool.QueueUserWorkItem— the same model as pub/sub (CompleteAsWorker) — so user code never runs on the socket reader, under the key table's all-stripe locks (flush queues under them, fires after), or on the evictor. Handler exceptions are swallowed per handler. Cost when unused: one null field per key-table node and a null check per death.Scope cuts, documented in code: a fill whose
CLIENT CACHING YESprefix was dropped enrolls nothing (no invalidation will ever come), error replies likewise, and the shared-fill fallback send has no dependencies to enroll. Under default admission the doorkeeper's first-miss read isn't a fill, so coverage starts when caching does.Tests
14 new tests in
RespCacheRemovalTestscover every contract point: each reason, one-shot/re-arm, the fill-race (an executor that invalidates mid-send), stale-grace announcement, two consumers, a throwing consumer not silencing another, prefix fidelity, compaction-is-not-death (real slab promotion churn), and residency at fire time. Full cache/tracking sweep: 284/284 across 17 classes; all TFMs build clean.