Per-key removal notification for the client-side cache

docs/ClientSideCaching.md

@@ -122,8 +122,9 @@ Controlling it
 | `TimeToLive` | 1 minute | The longest an entry is ever served, whether or not it was invalidated. A safety bound against missed invalidations and key expiry. |
 | `RefreshAfter` | off | Once an entry is this old, a read still gets it, *and* a refresh is started in the background - so a hot key never goes from "cached" to "missing" for everyone at once. |
 | `InvalidationGracePeriod` | off | After *another client's* change invalidates an entry, keep serving the old value for up to this long while a refresh runs. This deliberately serves a value the server has said is out of date, to protect a very hot key from every reader missing at the same instant. It never applies to your own writes. |
+| `RemovalHandler` | none | An observer told when a key read under this policy stops being servable - invalidated, replaced, evicted, expired or flushed - for layers that keep derived state coherent with cached keys. On the `DefaultPolicy` it hears about every cached read; on a per-caller policy (see [below](#per-caller-policy-withcachepolicy)) only about that caller's. |
 
-Entry ages - `TimeToLive`, `RefreshAfter` and `WithMaxCacheAge` - are measured on a coarse clock that is cheap enough to read on every hit, so they are accurate to a few milliseconds (about 16 ms on Windows). `InvalidationGracePeriod` is measured precisely.
+Everything in a `CachePolicy` is applied when an entry is *read*, never stamped when it is stored - the entry is shared, and one copy serves callers with different tolerances. Entry ages - `TimeToLive` and `RefreshAfter` - are measured on a coarse clock that is cheap enough to read on every hit, so they are accurate to a few milliseconds (about 16 ms on Windows). `InvalidationGracePeriod` is measured precisely.
 
 ### Serving stale while refreshing
 
@@ -139,7 +140,7 @@ How the window behaves, for both:
 - **A failed refresh is retried by a later read.** The old value goes on being served for the rest of the window, then the entry expires as usual.
 - **A change wins over a refresh in flight.** If the key changes while a refresh is on the wire, the refresh's reply is not stored, exactly as for an ordinary fill.
 - **The grace period is measured from the invalidation**, not from the first read to notice it, and it is the cap. A key nobody reads during the window simply expires - nothing resurrects it later - and on a key written faster than it can be refreshed, stale answers stop when the window ends rather than continuing indefinitely.
-- **Neither extends an entry's life.** `TimeToLive` remains the longest any entry is served, and a context's `WithMaxCacheAge` is honoured inside the window too: a caller who asked for fresher gets a miss, not a stale answer.
+- **Neither extends an entry's life.** `TimeToLive` remains the longest any entry is served, and a caller's own policy (`WithCachePolicy`) is honoured inside the window too: a caller who asked for fresher gets a miss, not a stale answer.
 
 When the window does **not** apply:
 
@@ -271,13 +272,24 @@ What that gives you:
   var hot = await db.Strings.GetAsync(key, CommandFlags.CacheOptIn);
   ```
 
-- **`WithMaxCacheAge(TimeSpan)`** - a context that accepts cached answers only up to a given age. It can only tighten `TimeToLive`, never extend it:
+- <span id="per-caller-policy-withcachepolicy">**`WithCachePolicy(CachePolicy)`**</span> - a context whose reads follow their own policy instead of the cache's `DefaultPolicy`. It *replaces* the default for those reads rather than merging with it, so derive it from the policy you mean (`policy with { ... }`), and it narrows rather than widens: the default policy's `TimeToLive` remains a ceiling, and how long the shared entry actually lives stays the cache's own business. A caller who wants very fresh answers, for instance:
 
   ```csharp
-  var fresh = db.Context.WithMaxCacheAge(TimeSpan.FromSeconds(1));
+  var fresh = db.Context.WithCachePolicy(new CachePolicy { TimeToLive = TimeSpan.FromSeconds(1) });
   var value = await fresh.Strings.GetAsync(key);
   ```
 
+  The same policy object carries a `RemovalHandler`, for a caller that wants to hear when the keys *it* read stop being current:
+
+  ```csharp
+  var observed = db.Context.WithCachePolicy(new CachePolicy
+  {
+      RemovalHandler = static (key, reason) => InvalidateDerivedState(key),
+  });
+  ```
+
+  Enrollment rides the read: each read arms one notification for that key, the first removal at or after it fires it (from the thread pool, never the connection's reader), and reading again re-arms. Contexts and policies are cheap and immutable - build them once and reuse them.
+
 - **`WithoutCache()`** - a context that bypasses the cache entirely:
 
   ```csharp
@@ -314,5 +326,5 @@ See also
 
 - [Using RESP3](Resp3)
 - [Configuration](Configuration)
-- [Command groups](Groups) - contexts, and where `WithMaxCacheAge` / `WithoutCache` fit
+- [Command groups](Groups) - contexts, and where `WithCachePolicy` / `WithoutCache` fit
 - [Redis: client-side caching](https://redis.io/docs/latest/develop/reference/client-side-caching/)

src/StackExchange.Redis/Caching/CacheKeyRemoval.cs

@@ -0,0 +1,65 @@
+using System;
+
+namespace StackExchange.Redis.Caching
+{
+    /// <summary>
+    /// Why a key the caller had read through the client-side cache stopped being servable.
+    /// </summary>
+    /// <remarks>
+    /// <para>
+    /// <b>A correctness-only consumer treats every reason as "drop".</b> The split exists for smarter
+    /// consumers, separating "the value changed, or may have" (<see cref="Invalidated"/>,
+    /// <see cref="LocalWrite"/>, <see cref="Replaced"/>) from cache hygiene the server knows nothing
+    /// about (<see cref="Evicted"/>, <see cref="Expired"/>) and from the bulk case
+    /// (<see cref="Flushed"/>).
+    /// </para>
+    /// </remarks>
+    public enum CacheKeyRemovalReason
+    {
+        /// <summary>The server said the key changed: an invalidation push or keyspace notification.</summary>
+        Invalidated,
+
+        /// <summary>This process wrote the key itself - a stronger statement than a server push, because it is a fact rather than a race.</summary>
+        LocalWrite,
+
+        /// <summary>The cached value was replaced by a fresh fill; the value that was read is gone.</summary>
+        Replaced,
+
+        /// <summary>The entry aged past the cache's lifetime - the backstop against missed invalidations - and was reclaimed.</summary>
+        /// <remarks>
+        /// Expiry is decided lazily, so this fires at physical reclamation rather than at the moment the
+        /// lifetime passed. A consumer with its own freshness bound should keep enforcing it; this reason
+        /// is the backstop's backstop, not a timer.
+        /// </remarks>
+        Expired,
+
+        /// <summary>The entry was evicted to stay inside the cache's budget; the server may still be tracking the key.</summary>
+        Evicted,
+
+        /// <summary>Everything went at once: a server flush, a lost connection, or tracking being torn down.</summary>
+        Flushed,
+    }
+
+    /// <summary>
+    /// Told when a key read under an enrolled policy stops being servable from the client-side
+    /// cache; see <see cref="CachePolicy.RemovalHandler"/>.
+    /// </summary>
+    /// <param name="key">
+    /// The key exactly as stored - prefixes included - valid only for the duration of the callback.
+    /// </param>
+    /// <param name="reason">Why the key stopped being servable.</param>
+    /// <remarks>
+    /// <para>
+    /// <b>Called on a thread-pool thread, never on the connection's reader</b> - the same dispatch model
+    /// as pub/sub handlers and connection events - so being notified cannot stall the connection.
+    /// Exceptions are swallowed: a notification is advice, and advice must not break the cache.
+    /// </para>
+    /// <para>
+    /// <b>One enrollment, one notification.</b> An enrollment is armed by a serve and consumed by the
+    /// first removal at or after it; reading the key again re-arms it. A notification can therefore
+    /// arrive after the consumer has already re-read the key - dropping derived state is always safe, so
+    /// a late notification costs a rebuild, never correctness.
+    /// </para>
+    /// </remarks>
+    public delegate void CacheKeyRemovedHandler(ReadOnlySpan<byte> key, CacheKeyRemovalReason reason);
+}

src/StackExchange.Redis/Caching/CachePolicy.cs

@@ -18,19 +18,33 @@ namespace StackExchange.Redis.Caching
     /// about the connection and live on <see cref="CacheOptions"/> instead.
     /// </para>
     /// <para>
-    /// The freshness tolerance a particular caller will accept rides on the context - see
-    /// <see cref="RespContext.WithMaxCacheAge"/> - and is read-time for exactly the same reason.
+    /// The same read-time nature is what lets a <b>whole policy ride on the context</b> - see
+    /// <see cref="RespDatabaseContext.WithCachePolicy"/> - so a caller with a different freshness
+    /// tolerance, or a removal observer to arm, states it per call without changing how the shared entry
+    /// behaves for anyone else. A per-read policy <b>replaces</b> the cache's
+    /// <see cref="CacheOptions.DefaultPolicy"/> for those reads rather than merging with it, so derive it
+    /// from the policy you mean: <c>policy with { TimeToLive = ... }</c>.
     /// </para>
     /// <para>
-    /// It also keeps <see cref="RespContext"/> at its 48 bytes: a context carries a reference to shared
-    /// policy plus at most one override, rather than a field per knob.
+    /// It also keeps <see cref="RespContext"/> at its 48 bytes: a context carries a reference to a shared
+    /// policy, rather than a field per knob.
     /// </para>
     /// </remarks>
-    public sealed class CachePolicy
+    public sealed record CachePolicy
     {
+        // BEFORE Default: statics initialise in declaration order, and Default's construction reads this
+        private static readonly TimeSpan DefaultTimeToLive = TimeSpan.FromMinutes(1);
+
         /// <summary>The default policy, used when none is given.</summary>
         public static CachePolicy Default { get; } = new();
 
+        private readonly TimeSpan _timeToLive = DefaultTimeToLive;
+        private readonly TimeSpan _refreshAfter;
+        private readonly TimeSpan _invalidationGracePeriod;
+        private readonly long _timeToLiveTicks = ToTicks(DefaultTimeToLive);
+        private readonly long _refreshAfterTicks;
+        private readonly long _serveStaleTicks;
+
         /// <summary>
         /// The longest an entry may be served after it was fetched, regardless of invalidation.
         /// </summary>
@@ -49,8 +63,27 @@ public sealed class CachePolicy
         /// seconds, so the default of one minute is comfortably longer than detection while still bounding
         /// the damage when detection itself fails.
         /// </para>
+        /// <para>
+        /// On a per-read policy (<see cref="RespDatabaseContext.WithCachePolicy"/>) this is the caller's
+        /// own freshness requirement, and it <b>narrows, never widens</b>: the
+        /// <see cref="CacheOptions.DefaultPolicy"/>'s lifetime remains a ceiling, and reclamation follows
+        /// the default policy alone - a read policy changes what its own reads will accept, never how long
+        /// the shared entry lives.
+        /// </para>
         /// </remarks>
-        public TimeSpan TimeToLive { get; init; } = TimeSpan.FromMinutes(1);
+        public TimeSpan TimeToLive
+        {
+            get
+            {
+                return _timeToLive;
+            }
+
+            init
+            {
+                _timeToLive = value;
+                _timeToLiveTicks = ToTicks(value);
+            }
+        }
 
         /// <summary>
         /// How old an entry may get before a read refreshes it in the background, while still being served.
@@ -75,7 +108,19 @@ public sealed class CachePolicy
         /// retained to make it possible.
         /// </para>
         /// </remarks>
-        public TimeSpan RefreshAfter { get; init; } = TimeSpan.Zero;
+        public TimeSpan RefreshAfter
+        {
+            get
+            {
+                return _refreshAfter;
+            }
+
+            init
+            {
+                _refreshAfter = value;
+                _refreshAfterTicks = ToTicks(value);
+            }
+        }
 
         /// <summary>
         /// A grace period after an invalidation, during which the old value may still be served while a
@@ -117,8 +162,61 @@ public sealed class CachePolicy
         /// which is otherwise a few nanoseconds wide and sees every key the server mentions - so the cost
         /// lands only on those who asked for the feature.
         /// </para>
+        /// <para>
+        /// That stamping decision belongs to the <see cref="CacheOptions.DefaultPolicy"/>, because the
+        /// invalidation path cannot know which context will read next. So on a per-read policy this window
+        /// only functions when the default policy enabled one too; given that, the window used is this
+        /// policy's own - a caller can opt out of stale serving, or use a different window, still inside
+        /// <see cref="TimeToLive"/>.
+        /// </para>
+        /// </remarks>
+        public TimeSpan InvalidationGracePeriod
+        {
+            get
+            {
+                return _invalidationGracePeriod;
+            }
+
+            init
+            {
+                _invalidationGracePeriod = value;
+                _serveStaleTicks = ToPreciseTicks(value);
+            }
+        }
+
+        /// <summary>
+        /// A removal observer: reads that follow this policy arm it on the keys they serve, and it is told
+        /// when such a key stops being servable from the client-side cache.
+        /// </summary>
+        /// <remarks>
+        /// <para>
+        /// <b>You hear about the keys you read, nothing else.</b> Enrollment rides the read: any serve
+        /// under this policy - a cache hit, or the fill a miss performs - arms the observer on the keys it
+        /// depended on, and the first removal at or after that consumes the enrollment. Reading again
+        /// re-arms it. A key may have entered the cache through a context with no observer at all: a later
+        /// hit under this policy still enrolls.
+        /// </para>
+        /// <para>
+        /// <b>Per enrollment, exactly one notification, and never a missed removal</b> - including a fill
+        /// that raced an invalidation, and a value served from the stale-grace window, both of which fire
+        /// immediately: a value that is already not current is announced, not silently handed over. A
+        /// notification can arrive after a racing re-read, so treat it as "drop derived state", which is
+        /// always safe; it is advice about staleness, not an ordered event stream.
+        /// </para>
+        /// <para>
+        /// The observer is told the key (exactly as stored, prefixes included, valid only for the
+        /// callback) and why it went. It is called from the thread pool, never from the connection's
+        /// reader. Keep it <b>long-lived and cheap</b>: build the policy once and reuse it (the handler
+        /// reference is the identity enrollment dedupes on), and do only bookkeeping in the callback - it
+        /// shares the thread pool with everything else. Exceptions are swallowed.
+        /// </para>
+        /// <para>
+        /// On the <see cref="CacheOptions.DefaultPolicy"/> it is <b>connection-wide</b>: every cached read
+        /// enrolls it, and so does a background refresh completing - so a watched key stays watched across
+        /// its own replacement, for as long as anything keeps reading it.
+        /// </para>
         /// </remarks>
-        public TimeSpan InvalidationGracePeriod { get; init; } = TimeSpan.Zero;
+        public CacheKeyRemovedHandler? RemovalHandler { get; init; }
 
         /// <summary>Whether this policy asks for background refresh at all.</summary>
         internal bool RefreshesEarly => RefreshAfter > TimeSpan.Zero && RefreshAfter < TimeToLive;
@@ -131,13 +229,13 @@ public sealed class CachePolicy
         /// the ages below: a grace window can be tens of milliseconds, and it is only consulted for an entry that
         /// has already been invalidated, so its clock read is off the hit path.
         /// </summary>
-        internal long ServeStaleTicks => ToPreciseTicks(InvalidationGracePeriod);
+        internal long ServeStaleTicks => _serveStaleTicks;
 
-        /// <summary><see cref="RefreshAfter"/> in <see cref="CacheClock"/> ticks.</summary>
-        internal long RefreshAfterTicks => ToTicks(RefreshAfter);
+        /// <summary><see cref="RefreshAfter"/> in <see cref="CacheClock"/> ticks; memoised, the hit path reads it per serve.</summary>
+        internal long RefreshAfterTicks => _refreshAfterTicks;
 
-        /// <summary><see cref="TimeToLive"/> in <see cref="CacheClock"/> ticks.</summary>
-        internal long TimeToLiveTicks => ToTicks(TimeToLive);
+        /// <summary><see cref="TimeToLive"/> in <see cref="CacheClock"/> ticks; memoised, the hit path reads it per serve.</summary>
+        internal long TimeToLiveTicks => _timeToLiveTicks;
 
         /// <summary>A span in <see cref="CacheClock"/> ticks: the coarse clock that entry ages are measured on.</summary>
         internal static long ToTicks(TimeSpan value)
@@ -189,24 +287,4 @@ internal static class CacheClock
         internal static long Now => Stopwatch.GetTimestamp();
 #endif
     }
-
-    /// <summary>
-    /// A per-context override of how stale an answer the caller will accept.
-    /// </summary>
-    /// <remarks>
-    /// The one piece of cache configuration that cannot be added to the existing surface:
-    /// <c>IDatabase.StringGet</c> cannot grow a parameter without a binary break, whereas the context
-    /// reaches every command without touching a single signature.
-    /// <para>
-    /// Applied when the entry is <b>read</b>, never stamped when it is stored - the entry is shared, so one
-    /// copy has to serve callers with different tolerances. Stamping at store time would force identical
-    /// replies to be cached once per distinct lifetime, which is the opposite of what a shared cache is for.
-    /// </para>
-    /// </remarks>
-    internal sealed class MaxCacheAgeService(TimeSpan maxAge)
-    {
-        internal TimeSpan MaxAge { get; } = maxAge;
-
-        internal long Ticks { get; } = CachePolicy.ToTicks(maxAge);
-    }
 }

src/StackExchange.Redis/Caching/RespClientCache.Slabs.cs

@@ -268,7 +268,12 @@ private bool TryEvictOldestSlab(ref int evicted)
                 }
                 else if (entries.Remove(new KeyValuePair<EntryKey, Entry>(key, entry)))
                 {
-                    Release(entry);
+                    // aged out, or live-but-unread in the slab being taken: Expired or Evicted says which;
+                    // an invalidated one already announced itself and Release says nothing for it
+                    var reason = CachePolicy.IsOlderThan(entry.FilledAt, lifetime)
+                        ? CacheKeyRemovalReason.Expired
+                        : CacheKeyRemovalReason.Evicted;
+                    Release(entry, reason);
                     evicted++;
                 }
             }
@@ -320,8 +325,8 @@ private void Promote(in EntryKey key, Entry entry, Slab from, ref int evicted)
             }
 
             // a fill re-added the key between the remove and the add; theirs is as good as ours, and the old entry
-            // is simply gone
-            Release(entry);
+            // is simply gone - superseded by that fill, which is what Replaced means
+            Release(entry, CacheKeyRemovalReason.Replaced);
             if (slab is not null) ReleaseSlab(slab);
             evicted++;
         }