diff --git a/frontend/content/caching.mdx b/frontend/content/caching.mdx index 2adc939..1f8ca82 100644 --- a/frontend/content/caching.mdx +++ b/frontend/content/caching.mdx @@ -6,57 +6,945 @@ category: "Concepts" ## Getting Started -Caching stores computed or retrieved data closer to the consumer so repeated requests are faster. It is a key technique for improving performance and scaling systems. +Caching is one of the most important performance techniques in modern software design. It reduces latency, decreases unnecessary computation, lowers infrastructure cost, and helps applications scale under high traffic. In practice, caching means storing a previously computed or retrieved result in a place that can be accessed more quickly than the original source of truth. + +Caching is not a single mechanism. It exists at several layers of the system stack, from the browser and CDN to application memory and distributed data stores. Each layer has different tradeoffs around freshness, consistency, capacity, and complexity. A strong caching strategy is about choosing the right layer, the right policy, and the right invalidation approach for the workload being served. + +A good cache improves user experience by making common reads feel instantaneous. A poor cache introduces subtle bugs, stale data, memory pressure, or invalidation churn. The goal is to make caching predictable and intentional rather than incidental. + +--- + +## Why caching matters + +- Caching reduces repeated database queries and remote API calls. +- It shortens response times for read-heavy workloads. +- It lowers backend load during traffic spikes and seasonal bursts. +- It can greatly improve throughput without immediately scaling infrastructure. +- It helps systems remain responsive when upstream services are slower than normal. +- It reduces the cost of repeated computation, serialization, and network transfer. +- It is often the first optimization to apply before introducing more complex architecture changes. +- It improves scalability by shifting work away from origin systems and toward nearby fast storage. +- It increases the effective capacity of the system without requiring a proportional increase in resources. +- It protects critical services from thundering herds caused by popular content or synchronized requests. +- It can reduce cold-start behavior in serverless, edge, and stateless deployments. +- It can increase reliability when upstream systems temporarily degrade. +- It can support graceful service degradation by serving recent data from cache while the source recovers. +- It can improve developer productivity by reducing the cost of repeated expensive operations in local development and testing. +- It can also create complexity when cache behavior is misunderstood or poorly managed. +- It is most effective when applied to stable or predictably repetitive access patterns. +- It is less useful for highly personalized, rapidly changing, or infrequently reused data. +- Good caching requires a clear understanding of read patterns, hot data, and the cost of recomputation. +- Bad caching often comes from treating all data as equally cacheable. +- Caching should be part of the system design from the start, not an afterthought. +- Cache design becomes more important as system scale increases. +- The right caching layer depends on latency targets, consistency expectations, and infrastructure constraints. +- Different consumers may require different cache policies even when they read the same underlying object. +- The same resource can be cached in multiple places at the same time, each with its own lifetime and invalidation rule. +- Every cache layer needs an eviction strategy, a freshness policy, and a clear failure mode. +- Caching is as much about tradeoffs as it is about speed. +- A cache is usually a performance optimization, not a source of truth. +- A cache that is never invalidated becomes a growing source of incorrect behavior. +- A cache that is invalidated too aggressively loses much of its value. +- The best cache policies are shaped by real traffic and data access characteristics. +- Caching is often used for API responses, HTML pages, database query results, object payloads, and computed aggregates. +- In distributed systems, cache coherence and consistency become essential design concerns. +- Effective caching often combines several techniques, such as CDNs, reverse proxies, application memory, and databases. +- Each layer can serve different kinds of data at different costs and granularities. +- The fastest cache is usually the one closest to the consumer, but also the most limited in scope. +- The most durable cache is usually the one farthest from the consumer, but also the least responsive. +- A layered cache strategy often provides the best balance between speed and correctness. +- The best systems treat caches as first-class infrastructure rather than incidental implementation details. --- ## Cache locations -- Browser cache -- CDN cache -- In-memory cache -- Distributed cache (Redis, Memcached) +- Browser cache stores responses locally in the client and serves them on subsequent visits. +- CDN cache stores content closer to users at edge locations around the world. +- Reverse proxy cache stores responses closer to the application server and can reduce origin traffic. +- In-memory cache stores data in process memory for extremely low-latency access. +- Distributed cache such as Redis or Memcached allows multiple application instances to share cached data. +- Database query cache can store repeated query results, although modern databases often manage this internally. +- Object storage and blob caches can store large media payloads for fast retrieval. +- Search index caches can accelerate repeated search and filter operations. +- Session stores often act as caches for user state and authentication data. +- Application-level caches are common for computed results, configuration objects, and frequently used references. +- Edge caches are powerful for static assets, public content, and read-heavy APIs. +- Local caches improve performance for single-node or single-request workflows. +- Shared caches improve consistency between multiple application instances. +- Hierarchical caches combine short-lived local caches with longer-lived shared caches. +- A browser cache is ideal for assets that do not change often and are safe to reuse. +- A CDN cache is ideal for media and public content that many users may request. +- An in-memory cache is ideal for hot objects that are read repeatedly in a short window. +- A distributed cache is ideal when multiple services need the same data quickly. +- The layer chosen should reflect the cost of a cache miss and the cost of serving stale data. +- Each cache layer should have a defined size, eviction policy, and lifetime. +- A cache miss should still be a manageable path rather than a correctness hazard. +- A cache hit should avoid unnecessary network hops and CPU work. +- Cache layers should ideally be transparent to the caller unless the caller explicitly needs freshness guarantees. +- Keeping too many layers can increase operational complexity and obscure where data originates. +- The most effective designs use the minimum set of layers needed to meet latency and scale goals. +- Some systems use a three-tier approach: browser, CDN, and application cache. +- Others rely on a local process cache plus a shared distributed cache. +- The architecture should reflect your data shape and request distribution. +- A single request may traverse several caches before reaching the origin service. +- Observing which layers serve the data is important when diagnosing performance issues. +- Cache placement should consider topology, user geography, and access frequency. +- Some workloads benefit from regional caches in addition to global caches. +- Caching is especially effective for immutable or semi-immutable content. +- It is less effective when each request is truly unique and cannot be shared. +- Public content can usually be cached aggressively, whereas private or sensitive data requires stricter controls. +- The closer the cache is to the consumer, the faster it is, but the less control it may have. +- A distributed cache is usually slower than local memory but offers better shared access. +- There is a natural tradeoff between locality, capacity, and consistency. --- ## Cache keys -Use stable, unique keys that clearly identify the cached item. +- A cache key is the identifier for a specific cached item. +- A key should be stable enough to reuse across requests and specific enough to avoid collisions. +- Keys should reflect the logical identity of the data rather than incidental implementation details. +- A simple key can be a string, but composite keys are common for multi-attribute queries. +- A user profile may be keyed by a user identifier plus a version or scope. +- A product page might be keyed by product identifier, locale, and currency. +- A query result might be keyed by the normalized query text, filters, pagination offset, and sorting criteria. +- A cache key should ideally be deterministic so the same request produces the same key. +- Avoid using mutable or non-serializable objects directly as keys. +- Normalize inputs before generating keys to prevent equivalent requests from producing different keys. +- Use a consistent serialization format for complex keys. +- Consider including a version segment when data shape changes significantly. +- Consider including a tenant or organization identifier when data is partitioned by tenant. +- Consider including a region or locale when content differs by geography or language. +- The key grammar should be documented so engineers can reason about cache behavior. +- Keys should be short enough to be efficient while still descriptive. +- Prefixing with a domain name or resource category can make debugging easier. +- A common pattern is a namespace like `user:`, `product:`, `session:`, or `config:`. +- Example: `user:42:profile` is clearer than a raw numeric identifier. +- Example: `search:query=red+shoes&page=2&sort=price` can be normalized for consistent lookup. +- Example: `catalog:sku:12345:en-US:USD` captures the important dimensions of cacheability. +- If the underlying data changes by version, incorporate that version into the key. +- A versioned key can help avoid serving stale objects after a schema or format change. +- Keys should be treated as part of the contract of the system, not as an implementation accident. +- Avoid leaking sensitive data into cache keys if the keys may appear in logs or metrics. +- Use caution with query parameters that are not semantically relevant. +- If one parameter does not affect the result, omit it from the key to reduce fragmentation. +- Cache key fragmentation occurs when similar requests produce many unique keys and the cache becomes less effective. +- Key cardinality matters because a very high number of unique keys can dilute hit rate. +- A high-cardinality cache key can lead to memory growth and more misses. +- Sometimes it is better to cache a broader grouping than a very narrow one. +- For example, caching a page by slug is often better than caching by every combination of headers and cookies. +- If the data depends on authorization, include identity information in the key or avoid caching the response entirely. +- If a request includes a user-specific header and the user-specific data should not be shared, use a narrower scope. +- User-specific caches can be scoped by user ID or tenant ID. +- Shared caches should avoid mixing data that should not be shared across users or contexts. +- When the same logical entity is represented in different forms, align the key on the canonical representation. +- Key generation should be centralized in a helper or library to avoid inconsistent patterns. +- Consider using HMAC or hash functions for long or complex keys if the storage backend requires it. +- Example JavaScript key generation: + +```js +const cacheKey = `user:${userId}:profile:${locale}:${version}`; +``` + +- Example Node.js with a helper: ```js -const cacheKey = `user:${userId}:profile`; +function makeKey(prefix, params) { + return `${prefix}:${JSON.stringify(params)}`; +} ``` +- Example for a normalized query string: + +```js +const key = `search:${Buffer.from(query).toString('base64')}`; +``` + +- The key is often the first place where cache correctness fails. +- A well-defined key strategy can dramatically improve cache hit rate. +- A poorly defined key strategy can create duplication, misses, and silent inconsistencies. +- Always think about what dimensions of the request affect the content being served. +- If the response changes when the user role changes, the key should include role or access context. +- If the response changes when the locale changes, the key should include locale. +- If the response changes when the currency changes, the key should include currency. +- If the response changes when the region changes, the key should include region. +- If data is versioned, include the version in the key to avoid crossing boundaries. +- If a request is likely to be stormed, a shared key can reduce repeated upstream load. +- Many systems use a prefix and a hash of the payload dimensions to keep keys compact. +- The key design should be reviewed like any other API contract. +- In distributed caches, a poor key can turn into a hot key problem where one key is overly popular. +- A hot key can cause contention, latency spikes, and unnecessary replication pressure. +- If a key can become hot, consider partitioning or precomputing the data. +- Key design decisions should reflect access patterns, not just entity names. + --- -## Expiration +## Expiration and TTL -Set TTL values to avoid stale data. +- TTL stands for time to live and defines how long a cached value remains valid. +- A short TTL is appropriate for quickly changing data. +- A long TTL is appropriate for stable content that does not change often. +- Setting a TTL avoids indefinite growth of the cache and limits stale data exposure. +- The correct TTL depends on the volatility of the underlying source data. +- A user profile might have a TTL of a few minutes if it changes frequently. +- A product listing might have a TTL of several minutes or hours. +- A static asset might have a TTL of days or months. +- A configuration object might be cached for long periods and refreshed through explicit invalidation. +- A generated report might be cached for a very short time because it is expensive and often outdated. +- TTL is often a compromise between freshness and efficiency. +- A very short TTL gives freshness but low hit rate. +- A very long TTL gives hit rate but risks serving stale data. +- The value should be tuned using actual read patterns and update cadence. +- Example: ```js cache.set(key, value, { ttl: 60 }); ``` -Use shorter TTL for frequently changing data and longer TTL for stable content. +- In many systems, TTLs are expressed in seconds. +- Some systems use wall-clock expiration, while others use sliding expiration. +- Sliding expiration refreshes the TTL when the item is accessed. +- Sliding expiration can be useful for popular items that should remain warm while still being reused. +- Absolute expiration is simpler and easier to reason about. +- Absolute expiration is often a better fit for content with a known freshness window. +- Example of a short TTL for dynamic data: + +```js +cache.set('dashboard', dashboardData, { ttl: 15 }); +``` + +- Example of a longer TTL for article content: + +```js +cache.set('article:42', articleContent, { ttl: 3600 }); +``` + +- Sometimes the TTL should be coupled to the data source update interval. +- If the source updates every five minutes, a TTL of thirty seconds may be too short. +- If the source updates every hour, a TTL of ten minutes may be acceptable. +- If the source updates only when deployed, a TTL of one day may be fine. +- TTL can be tuned per endpoint or per resource type. +- A general default TTL should exist, but exceptions should be possible. +- Include a maximum TTL to avoid accidental long-lived cache entries. +- Consider a minimum TTL to avoid thrashing when the cache is constantly refreshed. +- Cache expiration should be coupled to operations that are known to change the data. +- When a write occurs, that action can also trigger invalidation or a short TTL refresh. +- Some systems use a stale-while-revalidate approach where stale data is returned while a refresh is triggered. +- Others use stale-if-error to serve stale content when the origin is unavailable. +- A TTL is not absolute correctness. It is a freshness deadline. +- The cache should know when the item expires and how to treat it. +- Expired entries should either be lazily refreshed or proactively removed. +- Expired entries should not be returned as fresh data. +- For highly sensitive or time-critical data, an expiration of zero or near-zero may be appropriate. +- For near-static reference data, a very long TTL may be acceptable. +- TTL should be reviewed regularly as workload patterns change. +- Data that becomes more dynamic may require shorter TTLs. +- Data that becomes more stable may justify longer TTLs. +- Some systems use a jittered TTL so many entries do not expire at the same moment. +- Jitter helps avoid synchronized cache stampedes during refreshes. +- Cache stampedes occur when many clients miss at once and all try to recompute the same value. +- A small amount of randomization can reduce stampede behavior. +- Example of jitter: + +```js +const ttl = 60 + Math.floor(Math.random() * 10); +cache.set(key, value, { ttl }); +``` + +- Many caches provide expiration at the item level, but some systems rely on global expiration policies. +- A local in-memory cache can expire based on elapsed time since insertion. +- A distributed cache can expire based on absolute time or on touch time. +- Expiration interacts with eviction policies and memory pressure. +- An item that expires but remains in memory may still consume resources until cleanup occurs. +- Some caching systems keep expired entries in a grace period until the next access. +- Grace periods can soften the impact of short-lived misses. +- The right expiration policy should be documented and tested under load. --- ## Cache invalidation -Invalidate cached data when the underlying source changes. +- Invalidation is the process of removing or updating cached data when the underlying source changes. +- Invalidation is one of the hardest parts of caching because it determines whether the cache remains correct. +- Without invalidation, caches can become stale and misleading. +- With too much invalidation, the cache loses effectiveness and behaves like a pass-through. +- The right invalidation strategy depends on write patterns, read patterns, and consistency requirements. +- Write-through means updates write to the cache at the same time as the storage layer. +- Write-through keeps the cache fresh on writes and is simple to reason about. +- Write-through is useful when reads are much more common than writes. +- Write-through can increase write latency because the write must complete both operations. +- Cache-aside means the application loads data from the cache first and falls back to the source if missing. +- Cache-aside is common in application-level caching because it keeps the cache management logic close to the application. +- Cache-aside is flexible and often easier to adopt incrementally. +- Write-around means the write goes directly to the data store and bypasses the cache. +- Write-around prevents the cache from being polluted by data that is written but rarely read. +- Write-back means writes update the cache first and are later flushed to the data store. +- Write-back can improve write performance but adds risk because the data may be lost if the cache fails. +- Write-back is common in certain storage systems and some in-memory designs, though less so at the application layer. +- Time-based expiry is a simple invalidation strategy where entries expire after a TTL. +- Time-based expiry works well when the data is not updated very often or when approximate freshness is acceptable. +- Explicit invalidation is more precise than time-based expiry. +- Explicit invalidation can be done on create, update, delete, or business events. +- Versioned invalidation uses a version marker that changes when the underlying data changes. +- Versioned invalidation is useful when many related objects share the same cache representation. +- Event-driven invalidation is common in distributed systems and message-driven architectures. +- A write event can trigger a message to remove or refresh the related cache entries. +- Invalidation should ideally be coordinated with the actual write transaction. +- If invalidation is delayed, the cache may briefly serve stale data. +- If invalidation is too early, the cache may miss and cause unnecessary load. +- There is no universal invalidation strategy; each pattern has risks and benefits. +- A hybrid strategy often works best: short TTL plus explicit invalidation when updates are critical. +- Invalidation should be tested under concurrent write and read workloads. +- Race conditions are common when one request reads stale data while another updates the source. +- A common mitigation is to use versioning, compare-and-swap, or a write-through path. +- In systems with multiple replicas, invalidation needs to be propagated through the cluster. +- Invalidation messages can be lost if messaging infrastructure is unreliable. +- Observability around invalidation events is critical for debugging. +- A stale entry that is not invalidated can persist indefinitely if the TTL is long and no refresh occurs. +- A cache that is invalidated too aggressively can result in a high-miss rate and poor performance. +- The cost of a stale read should be weighed against the cost of a cache miss. +- Some systems accept slightly stale data in exchange for higher hit rate and lower latency. +- Others require strict freshness and therefore use shorter TTLs or more aggressive invalidation. +- Data with strong consistency requirements may need to bypass cache or use synchronous invalidation. +- Data used for financial transactions or authentication usually requires stricter handling. +- Business logic should define whether stale data is acceptable and for how long. +- A cache invalidation policy should be reviewed by domain experts when data correctness matters. +- The policy for public content can be more permissive than the policy for private content. +- Cache invalidation around user-specific data is more complicated because the write may affect many user views. +- Invalidation can be scoped to a resource family, a tenant, or a specific entity. +- When the source is eventually consistent, the cache should reflect that expectation. +- When the source is strongly consistent, the cache must follow stricter update semantics. +- Some systems use dependency graphs to know which cached entries should be invalidated by a change. +- Others keep invalidation simple by invalidating a broader object group. +- Broad invalidation can be safer but less efficient. +- Narrow invalidation can be more efficient but more error-prone. +- A common compromise is to invalidate by entity and scope rather than every possible combination. +- Cache invalidation should be part of incident response playbooks because stale data can appear during outages. +- When a deployment changes behavior or cache format, the cache should be flushed or versioned. +- A deployment that introduces a new structure can produce incompatible cached data if expected keys change. +- Explicit cache busting is often used for static assets after deployment. +- Cache busting can be done by changing a versioned URL or content hash in the asset reference. +- For APIs, an ETag or version header can trigger fresh retrievals without re-downloading the full payload. +- Invalidation is easier when the system uses immutable resources where updates create a new version. +- Immutable content patterns support very efficient caching because old versions can remain accessible. +- Mutable content patterns require more attention to update propagation and eviction. + +--- + +## Cache-control and HTTP freshness + +- Web caches use HTTP headers to decide whether a response can be stored and reused. +- The most common header is `Cache-Control`. +- `Cache-Control: public` tells shared caches such as proxies and CDNs that they may store the response. +- `Cache-Control: private` restricts caching to a single user’s browser or client. +- `Cache-Control: no-store` prevents storing the response anywhere. +- `Cache-Control: no-cache` allows caching but requires revalidation before reuse. +- `Cache-Control: max-age=3600` allows the response to be reused for up to one hour. +- `Cache-Control: s-maxage=3600` applies the freshness limit specifically to shared caches. +- `stale-while-revalidate` allows stale data to be served while a fresh copy is fetched in the background. +- `stale-if-error` allows stale data to be served if the origin is unavailable. +- `ETag` provides a strong identifier for a response version. +- `Last-Modified` provides a timestamp that can be used for conditional requests. +- A conditional request with `If-None-Match` or `If-Modified-Since` can verify freshness without transferring the entire body. +- Example: + +```http +Cache-Control: public, max-age=3600, s-maxage=86400, stale-while-revalidate=60 +``` + +- Example with revalidation: + +```http +ETag: "abc123" +Cache-Control: max-age=60, must-revalidate +``` + +- Example for static assets: + +```http +Cache-Control: public, max-age=31536000, immutable +``` + +- Example for sensitive user data: + +```http +Cache-Control: private, no-store +``` + +- Browsers and intermediaries may honor the headers differently depending on their implementation. +- Cache-control is not only about storage; it is also about revalidation behavior. +- A `no-cache` directive does not mean the response cannot be cached; it means it cannot be used without revalidation. +- A `no-store` directive is stronger and should be used for private or sensitive content. +- Many CDNs offer cache rules that can be expressed through headers or configuration. +- A content delivery network may override or extend the origin behavior based on policy. +- CDN edge cache behavior should be modeled along with browser cache behavior for accurate performance expectations. +- Cache headers interact with the `Vary` header. +- `Vary: Accept-Encoding` is common for compressed responses. +- `Vary: Authorization` is important when responses differ by authentication state. +- Omitting `Vary` when the response differs by request context can cause cache poisoning or incorrect reuse. +- Cache poisoning is when a cached response is served to a different user because the cache did not distinguish the request context. +- Adding `Vary` is a common mitigation for this issue. +- Example: + +```http +Vary: Accept-Language, Authorization +``` + +- If a resource depends on request headers, those headers should be part of the cache key or the response should be excluded from shared caching. +- Public caches should not store responses that contain user-specific or sensitive data. +- Shared caches need to be careful about cookies and authorization headers. +- Some frameworks automatically set cache headers for static content and SSR pages. +- The HTTP caching model is mature, but subtle mistakes can still cause stale or incorrect behavior. +- A cache miss should not cause a full re-render if the underlying content is already available elsewhere. +- When using SSR, cache headers should be chosen based on whether the page is public or personalized. +- Public marketing pages can often have long-lived cache headers, while dashboards should generally be private and short-lived. +- If a response is cached by a CDN, the CDN edge node may serve it even when the browser has a fresh copy. +- Browser and proxy caches can work together to reduce origin load but can also create confusing freshness states. +- The cache-control strategy should be tested in development and staging before production launch. +- The system should be able to observe the actual cache response headers in production telemetry. +- The surrounding infrastructure should not silently ignore or override the intended policy. +- Caching at the HTTP layer is efficient for whole responses, but it is not the same as object caching inside the application. +- Application caches often store precomputed objects and partial views, while HTTP caches store complete responses. +- Both layers are useful and complementary. +- A cache header policy should be documented and reviewed whenever an endpoint changes shape or privacy requirements change. + +--- + +## Application-level caching patterns + +- Application-level caching stores objects in the process memory or a shared cache backing store. +- It is often used for configuration, session data, computed summaries, and expensive lookups. +- A local cache can be extremely fast because it avoids network overhead. +- A shared application cache can be used by multiple instances of the same service. +- The most common pattern is cache-aside. +- In cache-aside, the application first attempts to read from the cache. +- If the cache has the item, it returns the value. +- If the cache does not have the item, the application fetches it from the source and writes it to the cache. +- Example: + +```js +async function getUserProfile(userId) { + const key = `user:${userId}:profile`; + const cached = await cache.get(key); + if (cached) return cached; + + const profile = await db.getUserProfile(userId); + await cache.set(key, profile, { ttl: 300 }); + return profile; +} +``` + +- Write-through is another common pattern. +- In write-through, writes update the cache as part of the same operation. +- It helps maintain cache freshness after writes. +- It can increase write latency because it performs more work per operation. +- Write-around is used when writes are not likely to be read immediately or frequently. +- It keeps the cache from filling up with data that is rarely reused. +- Read-through is similar to cache-aside but the library or wrapper does the cache handling. +- Read-through is common in ORMs, frameworks, and distributed cache integrations. +- Write-back is more advanced and is often used in storage systems rather than web application code. +- Write-back can reduce write amplification but introduces durability concerns. +- The application should know whether it is dealing with a cache hit or a miss for debugging and observability. +- Cache implementations often expose metrics such as hit count, miss count, eviction count, and latency. +- A cache hit rate is a useful performance indicator but should not be interpreted in isolation. +- A high hit rate with stale data can still be a problem if freshness is critical. +- A low hit rate might be acceptable if the cached objects are very expensive to fetch and the miss path is rare. +- Application-level caches should consider memory pressure and eviction behavior. +- A cache that grows without limit can create memory pressure and cause garbage collection churn. +- Most engines allow a maximum number of entries or a maximum memory budget. +- A cache may evict the least recently used, least frequently used, or random entries. +- The eviction policy should match the access pattern of the workload. +- Short-lived hot data often benefits from a simple LRU or LFU policy. +- Very large objects may need special handling to avoid consuming too much memory. +- Some systems split the cache by object size or priority to avoid one large entry displacing many small ones. +- Application caches should also think about serialization cost. +- Storing data in a compact format can reduce memory usage and CPU overhead. +- Some languages provide strong support for object caching with automatic serialization and expiration. +- Others require the application to manage key translation and object encoding manually. +- Caches should avoid storing mutable objects that can be mutated elsewhere without invalidation. +- If the application stores references to mutable objects, the cached value may be modified unexpectedly. +- Copy-on-read or deep-cloning can be used to preserve immutability. +- Caching should be explicit in the code path so engineers understand where it occurs. +- Explicit cache wrappers can make it easier to reason about and test. +- Transparent caching can be convenient but may hide important behavior from developers. +- A system with many implicit caches may be harder to debug than one with a few deliberate ones. +- The application should define a fallback strategy when the cache service is down. +- A cache outage should not crash the application if the origin is still available. +- In a degraded state, the application may serve stale content or bypass the cache entirely. +- The resilience plan should be documented so operators know what to expect. +- The application should also handle cache connection failures gracefully. +- Some systems use a local fallback cache if the distributed cache becomes unavailable. +- Others disable the cache temporarily and let the origin absorb the load. +- The tradeoff depends on the importance of the feature and the cost of the fallback strategy. +- Application caching is usually easiest to implement at the service boundary where the expensive operation occurs. +- A service can cache the output of a database query, an external API, or a complex transformation. +- The cache should be placed close to the operation whose cost it aims to reduce. +- Caching at too high a level can lead to poor locality and overly broad invalidation. +- Caching at too low a level can miss opportunities to reuse results across requests. +- The best granularity is usually aligned with natural business or data boundaries. + +--- + +## Distributed caches and shared state + +- Distributed caches are shared across multiple application instances. +- They are useful when several services or containers need to access the same cached object. +- Redis is a widely used distributed cache due to its rich feature set and performance. +- Memcached is simpler and often used for straightforward key-value caching. +- Distributed caches are often used for session data, rate limit counters, feature flags, and shared query results. +- A distributed cache adds network latency compared to local memory caches. +- The latency cost is often acceptable because sharing the cache across instances provides more value. +- Distributed caches should be monitored for memory usage, evictions, connection counts, and latency. +- Redis supports persistence, replication, and advanced data types, which can be useful in some architectural patterns. +- Memcached is often chosen when the application needs a simple memory store without persistence. +- Distributed caches can be sharded to improve scale and capacity. +- Sharding places different key ranges on different nodes. +- A distributed cache can experience hot key problems when one key is extremely popular. +- Hot keys can be mitigated by precomputing the result or by using a local cache in front of the distributed one. +- Distributed caches can be deployed as a separate service or integrated into a cluster. +- The application should handle cache dependency on the network and external service availability. +- A distributed cache outage should be treated as a first-class operational event. +- A fallback strategy is often necessary to prevent a complete outage in the face of cache unavailability. +- Example of using Redis in Node.js: + +```js +import { createClient } from 'redis'; + +const client = createClient(); +await client.connect(); + +await client.set('user:42:profile', JSON.stringify({ name: 'Ada' }), { EX: 300 }); +const value = await client.get('user:42:profile'); +``` + +- A distributed cache usually uses networked serialization and deserialization. +- This can add overhead compared with an in-process cache. +- The tradeoff is often worth it when multiple services need a consistent shareable store. +- Distributed caches are not ideal for very large objects if they increase serialization costs too much. +- Very large objects may be better stored in object storage and referenced by a small cache entry. +- Distributed caches should not be used as the only source of truth for critical business data. +- They are best used as acceleration layers on top of durable storage. +- Some systems use a tiered approach: local LRU cache, then distributed cache, then database or origin service. +- That tiered approach balances latency and shared access. +- Cache coherence is important when multiple application instances can mutate the same object. +- Use versioning or invalidation to prevent one instance from serving stale data after another updates it. +- Distributed caches can support pub/sub or event streams to propagate invalidations. +- Some systems rely on write-through semantics so that all instances see updates quickly. +- Others accept eventual consistency and rely on TTLs to eventually converge. +- Consistency requirements should be defined explicitly rather than inferred from the cache implementation. +- If data is used for authentication or authorization, stricter policies and bypass strategies are needed. +- A distributed cache that stores tokens or sessions should be secured and network-isolated. +- Access to the cache should be limited to authorized services. +- Cache keys should not contain sensitive values unless access is tightly controlled. +- Memory usage and eviction behavior should be reviewed regularly as the workload grows. +- A shared cache can become a bottleneck if too many operations hit it simultaneously. +- Partitioning and replication can help but also add operational complexity. +- Monitoring should include connection pool saturation, latency percentiles, and eviction rates. +- A distributed cache should be tested under failover and recovery conditions. +- Replication lag or failover events can create temporary inconsistency. +- The application should be prepared for cache misses during failover. +- An effective distributed cache deployment is often backed by automation and runbooks. +- The cache should be part of the capacity plan so it can be scaled before it becomes a bottleneck. + +--- + +## CDN and edge caching + +- CDNs cache content at edge locations that are geographically closer to end users. +- Edge caching is especially effective for static assets, images, documents, and public HTML pages. +- It reduces round-trip latency by eliminating the need to fetch from the origin for every request. +- It also reduces load on origin servers during traffic spikes. +- CDN caches are usually configured through origin headers, cache rules, and purge APIs. +- Many CDNs support automatic cache invalidation and origin shielding. +- Origin shielding places a regional cache in front of the origin and can reduce repeated fetches from upstream. +- Cache rules can be based on path, host, query parameters, headers, or file type. +- Static assets with content hashes are ideal for long-lived CDN caching. +- For example, a JavaScript bundle can be cached aggressively because its URL changes when content changes. +- A CDN can serve stale content if configured to do so during revalidation or outages. +- The caching policy should align with the update frequency of the assets being served. +- Public content can often be cached aggressively, while personalized content should not be broadly cached. +- The `Cache-Control` headers from the origin help the CDN decide how long to store content. +- The CDN may also apply its own caching rules, which can override or extend the origin policy. +- Edge caches should be tested for correctness under different query parameter combinations. +- Some responses may need `Vary` to prevent caching across languages, devices, or user agents. +- If the response changes by auth state, the cache should be configured carefully to avoid cross-user leakage. +- The CDN can cache compressed variants and serve the appropriate encoding based on the request. +- A misconfigured CDN cache can lead to stale content being served to many users simultaneously. +- Purging is an important operational capability for urgent updates. +- A purge can be selective, by URL, or broader, by host or path. +- Many systems use versioned asset URLs so the CDN can cache aggressively without needing frequent purges. +- Example: + +```html + +``` + +- The content hash in the URL ensures that a new asset is fetched when the file changes. +- Edge caches are ideal for low-change, high-reuse content. +- They are less suitable for highly dynamic pages that contain user-specific data. +- Application performance can be improved dramatically when edge and origin caches work together. +- The edge layer often creates a significant reduction in time-to-first-byte for global audiences. +- A CDN can also help protect the origin from DDoS traffic by absorbing a portion of the load. +- A CDN should be included in the system diagram because it changes request routing and failure modes. +- The origin should be able to serve the resource directly if the CDN is unavailable. +- Edge caching should be deployed with monitoring and rollback mechanisms. +- A broken cache rule can expose wrong content to large user segments. +- Shared caches at the edge should follow strict access controls and not cache sensitive or personal data by default. +- Policy should be tested with real traffic patterns to ensure the expected hit rate and freshness behavior. +- The CDN can cache not only HTML and assets but also API responses if the policy allows it. +- Some systems place an API gateway or reverse proxy in front of services to handle cached responses. +- That design is especially effective for public APIs with stable response formats. +- A cache at the edge can convert many network-bound operations into simple hits. +- The operational approach should include cache purge workflows, warmup strategies, and on-call playbooks. +- A long-lived edge cache can reintroduce stale data after a production issue unless there is a clear invalidation path. +- Monitoring the age of cached content and the hit rate by path is essential. +- The edge cache should be part of the incident response process, especially when content freshness matters. + +--- + +## Cache consistency and freshness + +- Consistency refers to how closely a cache reflects the source of truth. +- Freshness refers to whether a cached value is still valid for use. +- Strong consistency means every read sees the latest committed state. +- Eventual consistency means updates propagate over time rather than instantly. +- Most caches operate with some form of eventual consistency unless explicitly designed otherwise. +- Strong consistency is usually more expensive to maintain in distributed systems. +- Eventual consistency is acceptable for many read-heavy systems when stale reads are tolerable. +- The acceptable freshness window should be stated in the system requirements. +- A dashboard might tolerate stale data for a minute or two. +- An inventory count might tolerate only a brief delay. +- A banking balance may require much stricter guarantees. +- Consistency should be discussed in terms of business risk, not technical convenience. +- A cache can be correct even if it is not perfectly fresh if the stale window is acceptable. +- A cache can be incorrect even if it is fresh if the key selection or invalidation logic is wrong. +- Consistency should be validated with tests that simulate writes and reads interleaving. +- Read-after-write consistency is often a key requirement for interactive systems. +- Users typically expect that a change they just made is visible immediately. +- Achieving that expectation may require cache invalidation or direct write-through semantics. +- Write-through can help read-after-write consistency because the cache updates alongside the source. +- Event-driven invalidation can also support this but requires robust messaging. +- Some systems choose to bypass the cache for writes and for a short period afterward. +- This is one way to avoid stale reads immediately after updates. +- If a user updates their profile, the profile endpoint may be invalidated and refreshed immediately. +- If a product price changes, the pricing view may need to be updated and any cached representations invalidated. +- Cache invalidation should be coordinated with the write transaction to avoid visible inconsistency. +- A write should not appear to succeed while the cache still returns stale values for a long time. +- Monitoring should include the age of the cache entries and the time between write and invalidation. +- If the write path does not invalidate the cache, the system might deliver old content for too long. +- In some systems, the application can detect a write and set a short TTL to reduce staleness risk. +- Staleness budgets can be part of the service-level objectives. +- The cache policy should be measured against those objectives. +- The longer the acceptable staleness window, the more aggressive the caching can be. +- A low-latency application with a high read rate often benefits from a more aggressive cache policy. +- A high-consistency application may need a more conservative cache policy. +- The right balance depends on the user experience and the business impact of stale data. +- A system that serves public content can often tolerate staleness more than a system that serves sensitive data. +- The consistency and freshness requirements should be documented alongside the cache design. +- In practice, a hybrid approach often works best: cache aggressively but bound the staleness with TTL and invalidation. + +--- + +## Common anti-patterns + +- Caching everything is a common mistake because it increases complexity without improving the hit rate. +- Caching data that changes frequently with a long TTL produces stale responses and user confusion. +- Caching highly personalized content without isolating by user or tenant creates privacy and correctness issues. +- Using a single global key for multiple variants of data can cause one user to receive another user’s data. +- Using overly broad cache keys can create collision problems and unexpected reuse. +- Using overly narrow cache keys can create fragmentation and poor hit rate. +- Relying on a single cache layer when traffic patterns require multiple layers can create a bottleneck. +- Ignoring cache invalidation can turn a performance feature into a correctness bug. +- Returning stale data from a cache during an incident without informing users can damage trust. +- Failing to monitor hit rate, miss rate, and latency can leave cache issues undetected. +- Treating the cache as a database substitute is dangerous and can introduce data consistency problems. +- Using a cache without an eviction policy can lead to unbounded memory growth. +- Setting a TTL that is too long for dynamic data causes stale results to persist. +- Setting a TTL that is too short eliminates most of the benefit of caching. +- Using random TTLs without reason can make behavior unpredictable and hard to debug. +- Not considering serialization and deserialization overhead can make the cache slower than expected. +- Storing huge objects in the cache can displace more valuable entries and increase pressure. +- Bypassing the cache for writes but not for reads can create a load spike on the origin system. +- Failing to invalidate dependent objects after a write can leave a partially stale graph in the cache. +- Treating cache failures as fatal can make the application brittle and hard to recover. +- Relying on a distributed cache when the application can make do with a local cache is often unnecessary complexity. +- Using a distributed cache without connection pooling or timeouts can create cascading failures. +- Forgetting to include the `Vary` header for user-specific or language-specific content can cause cache poisoning. +- Caching responses that contain sensitive headers or authentication tokens can create a security problem. +- Generating cache keys from raw request objects without normalization can produce inconsistent behavior. +- Allowing different services to invent their own key patterns can make the cache impossible to reason about. +- Not distinguishing between public and private caching requirements leads to accidental leakage. +- Treating cache performance as purely a backend concern ignores browser and edge caching opportunities. +- Ignoring the operational cost of purging and warming caches can cause production incidents. +- Failing to include the cache in the incident playbook can prolong outages. +- Overly aggressive caching can create stale content that is hard to correct quickly. +- Overly conservative caching can prevent the system from scaling under load. +- Caching based solely on intuition rather than instrumentation is a common design flaw. +- The best systems test cache behavior in realistic traffic conditions, not only in unit tests. +- The most dangerous anti-pattern is not the existence of a cache; it is the absence of clear invariants around freshness and invalidation. + +--- + +## Security and privacy considerations + +- Sensitive data should not be cached unless the cache policy explicitly allows it. +- Responses that include personal information, credentials, or private account information should be treated carefully. +- Private content should usually use `Cache-Control: private, no-store`. +- Tokens, secrets, or session identifiers should never be stored in a cache unless strictly necessary and strongly protected. +- Cache keys should not expose sensitive data directly if they may be visible in logs or metrics. +- If a resource is only valid for a specific user, the cache should scope it appropriately. +- Multi-tenant systems should ensure tenant boundaries are part of the cache key or access controls. +- Authorization and role-based data should not be cached without enforcing the correct access semantics. +- A public cache should never store a response that is only valid for a specific authenticated session. +- A shared cache can inadvertently leak data if a key is not sufficiently scoped. +- Consider the possibility of cache poisoning, where unauthorized data is stored under a shared key. +- Protect cache endpoints with authentication and authorization if they are externally accessible. +- Cache servers should be network-isolated and monitored for unusual access patterns. +- Ingress and egress rules should limit who can read and write cache entries. +- If the cache is used for rate limiting or quotas, ensure the counters themselves are not easily manipulated. +- Cache entries should be encrypted if they hold sensitive material and the deployment environment requires it. +- Audit logs should capture cache invalidation and purge operations for compliance and tracing. +- Caches can become a source of stale secrets if they keep previously issued credentials or tokens. +- Token rotation strategies should include invalidating or expiring cached tokens appropriately. +- Some systems use short-lived caches for authorization data to reduce the impact of stale permissions. +- The cache policy should align with compliance requirements and privacy regulations. +- The data retention policy for cache entries should be defined and enforced. +- The application should consider the possibility that an object in the cache may outlive its intended privacy window. +- When caches are purged, the process should be documented and auditable. +- The system should avoid storing large amounts of private data in client-side caches that are outside the application’s control. +- The route and response policy should be reviewed whenever new endpoints are introduced. +- Security review should include whether the response can be cached and whether the cache key exposes sensitive values. +- Caching decisions should be reviewed by the team responsible for security and compliance when appropriate. + +--- + +## Monitoring and observability + +- Caching should be monitored like any other critical part of the platform. +- Track hit rate, miss rate, eviction count, memory usage, and request latency. +- Track the age of items in the cache and how often entries expire. +- Track invalidation frequency and purge volume so the team can see whether the cache is being refreshed too aggressively. +- Track the number of cache errors, timeouts, and connection failures. +- Track distribution of hit rates across endpoints and objects to find hot keys and bad policies. +- Track cache latency as a separate metric from origin latency. +- Watch for sudden changes in hit rate after deployments or traffic shifts. +- Correlate cache metrics with application errors, latency spikes, and API traffic. +- Use dashboards to show both system-level and endpoint-level behavior. +- Alerts should be configured for sudden drops in hit rate or spikes in miss or eviction counts. +- Ensure the cache service itself has an SLO and runbook. +- Logging should capture cache decisions where possible, especially for debugging and incident response. +- A trace should be able to show whether a request benefited from a cache hit. +- The observability plan should include whether cache misses are expected and why. +- Cache metrics are often more useful when segmented by key prefix, endpoint, or tenant. +- Some systems expose a `cache_status` header or debug field to indicate whether a response came from cache. +- That can be useful in development and support environments. +- The application should collect enough information to tell whether the stale data came from a cache or from the source. +- When a cache issue is suspected, inspect the hit rate, the key distribution, and the versioning discipline. +- A sudden rise in stale content can indicate a missing invalidation path or overly long TTL. +- Unexpectedly low hit rates often indicate poor key design or insufficient reuse. +- High memory usage can indicate a wide key space or oversized objects. +- Observability also includes understanding whether cache warming is needed after deploys or restarts. +- Cache warming is the process of preloading common entries so the system avoids cold misses. +- Warming can be helpful for startup or deployment events but should not substitute for a robust caching design. +- Some systems also monitor the cost of cache misses to quantify the value of the cache. +- The team should review cache metrics regularly as access patterns change. +- Observability helps the team move from guesswork to measurement. + +--- + +## Testing and validation -- write-through: update cache on writes -- cache-aside: load from cache, fall back to source -- time-based expiry +- Caching behavior should be tested at multiple levels, including unit tests, integration tests, and load tests. +- Unit tests should confirm that the correct key is generated and that TTL and invalidation logic work as intended. +- Integration tests should simulate a real cache store and verify that the application reads and writes through it correctly. +- Load tests should measure hit rate and latency under realistic traffic. +- Tests should include scenarios for cache misses, cache hits, evictions, timeouts, and invalidation. +- The tests should validate both the happy path and the failure path. +- A common issue is a cache entry that remains stale after an update; tests should explicitly cover that case. +- Tests should also cover multiple concurrent readers and writers to surface race conditions. +- A read-after-write test is essential when user expectations require immediate consistency. +- A stale-while-revalidate test ensures that the application can serve stale data safely while refreshing it. +- A failover test checks that the application handles cache outages without crashing. +- A cache stampede test can reveal whether the system recomputes the same value for many requests simultaneously. +- A key collision test ensures that different logical entities do not share the same cache slot by mistake. +- A `Vary` behavior test should confirm that different headers produce different cached responses when needed. +- A deployment test should verify that version changes or asset changes produce the expected cache miss or refresh. +- Testing should include memory pressure scenarios to ensure the cache behaves reasonably under pressure. +- Metrics from test runs should be compared against expected values and used to tune TTLs and cache sizes. +- A system that caches public content should test that stale content is not served longer than intended. +- A system that caches private content should test that authorization boundaries remain intact. +- Test environments should approximate production traffic patterns rather than only synthetic one-off requests. +- The cache should be tested with realistic data distributions, including hot keys and cold keys. +- The team should also test the purge and invalidation workflows because those are operationally important. +- A cache bug can be difficult to reproduce in production, so good tests reduce the chance of silent drift. +- The system should have a documented rollback plan if a caching change unexpectedly causes issues. --- -## Cache-control +## Practical examples + +- Example of a browser-friendly cache policy for static assets: + +```http +Cache-Control: public, max-age=31536000, immutable +``` + +- Example of a public API response that can be mildly stale: + +```http +Cache-Control: public, max-age=60, stale-while-revalidate=30 +``` -Web caches use headers to manage freshness. +- Example of a private dashboard that must not be shared: ```http -Cache-Control: public, max-age=3600, stale-while-revalidate=60 +Cache-Control: private, no-store ``` -Control how browsers and CDNs store responses. +- Example of a product detail page that can benefit from a CDN and application cache: + +```js +const productKey = `product:${productId}:en-US:${currency}`; +``` + +- Example of a distributed cache entry with expiration: + +```js +await redis.set(key, JSON.stringify(payload), { EX: 300 }); +``` + +- Example of a cache-aside flow in a service: + +```js +async function getCatalog() { + const cached = await cache.get('catalog'); + if (cached) return cached; + + const data = await fetchCatalogFromDatabase(); + await cache.set('catalog', data, { ttl: 600 }); + return data; +} +``` + +- Example of a read-through pattern with a wrapper: + +```js +const result = await cachedFetch('search:query', () => fetchSearchResults(query)); +``` + +- Example of a purge after a write: + +```js +await cache.delete(`product:${productId}`); +``` + +- Example of a versioned key after a schema change: + +```js +const key = `user-profile:v2:${userId}`; +``` + +- Example of a short-lived cache for volatile data: + +```js +await cache.set('recent-orders', orders, { ttl: 10 }); +``` + +- Example of a longer-lived cache for configuration: + +```js +await cache.set('config:app', config, { ttl: 3600 }); +``` + +- Example of using stale data while revalidating: + +```js +if (cached && !expired) { + return cached; +} +``` + +- Different applications will need different combinations of these patterns depending on urgency and consistency requirements. +- In practice, caching is most effective when it is integrated into the code path from the beginning. +- Long-lived caches should be accompanied by a strategy for warmup and recovery. +- A simple and well-documented policy is often more valuable than an over-engineered one. + +--- + +## Decision checklist + +- Start by asking whether the data is read frequently and reused often. +- Ask whether the data is expensive to compute or fetch. +- Ask whether the data changes rarely or predictably. +- Ask whether a stale result would degrade the user experience or create business risk. +- Ask whether the data should be shared across users, tenants, or instances. +- Ask whether the cache should be public, private, or shared. +- Ask whether the data is safe to cache at the browser, CDN, application, or distributed layer. +- Ask whether the response shape depends on headers, cookies, or authentication state. +- Ask whether the write path can invalidate the relevant cache entries. +- Ask whether the cache should be bypassed for some high-risk requests. +- Ask whether you need read-after-write consistency or eventual consistency. +- Ask whether you can tolerate a stale window and if so how long. +- Ask whether the cache key design is deterministic and simple enough to maintain. +- Ask whether eviction behavior is acceptable for the expected workload. +- Ask whether the system can operate gracefully if the cache is unavailable. +- Ask whether you have dashboards and alerts for cache health. +- Ask whether the team has a clear purge and invalidation runbook. +- Ask whether the design was reviewed for security and privacy concerns. +- Ask whether the policy should be adjusted as traffic and data patterns change. +- Ask whether the feature can be rolled back quickly if the cache configuration causes issues. +- Ask whether the code path is explicit and easy to reason about. +- Ask whether the team can explain the caching behavior in plain language to others. +- A cache that is invisible to the team is a risk, not an asset. +- A cache with clear semantics is usually easier to trust and maintain. + +--- + +## Final principles + +- Caching is a performance strategy that must be matched to workload and data characteristics. +- The best cache design balances speed, freshness, correctness, and complexity. +- Use the simplest effective caching layer and avoid premature optimization. +- Prefer deterministic keys, explicit invalidation, and measurable policies. +- Build around the real data access patterns rather than abstract principles alone. +- Treat cache behavior as part of the system contract and document it clearly. +- Monitor it, test it, and revisit it as the system evolves. +- A cache should make the system faster without making it less trustworthy. +- The right cache can be a force multiplier; the wrong one can quietly undermine the system. +- Good caching is disciplined engineering, not magic. +- The most effective systems combine several cache layers with clear responsibility boundaries. +- Each layer should have a specific purpose and should not be overused. +- High-quality caching is one of the clearest signs of mature systems design. +- The best teams treat cache decisions as first-class design decisions, not incidental implementation details. +- In the end, a successful cache is one that improves latency and resilience while preserving correctness. +- That is the standard by which every caching policy should be judged. +- When you can explain why a cache entry exists, how it expires, and what happens on a miss, you are on the right path. +- When you cannot explain that, the cache is likely to become a liability rather than an advantage. +- Caching should always support the business goal of making the service faster, more reliable, and more scalable. +- The best engineering teams understand that caching is both an optimization and an operational concern. +- Every cache policy should be grounded in data, measurement, and a realistic failure model. +- A system without a thoughtful cache strategy often pays for that omission in latency and cost at scale. +- A system with a thoughtful cache strategy can handle more load with less friction and less infrastructure spend. +- Caching is not only about speed; it is about making the system behave well under pressure. +- A well-designed cache makes traffic bursts manageable and normal operation smoother. +- A poorly designed cache makes the system harder to reason about and more vulnerable to stale data. +- The discipline of caching is one of the most reusable skills in modern software engineering. +- It touches web applications, APIs, databases, distributed systems, cloud architecture, and infrastructure. +- The knowledge you apply to caching will help you design systems that scale more gracefully. +- With a strong cache strategy, you can make a service feel instant even when the underlying systems are slower. +- That feeling of immediacy is a real competitive advantage for users and teams alike. +- The final lesson is simple: cache the right thing, at the right layer, with the right policy, and with the right invalidation strategy. +- If you can do that consistently, caching becomes a durable source of performance and resilience. +- If you cannot, the cache will become a source of confusion and hidden operational risk. +- A great caching strategy is therefore both a technical choice and a long-term maintenance commitment. +- It should be reviewed and updated as the product, traffic, and data change. +- In other words, caching is not a one-time decision; it is a continuing part of system stewardship. +- The most resilient systems evolve their caching policies as they evolve their products. +- That mindset is what separates a temporary optimization from a lasting architectural advantage. +- Keep the cache simple where possible, explicit where necessary, and measurable at all times. +- That principle is often enough to build a strong and durable caching design. +- The work of caching is rarely glamorous, but its impact is profound. +- It improves throughput, lowers cost, and makes the user experience feel effortless. +- That is exactly why caching remains a foundational practice in modern engineering. +