Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ The eviction policies share those bounds but spend metadata differently:
| CLOCK | The default uses one visited bit and a bounded shard-local hand. It keeps policy metadata and hit work small. |
| S3-FIFO | Uses small and main resident FIFOs, a metadata-only ghost FIFO, and a saturating two-bit frequency. Hits update frequency without moving queue nodes; repeated small-queue entries and ghost hits can enter the main queue. It spends additional fixed metadata to distinguish reused entries from one-hit traffic. |

Evicted values may remain alive while a caller owns them. Admission never waits for such retained values to release their memory charge; it selects another bounded victim or bypasses L1.
Evicted values may remain alive while a caller owns them. Admission never waits for such retained values to release their memory charge; it selects another bounded victim or bypasses L1. Replacement and eviction can reuse one detached, exclusively owned allocation when its combined key/value size matches the new entry. The allocation and its charge become the new resident value; there is no extra free-buffer pool, and caller-held values remain immutable. This reduces allocator churn for fixed-size turnover under both CLOCK and S3-FIFO.

### L2 index

Expand All @@ -97,14 +97,14 @@ This layout converts small foreground mutations into ordered, batched Region wri

### `put` and `put_l2`

1. Compute the key hash and encoded size once.
2. Enter bounded mutation admission and reserve an append position under one short manager try-lock.
1. Compute the key hash, encoded size, and payload checksum once.
2. Enter bounded mutation admission and try the hash-routed append shard once. Reserve an append position under one short manager try-lock.
3. Encode into the selected shard's current buffer.
4. For `put`, attempt bounded L1 admission. For `put_l2`, stage through Region and remove an older exact-key L1 entry best effort.
5. Return after bounded in-memory admission.
6. The shard worker writes sealed batches and publishes their L2 mappings only after write completion.

Full staging or short-path contention returns structured `ErrorKind::Overloaded`. Staging admission is shard-local; optional adaptive fill budgets are shared across shards. Success means accepted staging; `put_l2` becomes visible when publication completes. `drain` fences all mutations accepted before its operation barrier and waits for their Region writes and L2 publication, but does not issue the recovery durability syncs.
Full staging or lock contention returns structured `ErrorKind::Overloaded`. Every fill uses the same hash route during initial growth, capacity reclaim, and recovery; there is no neighboring-shard retry or routing mode switch. Reads follow the physical address in the index, so records written to another shard by an older runtime remain readable. Lock contention does not force an early flush; capacity and rotation pressure still wake the affected shard worker urgently. A Region capacity request remains in manager authority through its final write, allowing the worker to flush and rotate without waiting for another rejected put; duplicate notifications cannot rotate an empty new Region or retain obsolete rotation pressure. Staging admission is shard-local; optional adaptive fill budgets are shared across shards. Success means accepted staging; `put_l2` becomes visible when publication completes. `drain` fences all mutations accepted before its operation barrier and waits for their Region writes and L2 publication, but does not issue the recovery durability syncs.

### `get`

Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

## Unreleased

### Bug Fixes

- Lock contention during foreground staging returns overload without forcing partial Region flushes. `DetailedCacheSnapshot::write_buffer_rejections` excludes lock contention; `CacheSnapshot::write_rejections` continues to count all mutation rejections. Actual capacity and rotation pressure still request urgent progress.
- A full Region retains its capacity request through the final write and rotates without requiring another rejected put. Duplicate notifications do not rotate the new empty Region or retain obsolete flush pressure.
- `CacheSnapshot::served_bytes` counts the payload actually returned when L2 promotion selects a newer L1 value of a different size.

### Improvements

- L1 replacement and eviction reuse detached, exclusively owned allocations of the same size under CLOCK and S3-FIFO, reducing repeated allocation during fixed-size turnover without an additional buffer pool. Caller-held values remain immutable and retain their separate memory charge. When activity statistics are enabled, mutations with no eviction also avoid a shared zero-value counter update.

## v0.6.0 (2026-09-21)

### Breaking Changes
Expand Down
4 changes: 2 additions & 2 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,11 +223,11 @@ The controls operate at different stages:
| `write_flush_threshold_bytes` | Controls when a partial buffer requests publication | Smaller values issue more, smaller writes; larger values favor batching |
| Region size | Changes per-shard burst reservoir and rotation frequency | Multiplies staging and reclaim memory and changes eviction granularity |

Start with append shards near the number of genuinely concurrent foreground writers, then measure. More shards are not monotonic: excess shards fragment data across additional partial buffers and Active Regions, consume memory, and can reduce useful capacity. Hash skew can still overload one shard even when aggregate buffers look empty.
Start with append shards near the number of genuinely concurrent foreground writers, then measure. More shards are not monotonic: excess shards fragment data across additional partial buffers and Active Regions, consume memory, and can reduce useful capacity. Each fill tries its hash-routed shard once, with the same path during initial growth, reclaim, and recovery. Lock contention returns overload; capacity pressure requests progress on that shard. Hash skew can still overload one shard even when aggregate buffers look empty. Keep append-shard and I/O-worker counts fixed when comparing revisions, and tune them separately to the actual foreground concurrency and storage throughput.

Start write execution capacity near the append-shard count. Increase it only when write I/O in-flight peaks at the configured capacity or slot-wait time grows while the device still has headroom. More execution capacity cannot fix a foreground mutation-gate collision, a full shard-local staging pair, or a saturated SSD.

The flush threshold is per append shard, 4 KiB-aligned, and ranges from 4 KiB through 4 MiB. Lower it when publication latency matters more than write amplification. Keep it high for throughput and batching. Pressure, the bounded flush delay, rotation, and `drain` can flush below the threshold, so it is not a visibility deadline.
The flush threshold is per append shard, defaults to 4 MiB, is 4 KiB-aligned, and ranges from 4 KiB through 4 MiB. Smaller thresholds start writes sooner and reduce the final batch left for `drain`; larger thresholds reduce write-request overhead but can delay publication and capacity recovery. Compare accepted puts plus `drain` as well as continuous traffic when tuning it. Pressure, the bounded flush delay, rotation, and `drain` can flush below the threshold, so it is not a visibility deadline. When a record requires a new Region, its capacity request remains pending through the final write and the worker rotates as soon as it can safely do so, without requiring another rejected put.

`put` and `put_l2` intentionally do not wait for staging capacity. A larger Region or more appropriately placed shards can absorb more of a finite burst, but configuration does not turn immediate admission into an unbounded lossless queue. Applications that require higher acceptance must retry with their own deadline, bypass the cache, or split bursts with completion barriers. `put` attempts immediate L1 visibility; L2 visibility for both operations follows publication. `drain` waits for accepted Region writes and index publication.

Expand Down
Loading
Loading