You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Replace idle TTL folds with an opt-in Anthropic cache prompt transform (#1161)
* fix(compaction): limit idle recompress to the Anthropic cache
Guessed windows for other providers fold a cache that is still warm. Only Anthropic publishes a 5-minute expiry this client actually uses.
* feat(session): compact expired anthropic history before the next infer
A resumed process only knew the cache write time in memory, so the
first request rewrote the full prefix. Persist the Anthropic stamp
and fold through the existing compactor before that infer.
* fix(provider): resume anthropic cache stamps from the protocol
Runners persist the catalog id, which is not the messages adapter
name. Requiring that id to match the protocol dropped an expired
stamp, so the next request sent the untrimmed history.
* feat(session): shrink anthropic prompts once the cache write expires
* feat(session): gate anthropic prompt shrink behind settings
Cache expiry no longer compacts stored turns.
The prompt transform runs only when anthropicCachePrompt is set.
* fix(session): keep cache expiry from compacting stored turns
The settings fixture covers anthropicCachePrompt.
Unused governor clocks are gone.
* docs(architecture): drop the stale idle-recompress table note
The provider table is gone, so the sentence that explained its ollama row is gone too.
* fix(compaction): drop dead cache-ttl fold state and stale tests
## Summary
The TTL fold path is gone, so the governor's outstandingToolCalls
tracking was write-only and the stall-ping test merged from main
still expected a cache-ttl-recompress compact. Rewrite the
provider-aware describe block to pin the new contract — idle
re-entry never folds, whatever the provider window.
## Verification
- bun test src/agent/compaction.test.ts src/subagent/nudge-director.test.ts (117 pass)
- bun run check (the one local failure is a /tmp symlink artifact of
the worktree path; the same test passes on a real checkout)
Refs CL-8914
Copy file name to clipboardExpand all lines: docs/ARCHITECTURE.md
+1-14Lines changed: 1 addition & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -191,24 +191,11 @@ When a cycle's input tokens cross a threshold, the director compacts the inferen
191
191
192
192
-**Threshold at a tool pause** — Once over threshold, the follow-up `infer` after a tool batch is swapped for a `compact` cycle, and inference resumes via a host continuation message. After a compact that remains over the high watermark, the governor uses **growth hysteresis** (wait for usage to grow by ~10% of the window) instead of re-arming on every cycle; dropping under 60% is not required.
193
193
-**Idle (end-of-turn)** — An interactive turn can end with a reply and then sit idle with no tool batch to intercept; the governor requests a continuation at that pause and compacts when it arrives. An operator message that races the continuation still compacts first, then re-enters inference to answer it.
194
-
-**Idle recompress past the provider TTL** — The fold is a re-compress, not a cache play: provider KV caches expire on their own schedule, so a session compresses _after_ that expiry (the next turn is a cheaper write and later reads compound on the shrunk context) instead of only at 60% tokens. It is not limited to under-threshold sessions: once growth hysteresis has cleared `pending`, an over-threshold session in that gap can fire the same fold. Any live re-entry past the window fires it — an empty stall ping re-enters meter-only, a raced operator message re-infers — under reason `cache-ttl-recompress` through the same compactor, so the fresh tail stays raw exactly as in the threshold path. The interval follows provider cache economics (`src/provider/cache-ttl.ts`), not a global N minutes:
|`anthropic`, `zen-messages`, `opencode-go-messages`| 5 min | Default ephemeral cache TTL; hour-long breakpoints are never set |
199
-
|`openai-responses`, `codex-responses`, `openai-compatible`, `xai`| 10 min | In-memory prefixes typically evicted after 5–10 min idle; xAI assumed same economics |
200
-
|`gemini`| 15 min | No published implicit-cache eviction window; conservative |
201
-
|`deepseek`| 60 min | On-disk context cache persists for hours-to-days |
202
-
|`ollama`| never | Local inference has no remote cache to expire |
203
-
| anything else | 10 min | Assumed OpenAI-style in-memory economics; tune per upstream |
204
-
205
-
The `ollama` table key is the bare provider segment; the harness stamps slash-form on `sourceId` (`ollama/default` / `ollama/<instance>`), with `provider: openai-compatible` (`buildOpenAISource`) and a bare model (`llama3` / `qwen3`). Idle-recompress disable matches via `isOllamaProviderId` on that `sourceId` (`src/provider/ollama.ts`), not by looking up slash-form as a table key.
194
+
-**Idle recompress past the provider TTL** — Cache expiry does not compact. Stored turns stay. When settings set `anthropicCachePrompt`, and the saved Anthropic-protocol cache write is at least 5 minutes old, the pre-inference context transform stubs old tool-result bodies on the outgoing prompt only. `turns.jsonl` is not rewritten. The setting defaults off.
206
195
207
196
-**Operator `/compact`** — A first-class slash command folds now without waiting for the occupancy governor. Optional trailing instructions go to the summarizer (empty uses the default structured fold) and are stored on the compact record so later auto-folds still see them. Director rebuild (`/model`, interrupt, resume) restores those instructions from the latest compact record's parameters. An in-flight turn uses the same pair-safe compact-then-continue hop as threshold compact; an idle compact shows the fold and does not start a new turn.
208
197
-**Overflow recovery** — A `context_overflow` inference error would otherwise become a terminal error reply; the governor compacts and retries instead, bounded so a history the compactor cannot shrink does not loop forever. Overflow ignores hysteresis for the compact itself.
209
198
210
-
The TTL window is measured from the last `inference.done` (every inference rewrites the provider's prefix cache) with one fire per window per compact of any kind, and it never fires with a tool batch outstanding — stall pings can arrive mid-work. The consecutive-compact cap shared with the threshold path still bounds compact→infer→compact. Prompt-caching the summary call itself is an explicit non-goal: the fold carries no cache options.
211
-
212
199
The compaction control flow is shaped by a reactor invariant: a `compact` action runs in its own cycle (it cannot be paired with `infer`), and **the reactor delivers no event after a compact cycle**. A director that simply emitted `compact` in place of the follow-up `infer` would leave the loop idle forever — the cause of an earlier stall. Instead the governor pairs `compact` with an emit action (`custom.compaction.continue`), and the host answers that emission by delivering a content-less inbound message (`buildCompactionContinuationMessage()`) through the serial operation queue, generation-guarded like every other deliver. A hop superseded because interrupt already bumped generation and enqueued a rebuild is re-queued onto the replacement agent rather than consumed on the outgoing liveAgent. That message adds no turn (`createInboundTurn` returns `null` for empty content) but re-enters the loop, where the director issues the follow-up `infer` against the freshly truncated history. Each emission is answered at most once (a replayed duplicate of an already-answered emission is ignored), and an unsolicited continuation with neither resume flag set answers `wait` rather than burning a billable inference. The legacy `requestContinuation` closure survives only on the sub-agent path, which delivers directly with no host emit hop; idle arming keeps the two channels exclusive (closure fires and the arming flag stays false, or no closure and the flag reports the arming) so a caller honoring both cannot double-deliver.
213
200
214
201
Compaction replaces older turns with a structured, workflow-aware summary rather than a stats blob: sections for **What Happened / What We're Doing / Relevant Links / Action Items / Next Steps**, with the active workflow and step woven in so compacting mid-`/build` or mid-`/plan` preserves the contract. The summary is produced by a one-shot model call; on any failure it falls back to a deterministic summary so a compaction cycle never breaks the session.
0 commit comments