Repository navigation
inference: cache-stable two-block system prompt (don’t bust the KV prefix) #558
Description
Parent
#548 (agent session architecture)
Problem
OpenCode (and similar) pin a two-part system array so provider KV cache
stays valid across turns: if the first block is unchanged, later turns hit
cache instead of re-prefilling the world.
We already see cache working (1.2M cached on the usage chip) — the
prefix is enormous. Session-varying bits (cwd, attached skill bodies,
envelope-derived addenda, working notes once #550 exists) mixed into one
blob will invalidate that prefix whenever they twitch.
New-session “thinking hang” is still mostly serial I/O + cold start, but a
stable prefix is how we keep later turns from paying 1.2M uncached
again.
Goal
Assemble system as two (or few) stable blocks:
- Stable: tool schemas + persona snapshot + invariant harness law.
- Session-varying: cwd, skill catalog (skills: inject a catalog, not full bodies, every turn #557), working notes (agent memory: durable working notes / findings across turns (identity, not one-shot) #550),
turn-specific addenda.
Do not mutate block 1 mid-session unless the operator actually changed
persona or the tool surface.
Success: consecutive turns on the same model show a high cache-read /
input ratio on the occupancy/totals chip without stuffing new skill
bodies into block 1.
Constraints
- Gateway / AI SDK must actually accept a multi-part system (or a single
string with a documented stable prefix). Lock increate-planagainst
liverunAgent— do not inventcache_controlif the provider options
we send cannot express it. - Authoring meta-tools on every turn (meta(#531): built-in meta persona + skill authoring tools on /api/agent #533/meta(#532): built-in meta sandbox tools on /api/agent #535) still enlarge block 1.
Shrinking that surface is related but can be a later issue; this one
is ordering / stability, not “delete meta tools.” - Not status bar: show current context occupancy in addition to summed turn usage #556 (how we paint occupancy).
Maps from peer harnesses
OpenCode system.ts + concatenate-into-two-blocks for Anthropic cache;
plugin transform only if it does not smash the first element.
Suggested next
create-plan with #557 (catalog vs bodies) — same assembly site.
Orrery (2026-08-15)
Choice precedes assembly. Resolve model (operator catalog / Next) then tools, system blocks, history. No generic prompt with the id slapped on.
Layout, strictly: system → tool definitions → durable spec / notes (#550) → plan/todo → history (#549). Nothing above history may vary turn-to-turn for a given (session, model) except at a phase/compact boundary.
Stay-put is a routing decision. Next mid-session is a family switch: #551/#552 treat it as a cache event, not a free paint change.
Breakpoints (if Gateway/provider can express them) sit at the stable/volatile cut. Do not invent cache_control the live runAgent path cannot send.
Activity
btipling commentedon Aug 19, 2026
2026-08-19 source-level peer research
Two-block system is the right product. Name the peer mechanisms so the create-plan does not reinvent them:
Context Epoch / world-state diffs (steal the invariant)
WorldStatesectionssnapshot()+render_diff(previous). Unchanged AGENTS.md / tools / permissions → no new tokens. Replacement notice when they do change.systemmessages; baseline is rewritten only at compaction (or incompatible transition).agent/pre-stepafter compact shadows the old instruction event.Our persona snapshot (#485) is already an OpenCode baseline for standing orders. #558 generalizes that:
(session, model)until compact / persona-New / tool-surface change.systemmessage (Codexrender_diff), not a rewrite of block 1. Honor it at the same phase boundaries as compaction: model-capability context compaction (not API payload slices) #552, not mid-tool-streak (already in the Orrery note).OMP sticky-rules lesson: if we will not rebase personas mid-session (we will not — snapshot lock), re-attach the isolated persona/rules block near the current turn so A4 cannot bury standing orders. That is #670’s salience, durable across compact — not a live-body rewrite.
Handbook inject (#694)
If/when the later handbook ships,
memory_summarygoes in block 2 on New session, never block 1, and never mid-turn. OMPlearnwaits until the next session on purpose.Skip: inventing
cache_controlthe liverunAgentpath cannot send (already in the body).