Static system prompt: share the appended block across sessions via the prompt cache

[pufit]

Static system prompt: share the appended block across sessions via the prompt cache

What

build_system_prompt joins SOUL/TASK/IDENTITY/USER/AGENTS/TOOLS/MEMORY.md, a # Session Context block, the skills list and # Recalled Memories into one string that the Claude Code CLI receives as a single appended system-prompt block. That block carried the per-session Session ID and the per-session pre-recall list, so no two sessions ever had a byte-identical system prompt.

This PR makes the appended system prompt static across sessions of the same workspace / source / tool set and delivers the per-session parts in a <session-context> block prepended to the first user message of each native conversation.

• nerve/agent/prompts.py: build_system_prompt(..., static=True) no longer renders Session ID or # Recalled Memories (the arguments stay for signature compatibility; static=False renders the legacy shape). New build_session_preamble(session_id, source, recalled_memories) renders the per-session block and prepend_session_preamble() wraps it in <session-context>…</session-context> ahead of the user text. The static prompt states where the per-session details live and names mcp__nerve__session_context as the live fallback.
• nerve/agent/engine.py: recall + freeze logic is unchanged. The rendered preamble is parked per session at client build and popped by _run_inner into the FIRST user message of the native conversation — the same mechanism as the trailing Current time reminder (not persisted to the DB message, so the UI stays clean). It is delivered when the transcript is fresh (no resume target; codex resume-miss), on forks (new id), and once for resumable sessions that pre-date this change (preamble_sent marker in session metadata) — so persistent cron sessions and long-lived chats still see their id exactly once. A query-phase crash retry re-renders it from the base text, never stacking two blocks.
• agent.static_system_prompt (new, default true). false restores the old prompt shape and skips the preamble. Documented in config.example.yaml and docs/config.md; docs/memory.md, docs/architecture.md and the README pre-recall bullet updated.

Why

Measured on a production fleet: 814 cron session starts/day, ~118K-token appended block, cache-WRITTEN at every session start — about $847/day of cache writes. 87% of those session starts happen within 5 minutes of the previous one, i.e. inside the default 5-minute cache TTL, so with a byte-identical block nearly all of them become cache READS (0.1x base input instead of 1.25x for a write). Until now the API could only ever share the CLI's own ~29K preset block, because everything after it differed per session.

Mechanism

Anthropic prompt caching is exact-prefix: a cached prefix (tools → system → messages, in that order) is reused only when the new request's bytes match it exactly up to a breakpoint. The appended system prompt sits inside that prefix, so a single differing byte in it — a session id, a different recall list — invalidates the whole block for every other session. Moving the per-session bytes AFTER the system prompt (into the first user message) keeps the shared prefix identical; only the small per-session tail differs and is written once per session. The TTL policy in nerve/agent/cache_policy.py is untouched; the 1h TTL now also pays off across sessions, not just within one.

Risks / caveats

• Any edit to a workspace file that feeds the prompt (SOUL.md, MEMORY.md, TASK.md, …), to the skills list or to the tool set still changes the shared block: the next session start writes it once, later sessions read it. Unchanged from before, but now it is the only thing that invalidates the block.
• The Current date line rolls the block once a day (unchanged; deliberate — see the comment in prompts.py).
• The session id now arrives in the first user message instead of the system prompt. It replays on every later turn, but a CLI context compaction of a very long conversation could summarize it away; the static prompt tells the model to call mcp__nerve__session_context in that case. Scripts that grep the system prompt file for the id must read the transcript instead.
• Sessions created before this change receive the block once on their next turn (their system prompt loses the inline id on the first rebuild after upgrade).
• Source stays in the block as before, so the cache is shared per source (web / cron / telegram …). Sources also differ in model, so this splits nothing that could have been shared.
• Forked sessions carry the parent's block in the inherited history plus their own at the fork point; the newer one is the fork's id.

How to verify on a fleet

After two sessions of the same source have started within 5 minutes, compare the FIRST API request of the second session (session_usage, or the Langfuse / OTEL spans):

• before: cache_creation_input_tokens ≈ the full prefix (~118K appended block + ~29K CLI block) and cache_read_input_tokens ≈ 29K (the CLI block only);
• after: cache_read_input_tokens ≈ the full prefix and cache_creation_input_tokens ≈ only the small per-session tail (session-context block + user text).

Also spot-check a new session's native transcript: the first user message starts with <session-context> carrying Session ID:, and the second turn does not repeat it. agent.static_system_prompt: false is the one-line fallback.

Generated by Nerve