Skip to content

docs: stateless chat client live KV warm recipe - #956

Open
paride-cmd wants to merge 1 commit into
antirez:mainfrom
paride-cmd:fix/816-stateless-kv-readme
Open

docs: stateless chat client live KV warm recipe#956
paride-cmd wants to merge 1 commit into
antirez:mainfrom
paride-cmd:fix/816-stateless-kv-readme

Conversation

@paride-cmd

Copy link
Copy Markdown

Summary

  • Adds a README subsection under Disk KV Cache for keeping the live session warm from a stateless chat/completions client (exact assistant replay, `max_tokens: 0` prewarm, `cached_tokens` witness).
  • Notes that cold disk checkpoints trim to `kv_cache_chat_anchor_pos`, which is the usual disk hit when a client cannot replay.

Fixes #816

Test plan

AI usage disclosure: YES — assisted by Cursor agent; change is docs-only.

Made with Cursor

Document exact assistant replay, max_tokens:0 prewarm, cached_tokens
witness, and the cold chat-anchor fallback for clients that cannot replay.

Fixes antirez#816
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stateless chat clients never extend the live KV session on Flash/Metal: measurements, the anchor that saves them, and the replay recipe that fixes it

1 participant