feat(llm): add dynamic cache breakpoint on last message for Anthropic - #828
Merged
lizhengfeng101 merged 1 commit intoAug 10, 2026
Merged
Conversation
Contributor
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s). |
Mark the last content block of the final message with ephemeral cache_control so multi-turn conversations cache the growing history, not just the static system+tools prefix. Each turn then reads the previous full prefix and only writes the new delta.
yingjiexu2002
force-pushed
the
feat/anthropic-cache-breakpoint
branch
from
August 10, 2026 13:39
7b52673 to
29a9821
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Mark the last content block of the final message with ephemeral
cache_controlso multi-turn conversations cache the growing history, not just the static system+tools prefix. Each turn then reads the previous full prefix and only writes the new delta.Previously only two static breakpoints were set (last system block, last tool definition), so everything after the tools — the growing multi-turn conversation — was re-processed as fresh input on every turn of the per-file review loop.
This change adds a dynamic breakpoint on the last content block of the final message (user text,
tool_result, ortool_use— all handled generically via the SDK'sGetCacheControl()helper). Since request params are rebuilt from the append-only history on every turn, the marker never leaks back into stored history, and the breakpoint simply advances with each turn. Total breakpoints stay at 3, within Anthropic's limit of 4.Type of Change
How Has This Been Tested?
make testpasses locallyAdded
TestBuildAnthropicParams_DynamicCacheBreakpointcovering:tool_resultblock (the typical agent-loop turn shape) — the last block getscache_control: {"type":"ephemeral"}and earlier messages stay unmarkedAlso verified against the Anthropic API in a real multi-turn review session: turns after the first report
cache_readtokens for the full prior prefix and only write the delta.Checklist
go fmt,go vet)Related Issues