Repository navigation
feat(codex): append tools loaded by Claude Code's tool search instead of rewriting the tool prefix - #184
Draft
Butanium wants to merge 2 commits into
Draft
feat(codex): append tools loaded by Claude Code's tool search instead of rewriting the tool prefix#184Butanium wants to merge 2 commits into
Butanium wants to merge 2 commits into
Conversation
cyberphantom52
added a commit
to cyberphantom52/claude-code-proxy
that referenced
this pull request
Oct 22, 2026
…incremental tools With tool search on (ENABLE_TOOL_SEARCH), Claude Code sends deferred MCP tools with defer_loading: true once ToolSearch has loaded them, and marks the load with a tool_reference block in the tool_result. On the Responses Lite lane the proxy used to put every tool, deferred or not, in the leading additional_tools item, so each load rewrote the prefix and the next request missed the prompt cache. tool_reference blocks also reached the model as '[unsupported content block omitted: tool_reference]'. Now, on the lite lane, a deferred tool that a tool_reference loaded is appended as its own additional_tools item right after that tool_result, the way Codex's IncrementalTools feature appends catalog changes without rewriting history (core/tests/suite/scenarios_incremental_tools.rs). Deferred tools with no visible load (e.g. loaded before a compaction) stay in the prefix. The Anthropic-only DeferredToolPlaceholder stub is dropped on both lanes, and tool_reference renders as 'Tool loaded: <name>'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xmx9RE7u4ZLBzSUjpjqyBM
…odex Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Butanium
force-pushed
the
incremental-deferred-tools
branch
from
November 11, 2026 21:02
4aed636 to
40515bd
Compare
gongx030
added a commit
to gongx030/claude-code-proxy
that referenced
this pull request
Nov 12, 2026
) * feat(codex): append loaded deferred tools incrementally on the Lite lane With tool search enabled in Claude Code, a ToolSearch result carries tool_reference blocks and the loaded tools arrive with defer_loading. The Lite lane previously put every tool in the leading additional_tools item, so each load rewrote the request prefix, and tool_reference reached the model as an unsupported-block placeholder. Now a deferred tool loaded by a tool_reference is appended as its own additional_tools item right after the matching function_call_output, so each request's input is an exact prefix of the next. The Anthropic-only DeferredToolPlaceholder is dropped (only when also defer_loading) on both lanes, and tool_reference renders as "Tool loaded: <name>". Full lane tools and parallel_tool_calls are unchanged. Adds a 3-request sequence test on both lanes (fails on the prior code: the second request rewrote the prefix) and a byte-identity test whose goldens were captured before this change. Based on raine#184 (commit 8f326c0, request.rs only). Refs raine#182 Co-Authored-By: Clement Dumas <dumasclement2002@gmail.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs: document ENABLE_TOOL_SEARCH and incremental deferred tools on Codex Documents the opt-in ENABLE_TOOL_SEARCH setting and how the Codex Responses Lite lane now appends loaded deferred tools where they were loaded. Describes the mechanism only; no cost or cache-hit claim, and the proxy does not enable the setting itself. Adapted from raine#184 (commit 40515bd). Refs raine#182 Co-Authored-By: Clement Dumas <dumasclement2002@gmail.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Wuming Gong <gongw13@gene.com> Co-authored-by: Clement Dumas <dumasclement2002@gmail.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #182. With Claude Code's tool search on (
ENABLE_TOOL_SEARCH=true), a tool loaded throughToolSearchis now appended where it was loaded, instead of rewriting the leading tool list. Requests stay append-only across loads and keep their prompt cache. The load result now readsTool loaded: <name>, and Claude Code'sDeferredToolPlaceholderstub is no longer forwarded.Problem. Without tool search, Claude Code sends every MCP tool schema with every request. In our case that was 281 tools, about 150k of a ~270k-token request. With tool search on, the Lite lane put every loaded deferred tool into the single leading
additional_toolsitem. So each load changed the first input item, and the next request missed the cache for the whole conversation.Change (Responses Lite lane)
tool_referencein the history leaves the leading item. It is appended as its ownadditional_toolsitem right after thefunction_call_outputof the result that loaded it. This is what Codex'sIncrementalToolsdoes (test).tool_referencerenders asTool loaded: <name>.DeferredToolPlaceholderis dropped on both lanes.The full Responses lane keeps every loaded tool in
tools, as before.Result. Replaying a real Claude Code 2.1.293 session (load a tool, run a command, finish) through the proxy into a stub upstream, each translated request's
inputis an exact prefix of the next one's, including across the load. Against real Codex, the request right after a load read 99% of its input from the cache.Docs. "Configure Claude Code" gains an "MCP tool search" section recommending
ENABLE_TOOL_SEARCH=true. The Codex page describes the mapping.Details for reviewers and agents: implementation, edge cases, tests, live check
Implementation (
src/providers/codex/translate/request.rs)tool_reference_loadsmaps eachtool_result'stool_use_idto the tool names itstool_referencechildren load.split_loaded_deferred_toolskeeps loaded deferred tools out of the prefix and drops the placeholder.append_loaded_toolsinserts oneadditional_toolsitem after each matchingfunction_call_output. The order follows thetoolsarray and the message order, so output is deterministic. A tool loaded twice is appended once, at its first load.debug_assert!checks that every loaded tool was placed.Edge cases checked
tool_referencenaming a tool absent fromtools: no item is emitted.additional_toolsitems fall in theconversationslice ofsplit_input_envelope. After Claude Code compacts, the loads are no longer visible, so those tools return to the prefix.tools: None, and the new items land in the input delta.Tests
loaded_deferred_tool_is_appended_after_its_tool_search_resulttool_load_keeps_the_earlier_request_as_a_prefixdeferred_tool_without_a_visible_load_stays_in_the_prefixfull_lane_keeps_loaded_tools_and_drops_the_placeholdercargo fmt --all -- --check: clean.cargo clippy --all-targets --locked: no diagnostics.cargo test --all-targets --locked: 1121 passed, 0 failed. One earlier run failedbridge_resume_continues_after_tool_use_pauseintests/cursor_native.rs, which this change doesn't touch. That test file passed 15 of 15 times both on this branch and onmain, and the next full run passed.mainat fe10718 (2026-11-11), which adds the opt-in full Responses lane (codex: add fullLane config to escape Responses Lite for sol/terra (parallel tool calls) #77):cargo clippy --all-targets --locked -- -D warningsclean,cargo test --all --locked1175 passed, 0 failed. The full lane keeps loaded tools intoolsas before; only the Lite lane appends them mid-history.Live check. This was a real Claude Code 2.1.293 session with
ENABLE_TOOL_SEARCH=trueandgpt-6-astra, through our running build (this change applied on v0.1.44).ToolSearchloaded the deferredCronList, the model called it, then it ran a Bash command. Codex accepted the appendedadditional_toolsitem. Cache reads per request:Review. One automated review pass with a separate Claude agent found no correctness bug. It suggested the
debug_assert!. It could not determine from the source whether Codex accepts anid-lessadditional_toolsitem mid-input; the live check above covers that.Drafted by Claude Opus 5.5 in Claude Code on behalf of @Butanium; not reviewed by a human.
https://claude.ai/code/session_01Xmx9RE7u4ZLBzSUjpjqyBM