fix(server-tools): replay spent upstream bodies as real error statuses - #150
Merged
Merged
Conversation
An upstream error response was read twice on the server-tool path. The loop consumed the body to check whether the model asked for a search, then handed the same Response back; the route layer read it again to build the client's answer and threw "Body already used", so a rate limit surfaced as a 500 instead of its real status. Clone before reading in both loop iterations and rebuild the response from the drained body on failure, preserving non-JSON error details. The Anthropic streaming bridge also emitted a generic api_error at HTTP 200, hiding the upstream status from clients that key their retry decisions on rate_limit_error. It now maps the upstream status through the same status-to-type table the non-streaming path uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca6fa94ad2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #150 +/- ##
==========================================
- Coverage 95.43% 95.41% -0.02%
==========================================
Files 36 36
Lines 6329 6349 +20
Branches 1809 1813 +4
==========================================
+ Hits 6040 6058 +18
- Misses 289 291 +2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
…eams The previous fix covered a failure on the first upstream call only. When a server tool runs and a later call fails, `createInlineServerToolStream` has already returned a 200, so the failure travels as an SSE error frame and never reaches the handler's error branch. Two things were lost: - The status. `rejectStream` guessed the type from the message text, so a 429 became `invalid_request_error` — telling clients that key their retry decisions on `rate_limit_error` to stop retrying an exhausted quota. The error chunk now carries the upstream status, and the type is derived from it. - The detail. `readBufferedChatCompletionPayload` reported only the status code, dropping the upstream's own explanation — the rate-limit code and reset timestamp a client needs. It now prefers the upstream's message, un-nesting it so the client sees text rather than a JSON string. A payload with no message anywhere keeps the raw JSON, which is the only record of its `code`; an empty body falls through to the generic message instead of winning the slot on being non-null. `extractErrorMessage` moves to shared/http so the loop and the Anthropic handler share one definition of "the message" instead of diverging. Also fixes patch branch coverage, which CI measured at 75% against main: the previous run was measured against the wrong base and included an unrelated in-flight branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
orangeboyChen
force-pushed
the
fix/upstream-error-spent-body
branch
from
September 17, 2026 07:11
b2441bd to
53eaf6d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
An upstream rate limit surfaced to clients as
status_code=500, Body already usedinstead of the real 429:Cause
The
route: "/v1/chat/completions"in that log isfetchChatCompletion's internal label, so the 429 came from inside the server-tool loop, not a plain chat request.executeWebSearchLoopread the upstream body to check whether the model asked for a search, then returned that sameResponseobject. AResponsebody can only be consumed once, so the route layer's second read — the one that builds the answer the client sees — threwBody is unusable: Body has already been read, and the 500 replaced the real status.Confirmed against the unpatched tree: HTTP 500
{"type":"error","error":{"type":"api_error","message":"Body is unusable: Body has already been read"}}.Fix
web-search-loop.ts— clone before reading in both loop iterations, and rebuild the response from the drained body on failure so both reads work. The body is replayed verbatim rather than re-serialized, so an error detail that isn't valid JSON still reaches the client intact.content-length/content-encodingare dropped because the bytes are re-emitted, not re-encoded.anthropic.ts— the streaming bridge emitted a genericapi_errorat HTTP 200, hiding the upstream status from clients that key retry decisions onrate_limit_error. It now maps the upstream status through the same status→type table the non-streaming path already used, extracted asanthropicErrorType.Result
api_errorrate_limit_error+ upstream detailapi_errorrate_limit_error+ upstream detailBody already usedThe streaming HTTP status stays 200 because the Anthropic SSE envelope is already committed by the time the failure arrives — the error type now carries the information instead.
Verification
3 new regression tests in
tests/server/upstream-error-response.test.ts; all 3 fail on the unpatched tree and pass with the fix.bun run lint/format:check/typecheck/build— passbun run test:ci— 670 tests pass, 94.46% statementsbun run test:patch-branches— 92.80% (≥ 90% required)Note
Based directly on
main, not onfix/server-tool-loop-streaming. The fix cherry-picks cleanly and the full suite passes in both places, so it can merge independently of that in-flight work.🤖 Generated with Claude Code