Skip to content

fix(server-tools): replay spent upstream bodies as real error statuses - #150

Merged
orangeboyChen merged 2 commits into
mainfrom
fix/upstream-error-spent-body
Sep 17, 2026
Merged

orangeboyChen merged 2 commits into
mainfrom
fix/upstream-error-spent-body

Conversation

@orangeboyChen

Copy link
Copy Markdown
Owner

Problem

An upstream rate limit surfaced to clients as status_code=500, Body already used instead of the real 429:

[CodeBuddy2API] Upstream request failed {
  route: "/v1/chat/completions",
  status: 429,
  detail: "{"code":6004,"msg":"您的使用量已超出频率限制..."}"
}

Cause

The route: "/v1/chat/completions" in that log is fetchChatCompletion's internal label, so the 429 came from inside the server-tool loop, not a plain chat request.

executeWebSearchLoop read the upstream body to check whether the model asked for a search, then returned that same Response object. A Response body can only be consumed once, so the route layer's second read — the one that builds the answer the client sees — threw Body is unusable: Body has already been read, and the 500 replaced the real status.

Confirmed against the unpatched tree: HTTP 500 {"type":"error","error":{"type":"api_error","message":"Body is unusable: Body has already been read"}}.

Fix

  • web-search-loop.ts — clone before reading in both loop iterations, and rebuild the response from the drained body on failure so both reads work. The body is replayed verbatim rather than re-serialized, so an error detail that isn't valid JSON still reaches the client intact. content-length/content-encoding are dropped because the bytes are re-emitted, not re-encoded.
  • anthropic.ts — the streaming bridge emitted a generic api_error at HTTP 200, hiding the upstream status from clients that key retry decisions on rate_limit_error. It now maps the upstream status through the same status→type table the non-streaming path already used, extracted as anthropicErrorType.

Result

path before after
Anthropic non-stream 500 api_error 429 rate_limit_error + upstream detail
Anthropic stream 200 api_error 200 + rate_limit_error + upstream detail
Chat completions 500 Body already used 429 + upstream detail

The streaming HTTP status stays 200 because the Anthropic SSE envelope is already committed by the time the failure arrives — the error type now carries the information instead.

Verification

3 new regression tests in tests/server/upstream-error-response.test.ts; all 3 fail on the unpatched tree and pass with the fix.

  • bun run lint / format:check / typecheck / build — pass
  • bun run test:ci — 670 tests pass, 94.46% statements
  • bun run test:patch-branches — 92.80% (≥ 90% required)

Note

Based directly on main, not on fix/server-tool-loop-streaming. The fix cherry-picks cleanly and the full suite passes in both places, so it can merge independently of that in-flight work.

🤖 Generated with Claude Code

An upstream error response was read twice on the server-tool path. The
loop consumed the body to check whether the model asked for a search,
then handed the same Response back; the route layer read it again to
build the client's answer and threw "Body already used", so a rate limit
surfaced as a 500 instead of its real status.

Clone before reading in both loop iterations and rebuild the response
from the drained body on failure, preserving non-JSON error details.

The Anthropic streaming bridge also emitted a generic api_error at HTTP
200, hiding the upstream status from clients that key their retry
decisions on rate_limit_error. It now maps the upstream status through
the same status-to-type table the non-streaming path uses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca6fa94ad2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/server/proxy/anthropic.ts
@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.95918% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 95.41%. Comparing base (80f4125) to head (53eaf6d).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #150      +/-   ##
==========================================
- Coverage   95.43%   95.41%   -0.02%     
==========================================
  Files          36       36              
  Lines        6329     6349      +20     
  Branches     1809     1813       +4     
==========================================
+ Hits         6040     6058      +18     
- Misses        289      291       +2     
Flag Coverage Δ
unittests 95.41% <97.95%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…eams

The previous fix covered a failure on the first upstream call only. When a
server tool runs and a later call fails, `createInlineServerToolStream` has
already returned a 200, so the failure travels as an SSE error frame and
never reaches the handler's error branch. Two things were lost:

- The status. `rejectStream` guessed the type from the message text, so a 429
  became `invalid_request_error` — telling clients that key their retry
  decisions on `rate_limit_error` to stop retrying an exhausted quota. The
  error chunk now carries the upstream status, and the type is derived from
  it.
- The detail. `readBufferedChatCompletionPayload` reported only the status
  code, dropping the upstream's own explanation — the rate-limit code and
  reset timestamp a client needs. It now prefers the upstream's message,
  un-nesting it so the client sees text rather than a JSON string. A payload
  with no message anywhere keeps the raw JSON, which is the only record of
  its `code`; an empty body falls through to the generic message instead of
  winning the slot on being non-null.

`extractErrorMessage` moves to shared/http so the loop and the Anthropic
handler share one definition of "the message" instead of diverging.

Also fixes patch branch coverage, which CI measured at 75% against main: the
previous run was measured against the wrong base and included an unrelated
in-flight branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@orangeboyChen
orangeboyChen force-pushed the fix/upstream-error-spent-body branch from b2441bd to 53eaf6d Compare September 17, 2026 07:11
@orangeboyChen
orangeboyChen merged commit 7feb4ed into main Sep 17, 2026
7 checks passed
@orangeboyChen
orangeboyChen deleted the fix/upstream-error-spent-body branch September 17, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant