fix(anthropic): target-aware unsigned-thinking strip + forward image bytes (chat→messages 400s) - #860
Conversation
mcowger
left a comment
There was a problem hiding this comment.
Request changes: the unsigned-thinking drop cannot live unconditionally in the shared messages builder.
Plexus routes multiple providers through the Anthropic-shaped Messages wire format, but they do not share Anthropic's thinking semantics. In particular, Kimi's compatible endpoint requires unsigned thinking to remain present on historical assistant tool-call messages; Moonshot fixed this exact regression in this commit.
buildAnthropicRequest() does not know the concrete target provider, so dropping msg.thinking whenever signature is absent can break non-Anthropic Messages providers.
Suggested options:
- Preserve unsigned thinking in the shared builder, and strip it proactively only for targets known to require Anthropic signatures.
- Preserve it by default and extend the existing dispatcher auto-compat retry to match both
Invalid signature in thinking blockandthinking.signature: Field required. - Add an explicit provider/model capability or adapter for
thinking signature required, rather than inferring this solely from themessageswire type.
The registry-level auto_compat flag does not seem like the right abstraction; this is target wire compatibility, not generation-parameter translation.
My preferred implementation would be a hybrid:
- Shared builder: preserve unsigned thinking without emitting a
signaturefield. - Known strict Anthropic targets: proactively strip unsigned blocks using target-aware dispatch information.
- Fallback: use the existing bounded reactive auto-compat retry for strict or unknown gateways that return the missing-signature error.
- Tests: cover both strict Anthropic and Kimi-style unsigned-thinking tool replay, as well as native Messages pass-through.
That avoids an unnecessary first failed request for known Anthropic targets without imposing Anthropic behavior on every Messages-compatible provider.
Addresses review on mcowger#860: the shared Messages builder must not drop unsigned thinking unconditionally. Plexus routes several providers through the Anthropic-shaped Messages wire format and they do not share Anthropic's thinking semantics — Kimi's compatible endpoint requires unsigned thinking to remain on historical assistant tool-call messages, so stripping it in `buildAnthropicRequest()` (which cannot see the concrete target) would regress that provider. Replaced with the hybrid the review asked for: Shared builder — preserve unsigned thinking. The block is emitted with the `signature` key omitted rather than set to `undefined`, so code that inspects the built payload sees its true shape. The message-omission guard from the previous commit is gone with it; nothing is dropped here anymore. Known strict Anthropic targets — new `strip_unsigned_thinking` adapter, injected by `adapter-resolver.ts` under exactly the gate the tool-id normaliser already uses (outbound wire type is Anthropic Messages AND the target looks like Anthropic: anthropic.com URL, Anthropic OAuth, or Claude masking). Same override channel: `{ name, options: {}, enabled: true }` forces it on for a strict Anthropic-compatible gateway on another host, `{ name, enabled: false }` opts a detected-Anthropic route out. Only thinking blocks with no/empty signature are removed; signed blocks and `redacted_thinking` are untouched. The adapter reuses `stripThinkingSignatureBlocks` via a new optional `shouldStrip` predicate (default: strip all, preserving the existing stale-signature retry behaviour), so the empty-message handling — drop unless it would break user/assistant alternation or orphan a tool_result, else leave a `[reasoning elided]` placeholder — is shared rather than duplicated. Fallback — `matchThinkingSignatureError` now also matches `thinking.signature: Field required`, so the existing bounded reactive strip-and-retry covers strict or unknown gateways that were not detected up front. Tests cover: builder preserves unsigned thinking incl. the Kimi tool-call replay shape; adapter strips only unsigned blocks and keeps tool_use; copy-on-write; resolver injects for Anthropic targets and NOT for a Moonshot Messages URL; independent tombstone/force-enable; native messages -> messages pass-through keeps a signed block; the missing- signature 400 arms exactly one retry. Existing resolver gate tests updated for the second implicit adapter. Verified live: chat-completions history with `reasoning_content` -> builder emits unsigned thinking -> resolver injects the strip for an anthropic.com target -> Anthropic returns 200. Same builder output against a Moonshot target resolves no adapters and the block survives. Adds the adapter to docs/CONFIGURATION.md.
|
Thanks — agreed on all points, and the Kimi reference settled it. Reworked as the hybrid you described (c8117f4):
The adapter reuses Tests include the Kimi-style replay (unsigned thinking on a historical tool-call turn survives against a I also folded in a second, independent builder bug I hit right after (a537d89): |
Anthropic requires every `thinking` content block to carry the `signature`
it issued with it. A block without one is rejected outright:
400 invalid_request_error
messages.N.content.0.thinking.signature: Field required
The chat -> messages path emitted exactly that. `OpenAITransformer.parseRequest`
lifts an assistant message's `reasoning_content` into a unified `thinking`
block — necessarily without a signature, because the OpenAI chat-completions
wire format has no field for one. `buildAnthropicRequest` then pushed the
block unconditionally, `signature: undefined` and all.
In practice this means any client speaking chat-completions to Plexus whose
session history contains prior reasoning — produced by another model, or by
Claude through a translating proxy — 400s on every turn the moment it targets
a Claude alias. New sessions work; old sessions break. Observed with ZCode
switching a GLM-history session to claude-opus-5 / claude-fable-5-1.
Fix: only emit a `thinking` block when a signature is present. Prior-turn
thinking is optional on Anthropic's side, and an unsigned replay would never
have been accepted anyway, so dropping it is the only well-formed option. Text
and tool_use in the same message are preserved. If the unsigned block was the
message's *only* content, the message is omitted so we don't send an empty
content array (also rejected); the existing same-role merge keeps the
user/assistant alternation valid. That omission is scoped to the drop so
behaviour for other empty messages is unchanged.
Signed blocks arriving via the native messages -> messages path are
untouched — covered by a round-trip test.
Verified against the live API with the exact failing shape: 400 before,
200 after.
`buildAnthropicRequest` converted every unified `image_url` part into a
base64 image source with `data: ''`, discarding the actual image. Anthropic
rejects that outright:
400 invalid_request_error
messages.N.content.M.image.source.base64: image cannot be empty
so any chat-completions client attaching an image to a Claude alias 400'd on
the whole request. Observed with ZCode sending a screenshot to
claude-fable-5-1 through Plexus.
The unified schema carries images the OpenAI way — a single `url` that is
either a `data:` URL with the bytes inline or an http(s) URL. Map both onto
Anthropic's two source types:
data:<mime>;base64,<bytes> -> { type: 'base64', media_type, data }
http(s)://… -> { type: 'url', url }
`media_type` prefers the explicit part field, then the MIME in the data URL,
then image/jpeg. When there is nothing valid to send (no url, empty or
malformed data URL, non-base64 data URL) the part is dropped rather than
forwarded as a block we know upstream will reject; surrounding text and
tool blocks still go through. `cache_control` is preserved on the block.
This is the inverse of what content-mapper.ts already does on the parse
side (Anthropic base64 -> data URL), so a messages -> messages round-trip
now preserves image bytes exactly — covered by a test.
Verified against the live API with the exact failing shape: 400 before,
200 after (Claude read the pixel colour).
Addresses review on mcowger#860: the shared Messages builder must not drop unsigned thinking unconditionally. Plexus routes several providers through the Anthropic-shaped Messages wire format and they do not share Anthropic's thinking semantics — Kimi's compatible endpoint requires unsigned thinking to remain on historical assistant tool-call messages, so stripping it in `buildAnthropicRequest()` (which cannot see the concrete target) would regress that provider. Replaced with the hybrid the review asked for: Shared builder — preserve unsigned thinking. The block is emitted with the `signature` key omitted rather than set to `undefined`, so code that inspects the built payload sees its true shape. The message-omission guard from the previous commit is gone with it; nothing is dropped here anymore. Known strict Anthropic targets — new `strip_unsigned_thinking` adapter, injected by `adapter-resolver.ts` under exactly the gate the tool-id normaliser already uses (outbound wire type is Anthropic Messages AND the target looks like Anthropic: anthropic.com URL, Anthropic OAuth, or Claude masking). Same override channel: `{ name, options: {}, enabled: true }` forces it on for a strict Anthropic-compatible gateway on another host, `{ name, enabled: false }` opts a detected-Anthropic route out. Only thinking blocks with no/empty signature are removed; signed blocks and `redacted_thinking` are untouched. The adapter reuses `stripThinkingSignatureBlocks` via a new optional `shouldStrip` predicate (default: strip all, preserving the existing stale-signature retry behaviour), so the empty-message handling — drop unless it would break user/assistant alternation or orphan a tool_result, else leave a `[reasoning elided]` placeholder — is shared rather than duplicated. Fallback — `matchThinkingSignatureError` now also matches `thinking.signature: Field required`, so the existing bounded reactive strip-and-retry covers strict or unknown gateways that were not detected up front. Tests cover: builder preserves unsigned thinking incl. the Kimi tool-call replay shape; adapter strips only unsigned blocks and keeps tool_use; copy-on-write; resolver injects for Anthropic targets and NOT for a Moonshot Messages URL; independent tombstone/force-enable; native messages -> messages pass-through keeps a signed block; the missing- signature 400 arms exactly one retry. Existing resolver gate tests updated for the second implicit adapter. Verified live: chat-completions history with `reasoning_content` -> builder emits unsigned thinking -> resolver injects the strip for an anthropic.com target -> Anthropic returns 200. Same builder output against a Moonshot target resolves no adapters and the block survives. Adds the adapter to docs/CONFIGURATION.md.
c8117f4 to
ccb6ee9
Compare
Every Anthropic usage reader in Plexus looked for reasoning tokens in the
wrong place, so the recorded count was 0 for every real Anthropic request
regardless of how much the model thought:
- normalizeAnthropicUsage hard-coded reasoning_tokens: 0
- AnthropicTransformer.extractUsage read usage.thinkingTokens
- transformAnthropicResponse / transformAnthropicStream
ignored the reported count and imputed
from text length only
Anthropic reports it as `usage.output_tokens_details.thinking_tokens`, on
the non-streaming body and on the final streaming `message_delta`.
`thinkingTokens` is a Plexus-invented flat field that only Plexus's own
Anthropic-format responses ever emit.
Add `anthropicReasoningTokens(usage)` in usage-normalizer.ts — provider
field first, `thinkingTokens` fallback so a Plexus fronting another Plexus
still counts — and use it everywhere Anthropic usage is read. The response
and stream transformers now prefer the reported figure (output_tokens
already includes thinking, so the visible-text share is the remainder) and
keep the text-length imputation only as a fallback for compatible upstreams
that return thinking content without a count.
On the way out, the response and stream formatters emit
`output_tokens_details.thinking_tokens` alongside the existing
`thinkingTokens`, so Plexus's Anthropic-format output matches Anthropic's
own wire shape and round-trips through the new reader.
Observed on a live claude-fable-5-1 session: dashboard `reason` column 0 on
every turn while the API was returning thinking_tokens > 0. Also affects
cost, since tokensReasoning feeds the cost calculation. Fixture in the new
test is a verbatim live usage body.
|
Pushed one more Anthropic fix found while verifying thinking end-to-end (adb3013) — reasoning tokens were recorded as 0 for every Anthropic request. Every Anthropic usage reader looked in the wrong place: This affects cost too, since |
Summary
Two independent 400s hit any client that speaks OpenAI chat-completions to Plexus and targets a Claude alias, both caused by
transformers/anthropic/request-builder.tsemitting content Anthropic rejects outright:messages.N.content.0.thinking.signature: Field requiredthinkingblock pushed withsignature: undefinedmessages.N.content.M.image.source.base64: image cannot be emptyimage_urlpart became{ type: 'base64', data: '' }Observed with ZCode → Plexus (
/v1/chat/completions) →claude-opus-5/claude-fable-5-1. The thinking case makes old sessions 400 on every turn after a model switch (new sessions work, so it looks like a client bug); the image case makes any request with an attached screenshot 400.1. Unsigned thinking blocks — target-aware strip
Now the hybrid:
Shared builder — preserve. Unsigned thinking is emitted with the
signaturekey omitted (notundefined), so nothing is lost for providers that want it.Known Anthropic targets — proactive strip. New
strip_unsigned_thinkingadapter, injected byadapter-resolver.tsunder exactly the gatenormalize_anthropic_tool_idsalready uses: outbound wire type is Anthropic Messages and the target looks like Anthropic (anthropic.comURL, Anthropic OAuth, or Claude masking). Same override channel —{ name, options: {}, enabled: true }forces it on for a strict gateway on another host,{ name, enabled: false }opts out. Only blocks with a missing/empty signature are removed; signed blocks andredacted_thinkingare untouched.It reuses
stripThinkingSignatureBlocksvia a new optionalshouldStrippredicate (default unchanged: strip all), so the empty-message handling — drop, unless that breaks alternation or orphans atool_result, else[reasoning elided]placeholder — is shared, not duplicated.Fallback — reactive retry.
matchThinkingSignatureErrornow also matchesthinking.signature: Field required, so the existing bounded strip-and-retry covers strict or unknown gateways that weren't detected up front.Why proactive for known targets rather than reactive-only: it's a wasted round trip on every turn of an affected session, and Claude-Code masking signs the body after adapters run, so mutating on retry is the case the tool-id normaliser had to avoid too.
2. Empty image source — forward the bytes
The unified schema carries images the OpenAI way (one
url, eitherdata:orhttp(s)). Map both onto Anthropic's source types:media_typeprefers the explicit part field, then the data-URL MIME, thenimage/jpeg. An unusable part (no url, empty/malformed/non-base64 data URL) is dropped rather than sent as a block we know will be rejected; surrounding text and tools go through;cache_controlis preserved. This is the inverse of whatcontent-mapper.tsalready does on the parse side, so a messages → messages round-trip now preserves image bytes exactly.Tests
anthropic-unsigned-thinking.test.ts— builder preserves unsigned thinking incl. the Kimi tool-call replay shape; adapter strips only unsigned blocks, keepstool_use, copy-on-write, alternation placeholder; resolver injects for ananthropic.comtarget and not for a Moonshot Messages URL; independent tombstone / force-enable; native pass-through keeps a signed block; the missing-signature 400 arms exactly one retryanthropic-image-source.test.ts— data URL / http URL / MIME precedence / unusable inputs; chat-completions end-to-end; messages → messages byte-for-byte round-tripadapter-resolver.test.ts— existing gate cases updated for the second implicit adapter; new gate-independence casesbun run test,bun run typecheck, biome all green; pre-commit hooks ran clean on each commit.Verified against the live API
Same request shapes, unpatched vs patched, sent to
https://api.anthropic.com/v1/messages:Docs:
strip_unsigned_thinkingadded to the adapter table inCONFIGURATION.md.