Skip to content

fix(anthropic): target-aware unsigned-thinking strip + forward image bytes (chat→messages 400s) - #860

Open
invertibleMatrix wants to merge 4 commits into
mcowger:mainfrom
invertibleMatrix:fix/anthropic-unsigned-thinking-block
Open

fix(anthropic): target-aware unsigned-thinking strip + forward image bytes (chat→messages 400s)#860
invertibleMatrix wants to merge 4 commits into
mcowger:mainfrom
invertibleMatrix:fix/anthropic-unsigned-thinking-block

Conversation

@invertibleMatrix

@invertibleMatrix invertibleMatrix commented Sep 8, 2026

Copy link
Copy Markdown

Summary

Two independent 400s hit any client that speaks OpenAI chat-completions to Plexus and targets a Claude alias, both caused by transformers/anthropic/request-builder.ts emitting content Anthropic rejects outright:

Error from Anthropic Cause in the builder
messages.N.content.0.thinking.signature: Field required thinking block pushed with signature: undefined
messages.N.content.M.image.source.base64: image cannot be empty every image_url part became { type: 'base64', data: '' }

Observed with ZCode → Plexus (/v1/chat/completions) → claude-opus-5 / claude-fable-5-1. The thinking case makes old sessions 400 on every turn after a model switch (new sessions work, so it looks like a client bug); the image case makes any request with an attached screenshot 400.

1. Unsigned thinking blocks — target-aware strip

Revised per review: the first cut dropped unsigned thinking unconditionally in the shared builder. That's wrong — Kimi's Messages-compatible endpoint requires unsigned thinking to remain on historical assistant tool-call turns (MoonshotAI/kimi-code@13e0fff), and buildAnthropicRequest() doesn't know the concrete target.

Now the hybrid:

Shared builder — preserve. Unsigned thinking is emitted with the signature key omitted (not undefined), so nothing is lost for providers that want it.

Known Anthropic targets — proactive strip. New strip_unsigned_thinking adapter, injected by adapter-resolver.ts under exactly the gate normalize_anthropic_tool_ids already uses: outbound wire type is Anthropic Messages and the target looks like Anthropic (anthropic.com URL, Anthropic OAuth, or Claude masking). Same override channel — { name, options: {}, enabled: true } forces it on for a strict gateway on another host, { name, enabled: false } opts out. Only blocks with a missing/empty signature are removed; signed blocks and redacted_thinking are untouched.

It reuses stripThinkingSignatureBlocks via a new optional shouldStrip predicate (default unchanged: strip all), so the empty-message handling — drop, unless that breaks alternation or orphans a tool_result, else [reasoning elided] placeholder — is shared, not duplicated.

Fallback — reactive retry. matchThinkingSignatureError now also matches thinking.signature: Field required, so the existing bounded strip-and-retry covers strict or unknown gateways that weren't detected up front.

Why proactive for known targets rather than reactive-only: it's a wasted round trip on every turn of an affected session, and Claude-Code masking signs the body after adapters run, so mutating on retry is the case the tool-id normaliser had to avoid too.

2. Empty image source — forward the bytes

The unified schema carries images the OpenAI way (one url, either data: or http(s)). Map both onto Anthropic's source types:

data:<mime>;base64,<bytes>  →  { type: 'base64', media_type, data }
http(s)://…                 →  { type: 'url', url }

media_type prefers the explicit part field, then the data-URL MIME, then image/jpeg. An unusable part (no url, empty/malformed/non-base64 data URL) is dropped rather than sent as a block we know will be rejected; surrounding text and tools go through; cache_control is preserved. This is the inverse of what content-mapper.ts already does on the parse side, so a messages → messages round-trip now preserves image bytes exactly.

Tests

  • anthropic-unsigned-thinking.test.ts — builder preserves unsigned thinking incl. the Kimi tool-call replay shape; adapter strips only unsigned blocks, keeps tool_use, copy-on-write, alternation placeholder; resolver injects for an anthropic.com target and not for a Moonshot Messages URL; independent tombstone / force-enable; native pass-through keeps a signed block; the missing-signature 400 arms exactly one retry
  • anthropic-image-source.test.ts — data URL / http URL / MIME precedence / unusable inputs; chat-completions end-to-end; messages → messages byte-for-byte round-trip
  • adapter-resolver.test.ts — existing gate cases updated for the second implicit adapter; new gate-independence cases

bun run test, bun run typecheck, biome all green; pre-commit hooks ran clean on each commit.

Verified against the live API

Same request shapes, unpatched vs patched, sent to https://api.anthropic.com/v1/messages:

thinking — before: 400 "messages.1.content.0.thinking.signature: Field required"
thinking — after:  resolver injects [normalize_anthropic_tool_ids, strip_unsigned_thinking] → 200
thinking — Moonshot target: resolver injects [] → unsigned block preserved (not sent to Anthropic)

image    — before: 400 "messages.0.content.1.image.source.base64: image cannot be empty"
image    — after:  200, Claude read the pixel colour

Docs: strip_unsigned_thinking added to the adapter table in CONFIGURATION.md.

@mcowger mcowger left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes: the unsigned-thinking drop cannot live unconditionally in the shared messages builder.

Plexus routes multiple providers through the Anthropic-shaped Messages wire format, but they do not share Anthropic's thinking semantics. In particular, Kimi's compatible endpoint requires unsigned thinking to remain present on historical assistant tool-call messages; Moonshot fixed this exact regression in this commit.

buildAnthropicRequest() does not know the concrete target provider, so dropping msg.thinking whenever signature is absent can break non-Anthropic Messages providers.

Suggested options:

  1. Preserve unsigned thinking in the shared builder, and strip it proactively only for targets known to require Anthropic signatures.
  2. Preserve it by default and extend the existing dispatcher auto-compat retry to match both Invalid signature in thinking block and thinking.signature: Field required.
  3. Add an explicit provider/model capability or adapter for thinking signature required, rather than inferring this solely from the messages wire type.

The registry-level auto_compat flag does not seem like the right abstraction; this is target wire compatibility, not generation-parameter translation.

My preferred implementation would be a hybrid:

  • Shared builder: preserve unsigned thinking without emitting a signature field.
  • Known strict Anthropic targets: proactively strip unsigned blocks using target-aware dispatch information.
  • Fallback: use the existing bounded reactive auto-compat retry for strict or unknown gateways that return the missing-signature error.
  • Tests: cover both strict Anthropic and Kimi-style unsigned-thinking tool replay, as well as native Messages pass-through.

That avoids an unnecessary first failed request for known Anthropic targets without imposing Anthropic behavior on every Messages-compatible provider.

Repository owner deleted a comment from kody-ai Bot Sep 8, 2026
invertibleMatrix added a commit to invertibleMatrix/plexus that referenced this pull request Sep 8, 2026
Addresses review on mcowger#860: the shared Messages builder must not drop
unsigned thinking unconditionally. Plexus routes several providers through
the Anthropic-shaped Messages wire format and they do not share Anthropic's
thinking semantics — Kimi's compatible endpoint requires unsigned thinking
to remain on historical assistant tool-call messages, so stripping it in
`buildAnthropicRequest()` (which cannot see the concrete target) would
regress that provider.

Replaced with the hybrid the review asked for:

Shared builder — preserve unsigned thinking. The block is emitted with the
`signature` key omitted rather than set to `undefined`, so code that
inspects the built payload sees its true shape. The message-omission guard
from the previous commit is gone with it; nothing is dropped here anymore.

Known strict Anthropic targets — new `strip_unsigned_thinking` adapter,
injected by `adapter-resolver.ts` under exactly the gate the tool-id
normaliser already uses (outbound wire type is Anthropic Messages AND the
target looks like Anthropic: anthropic.com URL, Anthropic OAuth, or Claude
masking). Same override channel: `{ name, options: {}, enabled: true }`
forces it on for a strict Anthropic-compatible gateway on another host,
`{ name, enabled: false }` opts a detected-Anthropic route out. Only
thinking blocks with no/empty signature are removed; signed blocks and
`redacted_thinking` are untouched.

The adapter reuses `stripThinkingSignatureBlocks` via a new optional
`shouldStrip` predicate (default: strip all, preserving the existing
stale-signature retry behaviour), so the empty-message handling — drop
unless it would break user/assistant alternation or orphan a tool_result,
else leave a `[reasoning elided]` placeholder — is shared rather than
duplicated.

Fallback — `matchThinkingSignatureError` now also matches
`thinking.signature: Field required`, so the existing bounded reactive
strip-and-retry covers strict or unknown gateways that were not detected
up front.

Tests cover: builder preserves unsigned thinking incl. the Kimi tool-call
replay shape; adapter strips only unsigned blocks and keeps tool_use;
copy-on-write; resolver injects for Anthropic targets and NOT for a
Moonshot Messages URL; independent tombstone/force-enable; native
messages -> messages pass-through keeps a signed block; the missing-
signature 400 arms exactly one retry. Existing resolver gate tests updated
for the second implicit adapter.

Verified live: chat-completions history with `reasoning_content` ->
builder emits unsigned thinking -> resolver injects the strip for an
anthropic.com target -> Anthropic returns 200. Same builder output against
a Moonshot target resolves no adapters and the block survives.

Adds the adapter to docs/CONFIGURATION.md.
@invertibleMatrix invertibleMatrix changed the title fix(anthropic): drop unsigned thinking blocks instead of 400ing upstream fix(anthropic): target-aware unsigned-thinking strip + forward image bytes (chat→messages 400s) Sep 8, 2026
@invertibleMatrix

Copy link
Copy Markdown
Author

Thanks — agreed on all points, and the Kimi reference settled it. Reworked as the hybrid you described (c8117f4):

  • Shared builder: unsigned thinking is preserved; the signature key is omitted rather than emitted as undefined. The message-omission guard from the first commit is gone.
  • Known Anthropic targets: new strip_unsigned_thinking adapter, injected by adapter-resolver.ts under the exact same gate as normalize_anthropic_tool_ids (Messages wire type and anthropic.com / Anthropic OAuth / Claude masking), with the same enabled: true|false override channel. Only unsigned thinking is removed — signed blocks and redacted_thinking are untouched.
  • Fallback: matchThinkingSignatureError now also matches thinking.signature: Field required, so the existing bounded strip-and-retry covers strict/unknown gateways.

The adapter reuses stripThinkingSignatureBlocks via an optional shouldStrip predicate (default = current behaviour), so the alternation / orphaned-tool_result handling is shared rather than reimplemented.

Tests include the Kimi-style replay (unsigned thinking on a historical tool-call turn survives against a moonshot.ai Messages URL, is stripped against anthropic.com), native pass-through keeping a signed block, and the missing-signature 400 arming exactly one retry. Verified live against Anthropic: 200 with the adapter in the chain.

I also folded in a second, independent builder bug I hit right after (a537d89): image_url parts were always emitted as { type: 'base64', data: '' }, so any chat-completions request with an image 400'd with image.source.base64: image cannot be empty. Now maps data URLs → base64 source and http(s) → url source, inverse of what content-mapper.ts does on parse. Happy to split it into its own PR if you'd rather keep this one to the thinking change.

Anthropic requires every `thinking` content block to carry the `signature`
it issued with it. A block without one is rejected outright:

    400 invalid_request_error
    messages.N.content.0.thinking.signature: Field required

The chat -> messages path emitted exactly that. `OpenAITransformer.parseRequest`
lifts an assistant message's `reasoning_content` into a unified `thinking`
block — necessarily without a signature, because the OpenAI chat-completions
wire format has no field for one. `buildAnthropicRequest` then pushed the
block unconditionally, `signature: undefined` and all.

In practice this means any client speaking chat-completions to Plexus whose
session history contains prior reasoning — produced by another model, or by
Claude through a translating proxy — 400s on every turn the moment it targets
a Claude alias. New sessions work; old sessions break. Observed with ZCode
switching a GLM-history session to claude-opus-5 / claude-fable-5-1.

Fix: only emit a `thinking` block when a signature is present. Prior-turn
thinking is optional on Anthropic's side, and an unsigned replay would never
have been accepted anyway, so dropping it is the only well-formed option. Text
and tool_use in the same message are preserved. If the unsigned block was the
message's *only* content, the message is omitted so we don't send an empty
content array (also rejected); the existing same-role merge keeps the
user/assistant alternation valid. That omission is scoped to the drop so
behaviour for other empty messages is unchanged.

Signed blocks arriving via the native messages -> messages path are
untouched — covered by a round-trip test.

Verified against the live API with the exact failing shape: 400 before,
200 after.
`buildAnthropicRequest` converted every unified `image_url` part into a
base64 image source with `data: ''`, discarding the actual image. Anthropic
rejects that outright:

    400 invalid_request_error
    messages.N.content.M.image.source.base64: image cannot be empty

so any chat-completions client attaching an image to a Claude alias 400'd on
the whole request. Observed with ZCode sending a screenshot to
claude-fable-5-1 through Plexus.

The unified schema carries images the OpenAI way — a single `url` that is
either a `data:` URL with the bytes inline or an http(s) URL. Map both onto
Anthropic's two source types:

  data:<mime>;base64,<bytes>  ->  { type: 'base64', media_type, data }
  http(s)://…                 ->  { type: 'url', url }

`media_type` prefers the explicit part field, then the MIME in the data URL,
then image/jpeg. When there is nothing valid to send (no url, empty or
malformed data URL, non-base64 data URL) the part is dropped rather than
forwarded as a block we know upstream will reject; surrounding text and
tool blocks still go through. `cache_control` is preserved on the block.

This is the inverse of what content-mapper.ts already does on the parse
side (Anthropic base64 -> data URL), so a messages -> messages round-trip
now preserves image bytes exactly — covered by a test.

Verified against the live API with the exact failing shape: 400 before,
200 after (Claude read the pixel colour).
Addresses review on mcowger#860: the shared Messages builder must not drop
unsigned thinking unconditionally. Plexus routes several providers through
the Anthropic-shaped Messages wire format and they do not share Anthropic's
thinking semantics — Kimi's compatible endpoint requires unsigned thinking
to remain on historical assistant tool-call messages, so stripping it in
`buildAnthropicRequest()` (which cannot see the concrete target) would
regress that provider.

Replaced with the hybrid the review asked for:

Shared builder — preserve unsigned thinking. The block is emitted with the
`signature` key omitted rather than set to `undefined`, so code that
inspects the built payload sees its true shape. The message-omission guard
from the previous commit is gone with it; nothing is dropped here anymore.

Known strict Anthropic targets — new `strip_unsigned_thinking` adapter,
injected by `adapter-resolver.ts` under exactly the gate the tool-id
normaliser already uses (outbound wire type is Anthropic Messages AND the
target looks like Anthropic: anthropic.com URL, Anthropic OAuth, or Claude
masking). Same override channel: `{ name, options: {}, enabled: true }`
forces it on for a strict Anthropic-compatible gateway on another host,
`{ name, enabled: false }` opts a detected-Anthropic route out. Only
thinking blocks with no/empty signature are removed; signed blocks and
`redacted_thinking` are untouched.

The adapter reuses `stripThinkingSignatureBlocks` via a new optional
`shouldStrip` predicate (default: strip all, preserving the existing
stale-signature retry behaviour), so the empty-message handling — drop
unless it would break user/assistant alternation or orphan a tool_result,
else leave a `[reasoning elided]` placeholder — is shared rather than
duplicated.

Fallback — `matchThinkingSignatureError` now also matches
`thinking.signature: Field required`, so the existing bounded reactive
strip-and-retry covers strict or unknown gateways that were not detected
up front.

Tests cover: builder preserves unsigned thinking incl. the Kimi tool-call
replay shape; adapter strips only unsigned blocks and keeps tool_use;
copy-on-write; resolver injects for Anthropic targets and NOT for a
Moonshot Messages URL; independent tombstone/force-enable; native
messages -> messages pass-through keeps a signed block; the missing-
signature 400 arms exactly one retry. Existing resolver gate tests updated
for the second implicit adapter.

Verified live: chat-completions history with `reasoning_content` ->
builder emits unsigned thinking -> resolver injects the strip for an
anthropic.com target -> Anthropic returns 200. Same builder output against
a Moonshot target resolves no adapters and the block survives.

Adds the adapter to docs/CONFIGURATION.md.
@invertibleMatrix
invertibleMatrix force-pushed the fix/anthropic-unsigned-thinking-block branch from c8117f4 to ccb6ee9 Compare September 8, 2026 08:34
Every Anthropic usage reader in Plexus looked for reasoning tokens in the
wrong place, so the recorded count was 0 for every real Anthropic request
regardless of how much the model thought:

  - normalizeAnthropicUsage        hard-coded reasoning_tokens: 0
  - AnthropicTransformer.extractUsage  read usage.thinkingTokens
  - transformAnthropicResponse / transformAnthropicStream
                                   ignored the reported count and imputed
                                   from text length only

Anthropic reports it as `usage.output_tokens_details.thinking_tokens`, on
the non-streaming body and on the final streaming `message_delta`.
`thinkingTokens` is a Plexus-invented flat field that only Plexus's own
Anthropic-format responses ever emit.

Add `anthropicReasoningTokens(usage)` in usage-normalizer.ts — provider
field first, `thinkingTokens` fallback so a Plexus fronting another Plexus
still counts — and use it everywhere Anthropic usage is read. The response
and stream transformers now prefer the reported figure (output_tokens
already includes thinking, so the visible-text share is the remainder) and
keep the text-length imputation only as a fallback for compatible upstreams
that return thinking content without a count.

On the way out, the response and stream formatters emit
`output_tokens_details.thinking_tokens` alongside the existing
`thinkingTokens`, so Plexus's Anthropic-format output matches Anthropic's
own wire shape and round-trips through the new reader.

Observed on a live claude-fable-5-1 session: dashboard `reason` column 0 on
every turn while the API was returning thinking_tokens > 0. Also affects
cost, since tokensReasoning feeds the cost calculation. Fixture in the new
test is a verbatim live usage body.
@invertibleMatrix

Copy link
Copy Markdown
Author

Pushed one more Anthropic fix found while verifying thinking end-to-end (adb3013) — reasoning tokens were recorded as 0 for every Anthropic request.

Every Anthropic usage reader looked in the wrong place: normalizeAnthropicUsage hard-coded reasoning_tokens: 0, extractUsage read usage.thinkingTokens, and the response/stream transformers only imputed from text length. Anthropic actually reports usage.output_tokens_details.thinking_tokens. Added anthropicReasoningTokens() (provider field first, thinkingTokens fallback for Plexus-fronting-Plexus) and used it everywhere; the formatters now emit output_tokens_details.thinking_tokens alongside thinkingTokens so Plexus output matches Anthropic's wire shape. Test fixture is a verbatim live usage body.

This affects cost too, since tokensReasoning feeds the cost calc. Same offer as the image fix — happy to split any of these into their own PRs if you'd prefer smaller units.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants