Skip to content

$ llm codec: Cohere TIMEOUT and Bedrock malformed_* stops are served and billed as in 1.5.5 - #844

Draft
MattJackson wants to merge 1 commit into
predevfrom
lane-llm-billing-stops
Draft

MattJackson wants to merge 1 commit into
predevfrom
lane-llm-billing-stops

Conversation

@MattJackson

Copy link
Copy Markdown
Collaborator

What. The Cohere reader mapped finish_reason: TIMEOUT, and the Bedrock reader mapped stopReason: malformed_model_output | malformed_tool_use, to IrStopReason::Error. Through exchange::reply::wire::generation_failed that turned a translated buffered answer into a 502 that still carried its usage, and charged a lane fault. A Cohere TIMEOUT stream also raised an in-band error event. This PR restores 1.5.5: all three read as Other, so the answer is served and billed as a served answer.

Why. Row-159 audit, plane-llm L2#5 (HIGH). The 1.5.5 source is v1.5.5 crates/busbar/src/proto/cohere/mod.rs:179-191 and crates/busbar/src/proto/bedrock/mod.rs:705-716 (_ => S::Other). Nothing signs the change. The fidelity rule says "Billing never moves". §7 and #77 keep the money path at 1.5.5. CHANGELOG 1.6.0 (the "stream that dies mid-flight" entry) registers only Cohere's generic ERROR, and only on streams.

RED-able tests.

  • cohere/tests/ir_mapping_tests.rs::timeout_is_served_and_billed_as_in_1_5_5: generation_failed("cohere", …) is false, the usage is 3/1, and the stream has no Error event.
  • bedrock/tests/ir_mapping_tests.rs::bed10_context_window_and_malformed_stop_reasons_map: both malformed_* read Other and generation_failed("bedrock", …) is false.
  • cohere/tests/tests.rs: read_cohere_stop_reason("TIMEOUT") == Other.

Not changed: the generic Cohere ERROR (registered), and Bedrock model_context_window_exceeded → MaxTokens (a MEDIUM translation item, not money). No ledger, oracle cell or ceiling changes.

…and billed as in 1.5.5

Both readers mapped these stops to IrStopReason::Error, so a translated buffered answer became a
billed 502 with a lane fault, and a Cohere TIMEOUT stream raised an error frame. 1.5.5 read all
three as Other (v1.5.5 crates/busbar/src/proto/cohere/mod.rs:179-191, bedrock/mod.rs:705-716) and
served the answer. No owner ruling signs the change; CHANGELOG 1.6.0 registers only Cohere's
generic stream ERROR. RED tests: generation_failed() is false and the stream carries no error event.
@MattJackson

Copy link
Copy Markdown
Collaborator Author

row 155 B2 (#974) edits crates/busbar-plane-llm/src/codec/{bedrock,cohere}/mod.rs at fn claims and the DECL quota_exceeded_status / auth_failure_status_and_kind rows; disjoint from this PR's hunks.

@MattJackson
MattJackson marked this pull request as draft October 14, 2026 19:52
@MattJackson MattJackson added the alone ModernCI: judge and land alone (never batched) label Oct 14, 2026
@MattJackson
MattJackson marked this pull request as ready for review October 14, 2026 20:22
@MattJackson
MattJackson marked this pull request as draft October 14, 2026 22:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

alone ModernCI: judge and land alone (never batched)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant