[XieX]

Problem

With an Anthropic key that had no credits, a Claude Agents run returned { output: 'Credit balance is too low' } as a successful run. config().invoke tracked $ld:ai:generation:success, a caller would show the error text to a user as model output, and the trace showed an OK chat <synthetic> span.

The Claude Code CLI does not fail a run when an API call fails. Captured live from Agent SDK 0.3.220 with a rejected key (no credits takes the same path):

• an assistant message with model: "<synthetic>", error: "authentication_failed", zero usage, and the error text as content
• then a result with subtype: "success", is_error: true, api_error_status: 401, terminal_reason: "api_error", and the error text in result

The handlers checked only subtype. The agent SDK itself treats is_error as the failure signal and raises after the result, but the handler stops reading at the result and never sees that.

Fix

• _raise_if_run_failed fails a run when subtype != "success" or is_error is true. Both handler paths and to_claude_agents use it.
• to_claude_agents previously read message.result with no check at all, so it returned the error text as the answer and also ignored error subtypes. A failed root now raises and tracks $ld:ai:graph:invocation_failure; a failed sub-agent fails its tool call.
• An AssistantMessage whose model is <synthetic> gets no chat span, joins no conversation, and adds no usage. It keys on model, not the error field, because the CLI also emits a synthetic message after a response stops at max_output_tokens, and that run can still succeed.
• agents.md documents the rule.

Tests

Six new tests, each failing on main for the intended reason: an API-error result on the blocking and streaming paths, a synthetic message that shares a real response's id in a run that then succeeds, and the native graph root (API error, and error_max_turns) and sub-agent cases. A seventh pins that is_error: false stays a success.

Changed existing fixtures: _make_result_msg in test_native_graph.py now sets subtype, is_error, errors and api_error_status explicitly. Unset MagicMock attributes are truthy, so every existing mock result would otherwise read as an error.

Also checked end to end against the real CLI with a fake key and an empty CLAUDE_CONFIG_DIR: both paths reject with Claude agent run ended with an error result (HTTP 401): Invalid API key · Fix external API key, and the only span is invoke_agent with status ERROR.

Spec: TESTING.md §2.2 "Anthropic agents handler — run outcome from the result message" (launchdarkly/ai-sdks-monorepo#62).
Counterpart: launchdarkly/js-ai-sdk#138.

🤖 Generated with Claude Code

---

Note

Overview
Fixes a case where Anthropic API failures (bad key, no credits, etc.) were treated as successful runs because the Claude Code CLI returns ResultMessage with subtype: "success" and is_error: true, with the error text in result.

_raise_if_run_failed replaces the old subtype-only check: it raises on non-success subtypes (e.g. error_max_turns) and on success with is_error, including optional api_error_status in the error. Blocking invoke, streaming, and to_claude_agents all call it so callers get RuntimeError and generation/graph failure tracking instead of surfacing API error strings as model output.

Telemetry: assistant messages with model: "<synthetic>" (CLI-written errors or max-token notices) are ignored in InferenceSpans—no chat span, no conversation/usage merge—so error text cannot pollute a real response that shares the same message_id.

Docs in agents.md and expanded handler/native-graph tests cover API-error paths, streaming without a done event, and sub-agent failures.

^{Reviewed by Cursor Bugbot for commit a453733. Bugbot is set up for automated code reviews on this repo. Configure here.}

[jeffdupont]

Matches ai-sdks-monorepo#62 and js#138. _raise_if_run_failed writes usage to the root before it raises, on both paths and in _run_query. A streamed failure closes open AG-UI tool calls and re-raises with no done event. _is_synthetic keys on model, and the chat-span grouping uses message_id, which matches the SDK's AssistantMessage. The 7 new tests mirror the JS set; CI green.