fix(stream): parse streamed tool-call chunks (fixes 'dict' object has no attribute 'delta')#9
Merged
VickyXAI merged 3 commits intoJun 14, 2026
Conversation
…ribute 'delta')
Streaming a tool call crashed the SDK with:
AttributeError: 'dict' object has no attribute 'delta' (in _aiter_and_archive)
Root cause: ChatChunkDelta.tool_calls used the strict non-stream ToolCall
(id / function.name / arguments all required). OpenAI streams tool calls
incrementally — the first frame has id + name, later frames carry only an
arguments *fragment* — so those fragment frames failed validation,
ChatCompletionChunk(**chunk) raised, and _iter_sse_chunks fell back to
model_construct(), which doesn't parse nested models. choices were then raw
dicts, and the archive loop's `choice.delta.content` blew up. Plain chat
worked (text frames validate); any tool call broke. Hits LLMClient and
SolanaLLMClient, sync and async.
Fix:
- Add lenient streaming types ChatChunkFunctionCall / ChatChunkToolCall
(all fields optional + index) and use them in ChatChunkDelta, so streamed
tool-call frames parse into real objects instead of triggering the fallback.
- Harden the four archive loops (client.py + solana_client.py, sync + async)
with dict-tolerant accessors (stream_choice_content / _finish_reason /
chunk_usage_dict) so a model_construct fallback for any future reason can
no longer crash the stream.
Regression test (sync + async) feeds argument-fragment frames through the paid
streaming path (cost_usd > 0 so the archive loop runs); it crashes on the old
code and passes now. Full unit suite: 253 passed.
…ol-call type; export chunk types Review follow-ups to the streamed tool-call fix: - chunk.id/.model/.created in the four stream-archive loops were read raw, so a frame that fails strict validation AND omits the required top-level id (model_construct, which does not fill defaults) crashed the loop with AttributeError — the same failure class the new accessors were added to prevent. Route them through a dict/attr-tolerant chunk_meta() helper. - ChatChunkToolCall.type was Optional[Literal["function"]], which re-triggered the model_construct fallback for any non-"function" tool type. Loosen to Optional[str] (extra=allow does not relax a declared Literal field). - Export ChatChunkToolCall / ChatChunkFunctionCall from the package root, consistent with the already-exported sibling streaming types. - Add regression tests for both hardening cases; run black over the file.
…ide CI CI runs 'black --check .' and 'ruff check .' repo-wide; these files predate this branch and were already failing on main. Format-only + unused-import/var removal, plus a TYPE_CHECKING import so validation.py's PaymentError forward-ref resolves. No behavior change.
95f7bab to
14f3cb8
Compare
VickyXAI
pushed a commit
that referenced
this pull request
Jun 14, 2026
- Bump 1.4.0 -> 1.4.1 (1.4.0 was never published to PyPI). - Remove sunset model claude-fable-5: revert PREMIUM COMPLEX routing primary back to anthropic/claude-opus-4.8 (fable was never released, so no user-facing behavior change vs PyPI 1.3.0), and strip it from README + the unreleased 1.4.0 CHANGELOG note. - CHANGELOG 1.4.1: document the streamed tool-call archive-loop fix (#9).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Streaming a tool call crashes the SDK:
Plain chat streams fine; any tool call breaks. Hits
LLMClientandSolanaLLMClient, sync and async. Surfaced viablockrun-litellm's sidecar but the bug is here in the SDK.Root cause
ChatChunkDelta.tool_callsused the strict non-streamToolCall(id/function.name/argumentsall required). But OpenAI streams tool calls incrementally:index+id+function.name(+ empty args)index+function.argumentsfragments (no id, no name)Those argument-fragment frames fail
ToolCallvalidation →ChatCompletionChunk(**chunk)raises →_iter_sse_chunksfalls back toChatCompletionChunk.model_construct(...), which doesn't parse nested models, sochoicesstay as raw dicts → the archive loop'schoice.delta.contentblows up.Reproduced at the type level: a
{"tool_calls":[{"index":0,"function":{"arguments":"{\"ci"}}]}frame raisedValidationError, fell back tomodel_construct, andchoice.deltathen raisedAttributeError.Fix
ChatChunkFunctionCall/ChatChunkToolCall(all fields optional +index) and use them inChatChunkDelta.tool_calls. Streamed tool-call frames now parse into real objects instead of triggering themodel_constructfallback. The non-streamToolCall(used byChatMessage) stays strict._iter_and_archive/_aiter_and_archiveloops (client.py + solana_client.py, sync + async) now use dict-tolerant accessors (stream_choice_content/stream_choice_finish_reason/chunk_usage_dict), so amodel_constructfallback for any future reason can no longer crash the stream.Tests
New regression tests (sync + async) feed argument-fragment frames through the paid streaming path (so
cost_usd > 0and the archive loop actually runs). They crash on the old code (AttributeError: 'dict' object has no attribute 'delta') and pass with the fix; arguments reassemble to{"city":"Paris"}andfinish_reason=tool_calls.