Skip to content

fix(stream): parse streamed tool-call chunks (fixes 'dict' object has no attribute 'delta')#9

Merged
VickyXAI merged 3 commits into
BlockRunAI:mainfrom
KillerQueen-Z:fix/streamed-tool-call-chunk-parsing
Jun 14, 2026
Merged

fix(stream): parse streamed tool-call chunks (fixes 'dict' object has no attribute 'delta')#9
VickyXAI merged 3 commits into
BlockRunAI:mainfrom
KillerQueen-Z:fix/streamed-tool-call-chunk-parsing

Conversation

@KillerQueen-Z

Copy link
Copy Markdown
Contributor

Problem

Streaming a tool call crashes the SDK:

File ".../blockrun_llm/solana_client.py", in _aiter_and_archive
    if choice.delta.content:
AttributeError: 'dict' object has no attribute 'delta'

Plain chat streams fine; any tool call breaks. Hits LLMClient and SolanaLLMClient, sync and async. Surfaced via blockrun-litellm's sidecar but the bug is here in the SDK.

Root cause

ChatChunkDelta.tool_calls used the strict non-stream ToolCall (id / function.name / arguments all required). But OpenAI streams tool calls incrementally:

  • first frame: index + id + function.name (+ empty args)
  • later frames: only index + function.arguments fragments (no id, no name)

Those argument-fragment frames fail ToolCall validation → ChatCompletionChunk(**chunk) raises → _iter_sse_chunks falls back to ChatCompletionChunk.model_construct(...), which doesn't parse nested models, so choices stay as raw dicts → the archive loop's choice.delta.content blows up.

Reproduced at the type level: a {"tool_calls":[{"index":0,"function":{"arguments":"{\"ci"}}]} frame raised ValidationError, fell back to model_construct, and choice.delta then raised AttributeError.

Fix

  1. Lenient streaming tool-call types — add ChatChunkFunctionCall / ChatChunkToolCall (all fields optional + index) and use them in ChatChunkDelta.tool_calls. Streamed tool-call frames now parse into real objects instead of triggering the model_construct fallback. The non-stream ToolCall (used by ChatMessage) stays strict.
  2. Harden the archive loops (defense in depth) — the four _iter_and_archive / _aiter_and_archive loops (client.py + solana_client.py, sync + async) now use dict-tolerant accessors (stream_choice_content / stream_choice_finish_reason / chunk_usage_dict), so a model_construct fallback for any future reason can no longer crash the stream.

Tests

New regression tests (sync + async) feed argument-fragment frames through the paid streaming path (so cost_usd > 0 and the archive loop actually runs). They crash on the old code (AttributeError: 'dict' object has no attribute 'delta') and pass with the fix; arguments reassemble to {"city":"Paris"} and finish_reason=tool_calls.

  • Full unit suite: 253 passed.

KillerQueen-Z and others added 3 commits June 12, 2026 16:00
…ribute 'delta')

Streaming a tool call crashed the SDK with:

    AttributeError: 'dict' object has no attribute 'delta'   (in _aiter_and_archive)

Root cause: ChatChunkDelta.tool_calls used the strict non-stream ToolCall
(id / function.name / arguments all required). OpenAI streams tool calls
incrementally — the first frame has id + name, later frames carry only an
arguments *fragment* — so those fragment frames failed validation,
ChatCompletionChunk(**chunk) raised, and _iter_sse_chunks fell back to
model_construct(), which doesn't parse nested models. choices were then raw
dicts, and the archive loop's `choice.delta.content` blew up. Plain chat
worked (text frames validate); any tool call broke. Hits LLMClient and
SolanaLLMClient, sync and async.

Fix:
  - Add lenient streaming types ChatChunkFunctionCall / ChatChunkToolCall
    (all fields optional + index) and use them in ChatChunkDelta, so streamed
    tool-call frames parse into real objects instead of triggering the fallback.
  - Harden the four archive loops (client.py + solana_client.py, sync + async)
    with dict-tolerant accessors (stream_choice_content / _finish_reason /
    chunk_usage_dict) so a model_construct fallback for any future reason can
    no longer crash the stream.

Regression test (sync + async) feeds argument-fragment frames through the paid
streaming path (cost_usd > 0 so the archive loop runs); it crashes on the old
code and passes now. Full unit suite: 253 passed.
…ol-call type; export chunk types

Review follow-ups to the streamed tool-call fix:

- chunk.id/.model/.created in the four stream-archive loops were read raw,
  so a frame that fails strict validation AND omits the required top-level id
  (model_construct, which does not fill defaults) crashed the loop with
  AttributeError — the same failure class the new accessors were added to
  prevent. Route them through a dict/attr-tolerant chunk_meta() helper.
- ChatChunkToolCall.type was Optional[Literal["function"]], which re-triggered
  the model_construct fallback for any non-"function" tool type. Loosen to
  Optional[str] (extra=allow does not relax a declared Literal field).
- Export ChatChunkToolCall / ChatChunkFunctionCall from the package root,
  consistent with the already-exported sibling streaming types.
- Add regression tests for both hardening cases; run black over the file.
…ide CI

CI runs 'black --check .' and 'ruff check .' repo-wide; these files predate this
branch and were already failing on main. Format-only + unused-import/var removal,
plus a TYPE_CHECKING import so validation.py's PaymentError forward-ref resolves.
No behavior change.
@VickyXAI VickyXAI force-pushed the fix/streamed-tool-call-chunk-parsing branch from 95f7bab to 14f3cb8 Compare June 14, 2026 08:39
@VickyXAI VickyXAI merged commit e719e35 into BlockRunAI:main Jun 14, 2026
3 checks passed
VickyXAI pushed a commit that referenced this pull request Jun 14, 2026
- Bump 1.4.0 -> 1.4.1 (1.4.0 was never published to PyPI).
- Remove sunset model claude-fable-5: revert PREMIUM COMPLEX routing primary
  back to anthropic/claude-opus-4.8 (fable was never released, so no user-facing
  behavior change vs PyPI 1.3.0), and strip it from README + the unreleased
  1.4.0 CHANGELOG note.
- CHANGELOG 1.4.1: document the streamed tool-call archive-loop fix (#9).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants