Skip to content

fix(providers): stream OpenAI Responses tool calls and reasoning, mark it verified - #81

Merged
zhanghanduo merged 2 commits into
mainfrom
test/responses-stream-replay
Oct 14, 2026
Merged

zhanghanduo merged 2 commits into
mainfrom
test/responses-stream-replay

Conversation

@zhanghanduo

Copy link
Copy Markdown
Collaborator

Problem

OpenAIResponsesClient.stream() only mapped text / reasoning-summary / usage events, so a streamed Responses-API turn silently lost every function_call (the loop saw no tool call and stopped) and every encrypted_content reasoning item (the next request couldn't continue the model's reasoning state). protocol: responses was listed in UNVERIFIED_STREAM_PROTOCOLS, so the loop never streamed it on its own — but a profile that explicitly set stream_llm_tokens: true (e.g. to survive a gateway's first-byte timeout on long generations) hit this silently and lost tool calls.

Fix

  • stream() now forwards function-call argument deltas as they arrive (response.output_item.added / response.function_call_arguments.delta), and on response.completed hands the terminal response's parsed block list (reasoning items with encrypted_content + function calls, in provider order) to the assembler via StreamDelta.reasoning_blocks — the same content chat() already returns. Any call whose events a gateway didn't forward is completed from that terminal response (_unstreamed_call_deltas).
  • responses removed from UNVERIFIED_STREAM_PROTOCOLS (now just bedrock), so it streams automatically once an observer or the reasoning-only watchdog asks for deltas — matching anthropic's existing behavior.

Tests

  • New tests/test_responses_stream_replay.py: drives the real client through the real loop against a fake SDK, once per mode, over a two-call tool turn. Requires byte-identical follow-up requests and message history between streaming and non-streaming (both with and without per-item SDK events, to cover a gateway that forwards only deltas + the terminal event). Fails without the fix (confirmed via git stash).
  • tests/test_stream_llm_tokens_toggle.py updated: responses moved out of the "stays non-streaming, warns" parametrization into its own "streams when an observer wants deltas" test, mirroring the existing anthropic test.
  • Full suite: 2338 passed, 2 skipped. ruff check clean.

Live verification

Live-verified against two models on llm-hub's real /v1/responses endpoint via scripts/protocol_smoke/live_responses_stream_smoke.py (new):

  • gpt-4o (first key tried; only model that key's channel let through to a native Responses upstream): streaming correctly accumulates one tool call's arguments across 11 deltas, executes it, and produces a second-turn history structurally identical to the non-streaming run.
  • gpt-6.1-sol (after a key swap unlocked that channel): same tool-call check, plus a forced reasoning={"effort":"medium","summary":"auto"} run on a tool+reasoning turn — streaming and non-streaming both produce reasoning (with non-empty encrypted_content) → function_call(get_weather, same args) → text, with matching final answers.

Details in the companion repo's temp/2026-10-14_responses-stream-live-smoke.md (not part of this PR; AgentCore itself ships the reusable smoke script).

🤖 Generated with Claude Code

zhanghanduo and others added 2 commits October 14, 2026 13:20
…k it verified

OpenAIResponsesClient.stream() only mapped text / reasoning-summary /
usage events, so a streamed Responses turn lost every function_call
(the loop saw no tool call and stopped) and every encrypted_content
reasoning item (the next request couldn't continue the model's
reasoning state). stream_llm_tokens=True on protocol=responses was
silently broken.

Now stream() forwards function-call argument deltas as they arrive and,
on response.completed, hands the terminal response's parsed block list
(reasoning + function calls in provider order) to the assembler via
StreamDelta.reasoning_blocks -- the same content chat() returns. A call
whose events a gateway didn't forward is completed from that terminal
response.

responses is removed from UNVERIFIED_STREAM_PROTOCOLS (now just
bedrock), so it streams automatically once an observer or the
reasoning-only watchdog asks for deltas.

Added tests/test_responses_stream_replay.py: drives the real client
through the real loop with a fake SDK, once per mode, over a two-call
tool turn, and requires byte-identical follow-up requests and message
history between streaming and non-streaming. Fails without the fix.

Live-verified against gpt-4o on llm-hub (the only model there found to
both support native /v1/responses and not be blocked upstream as of
2026-10-14 -- see scripts/protocol_smoke/live_responses_stream_smoke.py):
streaming correctly accumulates one tool call's arguments across 11
deltas, executes it, and produces a second-turn history structurally
identical to the non-streaming run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@zhanghanduo
zhanghanduo merged commit 904a014 into main Oct 14, 2026
5 checks passed
@zhanghanduo
zhanghanduo deleted the test/responses-stream-replay branch October 14, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant