Repository navigation
fix(providers): stream OpenAI Responses tool calls and reasoning, mark it verified - #81
Merged
Merged
Conversation
…k it verified OpenAIResponsesClient.stream() only mapped text / reasoning-summary / usage events, so a streamed Responses turn lost every function_call (the loop saw no tool call and stopped) and every encrypted_content reasoning item (the next request couldn't continue the model's reasoning state). stream_llm_tokens=True on protocol=responses was silently broken. Now stream() forwards function-call argument deltas as they arrive and, on response.completed, hands the terminal response's parsed block list (reasoning + function calls in provider order) to the assembler via StreamDelta.reasoning_blocks -- the same content chat() returns. A call whose events a gateway didn't forward is completed from that terminal response. responses is removed from UNVERIFIED_STREAM_PROTOCOLS (now just bedrock), so it streams automatically once an observer or the reasoning-only watchdog asks for deltas. Added tests/test_responses_stream_replay.py: drives the real client through the real loop with a fake SDK, once per mode, over a two-call tool turn, and requires byte-identical follow-up requests and message history between streaming and non-streaming. Fails without the fix. Live-verified against gpt-4o on llm-hub (the only model there found to both support native /v1/responses and not be blocked upstream as of 2026-10-14 -- see scripts/protocol_smoke/live_responses_stream_smoke.py): streaming correctly accumulates one tool call's arguments across 11 deltas, executes it, and produces a second-turn history structurally identical to the non-streaming run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
OpenAIResponsesClient.stream()only mapped text / reasoning-summary / usage events, so a streamed Responses-API turn silently lost everyfunction_call(the loop saw no tool call and stopped) and everyencrypted_contentreasoning item (the next request couldn't continue the model's reasoning state).protocol: responseswas listed inUNVERIFIED_STREAM_PROTOCOLS, so the loop never streamed it on its own — but a profile that explicitly setstream_llm_tokens: true(e.g. to survive a gateway's first-byte timeout on long generations) hit this silently and lost tool calls.Fix
stream()now forwards function-call argument deltas as they arrive (response.output_item.added/response.function_call_arguments.delta), and onresponse.completedhands the terminal response's parsed block list (reasoning items withencrypted_content+ function calls, in provider order) to the assembler viaStreamDelta.reasoning_blocks— the same contentchat()already returns. Any call whose events a gateway didn't forward is completed from that terminal response (_unstreamed_call_deltas).responsesremoved fromUNVERIFIED_STREAM_PROTOCOLS(now justbedrock), so it streams automatically once an observer or the reasoning-only watchdog asks for deltas — matchinganthropic's existing behavior.Tests
tests/test_responses_stream_replay.py: drives the real client through the real loop against a fake SDK, once per mode, over a two-call tool turn. Requires byte-identical follow-up requests and message history between streaming and non-streaming (both with and without per-item SDK events, to cover a gateway that forwards only deltas + the terminal event). Fails without the fix (confirmed viagit stash).tests/test_stream_llm_tokens_toggle.pyupdated:responsesmoved out of the "stays non-streaming, warns" parametrization into its own "streams when an observer wants deltas" test, mirroring the existinganthropictest.2338 passed, 2 skipped.ruff checkclean.Live verification
Live-verified against two models on llm-hub's real
/v1/responsesendpoint viascripts/protocol_smoke/live_responses_stream_smoke.py(new):reasoning={"effort":"medium","summary":"auto"}run on a tool+reasoning turn — streaming and non-streaming both producereasoning (with non-empty encrypted_content) → function_call(get_weather, same args) → text, with matching final answers.Details in the companion repo's
temp/2026-10-14_responses-stream-live-smoke.md(not part of this PR; AgentCore itself ships the reusable smoke script).🤖 Generated with Claude Code