Repository navigation
fix(server-utils): Keep OpenAI streamed tool calls that have no index - #25622
RulaKhaled merged 4 commits into
Conversation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…an indexed one Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
RulaKhaled
left a comment
There was a problem hiding this comment.
Thanks for picking this up! the approach makes sense. ran the suite plus some edge cases locally, a couple of things inline around other shapes Gemini's been reported sending and test coverage for them
There was a problem hiding this comment.
could we also get a node-integration test for this? a Gemini style stream in dev-packages/node-integration-tests/suites/tracing/openai/openai-tool-calls/scenario.mjs (whole parallel calls, no index) so it goes through the real openai SDK, the unit tests call instrumentStream directly
…tool calls Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Thanks for the careful review. Pushed a follow-up commit:
This comment was prepared with AI assistance (Claude Code). |
RulaKhaled
left a comment
There was a problem hiding this comment.
Thanks for the follow up! a couple more things inline. also could you update the description? it still says deltas with an index are grouped exactly as before, but a new id on a taken index now opens a new call
| const key = keyByIndex[toolCall.index] ?? toolCall.index; | ||
| const existingId = calls[key]?.id; | ||
| if (existingId && toolCall.id && existingId !== toolCall.id) { | ||
| const next = findById(toolCall.id) ?? `index-${toolCall.index}-${Object.keys(calls).length}`; |
There was a problem hiding this comment.
m: the tool calls can come out of order now, index 0 call_a, index 0 call_b, then index 1 call_c gets recorded as [a, c, b].
Object.values puts integer keys ahead of string ones so index-0-1 lands last. the .sort() in the "later real index" test hides the same thing for no-index then indexed. could you key every call by a string (or use a Map) and assert the order in those tests?
| } | ||
|
|
||
| // Gemini's OpenAI-compatible endpoint streams whole, parallel tool calls without an `index` | ||
| async *_createGeminiStyleToolCallsStream(params) { |
There was a problem hiding this comment.
l: thanks for adding it! ran it locally and the gemini case fails on develop, so it catches the bug 👍 we can keep it, but could you also add the same stream to scenario-chat.mjs? that one uses the real OpenAI client against an express server sending SSE, so the SDK actually parses the chunks
…were opened Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Thanks, all three done in the follow-up commit:
The node-integration suite was not run locally (no built This comment was prepared with AI assistance (Claude Code). |
RulaKhaled
left a comment
There was a problem hiding this comment.
lgtm! ran the unit tests and both openai node-integration suites locally (they fail on develop and pass here), plus a few out of order index cases, all good 👍
Streamed Chat Completions tool calls without an
indexwere dropped by the OpenAI instrumentation (if (index === undefined) continue), so the span's tool-call attribute was empty or missing calls. OpenAI-compatible endpoints such as Gemini's send them that way, each call whole in its own delta.A delta with a numeric
indexis grouped by it, as before, except that a different non-emptyidarriving on an index that is already taken opens a new call (anidthat is already known continues its own call, and a delta with noidcontinues the latest call opened on that index). A delta without a usableindex(missing ornull) matches an existing call byid; a newid, or a function name without anid, starts a new call, and anything else (an arguments fragment with neither) continues the latest call. Calls are returned in the order they were opened.Added unit tests for the Chat Completions tool-call accumulation and a Gemini-style stream in the OpenAI node-integration scenarios (
openai-tool-callsandscenario-chat, the latter through the realopenaiclient).Fixes #25620