Skip to content

fix(server-utils): Keep OpenAI streamed tool calls that have no index - #25622

Merged
RulaKhaled merged 4 commits into
getsentry:developfrom
pcbeingused333:fix/openai-stream-tool-calls-without-index
Oct 28, 2026
Merged

RulaKhaled merged 4 commits into
getsentry:developfrom
pcbeingused333:fix/openai-stream-tool-calls-without-index

Conversation

@pcbeingused333

@pcbeingused333 pcbeingused333 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Streamed Chat Completions tool calls without an index were dropped by the OpenAI instrumentation (if (index === undefined) continue), so the span's tool-call attribute was empty or missing calls. OpenAI-compatible endpoints such as Gemini's send them that way, each call whole in its own delta.

A delta with a numeric index is grouped by it, as before, except that a different non-empty id arriving on an index that is already taken opens a new call (an id that is already known continues its own call, and a delta with no id continues the latest call opened on that index). A delta without a usable index (missing or null) matches an existing call by id; a new id, or a function name without an id, starts a new call, and anything else (an arguments fragment with neither) continues the latest call. Calls are returned in the order they were opened.

Added unit tests for the Chat Completions tool-call accumulation and a Gemini-style stream in the OpenAI node-integration scenarios (openai-tool-calls and scenario-chat, the latter through the real openai client).

Fixes #25620

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the external PR from an external contributor label Oct 21, 2026
…an indexed one

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@RulaKhaled
RulaKhaled self-requested a review October 26, 2026 09:13

@RulaKhaled RulaKhaled left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up! the approach makes sense. ran the suite plus some edge cases locally, a couple of things inline around other shapes Gemini's been reported sending and test coverage for them

Comment thread packages/server-utils/src/ai/openai/streaming.ts Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we also get a node-integration test for this? a Gemini style stream in dev-packages/node-integration-tests/suites/tracing/openai/openai-tool-calls/scenario.mjs (whole parallel calls, no index) so it goes through the real openai SDK, the unit tests call instrumentStream directly

…tool calls

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review. Pushed a follow-up commit:

  • index: null is now treated like a missing index, and a different non-empty id arriving on an index that is already taken starts a new call (an id that is already known still continues its own call, and a delta with no id continues the latest call opened on that index). Four unit test cases cover null indexes, index: 0 on every call, a fragment after a reused index, and a repeated id not splitting its call; three of them fail without the change.
  • Added a Gemini-style stream to the openai-tool-calls node-integration scenario (two whole parallel calls, no index), with the expectations in both the recording-enabled and recording-disabled tests. The node-integration suite was not run locally (no built @sentry/node), so that part has only been syntax-checked; CI will be its first run.

This comment was prepared with AI assistance (Claude Code).

@RulaKhaled
RulaKhaled marked this pull request as ready for review October 28, 2026 09:00
@RulaKhaled
RulaKhaled requested a review from a team as a code owner October 28, 2026 09:00
@RulaKhaled
RulaKhaled requested review from andreiborza and isaacs and removed request for a team October 28, 2026 09:00

@RulaKhaled RulaKhaled left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow up! a couple more things inline. also could you update the description? it still says deltas with an index are grouped exactly as before, but a new id on a taken index now opens a new call

const key = keyByIndex[toolCall.index] ?? toolCall.index;
const existingId = calls[key]?.id;
if (existingId && toolCall.id && existingId !== toolCall.id) {
const next = findById(toolCall.id) ?? `index-${toolCall.index}-${Object.keys(calls).length}`;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m: the tool calls can come out of order now, index 0 call_a, index 0 call_b, then index 1 call_c gets recorded as [a, c, b].

Object.values puts integer keys ahead of string ones so index-0-1 lands last. the .sort() in the "later real index" test hides the same thing for no-index then indexed. could you key every call by a string (or use a Map) and assert the order in those tests?

}

// Gemini's OpenAI-compatible endpoint streams whole, parallel tool calls without an `index`
async *_createGeminiStyleToolCallsStream(params) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: thanks for adding it! ran it locally and the gemini case fails on develop, so it catches the bug 👍 we can keep it, but could you also add the same stream to scenario-chat.mjs? that one uses the real OpenAI client against an express server sending SSE, so the SDK actually parses the chunks

…were opened

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Thanks, all three done in the follow-up commit:

  • Order: tool calls are now keyed by string in a Map, so they come out in the order they were opened. Added your case (index 0 call_a, index 0 call_b, index 1 call_c -> [a, b, c]), and the no-index-then-indexed test asserts the order instead of using .sort().
  • scenario-chat.mjs now streams the Gemini-style tool calls (two whole calls, no index) from the Express SSE server, so the real openai SDK parses the chunks; the three scenario-chat tests expect 9 spans and check the tool calls with recording on and off.
  • Updated the description.

The node-integration suite was not run locally (no built @sentry/node), only the unit tests (packages/server-utils ai tests pass) and a syntax check of the scenario, so CI will be its first run.

This comment was prepared with AI assistance (Claude Code).

@RulaKhaled RulaKhaled left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! ran the unit tests and both openai node-integration suites locally (they fail on develop and pass here), plus a few out of order index cases, all good 👍

@RulaKhaled
RulaKhaled merged commit be6a16f into getsentry:develop Oct 28, 2026
662 of 664 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external PR from an external contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI integration drops streamed tool calls that have no index

2 participants