Chat completions can return an unusable assistant turn: empty content, or tool calls the client never declared #30

Activity

1. serrade

   changed the title [-]Chat completions can return an empty assistant turn when the client declares tools[/-] [+]Chat completions can return an unusable assistant turn: empty content, or tool calls the client never declared[/+] on Jul 30, 2026
2. serrade commented on Jul 30, 2026

   

   serrade

   on Jul 30, 2026

   Author

   More actions

   Follow-up: there is a second path to the same user-visible failure, and it is the one Cline actually hits. Retitling the issue to cover both.

   I measured both against main by calling the response builders directly:

   Path 1 — the client declares tools and the model calls something unmappable (what the description above covers):

   content       = "" (empty)
   tool_calls    = []
   finish_reason = stop
   stream chunk  = empty Data(), nothing emitted, while the finish chunk still says tool_calls
   

   Path 2 — the client declares no tools at all:

   content       = null
   tool_calls    = ["shell"]      <- the SDK's own tool name, forwarded verbatim
   finish_reason = tool_calls
   

   In path 2, resolveToolCall takes the tools.isEmpty branch and passes the SDK tool call straight through:

   guard let tool = resolveToolSpec(...) else {
       guard tools.isEmpty else { return nil }
       return ResolvedToolCall(name: normalizedToolCall.name, arguments: normalizedToolCall.arguments)
   }

   Any OpenAI-compatible client will reject a tool_calls entry naming a tool it never declared, which is the "returned tool calls that Cline cannot process" half of Cline's error message. Path 1 produces the "empty or unparsable response" half.

   Why this matters for Cline specifically. I had assumed Cline sends native tool schemas, since nativeToolCallEnabled defaults on. It does not, for this transport. Native tool calling in Cline 4.0.12 is implemented per provider, and the two providers that can reach a local OpenAI-compatible endpoint both omit tools from the request:

   liteLlmModelId, liteLlmModelInfo             -> no tools key
   openAiApiKey, openAiBaseUrl, openAiModelId   -> no tools key
   groqApiKey / togetherApiKey / xaiApiKey ...  -> tools sent
   

   So Cline always uses its XML in-prompt tool protocol against this API, and always lands in path 2. There is no client-side setting that changes it.

   That has a second consequence worth noting: with no declared tools, nothing signals that the caller owns tool execution, so the SDK agent keeps the caller's real workspace as its working directory and does the file and shell work itself. The client then receives narration describing changes it never made, or a tool call it cannot execute.

   Scope of the open PRs. #33 fixes path 1. #34 fixes the harness competing with the client tool channel for callers that do declare tools. Neither addresses path 2, and I would rather not widen them mid-review. I am putting together a separate change for prompt-protocol clients: detect a caller that declared no tools, never hand it a tool call it did not declare, translate the SDK's tool calls into the protocol the caller's own prompt defines, and isolate the workspace on that signal instead of on the tools array. Happy to fold it into #34 instead if you would prefer one change.
3. serrade commented on Jul 30, 2026

   

   serrade

   on Jul 30, 2026

   Author

   More actions

   Correcting my previous comment. I claimed Cline cannot send tools over this transport under any
   setting. That is wrong, and the error was method: I read it out of the extension's minified bundle.
   Checking cline/cline at the version I was running (v4.0.12, f267bf4) shows the opposite.

   Cline's OpenAI Compatible provider does send native tools. OpenAiHandler.createMessage takes a
   tools argument and spreads it into the request:

   // apps/vscode/src/core/api/providers/openai.ts
   async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
     const stream = await client.chat.completions.create({
       model: modelId, messages: openAiMessages, temperature, max_tokens: maxTokens,
       reasoning_effort: reasoningEffort, stream: true, stream_options: { include_usage: true },
       ...getOpenAIToolParams(tools),
     })

   getOpenAIToolParams yields tools + tool_choice: "auto" + parallel_tool_calls: false whenever
   the array is non-empty. Two more corrections to that comment: nativeToolCallEnabled defaults to
   true (shared/storage/state-keys.ts), so nothing needs switching on; and tools are built only when
   the model ID selects a prompt variant labelled use_native_tools: 1, matched on the ID string the
   user types. claude-opus-5, claude-sonnet-5, grok-4.5 and gpt-5.6 all match; composer-2.5
   does not and falls through to the XML variant.

   What I got right by accident is that I was seeing tools=0, because I was using the LiteLLM
   provider. That one really cannot: LiteLlmHandler.createMessage(systemPrompt, messages) takes two
   parameters, ignores the third, and never parses delta.tool_calls. "litellm" is also absent from
   isNextGenModelProvider. So the limitation is one provider, not the client.

   Consequences for this issue. Path 2 is still a real bug, but the population is narrower than I
   described: genuinely tool-less callers, which includes plain curl and Cline's LiteLLM provider, not
   Cline as a whole. I am withdrawing the prompt-protocol translation change I said I was preparing —
   it would have solved a problem this client does not have. If path 2 is worth fixing, it looks much
   smaller than what I proposed: never forward a tool call the caller did not declare.

   On the plus side, #33 and #34 turn out to be exactly what the native path needs. Driving the real
   extension (Cline 4.0.12, OpenAI Compatible, model id claude-opus-5) rather than a reconstruction of
   it, a task now runs start to finish: Cline sends 38 native tools, claude-opus-5 calls its
   read_file, replace_in_file and execute_command by name, the diffs render and get approved in
   Cline's own UI, and the task closes on attempt_completion. Every turn logged executor=client with
   the harness in an empty scratch directory, so the model had no route to the workspace except back
   through the client — which is the behaviour #34 is meant to enforce.

   The same session also produced a clean demonstration of path 2, by accident. One turn went out as
   composer-2.5, which does not match any of Cline's native-tool model families, so Cline sent no
   tools at all:

   model=claude-opus-5  executor=client   cwd=/tmp/.../workspaces/98e0205…   (x10)
   model=composer-2.5   executor=harness  cwd=/Users/alex/…/cline_test      (x1)
   

   With no declared tools nothing signals that the caller owns execution, so the harness kept the real
   project directory. That is path 2 in the wild: not a client that cannot be fixed, but any caller that
   genuinely declares nothing.

   Sorry for the noise on this one. Testing the corrected path did shake out further bugs of its own; I
   will file those separately rather than pile more onto this thread.