Skip to content

[Bug]: OpenAI-compatible streaming sends finish_reason=stop on intermediate thought/tool chunks #403

Description

@IlusiveAtlas

Summary

OpenAI-compatible streaming responses (/antigravity/v1/chat/completions, and likely /v1/chat/completions) send "finish_reason": "stop" on every SSE chunk, including intermediate thought/reasoning chunks and tool-call chunks. OpenAI-compatible clients (e.g. DeepSeek Harness / DSH) treat the first non-null finish_reason as the end of the stream, so they stop immediately after the first reasoning_content or tool_calls chunk and never consume the actual content / continue the tool loop.

Environment

  • gcli2api latest upstream: 2b5d78f / 69638c8 (2026-08-24)
  • Route: POST /antigravity/v1/chat/completions
  • Model: gemini-3.7-flash-high
  • Auth: Bearer token
  • Client: DeepSeek Harness (OpenAI-compatible streaming), also reproducible with curl

Steps to reproduce

  1. Start gcli2api.
  2. Send a streaming request that triggers thinking:
curl -N http://127.0.0.1:7861/antigravity/v1/chat/completions \
  -H 'Authorization: Bearer <password>' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemini-3.7-flash-high",
    "messages": [{"role": "user", "content": "请一步步推理:17*23+45=?"}],
    "stream": true
  }'
  1. Observe the first SSE chunk:
{"choices":[{"index":0,"delta":{"reasoning_content":"..."},"finish_reason":"stop"}]}

Expected OpenAI-compatible behavior: the first chunk should have "finish_reason": null; only the final chunk should have "finish_reason": "stop".

Actual behavior

Every streamed chunk contains "finish_reason": "stop" because:

  • The upstream Antigravity/Gemini SSE stream only sets finishReason on the final chunk (verified via the native /antigravity/v1/models/{model}:streamGenerateContent endpoint).
  • src/converter/openai2gemini.py -> convert_gemini_to_openai_stream() calls _map_finish_reason(gemini_finish_reason) even when gemini_finish_reason is None.
  • _map_finish_reason() defaults any None/unknown value to "stop".

This causes OpenAI clients that honor finish_reason to stop after the first thought/tool chunk.

Impact

  • Streaming chat responses that include reasoning_content appear to “think then stop”.
  • Streaming tool-call responses appear to return only the first tool call and never continue the agent loop.
  • This breaks agentic use-cases through DSH and likely other strict OpenAI-compatible clients.

Suggested fix

In convert_gemini_to_openai_stream(), only map an explicit upstream finishReason; use null when it is absent:

gemini_finish_reason = candidate.get("finishReason")
if gemini_finish_reason:
    finish_reason = _map_finish_reason(gemini_finish_reason)
else:
    finish_reason = None

I verified this locally: intermediate chunks now have "finish_reason": null, and only the final empty chunk has "finish_reason": "stop". The tool-call stream also correctly emits null on the tool-call chunk and stop on the terminal chunk.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions