Skip to content

feat(interactions): add automatic_continuation to interactions.create in the Python SDK - #135

Open
copybara-service[bot] wants to merge 1 commit into
mainfrom
copybara/995364951
Open

copybara-service[bot] wants to merge 1 commit into
mainfrom
copybara/995364951

Conversation

@copybara-service

Copy link
Copy Markdown

feat(interactions): add automatic_continuation to interactions.create in the Python SDK

Foreground interactions.create() calls are capped per request by the server (32k output tokens unary, 64k streaming) and stop with status='requires_continuation' plus a continuation_token. GeminiNextGenInteractions.create and AsyncGeminiNextGenInteractions.create now accept automatic_continuation (default True) and chain those continuation hops client-side so a plain create() returns the full decode.

Request chaining:

  • Each hop after the first sends the continuation_token from the previous hop. With store=True (default) it sets previous_interaction_id to the hop that issued the token and omits input; with store=False it resends the original input.
  • Unary foreground model calls default generation_config.max_output_tokens to 65536 when unset (streaming is left to the server's 64k default). Explicit values are never overridden.
  • Stops on completed, incomplete (token preserved for manual resumption), requires_action, failed, cancelled, a missing token, or a repeated token. Skipped for agent / agent_config requests (server-managed loops) and background=True; background=True with automatic_continuation=False raises ValueError.

Unary (stream=False) merging:

  • Hops are merged into one Interaction: steps concatenated in hop order (leading user_input steps dropped on later hops), usage counts and per-modality breakdowns summed across hops with usage.initial_input_tokens taken from hop 1, id / status / continuation_token / updated from the final hop, and previous_interaction_id / created / input from hop 1.
  • continuation_ids lists every hop id in order (the last equals id), de-duplicated with any server-provided ids. output_text skips intervening thought steps across hop boundaries.

Streaming (stream=True):

  • Events from all hops are yielded verbatim on the returned Stream / AsyncStream. Consumers should treat interaction.completed with status='requires_continuation' as intermediate.
  • Mid-hop transport drops or premature EOF are resumed from the latest interaction.status_update checkpoint token when no content has been yielded since that checkpoint, with at most 3 no-progress reconnects. SSE error events are terminal. HTTP status retries stay with http_options.retry_options.

Timeouts and errors:

  • A numeric per-call timeout bounds the whole create() call across hops: each hop is sent the remaining budget and no further hop starts once it expires. http_options.timeout, httpx.Timeout and http_options.retry_options continue to apply per hop.
  • A failure (or timeout) on a hop after earlier hops already produced output raises errors.ContinuationInterruptedError (underlying error as __cause__) carrying partial_interaction (unary: the hops merged so far, status requires_continuation), continuation_token, interaction_id (to send as previous_interaction_id) and continuation_ids, so billed work is not lost and the call can be resumed manually. Failures on the first hop propagate unchanged.

Opt-out:

  • automatic_continuation=False returns the single server response unchanged (no budget injection, no merging). A one-time UserWarning is emitted when such a response stops at requires_continuation and the caller did not supply a continuation_token themselves.

Tests: tests/interactions/test_automatic_continuation.py (unit and edge cases, sync and async) and tests/interactions/test_continuation_conformance.py (cross-language conformance suite).

… in the Python SDK

Foreground `interactions.create()` calls are capped per request by the server (32k output tokens unary, 64k streaming) and stop with `status='requires_continuation'` plus a `continuation_token`. `GeminiNextGenInteractions.create` and `AsyncGeminiNextGenInteractions.create` now accept `automatic_continuation` (default `True`) and chain those continuation hops client-side so a plain `create()` returns the full decode.

Request chaining:
- Each hop after the first sends the `continuation_token` from the previous hop. With `store=True` (default) it sets `previous_interaction_id` to the hop that issued the token and omits `input`; with `store=False` it resends the original `input`.
- Unary foreground model calls default `generation_config.max_output_tokens` to 65536 when unset (streaming is left to the server's 64k default). Explicit values are never overridden.
- Stops on `completed`, `incomplete` (token preserved for manual resumption), `requires_action`, `failed`, `cancelled`, a missing token, or a repeated token. Skipped for `agent` / `agent_config` requests (server-managed loops) and `background=True`; `background=True` with `automatic_continuation=False` raises `ValueError`.

Unary (`stream=False`) merging:
- Hops are merged into one `Interaction`: `steps` concatenated in hop order (leading `user_input` steps dropped on later hops), `usage` counts and per-modality breakdowns summed across hops with `usage.initial_input_tokens` taken from hop 1, `id` / `status` / `continuation_token` / `updated` from the final hop, and `previous_interaction_id` / `created` / `input` from hop 1.
- `continuation_ids` lists every hop id in order (the last equals `id`), de-duplicated with any server-provided ids. `output_text` skips intervening thought steps across hop boundaries.

Streaming (`stream=True`):
- Events from all hops are yielded verbatim on the returned `Stream` / `AsyncStream`. Consumers should treat `interaction.completed` with `status='requires_continuation'` as intermediate.
- Mid-hop transport drops or premature EOF are resumed from the latest `interaction.status_update` checkpoint token when no content has been yielded since that checkpoint, with at most 3 no-progress reconnects. SSE `error` events are terminal. HTTP status retries stay with `http_options.retry_options`.

Timeouts and errors:
- A numeric per-call `timeout` bounds the whole `create()` call across hops: each hop is sent the remaining budget and no further hop starts once it expires. `http_options.timeout`, `httpx.Timeout` and `http_options.retry_options` continue to apply per hop.
- A failure (or timeout) on a hop after earlier hops already produced output raises `errors.ContinuationInterruptedError` (underlying error as `__cause__`) carrying `partial_interaction` (unary: the hops merged so far, status `requires_continuation`), `continuation_token`, `interaction_id` (to send as `previous_interaction_id`) and `continuation_ids`, so billed work is not lost and the call can be resumed manually. Failures on the first hop propagate unchanged.

Opt-out:
- `automatic_continuation=False` returns the single server response unchanged (no budget injection, no merging). A one-time `UserWarning` is emitted when such a response stops at `requires_continuation` and the caller did not supply a `continuation_token` themselves.

Tests: tests/interactions/test_automatic_continuation.py (unit and edge cases, sync and async) and tests/interactions/test_continuation_conformance.py (cross-language conformance suite).
PiperOrigin-RevId: 995364951
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant