Skip to content

feat(interactions): add automatic_continuation to interactions.create in the TypeScript SDK - #137

Open
copybara-service[bot] wants to merge 1 commit into
mainfrom
copybara/995657093
Open

copybara-service[bot] wants to merge 1 commit into
mainfrom
copybara/995657093

Conversation

@copybara-service

Copy link
Copy Markdown

feat(interactions): add automatic_continuation to interactions.create in the TypeScript SDK

Foreground interactions.create() calls are capped per request by the server (32k output tokens unary, 64k streaming) and stop with status='requires_continuation' plus a continuation_token. GeminiNextGenInteractions.create now accepts automatic_continuation (default true) on params and chains continuation hops client-side so a plain create() returns the full decode.

Request chaining:

  • Each hop after the first sends the continuation_token from the previous hop. With store !== false (default) it sets previous_interaction_id to the hop that issued the token and omits input; with store: false it resends the original input.
  • Unary foreground model calls default generation_config.max_output_tokens to 65536 when unset without mutating caller objects (streaming uses the server's 64k default). Explicit values are never overridden.
  • Stops on completed, incomplete (token preserved for manual resumption), requires_action, failed, cancelled, a missing token, or a repeated token. Skipped for agent / agent_config requests and background: true; background: true with automatic_continuation: false throws an error upfront.

Unary (stream: false) merging:

  • Hops are merged into one Interaction: steps concatenated in hop order (leading user_input steps dropped on later hops), usage counts and per-modality breakdowns summed across hops with usage.initial_input_tokens taken from hop 1, id / status / continuation_token / updated from the final hop, and previous_interaction_id / created / input from hop 1.
  • continuation_ids lists every hop id in order, de-duplicated with any server-provided ids. output_text skips intervening thought steps across hop boundaries.

Streaming (stream: true):

  • Events from all hops are yielded verbatim on a pull-on-demand Stream<InteractionSSEEvent>. Consumers should treat interaction.completed with status='requires_continuation' as intermediate.
  • Mid-hop transport drops or premature EOF resume from the latest interaction.status_update checkpoint token when no content was yielded since that checkpoint (up to 3 no-progress reconnects). SSE error events are terminal; HTTP status retries stay with options.retries.

Timeouts and errors:

  • Numeric per-call options.timeout_ms (and alias options.timeout) bounds the whole create() call across hops via CallDeadline. Client-level httpOptions.timeout, options.retries, and AbortSignal apply per hop.
  • A failure or timeout after progress throws ContinuationInterruptedError (cause chained) carrying partialInteraction (unary), continuationToken, interactionId, and continuationIds so billed work is preserved for manual resumption. First-hop failures propagate unchanged.

Opt-out:

  • automatic_continuation: false returns the single server response untouched (no budget injection, no merging) and emits a one-time console.warn if it stops at requires_continuation without a caller-supplied continuation_token.

Tests: test/unit/interactions_continuation_conformance_test.ts (52 conformance cases) and test/unit/interactions_continuation_test.ts (whole-call timeout, wire hygiene, stream cancellation, and exports).

… in the TypeScript SDK

Foreground `interactions.create()` calls are capped per request by the server (32k output tokens unary, 64k streaming) and stop with `status='requires_continuation'` plus a `continuation_token`. `GeminiNextGenInteractions.create` now accepts `automatic_continuation` (default `true`) on `params` and chains continuation hops client-side so a plain `create()` returns the full decode.

Request chaining:
- Each hop after the first sends the `continuation_token` from the previous hop. With `store !== false` (default) it sets `previous_interaction_id` to the hop that issued the token and omits `input`; with `store: false` it resends the original `input`.
- Unary foreground model calls default `generation_config.max_output_tokens` to 65536 when unset without mutating caller objects (streaming uses the server's 64k default). Explicit values are never overridden.
- Stops on `completed`, `incomplete` (token preserved for manual resumption), `requires_action`, `failed`, `cancelled`, a missing token, or a repeated token. Skipped for `agent` / `agent_config` requests and `background: true`; `background: true` with `automatic_continuation: false` throws an error upfront.

Unary (`stream: false`) merging:
- Hops are merged into one `Interaction`: `steps` concatenated in hop order (leading `user_input` steps dropped on later hops), `usage` counts and per-modality breakdowns summed across hops with `usage.initial_input_tokens` taken from hop 1, `id` / `status` / `continuation_token` / `updated` from the final hop, and `previous_interaction_id` / `created` / `input` from hop 1.
- `continuation_ids` lists every hop id in order, de-duplicated with any server-provided ids. `output_text` skips intervening thought steps across hop boundaries.

Streaming (`stream: true`):
- Events from all hops are yielded verbatim on a pull-on-demand `Stream<InteractionSSEEvent>`. Consumers should treat `interaction.completed` with `status='requires_continuation'` as intermediate.
- Mid-hop transport drops or premature EOF resume from the latest `interaction.status_update` checkpoint token when no content was yielded since that checkpoint (up to 3 no-progress reconnects). SSE `error` events are terminal; HTTP status retries stay with `options.retries`.

Timeouts and errors:
- Numeric per-call `options.timeout_ms` (and alias `options.timeout`) bounds the whole `create()` call across hops via `CallDeadline`. Client-level `httpOptions.timeout`, `options.retries`, and `AbortSignal` apply per hop.
- A failure or timeout after progress throws `ContinuationInterruptedError` (`cause` chained) carrying `partialInteraction` (unary), `continuationToken`, `interactionId`, and `continuationIds` so billed work is preserved for manual resumption. First-hop failures propagate unchanged.

Opt-out:
- `automatic_continuation: false` returns the single server response untouched (no budget injection, no merging) and emits a one-time `console.warn` if it stops at `requires_continuation` without a caller-supplied `continuation_token`.

Tests: `test/unit/interactions_continuation_conformance_test.ts` (52 conformance cases) and `test/unit/interactions_continuation_test.ts` (whole-call timeout, wire hygiene, stream cancellation, and exports).
PiperOrigin-RevId: 995657093
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant