Skip to content

Question: expose raw Gemini Live UsageMetadata in the Node.js Google realtime plugin #2490

Description

@yagince

Context

The Gemini Live API exposes usage metadata including promptTokenCount, responseTokenCount, thoughtsTokenCount, and totalTokenCount.

The official @google/genai SDK exposes this through LiveServerMessage.usageMetadata in the live.connect onmessage callback:

The Gemini 3.1 Flash Live pricing page also states that thinking tokens are included in output pricing:

Current behavior

With the Node.js Google realtime plugin, the provider Usage message is handled internally and normalized into RealtimeModelMetrics.

Current plugin source maps promptTokenCount, responseTokenCount, and totalTokenCount to inputTokens, outputTokens, and totalTokens, but does not expose thoughtsTokenCount or whether optional usage fields were present:

Environment used for the observation:

  • @livekit/agents 1.8.1
  • @livekit/agents-plugin-google 1.8.1
  • @google/genai 2.22.0
  • Gemini Developer API
  • gemini-3.1-flash-live-preview

In a short synthetic session, a provider message like this was observed:

{
  "usageMetadata": {
    "promptTokenCount": 397,
    "responseTokenCount": 49,
    "thoughtsTokenCount": 26,
    "totalTokenCount": 446
  }
}

The corresponding LiveKit metric exposed input/output/total values but did not contain the thoughts count or field presence.

This prevents consumers from distinguishing:

  • thoughtsTokenCount was explicitly 0
  • the provider omitted the field
  • the plugin discarded the field

The request is for numeric usage metadata only. Thought text and thought summaries should not be forwarded to user-visible transcripts or TTS.

Question

Is exposing raw Gemini Live UsageMetadata (especially thoughtsTokenCount) in the Node.js Google realtime plugin the right scope for this repository, or should this feature be proposed in the canonical livekit/agents repository shown by the issue chooser?

Possible API shape

If this belongs in agents-js, a small provider-specific API could expose the raw usage before normalization, for example:

  1. an optional usage callback on the Google realtime model, or
  2. a provider-specific usage event on the realtime session.

The callback/event would provide the read-only provider usage object together with the generation/request identity. Existing RealtimeModelMetrics fields should remain backward compatible.

The implementation should preserve optional field presence and should not require consumers to infer reasoning tokens as:

totalTokens - inputTokens - outputTokens

The Live API reference and Google SDK type definitions currently describe totalTokenCount differently, so the provider field is preferable to a derived value.

Related issues checked

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions