feat(voice): expose a bounded session-level provider request ledger

[Nisarg38]

Feature Type

Would make my life easier

Problem

A production application often needs to attach provider-call correlation records to its own call/session object, even when it does not use LiveKit Cloud Observability or an OpenTelemetry exporter.

The SDK has useful but incomplete ingredients today:

• LLMMetrics, STTMetrics, TTSMetrics, and realtime metrics carry a request_id.
• Some TTS plugins add provider-known IDs to the lk.provider_request_ids trace attribute.
• metrics_collected can currently aggregate the active session's component metrics, but that session-level event is deprecated.
• SessionReport.to_dict() excludes individual metric events.

As a result, an application must maintain private bookkeeping to answer a basic post-call support question:

For this application call, which STT/TTS/LLM provider attempts ran, which provider-correlatable IDs belong to each attempt, and did an attempt succeed, fail, get cancelled, or trigger fallback?

There is no provider-independent, non-Cloud API for that today. In particular, component request_id values do not have consistent semantics:

• some are provider server request/response IDs;
• some are SDK-local stream IDs;
• some provider support flows use a distinct trace/context ID;
• errors can carry a provider request ID before a normal metric is emitted.

Related work and scope

This deliberately builds on, rather than duplicates:

• expose xAI realtime provider correlation ids through a non-deprecated API #6333 / feat(voice): expose realtime provider request IDs #6501, which expose a successful realtime provider response ID on assistant ChatMessage.metrics;
• Cannot correlate TTSMetrics.ttfb with ChatMessage.metrics.tts_node_ttfb #4387 / feat(voice): add speech_id to assistant message metrics #6787, which correlate assistant messages with speech_id;
• existing lk.provider_request_ids OpenTelemetry trace attributes.

Those are valuable per-turn and trace surfaces. The remaining gap is a framework-owned, session-scoped lifecycle record that an application can snapshot and persist with its own call entity. It must cover STT, TTS, LLM, and realtime paths, and must include failures/retries/fallbacks that never create an assistant message.

Proposed direction

Add an opt-in, bounded request ledger owned by AgentSession (exact API shape is open for discussion). For example:

ledger = ProviderRequestLedger(capacity=256)
session = AgentSession(..., provider_request_ledger=ledger)

# At call cleanup, no Cloud/OTel dependency:
call.provider_requests = ledger.snapshot()

And/or a non-deprecated event such as:

@session.on("provider_request_completed")
def on_provider_request_completed(ev):
    call.provider_requests.append(ev.attempt)

Each immutable record should distinguish identifiers instead of overloading request_id:

ProviderRequestAttempt(
    component="stt" | "tts" | "llm" | "realtime",
    provider="...",
    model="..." | None,
    sdk_request_id="..." | None,
    provider_request_ids=["..."],      # only IDs known to the provider
    provider_trace_ids=["..."],        # e.g. provider trace/context identifiers
    speech_id="..." | None,
    started_at=...,
    completed_at=...,
    outcome="success" | "error" | "cancelled",
    retry_index=...,
    fallback_index=...,
    error_type="..." | None,
    status_code=... | None,
)

The proposed names are illustrative. The key contract is that a caller can reliably tell a locally generated SDK ID from a provider-side correlation ID.

Requirements

• Works completely in-process; no LiveKit Cloud, OTel exporter, or external observability backend required.
• Bounded retention with deterministic eviction or an application-supplied sink. It must not grow unbounded during a long session.
• No request/response bodies, transcripts, audio, prompts, API keys, or arbitrary error bodies in the default record.
• Records terminal outcomes for success, cancellation, provider errors, retries, and fallback attempts.
• Preserves existing speech_id / turn correlation where available.
• Plugin authors can supply provider-known IDs without every plugin inventing a private convention.
• Missing provider IDs are normal and represented explicitly as absent, not fabricated.

Candidate acceptance tests

1. A normal pipeline call yields STT, LLM, and TTS records that can be attached to an application call object.
2. A fallback LLM/TTS attempt records both the failed primary and the successful fallback with stable lineage.
3. A cancelled TTS/realtime attempt is retained as cancelled.
4. A plugin-local ID is not presented as a provider server ID.
5. A provider-known ID, such as a streaming STT request ID or TTS context/trace ID, survives into the record where the plugin exposes it.
6. The ledger stays within its configured capacity and contains no content-bearing request/response data.

Workarounds / Alternatives

Applications can currently subscribe to deprecated session.metrics_collected, subscribe to individual plugin metric events, scrape OTel spans, or monkey-patch provider plugins. All require application-specific joins between component metrics, errors, fallbacks, provider-specific identifier semantics, and the application's call record.

Additional Context

This is especially useful for self-hosted and telephony agents, where a provider support ticket may need to be correlated to one customer call after the fact. It should improve supportability without making LiveKit Cloud Observability a prerequisite.