fix(google): restart the realtime session on abnormal WebSocket close - #2366
Open
anzemur wants to merge 1 commit into
Open
fix(google): restart the realtime session on abnormal WebSocket close#2366anzemur wants to merge 1 commit into
anzemur wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: b616a87 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
anzemur
marked this pull request as ready for review
August 28, 2026 14:06
Contributor
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When the Gemini Live WebSocket dies abnormally, the session never recovers. In our production voice agent this branch fires for two distinct error groups: bare
WebSocket closed with code 1006(a few per day) and closes carrying Google'sThe operation was aborted.reason (TLS connection dropped server-side) - the latter at ~2,800 events / ~1,800 users over two months, ~30 live sessions dying per day. Every one of them passed our teardown filter, i.e. genuinely mid-conversation:onerror(network-level failures) callsmarkRestartNeeded(), so#mainTask's reconnect loop tears down, reconnects, and re-seeds the chat context.onclosewith a non-normal code only logs, emits an unrecoverable error, and marks the generation done. Nothing setssessionShouldClose, so the main task stays parked onsessionShouldClose.wait()with a dead socket — the user keeps talking into a session that will never answer. Outbound sends on the closedwssocket don't throw, so the retry path in#mainTask's catch never triggers either.Fix
Mirror
onerrorinonclose: on a non-normal close while the session isn't already closing, callmarkRestartNeeded()so the main task reconnects and re-seeds history. The emitted error is markedrecoverable: truein that case — a restart is coming, and apps that treat unrecoverable errors as fatal shouldn't tear down a session that's about to recover. When the session is already closing (sessionShouldCloseset), behavior is unchanged (recoverable: false, no restart).Same as the existing
onerrorpath, these restarts don't consume the retry budget; a repeatedly failing endpoint behaves as it does today for network errors.Tests
realtime_reconnect.test.tsdrives the live-connect callbacks through a mocked@google/genai:connecthappens and the error is emitted withrecoverable: true(fails without the fix: no reconnect,recoverable: false),Full
@livekit/agents-plugin-googlesuite passes (59 tests).Live verification
Applied this change to the installed 1.7.0 dist of a production voice agent and killed the real Gemini socket mid-session (
ws.terminate()on the underlyingws@8.19.0socket — the client observes exactly a 1006), with a probe question whose answer (47) existed only in the seeded chat history:WebSocket closed with code 1006logged, no reconnect ever, and one step worse than the parked loop: therecoverable: falseerror makesAgentSessionclose outright (AgentSession is closing due to an unrecoverable error—ev.typeis undefined so_onError'smaxUnrecoverableErrorscounting is skipped). The session dies mid-conversation.sessionShouldClose:true→ newConnecting to Gemini Realtime API...),recoverable: truekeeps the AgentSession alive, and the model answers47.— proving both the reconnect and the chat-context re-seed.So beyond un-parking the main task,
recoverable: truealso prevents the framework-level session teardown on a transient socket drop.