Skip to content

fix(realtime): subscribe to the server's audio track only when the client publishes audio - #219

Merged
tomershlasky merged 2 commits into
mainfrom
fix/video-only-remote-subscribe
Oct 8, 2026
Merged

tomershlasky merged 2 commits into
mainfrom
fix/video-only-remote-subscribe

Conversation

@tomershlasky

@tomershlasky tomershlasky commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Why

A video-only React Native client on iOS gets a microphone permission prompt on a fresh install even though it never publishes audio. The realtime server publishes an audio track at join on every session (it is what libwebrtc's initial bandwidth probe rides on, so it stays), and the SDK joined the LiveKit room with the default autoSubscribe: true, so it subscribed to that track. On iOS, playing out a remote audio track starts the WebRTC audio engine, which is what triggers the microphone prompt. The server's audio track is the client's own audio delayed to stay in sync with the transformed video, so a video-only client was only ever receiving silence.

What

  • The room now joins with autoSubscribe: false and the SDK subscribes per track: always to the server's video, and to its audio exactly when the local stream carries an audio track. No new option; the rule is derived from what the app publishes (fix(realtime): derive the remote-audio subscription from the local stream; drop the remoteAudio option #220, merged into this branch).
  • The subscribe sweep runs after connect (publications already in the room never emit TrackPublished), on every later TrackPublished, and on RoomEvent.Reconnected (a full reconnect lands on a fresh SFU session). setSubscribed(true) is idempotent, so repeating it is safe. Signal resumes already re-send the subscribed set via livekit-client's sync state.
  • README "Remote audio" section describes the behavior and why to play the remote audio rather than the local mic.
  • The watch-stream subscribe client is unchanged.

Behavior change

Apps that publish audio keep today's behavior. Apps that publish video only now receive a remote stream with no audio track (it was a silent track before). A model that one day emits audio the client never sent would need an SDK change driven by a server capability, not an app flag. Suggest shipping as a minor release with a changelog note.

Tests

  • New unit tests in tests/realtime.unit.test.ts: video-only client subscribes video only (join sweep, late publication, and a stray TrackSubscribed audio never reaches the consumer), audio-publishing client gets audio, no local stream at all, re-sweep on Reconnected, non-server participants ignored.
  • The React Native compat fixture gained getAudioTracks and remoteParticipants; connect assertions now include { autoSubscribe: false }.
  • pnpm test, pnpm typecheck and biome check are clean. Not yet verified end to end on an iOS device.

Note

Medium Risk
Changes WebRTC subscription behavior for all realtime publishers; video-only apps lose the (silent) remote audio track, while audio sessions should match prior behavior.

Overview
Fixes spurious iOS microphone permission on video-only realtime clients by stopping blind subscription to the inference server’s always-present audio track.

LiveKit connect now uses { autoSubscribe: false }. The SDK selectively subscribes to the inference server’s video always, and to audio only when the local MediaStream has audio tracks (wantsRemoteAudio()). A post-connect sweep plus TrackPublished and Reconnected handlers re-apply subscriptions; TrackSubscribed still omits audio from the emitted remoteStream when the client is video-only.

README adds a Remote audio section: play the delayed server audio (not the local mic) when you publish audio; video-only sessions get no remote audio track.

Tests cover video-only vs audio publishing, reconnect re-sweep, and non-server participants. Watch/subscribe client behavior is unchanged.

Reviewed by Cursor Bugbot for commit e44b601. Bugbot is set up for automated code reviews on this repo. Configure here.

@tomershlasky
tomershlasky force-pushed the fix/video-only-remote-subscribe branch from c10b66f to 4b7275d Compare October 7, 2026 13:12
@pkg-pr-new

pkg-pr-new Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@decartai/sdk@219

commit: e44b601

…ream; drop the remoteAudio option

Stacked on #219. The server's audio track is the client's own audio played
back delayed to stay in sync with the transformed video, so the only correct
subscription rule is "subscribe exactly when we publish audio", which #219
already uses as the default. The explicit `remoteAudio` override had no
caller: `true` would serve a model that emits audio the client never sent
(none is served), `false` would serve an app that sends its microphone but
does not want the synced voice back. Removing it keeps the fix transparent to
apps and avoids a public knob for a hypothetical.

The connect-time decision is stable for the life of the session: the SDK's
only post-connect change to the local stream is replaceVideoTrack, which
preserves the audio tracks.

Tests: the two option tests go; a null-localStream case pins the video-only
default for sessions without a camera stream.
@tomershlasky
tomershlasky merged commit 18f2880 into main Oct 8, 2026
5 checks passed
@tomershlasky
tomershlasky deleted the fix/video-only-remote-subscribe branch October 8, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants