Repository navigation
Add OpenAI Realtime SDK example (Next.js) - #259
Conversation
Ported from elevenlabs/elevenlabs-dx#2577. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR SummaryLow Risk Overview The demo uses The client hook streams mic PCM with wavtools (SDK audio is WebRTC-only), plays agent audio, builds an ordered transcript from transport events, and tears down on disconnect with explicit AudioContext cleanup and Safari-friendly Reviewed by Cursor Bugbot for commit e6006e0. Bugbot is set up for automated code reviews on this repo. Configure here. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Close the AudioContexts wavtools leaves open, so repeated Start/Stop cycles don't accumulate them. - Tear down when the transport disconnects, so the mic stops and the UI leaves "Connected". - Connect the player before the token fetch, so Safari creates its AudioContext within the click. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A disconnect-triggered teardown read sessionRef after awaiting audio cleanup, so a Stop → Start during that window could close the new session and leave the UI showing disconnected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A disconnect during session.connect or recorder.record let the in-flight start() mark the UI connected, or tear down a newer session from its catch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7649ff0. Configure here.
A socket closed before open rejects session.connect() right after emitting "disconnected", so the torn-down attempt skipped setError. Errors are now hidden only when a newer start() has begun. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Developers with an existing OpenAI Realtime client need a runnable example of pointing it at ElevenLabs Realtime, to go with the OpenAI Realtime API compatibility guide.
Merging this PR will:
agents/nextjs/openai-realtime(PROMPT.md,setup.sh,README.md, generatedexample/): a Next.js voice demo using@openai/agents-realtimeover WebSocket againstwss://api.elevenlabs.io/v1/convai/realtime.app/api/tokenwithclient.conversationalAi.realtime.createClientSecret.README.md.Additional context intended for coding agents and bots
Origin
Ported from the Vite demo in elevenlabs/elevenlabs-dx#2577 onto
templates/nextjs, following the repo's prompt-driven layout (modelled onagents/nextjs/guardrails).Changes relative to the source demo
@elevenlabs/elevenlabs-js≥ 2.74.0) instead of a rawfetchin a Vite dev middleware, and returns only{ value }to the browser. The/api/tokenpath matches the guide's browser snippet.wavtoolsAudioContexts are closed on teardown, the session tears down on transportconnection_change: "disconnected"(not on every non-fatalerrorevent), andplayer.connect()runs before the token fetch so Safari creates the playback context within the click's user activation.DESIGN.md(neutral, flat, template header preserved) with plain buttons instead of a shadcnButtonand custom theme.Known issue (not introduced here)
With pnpm 12,
setup.shexits withERR_PNPM_IGNORED_BUILDSfor template deps (sharp,msw,unrs-resolver) after installing packages; this applies to every Next.jssetup.shin the repo.Test plan and verification
tsc --noEmit(clean)eslint(clean)next build(succeeds;/static,/api/tokendynamic)next startsmoke test:/server-renders;/api/tokenwithout a key returns500 {"error": "Server misconfiguration: ELEVENLABS_API_KEY is not set."}npx prettier . --check(matches CI)ELEVENLABS_API_KEY, including Stop → Start cycles (verified manually)🤖 Generated with Claude Code