feat(assemblyai): add universal-3-6-pro streaming model - #2409
feat(assemblyai): add universal-3-6-pro streaming model#2409rosetta-livekit-bot[bot] wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: bfef640 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Note
This report is out of date. Scroll down for Devin Review's latest report on this PR.
Devin Review found 3 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if (opts.continuousPartials !== undefined) { | ||
| configMsg.continuous_partials = opts.continuousPartials; | ||
| } | ||
| if (opts.interruptionDelay !== undefined) { | ||
| configMsg.interruption_delay = opts.interruptionDelay; | ||
| } |
There was a problem hiding this comment.
🟡 Unsupported live options end transcription
On standard models, updateOptions forwards continuous partials or interruption delay without the constructor’s family check. AssemblyAI can reject the update and end transcription.
Prompt for agents
Validate continuousPartials and interruptionDelay against the effective speech model before mutating STT or SpeechStream state, matching the constructor's U3_PRO_ONLY_PARAMS gate. Cover both STT.updateOptions and the exported SpeechStream.updateOptions, and add tests showing standard models reject these options before any UpdateConfiguration message is queued.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if (opts.languageCodes !== undefined) { | ||
| if (!isU3ProModel(this.#opts.speechModel)) { | ||
| throw new Error( | ||
| `The 'languageCodes' parameter is only supported with the ${U3_PRO_MODELS.join(', ')} models.`, | ||
| ); | ||
| } | ||
| opts.languageCodes = normalizeLanguageCodes(opts.languageCodes); |
There was a problem hiding this comment.
🟡 Combined model and language update fails
When updateOptions changes a standard model to Pro and sets languages together, each stream validates against its old model and throws. The parent retains new settings.
Prompt for agents
Make STT.updateOptions atomic when speechModel and languageCodes arrive together. Validate the complete effective option set before mutating parent or stream state, and have SpeechStream validate languageCodes against the effective incoming speechModel rather than only its old model. Add a test with an existing standard-model stream updated to universal-3-6-pro plus languageCodes.
Was this helpful? React with 👍 or 👎 to provide feedback.
| function validateAgentContext(agentContext: string | undefined): void { | ||
| if (agentContext !== undefined && agentContext.length > MAX_AGENT_CONTEXT_CHARS) { | ||
| throw new Error( | ||
| `agentContext exceeds maximum length of ${MAX_AGENT_CONTEXT_CHARS} characters (got ${agentContext.length})`, | ||
| ); |
There was a problem hiding this comment.
🟡 Emoji contexts hit the wrong limit
For non-BMP context, validateAgentContext counts UTF-16 units instead of characters. Valid text can be rejected, while automatic truncation can split a character.
Prompt for agents
Apply the 1,750-character limit by Unicode code points rather than JavaScript UTF-16 code units in validateAgentContext and _pushConversationItem. Ensure tail truncation cannot split a surrogate pair, and add tests using emoji at and around the limit.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
1 participant
Summary
universal-3-6-proto the AssemblyAI streaming STT model type and Universal-3 Pro family.Upstream: livekit/agents#7100
Source diff coverage
livekit-plugins/livekit-plugins-assemblyai/livekit/plugins/assemblyai/stt.pytoplugins/assemblyai/src/models.tsandplugins/assemblyai/src/stt.ts; TypeScript separates the public model union from the STT implementation. Addeduniversal-3-6-proto the model type and Pro-family behavior, and ported missing target infrastructure required by the source behavior (continuousPartials,interruptionDelay,languageCodes, safe default chat-context carryover). Updated the generated counterpartplugins/assemblyai/etc/agents-plugin-assemblyai.api.md.tests/test_plugin_assemblyai_stt.pytoplugins/assemblyai/src/stt.test.ts; ported model acceptance, Pro-only parameter forwarding, connect-time defaults, family-wide Voice Focus/mode/language-code acceptance, and default chat-context carryover coverage using the target WebSocket test harness.Testing
pnpm vitest run plugins/assemblyai(11 passed, 1 credential-dependent skip)pnpm buildpnpm --filter @livekit/agents-plugin-assemblyai lintpnpm --filter @livekit/agents-plugin-assemblyai api:checkpnpm exec prettier --check "plugins/assemblyai/src/**/*.{ts,tsx,md,json}" ".changeset/wise-cats-stream.md"A supplemental
pnpm lintmonorepo run reaches the AssemblyAI task successfully but is blocked by a pre-existing Prettier violation inplugins/inworld/src/tts.ts:43. That unrelated file is not modified here.Ported from livekit/agents#7100
Original PR description
What
Adds
universal-3-6-proto the AssemblyAI streaming STT plugin as a supportedmodelvalue.universal-3-6-prois the next Universal-3 Pro release. It is served by its own ASR deployment and shares all Universal-3 Pro behavior:prompt,agent_context,previous_context_n_turns,continuous_partials,interruption_delay,voice_focus/voice_focus_threshold,mode,language_codes, and formatted output. It introduces no new connection orUpdateConfigurationparameters.speech_model/modelLiterals and to_U3_PRO_MODELS, so the model gets the Pro-family parameter gating, the family's connect-time defaults (100 ms min/max turn silence unlessmodeis set, language detection on), and chat-context carryover.universal-3-5-proremains the default, and the deprecatedu3-proalias is unchanged.Testing
tests/test_plugin_assemblyai_stt.py: newuniversal-3-6-problock (accepted as a model, accepts the Pro-only params, connect query names the model and applies the family defaults), and the family-loop tests (voice_focus,mode,language_codes, carryover default) now include it. Full file: 111 passed; ruff and strict mypy clean.speech_model=universal-3-6-proconnection returns aBeginsession message, identical touniversal-3-5-pro.