feat(smallestai): add TTS continuations protocol support - #7111
Open
harshitajain165 wants to merge 2 commits into
Open
feat(smallestai): add TTS continuations protocol support#7111harshitajain165 wants to merge 2 commits into
harshitajain165 wants to merge 2 commits into
Conversation
Smallest AI's WebSocket TTS API recently added a "continuations" protocol (context_id-based) that releases buffered text at natural sentence boundaries instead of holding everything until an explicit flush, and primes each release with the previous one's audio so prosody stays continuous across fragments. SynthesizeStream now defaults to this protocol instead of the legacy continue/flush fields: - New use_continuations (default True) and max_buffer_delay_ms options; max_buffer_flush_ms is kept for the use_continuations=False fallback. - Continuations can emit multiple `complete` frames per context with no terminal marker, so _recv_task drains on a short idle gap after the closing fragment instead of returning on the first `complete`. - Best-effort cancel_request cleanup if a stream is interrupted (e.g. barge-in) mid-context, so an open context isn't left behind on a pooled connection that gets reused for the next turn.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Comment on lines
+461
to
+468
| timeout = ( | ||
| _CONTINUATIONS_IDLE_TIMEOUT if drained_after_close else self._conn_options.timeout | ||
| ) | ||
| try: | ||
| msg = await ws.receive(timeout=timeout) | ||
| except asyncio.TimeoutError: | ||
| if drained_after_close: | ||
| return |
Contributor
There was a problem hiding this comment.
🔴 Idle gaps truncate continuation audio
After closure, _recv_one treats the first 600 ms response gap as completion without receiving a completion frame. Slow synthesis loses pending audio.
Was this helpful? React with 👍 or 👎 to provide feedback.
Contributor
Author
There was a problem hiding this comment.
Fixed in 8dd9247: connection is now discarded (pool.remove()) instead of reused whenever continuations completion is inferred rather than protocol-confirmed, so a still-draining context can't leak into the next request.
…in continuations Addresses two review findings on the continuations receive loop: - The completion check depended on ctx_state["finalized"] being set at the exact moment a `complete` frame was processed. If the server's reply for the closing fragment was scheduled before send_task's continuation set that flag, the signal was silently dropped and the stream would hang until the full connection timeout. Fixed by no longer reacting to which `complete` event arrives when; the continuations loop now polls with a short idle timeout and re-checks `finalized` fresh on every tick, which is immune to task-scheduling order. - The idle-drain heuristic has no way to be certain a context has fully drained (continuations gives no terminal marker), so a connection that finished this way could still have trailing frames in flight when returned to the pool, corrupting whatever request reuses it next. _recv_task now reports whether completion was protocol-confirmed (legacy) or merely inferred (continuations), and _run explicitly removes the connection from the pool instead of returning it whenever it wasn't confirmed.
Comment on lines
+539
to
+542
| elif status == "error": | ||
| raise APIConnectionError( | ||
| f"SmallestAI TTS error: {event.get('message', 'unknown error')}" | ||
| ) |
Contributor
There was a problem hiding this comment.
🟨 Provider errors expose customer content
_parse_status_event copies provider text into an exception recorded by TTS telemetry. Provider errors can expose customer speech without PII tagging.
Was this helpful? React with 👍 or 👎 to provide feedback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Smallest AI's WebSocket TTS API recently added a "continuations" protocol (
context_id-based) that releases buffered text at natural sentence boundaries instead of holding everything until an explicit flush, and primes each release with the previous one's audio so prosody stays continuous across fragments.SynthesizeStreamnow defaults to this protocol instead of the legacycontinue/flushfields:use_continuations(defaultTrue) andmax_buffer_delay_msoptions;max_buffer_flush_msis kept for theuse_continuations=Falsefallback.completeframes per context with no terminal marker, so_recv_taskdrains on a short idle gap after the closing fragment instead of returning on the firstcomplete.cancel_requestcleanup if a stream is interrupted (e.g. barge-in) mid-context, so an open context isn't left behind on a pooled connection that gets reused for the next turn.Testing
ruff format/ruff check/mypypass.tests/test_plugin_smallestai_tts.py, kept local): legacy protocol path, continuations-is-default path, and a new test confirmingcancel_requestis sent when a stream is interrupted mid-context.