perf(rime): stream sentence batches and reuse WebSocket connections - #2426
perf(rime): stream sentence batches and reuse WebSocket connections#2426chasef07 wants to merge 1 commit into
Conversation
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if (data.type === 'error') { | ||
| // Provider error text can echo input or credentials. Keep it out of logs. | ||
| fail('Rime WebSocket synthesis failed'); | ||
| } else if (activeContext && data.contextId === activeContext) { | ||
| void messages.write(data).catch(() => {}); |
There was a problem hiding this comment.
🟡 Stale errors abort later speech
When a reused socket delivers an old request's error, onMessage fails the current speech before checking its context. Late errors can therefore discard unrelated speech on the same connection.
| if (data.type === 'error') { | |
| // Provider error text can echo input or credentials. Keep it out of logs. | |
| fail('Rime WebSocket synthesis failed'); | |
| } else if (activeContext && data.contextId === activeContext) { | |
| void messages.write(data).catch(() => {}); | |
| if (data.type === 'error') { | |
| // Provider error text can echo input or credentials. Keep it out of logs. | |
| if (activeContext && (!data.contextId || data.contextId === activeContext)) { | |
| fail('Rime WebSocket synthesis failed'); | |
| } | |
| } else if (activeContext && data.contextId === activeContext) { | |
| void messages.write(data).catch(() => {}); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
|
@naszzz can you look and test this aap and if you want it to be default? |
1 participant
Description
With Rime WebSocket
segment: 'never', the current plugin sends its provider flush only after all input text has arrived and closes the socket after each speech stream. This delays synthesis even when tokenized sentences are already available and repeats connection setup on later replies.Add opt-in
flushSentencesandreuseWebsocketoptions. The first synthesizes available tokenizer sentences sequentially; the second reuses a fully drained connection. Existing HTTP and WebSocket behavior remains unchanged unless enabled.Changes Made
stream.flush()boundaries with separate final frames and metrics, including already-queued segments and whitespace-only input. Retain at most one real PCM sample for the final frame so short audio is delivered promptly.TTS.close().Pre-Review Checklist
Testing
ttsNode, baseline behavior, serialized batches, exact PCM conservation, timestamps, explicit and queued SDK segments, metrics, context tracing, reuse/concurrency, cancellation, option changes, timeout, idle expiry, and shutdown.pnpm test agents plugins/rime --maxWorkers 2 --silent— 153 files passed; 2,497 tests passed and 6 skipped.restaurant_agent.ts/realtime_agent.ts: Not run; this change is isolated to the Rime plugin and its opt-in WebSocket path.Local checks used Node 24 and the repository-pinned pnpm 11.13.1: full monorepo
pnpm build,pnpm lint,pnpm typecheck,pnpm format:check,pnpm throws:check, and Rimeapi:check.An earlier run alongside concurrent build/lint work had two timing failures in unchanged core tests. Both affected files passed on targeted rerun, and the complete final run above passed with two test workers.
The regression fixture was also run against unchanged upstream source: early audio, socket reuse, and default
ttsNodeearly delivery failed before the change; the compatibility test passed. The explicit/queued SDK segment metrics failures were reproduced before their fixes.Additional Notes
The underlying transport change was exercised against the real Rime Coda
/ws3endpoint in ten synthetic-text scenarios, including English/Spanish, cancellation/recovery, connection reuse, and shutdown. No credentials, patient data, or private application code are included in this PR.With a sentence-ready tokenizer and a deliberately paced two-second text-input schedule, English first PCM arrived at 2.27–2.36 seconds on the original path versus 0.31–0.65 seconds on the optimized path. The second optimized reply opened no new connection. These controlled measurements demonstrate removal of client buffering; they are not a production latency distribution or a claim about the default tokenizer's latency.
Rime can coalesce overlapping flushes, so this implementation waits for each matching
donebefore submitting the next batch. See the segmentation contract and Coda JSON protocol.Live word timestamps sometimes exceeded returned PCM duration on both original and optimized paths. This PR preserves/rebases provider timing; it does not attempt to correct provider alignment accuracy. Sentence prosody and full SIP-call behavior still require listening/call validation.
Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.