feat(telnyx): add Telnyx STT plugin - #622
Conversation
Streams audio to the Telnyx transcription WebSocket as raw linear16 frames and emits transcripts as they arrive. The configurable sample rate lets telephony audio from TelnyxMediaStream pass through at 8kHz without an upsample. Follows the plugins/sarvam precedent of one vendor plugin covering LLM, STT, and TTS.
📝 WalkthroughWalkthroughAdds Telnyx streaming STT over an authenticated aiohttp WebSocket. It validates configuration, resamples and sends PCM audio, manages connection cleanup, parses final and interim transcripts, normalizes confidence values, and reports transport or payload errors. STT is publicly exported, aiohttp is added as a dependency, documentation describes usage, and unit and conditional integration tests cover the implementation. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 63850d89-6127-4285-8df0-2235aac604d3
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (5)
plugins/telnyx/README.mdplugins/telnyx/pyproject.tomlplugins/telnyx/tests/test_telnyx_stt.pyplugins/telnyx/vision_agents/plugins/telnyx/__init__.pyplugins/telnyx/vision_agents/plugins/telnyx/stt.py
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
plugins/telnyx/tests/test_telnyx_stt.py (1)
122-124: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDuplicate
participantfixture.Identical to lines 17-19. Move it to module scope or
conftest.py.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 28597dbf-cea9-4b9d-aeee-b95bd77e4fa9
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (5)
plugins/telnyx/README.mdplugins/telnyx/pyproject.tomlplugins/telnyx/tests/test_telnyx_stt.pyplugins/telnyx/vision_agents/plugins/telnyx/__init__.pyplugins/telnyx/vision_agents/plugins/telnyx/stt.py
🚧 Files skipped from review as they are similar to previous changes (1)
- plugins/telnyx/README.md
| "numpy>=1.24.0", # capped at <2.0 via workspace override in root pyproject.toml | ||
| "cryptography>=44.0.0", | ||
| "fastapi>=0.135.1", | ||
| "aiohttp>=3.13.3", |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
curl -s https://pypi.org/pypi/aiohttp/json | jq '.info.version, (.releases | keys | map(select(startswith("3.13"))))'
gh api graphql -f query='
{
securityVulnerabilities(first: 10, ecosystem: PIP, package: "aiohttp") {
nodes {
advisory { summary severity publishedAt }
vulnerableVersionRange
firstPatchedVersion { identifier }
}
}
}'Repository: GetStream/Vision-Agents
Length of output: 2667
Raise the aiohttp floor to cover open advisories.
plugins/telnyx/pyproject.toml:L17 pins aiohttp>=3.13.3, but the existing aiohttp advisories mark 3.13.3 as vulnerable. Bump the requirement to the documented first patched version, e.g. aiohttp>=3.14.1, unless this plugin has an explicit reason to accept the lower bound.
There was a problem hiding this comment.
Not changing this one here. Every aiohttp-using plugin in the repo pins aiohttp>=3.13.3 (plugins/assemblyai, plugins/nvidia, plugins/sarvam, plugins/xai), and the resolved version comes from the shared uv.lock anyway, so bumping only the Telnyx floor would make the plugin inconsistent without changing what actually gets installed. This is worth a repo-wide bump in its own PR.
There was a problem hiding this comment.
@a692570 The shared lockfile protects workspace installs, but the published plugin dependency floor still permits a vulnerable aiohttp version for independent consumers.
Deferring this to a coordinated repo-wide bump is reasonable. Would you like me to open a follow-up GitHub issue for updating all affected plugin constraints and regenerating uv.lock?
✏️ Learnings added
Learnt from: a692570
Repo: GetStream/Vision-Agents PR: 622
File: plugins/telnyx/pyproject.toml:17-17
Timestamp: 2026-07-30T04:32:49.180Z
Learning: In the Vision-Agents repository, the aiohttp-using plugins `plugins/assemblyai`, `plugins/nvidia`, `plugins/sarvam`, `plugins/telnyx`, and `plugins/xai` consistently declare `aiohttp>=3.13.3`; upgrades to their aiohttp minimum version should be coordinated repository-wide with the shared `uv.lock` update rather than made in only one plugin.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
You are interacting with an AI system.
There was a problem hiding this comment.
Thanks, but no need to open the issue from here. That is a maintainer call about their own dependency policy, and a bot-filed issue against four other plugins from a contributor PR would be presumptuous. The learning you recorded captures the context if a maintainer wants to pick it up.
There was a problem hiding this comment.
@a692570 Understood. No issue will be opened. The coordinated dependency-policy context is retained in the existing learning.
🧠 Learnings used
Learnt from: a692570
Repo: GetStream/Vision-Agents PR: 622
File: plugins/telnyx/pyproject.toml:17-17
Timestamp: 2026-07-30T04:32:49.180Z
Learning: In Vision-Agents, all aiohttp-using plugins in `plugins/assemblyai`, `plugins/nvidia`, `plugins/sarvam`, `plugins/telnyx`, and `plugins/xai` should keep a consistent minimum dependency on `aiohttp` (e.g., `aiohttp>=3.13.3`). If you need to raise the aiohttp minimum version in one plugin, coordinate the change across the other aiohttp-using plugins in the same PR and update the shared `uv.lock` together, instead of updating only a single plugin’s constraint.
You are interacting with an AI system.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
plugins/telnyx/vision_agents/plugins/telnyx/stt.py (1)
121-129: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winRelease audio waiters when startup fails or shutdown begins.
process_audio()waits at Line 145. A failed connection never signals the event, and Line 187 clears it during shutdown; audio submitted during startup can wait forever instead of reaching the closed-WebSocket guard. Signal a terminal connection state on failure and teardown, then re-check it after waiting.
plugins/telnyx/vision_agents/plugins/telnyx/stt.py#L121-L129: wake blocked audio tasks after failed connection cleanup.plugins/telnyx/vision_agents/plugins/telnyx/stt.py#L187-L187: do not leave existing waiters blocked during teardown.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: fa7405b0-5c5a-412e-9307-a10e730e0165
📒 Files selected for processing (2)
plugins/telnyx/tests/test_telnyx_stt.pyplugins/telnyx/vision_agents/plugins/telnyx/stt.py
🚧 Files skipped from review as they are similar to previous changes (1)
- plugins/telnyx/tests/test_telnyx_stt.py
2 participants
Adds streaming Telnyx STT to
plugins/telnyx/, which today only contains themedia streaming transport from #594. The pattern precedent is
plugins/sarvam/,which keeps
llm.py,stt.py, andtts.pyin one vendor plugin. LLM and TTSare separate PRs so each lands independently.
What it adds
plugins/telnyx/vision_agents/plugins/telnyx/stt.pySTTexport in the plugin__init__.pyaiohttpdependency in the pluginpyproject.tomlplugins/telnyx/tests/test_telnyx_stt.pySample rate and the existing transport
TelnyxMediaStreamdecodes PCMU telephony audio to 8 kHzPcmDatatoday. Theplugin takes a
sample_rateargument, resamples each chunk to it, and sendsraw
linear16frames, so a phone agent can runtelnyx.STT(sample_rate=8000)and avoid an upsample to 16 kHz that adds nothing back.
The default is 16000, which suits non telephony sources. Telnyx requires an
explicit
sample_ratefor raw encodings, so there is no negotiate-from-streampath on the wire. If you would rather the plugin read the rate off an attached
TelnyxMediaStreaminstead of taking it as an argument, say so and I willchange it, but that couples STT construction to a live call and the current
shape matches how the other STT plugins are configured.
Turn detection
turn_detectionis left at the base default ofFalse. This endpoint sendsonly
{"transcript", "confidence", "is_final"}and no VAD signals, so unlikesarvam there is nothing to derive
TurnStartedandTurnEndedfrom, and theagent's own turn detection handles it. Flagging in case a maintainer knows of a
VAD parameter on this endpoint that is not surfaced in the errors the server
returns for unknown parameters.
Protocol note
The query parameter for partial transcripts is
interim_results.partial_resultsis accepted by the endpoint but ignored, which silently givesfinals only. There is a unit test pinning this so it does not regress.
The Telnyx edge rejects a WebSocket handshake carrying an
Originheader.aiohttp does not send one, so no workaround is needed here, but there is a
comment at the connect site because it is not obvious.
Testing
Verified against the live Telnyx API, not from docs alone. Supported values for
transcription_engineandinput_formatwere read off the errors the serverreturns for unknown values rather than guessed.
uv run ruff check .andruff format --check .passuv run dev.py mypyanddev.py mypy-pluginspassuv run dev.py validate-extraspassesuv run pytest -m "not integration" plugins/telnyx/tests/passes, 41 testsuv run pytest -m integration plugins/telnyx/tests/test_telnyx_stt.pypasseswith a real
TELNYX_API_KEY, 2 tests transcribing the sharedmia.mp3fixture, one at 16 kHz and one at the 8 kHz telephony rate
uv.lockchanges by two lines, for the new dependency.Relation to the other two Telnyx PRs
This is one of three independent PRs, each branched off
main, not stacked:#620 LLM, #621 TTS, #622 STT. Each touches the plugin
__init__.py,pyproject.toml,README.md, and two lines ofuv.lock, so whichever landsfirst leaves the other two with small conflicts in those four files. Happy to
rebase on request, or to fold all three into one PR if you would rather review
them together.