Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -900,6 +900,37 @@ function Chat() {

Use `isToolContinuation` when your UI should distinguish a fresh user submit from a continuation after a tool result. For example, show a typing indicator only for `status === "submitted" && !isToolContinuation`, while keeping loading controls disabled whenever `isStreaming` is true.

### Non-React clients

`useAgentChat` is React-specific. For Vue, Svelte, or vanilla JavaScript, `agents/chat/transport` exports `WebSocketChatTransport`, which adapts an `AgentClient` WebSocket connection to the AI SDK transport interface. This entry point requires no React peer dependency.

<TypeScriptExample>

```ts
import { useChat } from "@ai-sdk/vue";
import { AgentClient } from "agents/client";
import { WebSocketChatTransport } from "agents/chat/transport";

const agent = new AgentClient({
agent: "ChatAgent",
name: "user-123",
host: window.location.host,
});

const { messages, sendMessage, status } = useChat({
transport: new WebSocketChatTransport({
agent,
cancelOnClientAbort: true,
}),
});
```

</TypeScriptExample>

The transport covers new turns, regenerated turns, and stream cancellation. It is a lower-level primitive than `useAgentChat`: loading persisted history, automatic stream resume after a reconnect, cross-tab transcript synchronization, and client-side tool continuations remain the React hook's responsibility. Implement whichever of those your client needs on top of the transport.

The [`vue-chat` example](https://github.com/cloudflare/agents/tree/main/examples/vue-chat) shows a minimal Vue client, and [`ai-chat`](https://github.com/cloudflare/agents/tree/main/examples/ai-chat) shows the full React integration for comparison.

## Tools

`AIChatAgent` supports three tool patterns, all using the AI SDK's `tool()` function:
Expand Down Expand Up @@ -1706,6 +1737,7 @@ The originating client receives the streaming response. All other clients receiv
| `@cloudflare/ai-chat/react` | `useAgentChat`, `extractClientToolSchemas`, `getToolPartState`, `getToolCallId`, `getToolInput`, `getToolOutput`, `getToolApproval` |
| `@cloudflare/ai-chat/types` | `MessageType`, `OutgoingMessage`, `IncomingMessage` |
| `agents/chat` | Shared advanced chat primitives such as `SaveMessagesResult`, `SaveMessagesOptions`, `CHAT_MESSAGE_TYPES`, `ROW_MAX_BYTES`, and `isReplayChunk()` |
| `agents/chat/transport` | `WebSocketChatTransport` and its `AgentConnection` connection types, for non-React clients |

### WebSocket protocol

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -614,6 +614,25 @@ client.addEventListener("message", () => {});

</TypeScriptExample>

### Chat from a non-React client

An `AgentClient` connection can drive an AI SDK chat UI in any framework. `agents/chat/transport` exports `WebSocketChatTransport`, which requires no React peer dependency:

<TypeScriptExample>

```ts
import { AgentClient } from "agents/client";
import { WebSocketChatTransport } from "agents/chat/transport";

const transport = new WebSocketChatTransport({
agent: new AgentClient({ agent: "ChatAgent", name: "user-123" }),
});
```

</TypeScriptExample>

Refer to [Non-React clients](/agents/communication-channels/chat/chat-agents/#non-react-clients) for the behaviors this transport does and does not cover.

## Agent-tool events

If your chat UI renders retained child runs from [Agents as tools](/agents/runtime/execution/agent-tools/), use `useAgentToolEvents()` alongside `useAgent()` and `useAgentChat()`. The hook subscribes to the parent connection, replays retained child timelines, and groups runs by parent tool call ID.
Expand Down
104 changes: 94 additions & 10 deletions src/content/docs/agents/communication-channels/voice.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ products:
---

import {
CardGrid,
InlineBadge,
LinkCard,
TypeScriptExample,
WranglerConfig,
PackageManagers,
Expand Down Expand Up @@ -607,11 +609,13 @@ export class CustomAgent extends VoiceAgent<Env> {

### Third-party providers

| Package | Class | Description |
| ------------------------------ | --------------- | ----------------------- |
| `@cloudflare/voice-deepgram` | `DeepgramSTT` | Continuous STT |
| `@cloudflare/voice-elevenlabs` | `ElevenLabsTTS` | High-quality TTS |
| `@cloudflare/voice-twilio` | `TwilioAdapter` | Telephony (phone calls) |
| Package | Class | Description |
| ------------------------------ | ------------------------ | ----------------------- |
| `@cloudflare/voice-deepgram` | `DeepgramSTT` | Continuous STT |
| `@cloudflare/voice-elevenlabs` | `ElevenLabsTTS` | High-quality TTS |
| `@cloudflare/voice-telnyx` | `TelnyxSTT`, `TelnyxTTS` | STT, TTS, and telephony |
| `@cloudflare/voice-twilio` | `TwilioAdapter` | Telephony (phone calls) |
| `@cloudflare/voice-plivo` | `PlivoAdapter` | Telephony (phone calls) |

**ElevenLabs TTS:**

Expand Down Expand Up @@ -648,21 +652,101 @@ export class MyAgent extends VoiceAgent<Env> {

</TypeScriptExample>

## Telephony (Twilio)
**Telnyx STT and TTS:**

Connect phone calls to your voice agent using the Twilio adapter:
Import from the `/stt` and `/tts` subpaths, which are server-safe:

<TypeScriptExample>

```ts
import { TelnyxSTT } from "@cloudflare/voice-telnyx/stt";
import { TelnyxTTS } from "@cloudflare/voice-telnyx/tts";

export class MyAgent extends VoiceAgent<Env> {
transcriber = new TelnyxSTT({
apiKey: this.env.TELNYX_API_KEY,
engine: "Telnyx", // or "Deepgram"
interimResults: true,
});
tts = new TelnyxTTS({
apiKey: this.env.TELNYX_API_KEY,
voice: "Telnyx.NaturalHD.astra",
});
}
```

</TypeScriptExample>

`TelnyxTTS` defaults to `backend: "rest"`. Set `backend: "websocket"` for lower time-to-first-audio; that backend requires the Workers runtime.

## Telephony

Telephony connects phone calls to the same `withVoice` agent that serves your browser clients. The call shares that agent instance's conversation history, state, tools, and schedules, so one agent can answer the phone and the web.

Providers take one of two approaches, which determines where call audio arrives and what you have to deploy:

| Provider | Approach | Call audio arrives at | Best for |
| -------- | ---------------------------------- | --------------------- | ---------------------------------------------- |
| Twilio | Server-side adapter in your Worker | Your Worker | Inbound numbers answered server-side |
| Plivo | Server-side adapter in your Worker | Your Worker | Inbound numbers answered server-side |
| Telnyx | Browser WebRTC bridge | The browser | Softphone and click-to-call in an app you ship |

### Server-side adapters (Twilio and Plivo)

```sh
npm install @cloudflare/voice-twilio
# or
npm install @cloudflare/voice-plivo
```

The adapter bridges Twilio Media Streams to your VoiceAgent:
The adapter terminates the provider's audio WebSocket in your Worker and converts between the provider's 8 kHz mulaw audio and the agent's 16 kHz PCM protocol:

```txt
Phone → Twilio → WebSocket → TwilioAdapter → WebSocket → VoiceAgent
Phone → provider → WebSocket → adapter → WebSocket → VoiceAgent
```

`WorkersAITTS` returns MP3, which cannot be decoded to PCM in the Workers runtime. When using the Twilio adapter, use a TTS provider that outputs raw PCM (for example, ElevenLabs with `outputFormat: "pcm_16000"`).
No browser is involved. Each adapter exposes a `handleRequest()` method that you call from your `fetch` handler for the provider's WebSocket path, and by default each call gets its own agent instance named after the provider's call identifier.

Beyond that path, the two providers differ in what they need from you. Twilio is configured with TwiML that points at your Worker. Plivo needs a second route — an answer URL returning XML that tells Plivo where to open the audio WebSocket — plus an application that links your phone number to that answer URL.

### Browser WebRTC bridge (Telnyx)

```sh
npm install @cloudflare/voice-telnyx
```

Telnyx bridges the PSTN call through WebRTC in the browser and reuses your existing voice client transport:

```txt
Phone ↔ Telnyx ↔ WebRTC ↔ browser bridge ↔ WebSocket → VoiceAgent
```

Because the browser holds the WebRTC session, it needs a short-lived Telnyx credential — never your API key. `TelnyxJWTEndpoint` mints those tokens server-side and requires an `authorize` callback, so a public route cannot mint credentials for arbitrary callers. Telephony needs `TELNYX_CREDENTIAL_CONNECTION_ID` alongside `TELNYX_API_KEY`.

Telnyx also provides STT and TTS, so it can supply the whole pipeline. Refer to [Third-party providers](#third-party-providers) for those.

### PCM output for telephony

`WorkersAITTS` returns MP3, which cannot be decoded to PCM in the Workers runtime. With the Twilio or Plivo adapter, use a TTS provider that outputs raw PCM — for example ElevenLabs with `outputFormat: "pcm_16000"`, or a Workers AI model called with `encoding: "linear16"` and `container: "none"`.

This constraint does not apply to Telnyx, where the browser decodes audio before playback.

### Complete examples

Each adapter ships a runnable example with the Worker routes, provider configuration, and deployment steps:

<CardGrid>
<LinkCard
title="Plivo voice agent"
href="https://github.com/cloudflare/agents/tree/main/examples/plivo-voice-agent"
description="Answer inbound Plivo calls in a Worker, including the answer URL and application setup."
/>
<LinkCard
title="Telnyx voice agent"
href="https://github.com/cloudflare/agents/tree/main/examples/telnyx-voice-agent"
description="Bridge PSTN calls through the browser, including the JWT endpoint and client wiring."
/>
</CardGrid>

## Text messages

Expand Down
29 changes: 29 additions & 0 deletions src/content/docs/agents/runtime/communication/routing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,35 @@ export default {

</TypeScriptExample>

## Build Agent URLs

Use `buildAgentPath()` to create a pathname for a known Agent identity. The function handles the root route and each nested `/sub/` route.

<TypeScriptExample>

```ts
import { buildAgentPath, buildAgentUrl } from "agents";

const address = [
{ className: "Inbox", name: userId },
{ className: "Chat", name: chatId },
];

buildAgentPath(address, { leafPath: "/callbacks/job" });
// /agents/inbox/{userId}/sub/chat/{chatId}/callbacks/job

buildAgentUrl("https://app.example.com", address, {
leafPath: "/callbacks/job",
});
// URL("https://app.example.com/agents/inbox/...")
```

</TypeScriptExample>

Inside an Agent, `this.selfPath` provides the required root-first identity. If the root Durable Object binding name differs from its class name, pass the binding name as `rootBinding`. The pathname supports both HTTP requests and WebSocket connections.

For a custom route prefix, pass the same `prefix` to `buildAgentPath()` and `routeAgentRequest()`. Refer to [Sub-agents](/agents/runtime/execution/sub-agents/#direct-http-and-websocket-urls) for callback and webhook examples.

## Instance naming patterns

The instance name (the last part of the URL) determines which agent instance handles the request. Each unique name gets its own isolated agent with its own state.
Expand Down
55 changes: 54 additions & 1 deletion src/content/docs/agents/runtime/execution/sub-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,59 @@ const chat = useAgent({

The hook builds a URL like `/agents/inbox/user-123/sub/chat/chat-abc` and opens a direct WebSocket to the `Chat` child. Every other `useAgent` feature works as usual: state sync, `stub` calls, `@callable` RPC, and `useAgentChat` on top of the returned socket.

### Direct HTTP and WebSocket URLs

Use `buildAgentPath()` to create a canonical pathname for an Agent identity. The same pathname supports HTTP requests and WebSocket connections.

<TypeScriptExample>

```ts
import { buildAgentPath } from "agents";

const path = buildAgentPath(
[
{ className: "Inbox", name: userId },
{ className: "Chat", name: chatId },
],
{ leafPath: "/callbacks/job" },
);

// /agents/inbox/{userId}/sub/chat/{chatId}/callbacks/job
```

</TypeScriptExample>

Inside an Agent, pass `this.selfPath` directly. If the root Durable Object binding name differs from its class name, also pass `rootBinding` in the options. Use `buildAgentUrl()` to add a public origin for callbacks, webhooks, approvals, or asynchronous job completion.

<TypeScriptExample>

```ts
import { buildAgentUrl } from "agents";

export class Chat extends Agent<Env> {
callbackUrl() {
return buildAgentUrl(this.env.PUBLIC_ORIGIN, this.selfPath, {
leafPath: "/callbacks/job",
});
}

override async onRequest(request: Request) {
if (new URL(request.url).pathname === "/callbacks/job") {
return this.handleJobCallback(request);
}
return new Response("Not found", { status: 404 });
}
}
```

</TypeScriptExample>

Pass the incoming request to `routeAgentRequest()`. Each ancestor runs `onBeforeSubAgent` before the destination receives the request. For a sub-agent destination, routing removes the nested `/sub/` segments, so its pathname is the `leafPath` suffix.

`buildAgentUrl()` accepts an HTTP(S) or WS(S) origin. The origin cannot contain credentials, a pathname, a query, or a fragment. Add callback query parameters through the returned URL `searchParams` property.

Root Agent names must already be valid pathname segments. The `sub` segment is reserved in routing prefixes, class and binding names, and root Agent names. The helper URL-encodes descendant names, including spaces, Unicode characters, `/`, and other URL-reserved characters.

### Custom HTTP routing

For fetch handlers that do their own top-level URL parsing, use `routeSubAgentRequest()` to dispatch a request into a sub-agent from an already-resolved parent stub:
Expand All @@ -311,7 +364,7 @@ export default {

</TypeScriptExample>

`fromPath` takes the sub-agent tail, such as `/sub/chat/chat-abc`. The helper parses it, runs the parent's `onBeforeSubAgent` hook, and forwards the request into the facet.
`fromPath` takes any pathname that contains a sub-agent tail, such as `/sub/chat/chat-abc`. You can pass the result of `buildAgentPath()` directly. The helper parses it, runs the parent `onBeforeSubAgent` hook, and forwards the request into the facet.

### External typed RPC

Expand Down
Loading