diff --git a/README.md b/README.md index a56ba175..16ea9519 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ Shared base templates live in `templates/` (Expo, Next.js, Python, TypeScript). - [Real-Time Voice Agent (Next.js)](agents/nextjs/quickstart/example/README.md) — Live voice conversations with the ElevenLabs Agents Platform using the React Agents SDK. - [Voice Agent Guardrails Demo (Next.js)](agents/nextjs/guardrails/example/README.md) — Demonstrate custom guardrails and the `guardrail_triggered` client event in a live voice agent. - [OpenAI Realtime SDK with ElevenLabs (Next.js)](agents/nextjs/openai-realtime/example/README.md) — Talk to ElevenLabs Realtime using OpenAI's Realtime SDK pointed at the ElevenLabs OpenAI-compatible endpoint. +- [LiveKit Agents with Eleven Realtime (Python)](agents/python/livekit/example/README.md) — Talk to a LiveKit Agents console voice agent powered by ElevenLabs Realtime, with a function tool. - [Voice Isolator (Next.js)](voice-isolator/nextjs/quickstart/example/README.md) — Record your voice in the browser and remove background noise with the Voice Isolator API. - [Dubbing Recorder (Next.js)](dubbing/nextjs/quickstart/example/README.md) — Record your voice in the browser, dub it into another language, and play or download the result. diff --git a/agents/python/livekit/PROMPT.md b/agents/python/livekit/PROMPT.md new file mode 100644 index 00000000..34afb9bd --- /dev/null +++ b/agents/python/livekit/PROMPT.md @@ -0,0 +1,14 @@ +Before writing any code, invoke the `/agents` skill to learn the correct ElevenLabs SDK patterns. + +Build a LiveKit Agents console voice agent that uses ElevenLabs Realtime through `livekit-plugins-elevenlabs`. + +## `main.py` + +- Import everything at the top, then call `load_dotenv()`. The plugin reads `ELEVEN_API_KEY` from the environment. +- Create an `AgentServer` and an `@server.rtc_session()` entrypoint that sets `ctx.log_context_fields = {"room": ctx.room.name}`. +- Build an `AgentSession` with `llm=elevenlabs.realtime.RealtimeModel(model="eleven_realtime_v1_mini", voice="brian", language="en")`. +- Subclass `Agent` with short voice-first instructions (no formatting, emojis or symbols) that tell it to use `lookup_weather` for weather questions. +- Greet the user in `on_enter` with `self.session.generate_reply()`. +- Add a `lookup_weather(context: RunContext, location: str)` `@function_tool` with a Google-style docstring (it becomes the tool schema) that logs the lookup and returns a canned sunny forecast. +- Call `await session.start(agent=Assistant(), room=ctx.room)`, then `await ctx.connect()`. +- Run with `cli.run_app(server)` under `if __name__ == "__main__":`. diff --git a/agents/python/livekit/README.md b/agents/python/livekit/README.md new file mode 100644 index 00000000..5d73570e --- /dev/null +++ b/agents/python/livekit/README.md @@ -0,0 +1,42 @@ +# LiveKit Agents with Eleven Realtime (Python) + +Talk to a [LiveKit Agents](https://docs.livekit.io/agents/) voice agent powered by ElevenLabs Realtime, with a function tool, from your terminal. + +> This example depends on the unreleased `elevenlabs.realtime` module from [livekit/agents#8075](https://github.com/livekit/agents/pull/8075), so `requirements.txt` installs `livekit-plugins-elevenlabs` from that pull request's branch. + +## Setup + +1. Copy the environment file and add your API key: + + ```bash + cp .env.example .env + ``` + + Then edit `.env` and set `ELEVEN_API_KEY` to your [ElevenLabs API key](https://elevenlabs.io/app/settings/api-keys). Note the name: `livekit-plugins-elevenlabs` reads `ELEVEN_API_KEY`, not the `ELEVENLABS_API_KEY` the other examples use. + +2. Create a virtual environment and install dependencies: + + ```bash + python3 -m venv .venv + .venv/bin/pip install -r requirements.txt + ``` + +## Run + +```bash +.venv/bin/python main.py console +``` + +Console mode uses your local microphone and speakers, so no LiveKit server is needed. + +## Usage + +- The agent greets you when the session starts. +- Talk; the agent replies by voice. +- Ask about the weather somewhere to have the agent call the `lookup_weather` tool. +- Press `Ctrl+C` to end the session. + +## How it works + +- `elevenlabs.realtime.RealtimeModel` from `livekit-plugins-elevenlabs` is the session's `llm`, handling speech in and speech out with `eleven_realtime_v1_mini` and the `brian` voice. +- `lookup_weather` is a `@function_tool`; its docstring becomes the tool description the model sees. diff --git a/agents/python/livekit/example/.env.example b/agents/python/livekit/example/.env.example new file mode 100644 index 00000000..034425ad --- /dev/null +++ b/agents/python/livekit/example/.env.example @@ -0,0 +1,2 @@ +# livekit-plugins-elevenlabs reads ELEVEN_API_KEY, not the usual ELEVENLABS_API_KEY. +ELEVEN_API_KEY= diff --git a/agents/python/livekit/example/README.md b/agents/python/livekit/example/README.md new file mode 100644 index 00000000..5d73570e --- /dev/null +++ b/agents/python/livekit/example/README.md @@ -0,0 +1,42 @@ +# LiveKit Agents with Eleven Realtime (Python) + +Talk to a [LiveKit Agents](https://docs.livekit.io/agents/) voice agent powered by ElevenLabs Realtime, with a function tool, from your terminal. + +> This example depends on the unreleased `elevenlabs.realtime` module from [livekit/agents#8075](https://github.com/livekit/agents/pull/8075), so `requirements.txt` installs `livekit-plugins-elevenlabs` from that pull request's branch. + +## Setup + +1. Copy the environment file and add your API key: + + ```bash + cp .env.example .env + ``` + + Then edit `.env` and set `ELEVEN_API_KEY` to your [ElevenLabs API key](https://elevenlabs.io/app/settings/api-keys). Note the name: `livekit-plugins-elevenlabs` reads `ELEVEN_API_KEY`, not the `ELEVENLABS_API_KEY` the other examples use. + +2. Create a virtual environment and install dependencies: + + ```bash + python3 -m venv .venv + .venv/bin/pip install -r requirements.txt + ``` + +## Run + +```bash +.venv/bin/python main.py console +``` + +Console mode uses your local microphone and speakers, so no LiveKit server is needed. + +## Usage + +- The agent greets you when the session starts. +- Talk; the agent replies by voice. +- Ask about the weather somewhere to have the agent call the `lookup_weather` tool. +- Press `Ctrl+C` to end the session. + +## How it works + +- `elevenlabs.realtime.RealtimeModel` from `livekit-plugins-elevenlabs` is the session's `llm`, handling speech in and speech out with `eleven_realtime_v1_mini` and the `brian` voice. +- `lookup_weather` is a `@function_tool`; its docstring becomes the tool description the model sees. diff --git a/agents/python/livekit/example/main.py b/agents/python/livekit/example/main.py new file mode 100644 index 00000000..3cef1288 --- /dev/null +++ b/agents/python/livekit/example/main.py @@ -0,0 +1,62 @@ +import logging + +from dotenv import load_dotenv +from livekit.agents import ( + Agent, + AgentServer, + AgentSession, + JobContext, + RunContext, + cli, + function_tool, +) +from livekit.plugins import elevenlabs + +logger = logging.getLogger("agent") + +load_dotenv() + + +class Assistant(Agent): + def __init__(self) -> None: + super().__init__( + instructions="""You are a helpful voice AI assistant. The user is interacting with you via voice. + Keep your answers short and conversational, without any formatting, emojis or symbols. + When the user asks about the weather, use the lookup_weather tool.""", + ) + + async def on_enter(self) -> None: + self.session.generate_reply() + + @function_tool + async def lookup_weather(self, context: RunContext, location: str) -> str: + """Look up the current weather for a location. + + Args: + location: The city or place to look up. + """ + logger.info(f"looking up weather for {location}") + return f"It is sunny and 21 degrees Celsius in {location}." + + +server = AgentServer() + + +@server.rtc_session() +async def my_agent(ctx: JobContext) -> None: + ctx.log_context_fields = {"room": ctx.room.name} + + session = AgentSession( + llm=elevenlabs.realtime.RealtimeModel( + model="eleven_realtime_v1_mini", + voice="brian", + language="en", + ), + ) + + await session.start(agent=Assistant(), room=ctx.room) + await ctx.connect() + + +if __name__ == "__main__": + cli.run_app(server) diff --git a/agents/python/livekit/example/requirements.txt b/agents/python/livekit/example/requirements.txt new file mode 100644 index 00000000..b242dca7 --- /dev/null +++ b/agents/python/livekit/example/requirements.txt @@ -0,0 +1,3 @@ +livekit-agents>=1.8.10 +livekit-plugins-elevenlabs @ git+https://github.com/kraenhansen/livekit-agents@kraen/elevenlabs-realtime#subdirectory=livekit-plugins/livekit-plugins-elevenlabs +python-dotenv>=1.2.1 diff --git a/agents/python/livekit/setup.sh b/agents/python/livekit/setup.sh new file mode 100755 index 00000000..f10b016f --- /dev/null +++ b/agents/python/livekit/setup.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +set -euo pipefail + +DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$DIR/../../.." && pwd)" +cd "$DIR" + +if [ -d example ]; then + find example -mindepth 1 -maxdepth 1 ! -name .venv -exec rm -rf {} + +fi +mkdir -p example + +rsync -a \ + --exclude .venv \ + --exclude example \ + "$REPO_ROOT/templates/python/" example/ + +cp README.md example/README.md + +# TODO: pin a released livekit-plugins-elevenlabs once livekit/agents#8075 ships +cat > example/requirements.txt <<'EOF' +livekit-agents>=1.8.10 +livekit-plugins-elevenlabs @ git+https://github.com/kraenhansen/livekit-agents@kraen/elevenlabs-realtime#subdirectory=livekit-plugins/livekit-plugins-elevenlabs +python-dotenv>=1.2.1 +EOF +cat > example/.env.example <<'EOF' +# livekit-plugins-elevenlabs reads ELEVEN_API_KEY, not the usual ELEVENLABS_API_KEY. +ELEVEN_API_KEY= +EOF + +if [ -f "$DIR/.env" ]; then + cp "$DIR/.env" example/.env +fi + +cd example +python3 -m venv .venv +.venv/bin/pip install --upgrade pip +.venv/bin/pip install -r requirements.txt