From 6d0fd1a0540ff74b79ae938e1f28c7fc2444cb4a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kr=C3=A6n=20Hansen?= Date: Fri, 2 Oct 2026 14:34:29 +0200 Subject: [PATCH 1/6] Add community health files for the public repo Issue forms, PR template, CODEOWNERS and SECURITY.md (under .github/ so Unity doesn't import them), plus Support and License sections in the README. Templates follow elevenlabs-python's issue forms, adapted for Unity versions, build targets and logs. Closes #20 Co-Authored-By: Claude Opus 5.5 --- .github/CODEOWNERS | 1 + .github/ISSUE_TEMPLATE/1-bug_report.yml | 79 ++++++++++++++++++++ .github/ISSUE_TEMPLATE/2-feature_request.yml | 37 +++++++++ .github/ISSUE_TEMPLATE/config.yml | 11 +++ .github/SECURITY.md | 24 ++++++ .github/pull_request_template.md | 23 ++++++ README.md | 12 ++- 7 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 .github/CODEOWNERS create mode 100644 .github/ISSUE_TEMPLATE/1-bug_report.yml create mode 100644 .github/ISSUE_TEMPLATE/2-feature_request.yml create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 .github/SECURITY.md create mode 100644 .github/pull_request_template.md diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 0000000..4c965b9 --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1 @@ +* @kraenhansen diff --git a/.github/ISSUE_TEMPLATE/1-bug_report.yml b/.github/ISSUE_TEMPLATE/1-bug_report.yml new file mode 100644 index 0000000..620d777 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/1-bug_report.yml @@ -0,0 +1,79 @@ +name: Bug report +description: Create a bug report for the ElevenAgents Unity SDK +labels: [bug] +body: + - type: markdown + attributes: + value: | + Thank you for reporting a bug with the **ElevenAgents Unity SDK**. Please fill out the sections below to help us understand and resolve the issue. + + **Never paste an ElevenLabs API key, signed URL, or conversation token** into this issue. + + - type: textarea + attributes: + label: Description + description: | + **Describe the bug in detail and provide clear steps to reproduce it.** + + Include information such as: + - What you were trying to achieve. + - What happened instead. + - Any relevant `ConversationOptions` or agent configuration. + placeholder: | + **Steps to reproduce:** + 1. ... + 2. ... + + **Expected behavior:** + + **Actual behavior:** + validations: + required: true + - type: input + attributes: + label: Package version + description: The `io.elevenlabs.agents` version, or the git commit from `Packages/packages-lock.json`. + placeholder: 0.1.0 (or a commit SHA) + validations: + required: true + - type: input + attributes: + label: Unity version + placeholder: 6000.3.6f1 + validations: + required: true + - type: dropdown + attributes: + label: Where does it happen? + multiple: true + options: + - Editor (Play Mode) + - Standalone (macOS) + - Standalone (Windows) + - Standalone (Linux) + - WebGL + - iOS + - Android + - XR + - Other + validations: + required: true + - type: textarea + attributes: + label: Code example + description: A minimal snippet (or a link to a small project) that reproduces the problem. + render: csharp + - type: textarea + attributes: + label: Logs + description: | + Relevant output from the Unity Console, `Editor.log` / `Player.log`, or the browser console for WebGL builds. + render: text + - type: textarea + attributes: + label: Additional context + description: Add any other context or screenshots about the problem here. + placeholder: | + - Related issues: + - Possible workaround: + - ... diff --git a/.github/ISSUE_TEMPLATE/2-feature_request.yml b/.github/ISSUE_TEMPLATE/2-feature_request.yml new file mode 100644 index 0000000..81ee730 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/2-feature_request.yml @@ -0,0 +1,37 @@ +name: Feature request +description: Propose a new feature for the ElevenAgents Unity SDK +labels: [enhancement] +body: + - type: markdown + attributes: + value: | + Thank you for taking the time to propose a new feature for the **ElevenAgents Unity SDK**. Please provide detailed information below to help us understand your proposal. + + If the feature exists in another ElevenLabs SDK (such as `@elevenlabs/client`, Swift, or Android), mention it below. We aim for parity across SDKs. + + - type: textarea + attributes: + label: Feature description + description: What is the feature, what problem does it solve, and how do you envision it working? + placeholder: | + - What is the feature? + - What problem does it solve? + - How do you envision it working? + validations: + required: true + - type: textarea + attributes: + label: Use case + description: Describe the game or app scenario where this feature would be useful, including the target platforms. + - type: textarea + attributes: + label: Alternatives considered + description: Other ways you've tried or considered solving the problem, and why they fall short. + - type: textarea + attributes: + label: Additional context + description: Any extra information, references, or screenshots that might help us understand your feature request. + placeholder: | + - Related issues or discussions: + - Equivalent API in another ElevenLabs SDK: + - Relevant links: diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..1e1e0d8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,11 @@ +blank_issues_enabled: true +contact_links: + - name: Report a security vulnerability + url: https://github.com/elevenlabs/unity/security/advisories/new + about: Please report vulnerabilities privately, not as a public issue. See SECURITY.md. + - name: ElevenAgents documentation + url: https://elevenlabs.io/docs/eleven-agents/overview + about: Agent configuration, voices, tools, and platform questions. + - name: Ask a question (Discord) + url: https://discord.com/invite/elevenlabs + about: General questions and community help. diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 0000000..a8c8930 --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,24 @@ +# Security policy + +## Reporting a vulnerability + +Please do not report security vulnerabilities through public GitHub issues, discussions, or pull requests. + +Report them privately through either channel: + +- **GitHub:** [Report a vulnerability](https://github.com/elevenlabs/unity/security/advisories/new) on this repository (Security tab → _Report a vulnerability_). +- **ElevenLabs Vulnerability Disclosure Program:** [elevenlabs.io/security/report](https://elevenlabs.io/security/report) or [vulnerability_disclosure_program@elevenlabs.io](mailto:vulnerability_disclosure_program@elevenlabs.io). + +Include the affected package version (or commit), the Unity version and build target, and steps to reproduce. We will acknowledge your report, investigate, and keep you updated on the fix. + +ElevenLabs does not offer cash rewards for vulnerability reports at this time. + +## Supported versions + +The SDK is pre-1.0. Security fixes land on `main` and ship in the next release; older releases are not patched. + +## Scope + +This policy covers the code in this repository: the Unity package (`Runtime/`, `Editor/`, `Plugins/`, `Samples~/`) and its build tooling. Vulnerabilities in the ElevenLabs platform or API itself should go to the ElevenLabs Vulnerability Disclosure Program above. + +Never commit or share an ElevenLabs API key in a Unity project. Players can extract anything shipped in a build, so use a public agent or have your own backend mint signed URLs or conversation tokens. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..9ca114c --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,23 @@ + + +## Changes + + + +- + +## Verification + + + +- [ ] `dotnet csharpier check .` +- [ ] `Bridge~` checks (`format:check`, `lint`, `typecheck`, `test`, `verify:primitives`, `verify:connection`) +- [ ] `Codegen~` checks (`verify:protocol-dtos`, `round-trip`) +- [ ] Edit Mode tests (`pnpm --dir TestProject run test`) +- [ ] WebGL builds + browser integration tests (`IntegrationTests~`) +- [ ] Manually tested in a Unity project. Platforms and Unity version: + +## Checklist + +- [ ] `CHANGELOG.md` updated under `[Unreleased]` for user-facing changes +- [ ] Public API changes are called out above diff --git a/README.md b/README.md index db78d5b..abddf39 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ > [!CAUTION] > ⚠️ **Early-stage SDK:** This package is under active development and APIs may change without notice. If you hit a bug or rough edge, please [open a GitHub issue](https://github.com/elevenlabs/unity/issues/new) - your reports directly shape what we fix next. -Drop ElevenLabs voice agents into a Unity scene. Walk up to a cube, it greets you in character; walk out, the session ends — all driven by the same `Conversation` API on standalone, mobile, the Editor, and WebGL. +Drop ElevenLabs voice agents into a Unity scene. Walk up to a cube, it greets you in character; walk out, the session ends — all driven by the same `Conversation` API in the Editor, desktop standalone players, and WebGL. @@ -38,3 +38,13 @@ Import via **Window → Package Manager → ElevenAgents → Samples**. - [`Docs~/ARCHITECTURE.md`](./Docs~/ARCHITECTURE.md) — internal layout (transports, audio pipeline, codegen) for contributors. - [`CONTRIBUTING.md`](./CONTRIBUTING.md) — how to build, test, and submit changes. - [`CHANGELOG.md`](./CHANGELOG.md) — release notes. + +## Support + +- Bugs and feature requests: [GitHub issues](https://github.com/elevenlabs/unity/issues/new/choose). +- Security vulnerabilities: report privately per [`SECURITY.md`](./.github/SECURITY.md). +- Agent configuration and platform questions: [ElevenAgents docs](https://elevenlabs.io/docs/eleven-agents/overview) or [Discord](https://discord.com/invite/elevenlabs). + +## License + +[MIT](./LICENSE) From d0fd56129fc4ab01124229b1d69673c595d7b758 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kr=C3=A6n=20Hansen?= Date: Fri, 2 Oct 2026 14:34:30 +0200 Subject: [PATCH 2/6] Bring public docs in line with the shipped SDK CHANGELOG 0.1.0 lists the API that actually exists, and folds in the Unity 6.3 minimum (never released at 6000.0). ARCHITECTURE, CONTRIBUTING and COMPATIBILITY describe the shipped native transport, asmdef layout, test layers and builds, and limit platform claims to what's tested. CONTRIBUTING gains a "Submitting changes" section. Co-Authored-By: Claude Opus 5.5 --- .claude/CLAUDE.md | 4 +- CHANGELOG.md | 23 +-- COMPATIBILITY.md | 20 +- CONTRIBUTING.md | 86 ++++++--- Docs~/ARCHITECTURE.md | 346 ++++++++++++++++++---------------- Samples~/QuickStart/README.md | 9 +- 6 files changed, 276 insertions(+), 212 deletions(-) diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index b4d1947..32cb8bb 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -46,7 +46,7 @@ pnpm --dir Codegen~ run format:check # Prettier pnpm --dir Codegen~ run lint # ESLint pnpm --dir Codegen~ run typecheck # TypeScript pnpm --dir Codegen~ run generate # Emit C# DTOs + CSharpier-format them -pnpm --dir Codegen~ run round-trip # Serialize/deserialize each DTO via System.Text.Json +pnpm --dir Codegen~ run round-trip # Serialize/deserialize each DTO via Newtonsoft.Json pnpm --dir Codegen~ run verify:protocol-dtos # Regenerate and assert no git drift ``` @@ -61,7 +61,7 @@ emit the models (same stack the JS SDK's codegen uses). On top of that: `{ type: string, const: foo }`, strip `enum` from string fields (keep them plain `string` instead of generating C# enums), break cycles introduced by the parser's reference-based inlining of recursive schemas. -- Custom C# preset: PascalCase property names, `[JsonPropertyName("…")]` +- Custom C# preset: PascalCase property names, `[JsonProperty("…")]` attributes, `const` schemas → `init`-only string literals, required reference types get `= null!` to silence CS8618. - Post-process: inject `: IncomingSocketEvent` / `: OutgoingSocketEvent` for diff --git a/CHANGELOG.md b/CHANGELOG.md index 8cc6b75..828e7bc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,21 +7,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -### Changed - -- **Breaking:** Minimum Unity version bumped from `6000.0` to `6000.3` (Unity 6.3 LTS). Required for the new native audio output engine built on [`Audio.IAudioGenerator`](https://docs.unity3d.com/6000.3/Documentation/ScriptReference/Audio.IAudioGenerator.html) / [`Audio.GeneratorInstance`](https://docs.unity3d.com/6000.3/Documentation/ScriptReference/Audio.GeneratorInstance.html), which eliminates the streaming-`AudioClip` pre-fill latency that older Unity versions imposed structurally. Unity 6.3 LTS is supported through December 2027. - ## [0.1.0] - Unreleased ### Added -- Core session management (`StartSession`, `EndSession`) -- WebSocket transport for native platforms (desktop, mobile, XR) -- `@elevenlabs/client` bridge transport for WebGL -- Audio capture and playback (default mode) across all platforms -- Core conversation events: `OnConnect`, `OnDisconnect`, `OnMessage`, `OnError`, `OnStatusChange`, `OnModeChange` -- WebGL bridge primitives: Promise-as-Task, Observer, JS-initiated Promise -- UPM distribution via git URL +- `Conversation` session API: `Conversation.StartSessionAsync(ConversationOptions)` / `EndSession()`, with `ConversationId`, `Status`, `Mode`, and `CanSendFeedback`. Async members return Unity's `Awaitable`. +- `ConversationOptions`: `AgentId` / `SignedUrl` / `ConversationToken`, `ConnectionType` (WebSocket, or WebRTC on WebGL), `Input` / `Output` device config, `OutputAudioSource`, `DynamicVariables`, `Overrides`, `UserId`, `CustomLlmExtraBody`, `EnableDebugLogging`. +- Events: `Connected`, `Disconnected`, `ErrorOccurred`, `StatusChanged`, `ModeChanged`, `CanSendFeedbackChanged`, `InitiationMetadataReceived`, `UserTranscriptReceived`, `AgentResponded`, `AgentResponseCorrected`, `AgentResponseCompleted`, `AgentChatResponsePartReceived`, `AudioReceived`, `Interrupted`, `VadScoreUpdated`, `GuardrailTriggered`, `UnhandledClientToolCall`, `AgentToolRequested`, `AgentToolResponded`, `MCPToolCallReceived`, `MCPConnectionStatusChanged`. +- Messaging: `SendUserMessage`, `SendContextualUpdate`, `SendUserActivity`, `SendMultimodalMessage`, `UploadFileAsync`, `SendFeedback`, `SendMCPToolApprovalResult`. +- Client tools: `RegisterTool` (sync and `Awaitable` handlers), `UnregisterTool`, and `ClientToolException` for reporting tool errors to the agent. +- Audio control: `SetVolume`, `SetMicMuted`, `ChangeInputDevice`, `ChangeOutputDevice`, `GetInputVolume` / `GetOutputVolume`, `GetInputByteFrequencyData` / `GetOutputByteFrequencyData`. +- Native transport for the Editor and desktop standalone players: WebSocket via `ClientWebSocket`, microphone capture via `UnityEngine.Microphone`, and low-latency playback through Unity 6.3's `IAudioGenerator`, optionally through a supplied (spatial) `AudioSource`. +- WebGL transport wrapping `@elevenlabs/client` (WebSocket and WebRTC). WebSocket audio plays through a Web Audio graph that mirrors the supplied `AudioSource`'s volume, position, and spatial settings. +- WebGL JS↔C# bridge primitives: `JsBridge`, `JsObject`, `JsFunction`, `BridgeCallback`, `BridgeException`. +- Protocol DTOs and typed event-args records (`ElevenLabs.Protocol`), generated from the Agents AsyncAPI spec. +- Samples: GettingStarted (walk-up-and-talk with spatial audio, dynamic variables, and overrides) and QuickStart (minimal transcript). +- UPM distribution via git URL. Requires Unity 6.3 LTS (`6000.3`) or later; depends on Input System 1.18.0 and Newtonsoft.Json 3.2.1. [Unreleased]: https://github.com/elevenlabs/unity/compare/v0.1.0...HEAD [0.1.0]: https://github.com/elevenlabs/unity/releases/tag/v0.1.0 diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 5d48f45..f0fd692 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -20,7 +20,19 @@ The SDK reads JS→C# callback payloads on WebGL via [`Marshal.PtrToStringUTF8`] ## Platform support -The public API (`Runtime/`) is cross-platform. WebGL is the most mature target today; native targets (desktop, mobile, XR) are implemented under `Runtime/Native/` and ship in the same package — the right implementation is selected at compile time per platform. +The public API (`Runtime/Core/`) is cross-platform. Two transports ship in the package, and the one matching the active build target is registered automatically: + +| Target | Transport | Connection types | Status | +|---|---|---|---| +| WebGL | `@elevenlabs/client` via the JS bridge | WebSocket, WebRTC | Supported, CI-tested in Chromium | +| Editor (Play Mode), desktop standalone (macOS, Windows, Linux) | Native (`ClientWebSocket`, `UnityEngine.Microphone`, `IAudioGenerator`) | WebSocket only — `ConnectionType.WebRTC` throws `NotSupportedException` | Supported | +| iOS, Android, XR | Native | WebSocket only | Untested. The SDK doesn't request microphone permission itself. | + +Editor Play Mode while the active build target is WebGL isn't supported yet ([#16](https://github.com/elevenlabs/unity/issues/16)); switch the build target to a desktop platform to iterate in the Editor. + +## Package dependencies + +Installed automatically by the Package Manager: `com.unity.inputsystem` 1.18.0 and `com.unity.nuget.newtonsoft-json` 3.2.1. ## Additional WebGL setup @@ -51,9 +63,9 @@ When you pass an `AudioSource` to `ConversationOptions.OutputAudioSource`, the f | `maxDistance` | `PannerNode.maxDistance` | Direct | | `rolloffMode` (Linear / Logarithmic) | `PannerNode.distanceModel` | `AudioRolloffMode.Custom` falls back to Logarithmic with a one-time warning | | `panStereo` | `StereoPannerNode.pan` | Effective when `spatialBlend < 1` | -| `dopplerLevel` | Sampled per-frame velocity → `PannerNode` position pre-emphasis | Approximate | +| `dopplerLevel` | ❌ Not supported | Forwarded but has no effect — Web Audio has no doppler control | | `outputAudioMixerGroup` | ❌ Not supported | One-time warning when first observed as non-null | -| Custom rolloff `AnimationCurve` | ❌ Not supported | One-time warning; falls back to `rolloffMode` | +| Custom rolloff `AnimationCurve` | ❌ Not supported | Falls back to Logarithmic (see `rolloffMode` above) | | `bypassEffects`, `bypassListenerEffects`, `bypassReverbZones` | ❌ Not supported | Silently ignored — no Web Audio analog | | `priority`, `spread`, `reverbZoneMix` | ❌ Not supported | Silently ignored | @@ -70,6 +82,6 @@ Use the SDK's cross-platform reading APIs instead — they're tuned to return co | What you want | Native-only API (silent on WebGL) | Cross-platform SDK API | |---|---|---| | Scalar volume `[0, 1]` for an envelope / meter / bob | `audioSource.GetOutputData(buf); rms(buf);` | `conversation.GetOutputVolume()` | -| Frequency-domain magnitudes | `AudioSource.GetSpectrumData(...)` | `conversation.GetByteFrequencyData(buf)` | +| Frequency-domain magnitudes | `AudioSource.GetSpectrumData(...)` | `conversation.GetOutputByteFrequencyData(buf)` | The shipped samples under `Samples~/` follow this rule — copy from them. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 65640eb..e9bcef8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,20 +11,24 @@ Runtime/ C# source compiled into the shipped package internal IConnection / IInputController / IOutputController abstractions the platform impls satisfy, and the generated protocol DTOs under Protocol/*.g.cs. No platform restrictions. - WebGL/ Editor + WebGL asmdef `ElevenLabs.Agents.WebGL`. Carries the bridge - primitives (JsObject, JsFunction, BridgeCallback handle types + their + WebGL/ Bridge primitives (JsObject, JsFunction, BridgeCallback handle types + their IJsObject / IJsFunction interface seams) and the Bridged/ folder of IConnection / IInputController / IOutputController implementations that - wrap @elevenlabs/client. - (Native/ Phase 7 — not yet present. Will mirror Runtime/WebGL/ as an - excludePlatforms=WebGL asmdef carrying native IConnection / - IInputController / IOutputController implementations.) - -Editor/ Editor-only C# (HostBuild entry points for WebGL builds, inspectors, - build post-processors, validation). + wrap @elevenlabs/client. Compiled by the Editor + WebGL asmdef + `ElevenLabs.Agents.WebGL`, which sits at `Runtime/` itself — so a new file + dropped directly into `Runtime/` lands in that assembly. + Native/ excludePlatforms=WebGL asmdef `ElevenLabs.Agents.Native`: ClientWebSocket + connection, UnityEngine.Microphone input, IAudioGenerator-based output + (UnityAudioSourceOutput + the IAudioOutputEngine implementations), and + NativeSessionLauncher. + +Editor/ Editor-only C#: HostBuild entry points for the WebGL and standalone smoke + builds, a build preprocessor that validates WebGL Player Settings, and the + link.xml injector. Tests/ - Editor/ Unity Test Runner tests (Edit Mode — run headlessly in CI). + Editor/ Edit Mode tests (run headlessly in CI). Native/ is its own asmdef. + Runtime/Native/ PlayMode characterization tests for Unity's audio APIs (run manually). Plugins/ WebGL/ .jslib bundles — Unity's required location for WebGL native plugins. @@ -70,7 +74,7 @@ TestProject/ Embedded dev Unity host project. Used to (a) run the Edit Mode integration tests. Not shipped with the package. Most of Assets/ is gitignored so a fresh clone always opens to a known-good state. -IntegrationTests~/ Vitest + Playwright tests that drive the two WebGL builds produced by +IntegrationTests~/ Vitest + Playwright tests that drive the WebGL builds produced by TestProject/ from a headless Chromium. Runs entirely outside Unity. Docs~/ Design documents and RFCs. Read these before making architectural changes; @@ -102,7 +106,7 @@ with **pnpm**. Run commands with `pnpm --dir run