-
Notifications
You must be signed in to change notification settings - Fork 17
Multiple Backends
Kai supports four agent backends. Instead of being locked to Anthropic's Claude Code CLI, Kai can route conversations through Goose or OpenCode (both via the Agent Client Protocol), or OpenAI's Codex CLI (via its app-server protocol). Between them they cover providers like OpenAI, Google, DeepSeek, OpenRouter, and local models via Ollama. This page explains why that matters, how the backends differ, and how to configure them.
Kai was built on Claude Code and it remains the default. Claude is an excellent coding agent. But running a personal assistant on a single vendor's subscription plan means inheriting every problem that vendor has, with no fallback. Over the past year, those problems have stacked up.
The most persistent complaint in the Claude ecosystem is the perception that model quality degrades over time without announcement. Users describe a pattern: a model works well for weeks, then responses become shallower, tool use gets sloppier, and code changes are incomplete or wrong. Anthropic has never confirmed intentional downgrades, but the complaints are widespread and recurring across Reddit, Hacker News, and GitHub.
March 2026 was particularly bad. Anthropic's own status page documented multiple infrastructure incidents across the month. Users reported Claude Code "lying about changes it made to code" - claiming edits were complete when they were partial or incorrect. GitHub issues described sessions producing 50+ consecutive failures, with some hitting thousands of errors in a single session. The community characterized it as a shift from "collaborating with a senior expert" to "supervising an unreliable worker who lies about the work."
Whether these episodes reflect genuine model changes, infrastructure degradation, or capacity-related routing to weaker model variants is unknowable from the outside. What matters for a tool like Kai is that they happen, they're unpredictable, and when they happen, your assistant is useless until they resolve.
Claude's subscription plans use rolling 5-hour token windows rather than monthly quotas:
| Plan | Price | Approximate token budget per 5-hour window |
|---|---|---|
| Pro | $20/month | ~44,000 tokens |
| Max 5x | $100/month | ~88,000 tokens |
| Max 20x | $200/month | ~220,000 tokens |
When you hit the limit, new prompts are blocked until the window resets. There is no graceful degradation - it is a hard cutoff.
The numbers above are approximate because Anthropic does not publish exact token budgets. Both Pro and Max plans also have weekly limits across all models combined, and Max plans carry a separate weekly Sonnet-specific cap. Anthropic reserves the right to impose additional caps at its discretion. In July 2025, multiple Max tier users ($200/month) reported that their usage limits were quietly lowered without announcement, hitting "Claude usage limit reached" errors during what appeared to be infrastructure disruptions.
A separate finding from early 2026 showed Claude Code rate limits hitting developers at roughly 6% of their apparent quota, suggesting that the effective limits for agentic workloads (where each "message" involves many internal tool calls) are much lower than what the raw token numbers imply.
For Kai specifically, rate limits are a structural problem. An always-on personal assistant that handles PR reviews, scheduled jobs, and interactive conversations throughout the day will consume tokens continuously. A hard cutoff during a PR review or a time-sensitive task is not just inconvenient - it is a service outage.
On March 31, 2026, Anthropic accidentally published the entire Claude Code source - 512,000 lines across 1,906 TypeScript files - to the public npm registry. The code exposed 44 internal feature flags, anti-distillation measures designed to poison competitor training data, an "Undercover Mode" for leak prevention, and various internal codenames. Anthropic called it human error. The code was mirrored within hours and remains available despite takedown attempts.
The leak itself did not affect Kai's operation, but it eroded trust in Anthropic's engineering practices and raised questions about what other undisclosed behaviors might exist in the models and tooling.
For individual developers, subscription plans are more economical than API usage if your consumption is consistent. The break-even point is roughly $100/month in API-equivalent usage for Max 5x, and $200/month for Max 20x. Below those thresholds, or for bursty workloads, direct API access with pay-per-token pricing can be cheaper and comes without the rolling window restrictions.
But the comparison is not just Anthropic vs. Anthropic. Competing services like OpenAI and Google offer their own models at different price points with different rate limit structures. OpenRouter aggregates dozens of providers and models behind a single API. Ollama runs models locally with no API costs or rate limits at all (at the expense of hardware requirements and model capability). Having a single backend locks you into one vendor's pricing and limits even when alternatives might better suit a particular workload.
Every issue above - quality regressions, rate limits, pricing changes - is a problem you can work around. The one you cannot work around is Anthropic deciding to change or remove the access model entirely.
Kai depends on the Claude Code CLI, which runs under Anthropic's subscription plans. That CLI access is not guaranteed by any long-term contract. Anthropic can change the terms, restrict the feature set, or retire the subscription model at any point. They have already demonstrated willingness to make breaking changes to third-party tooling: in February 2026, they banned OAuth-based access for tools that spoofed Claude Code headers, cutting off several popular wrappers overnight. Kai was not affected (it runs the real CLI as a subprocess, which is explicitly permitted), but the episode showed how quickly the rules can shift.
The scenario worth planning for is not dramatic. Anthropic does not need to shut down Claude Code. They could move agentic CLI access to an enterprise tier. They could require interactive authentication that breaks headless usage. They could introduce per-session fees that make always-on operation uneconomical. They could deprecate stream-json mode in favor of a protocol that requires a different integration. Any of these would effectively kill a single-backend Kai, not through malice, but through ordinary product evolution that does not account for this use case.
This is not speculation about unlikely events. It is the normal lifecycle of a platform dependency. Companies pivot, reprice, and sunset features constantly. The only reliable mitigation is not depending on a single one.
None of the above means Claude is a bad model. For many tasks it remains the strongest coding agent available. The argument for multiple backends is not "Claude is bad" - it is "depending entirely on one vendor for an always-on assistant is fragile." When Claude has a bad day, you want the option to route to GPT or Gemini without reconfiguring your entire setup. When rate limits bite during a PR review, you want a fallback that keeps working. When a provider's pricing changes, you want leverage.
Kai's multi-backend architecture makes the provider a configuration choice rather than a structural dependency. You can run Claude as your daily driver and keep another backend configured for a specific user. You can run Ollama locally for tasks that do not need frontier intelligence. You can switch models mid-conversation with /model. The plumbing is the same either way.
Kai's multi-backend system is built around a single abstraction: the AgentBackend interface. Every backend - Claude Code CLI, Codex, Goose, OpenCode - implements the same set of methods. The bot and the subprocess pool never interact with a specific backend directly. They program against the interface, and the concrete implementation handles the protocol details underneath.
This section walks through each layer of the architecture, from the abstract interface at the top to the wire protocols at the bottom.
The AgentBackend abstract base class (defined in backend.py) is the contract that all backends must fulfill. It defines:
Mutable state set by the pool during operation:
-
model- the current LLM model name (e.g., "sonnet", "gpt-5.5") -
workspace- the directory the backend operates in -
home_workspace- Kai's home directory (for context injection decisions) -
provider- the LLM provider identifier (e.g., "anthropic", "openai") -
timeout_seconds- tuning knob -
max_session_hours- session-age recycling limit (the age helpers are concrete methods on the base class)
Abstract methods every backend must implement:
-
send(prompt)- the core interface. Accepts a string or structured prompt, yieldsStreamEventobjects as the response streams in, and produces a finalAgentResponsewith the complete text. -
change_workspace(path)- kills the running process and points the backend at a new directory. -
restart()- kills the process so the nextsend()spawns a fresh one. -
shutdown()- graceful teardown. -
force_kill()- immediate SIGKILL, safe to call without holding the lock.
Protocol types shared across all backends:
-
StreamEvent- a partial update with accumulated text so far, adoneflag, and an optionalAgentResponseon the final event. -
AgentResponse- the final result: success/failure, full text, session ID, and error message if something went wrong.
The interface is deliberately minimal. It does not expose tool use, file edits, or any backend-specific features. From the pool's perspective, a backend is a box that accepts prompts and streams text back.
The two ACP backends (Goose and OpenCode) additionally share a concrete base class, AcpBackend (in acp.py), which owns the JSON-RPC transport, lifecycle, timeouts, per-user sudo isolation, and cross-user process cleanup. Each adapter overrides only a narrow hook surface (argv, env, handshake params, stream parsing), so the two ACP backends cannot drift in their transport behavior.
SubprocessPool (in pool.py) manages one backend instance per user. It handles:
Lazy creation. Instances are not created until a user sends their first message. At that point, the pool reads the user's configuration (from users.yaml and the database) to decide which backend to instantiate and with what settings.
Backend selection. The pool checks the user's backend setting (per-user override or global default) and instantiates the matching backend: claude (default), codex, goose, or opencode.
Model resolution. The model for a new instance follows a cascade:
- Per-user database override (set via
/modelor/settings model) - Per-user YAML config (from
users.yaml) - Workspace config (from
workspaces.yaml) - Global default model (from
.env) - Per-backend or per-provider default (e.g., "sonnet" for claude, "gpt-5.5" for codex)
Workspace restore. On first use after a restart, the pool loads the user's saved workspace from the database, validates that they still have access to it, applies any workspace-specific overrides, and hands it to the backend.
Idle eviction. A background loop checks every 60 seconds for instances that have been idle longer than AGENT_IDLE_TIMEOUT (default: 30 minutes). Idle instances are gracefully shut down to free resources. This matters on hardware like a Mac mini where memory is limited.
Session-age recycling. When AGENT_MAX_SESSION_HOURS is set, every backend's subprocess is recycled after the configured age (checked between interactions, never mid-response). Long-lived agent CLI processes can accumulate memory; the next message after a recycle starts a fresh session.
Message routing. When a message arrives, pool.send() acquires the user's instance, restores the workspace if needed, and delegates to instance.send(). The pool does not know or care which backend the instance is.
One of the key design decisions is that context injection is shared across all backends. The functions that build the session prefix live in backend.py, not in any specific backend implementation.
When a backend processes its first message in a new session, it prepends a block of context that includes:
-
Identity - Kai's
CLAUDE.mdfrom the home workspace, which defines personality, rules, and capabilities. -
Memory - the per-user
MEMORY.mdcontents when semantic memory is disabled; when it is enabled, relevant facts are retrieved from the vector store per turn instead (see Memory). - Conversation history - recent messages from the JSONL history log, so the backend has continuity even after a process restart.
- Workspace system prompt - if the current workspace has a custom system prompt configured.
- API documentation - instructions for the scheduling API, messaging API, file API, and external services, so the backend knows how to use Kai's infrastructure.
On every subsequent message (not just the first), if the backend is operating in a workspace other than home, a foreign workspace reminder is prepended. This reminder tells the backend which workspace it is in and reinforces behavioral rules for non-home contexts.
This unified injection means that switching a user between backends does not change their identity, memory, or conversation continuity. The backend is the engine; the context is the personality.
The same uniformity applies to Kai's one-shot agents: PR review, issue triage, and memory extraction all dispatch per user through the backend the user is configured for. No one-shot surface falls back to a different backend's CLI.
ClaudeCodeBackend (in claude.py) wraps Anthropic's Claude Code CLI in stream-json mode. It is the original backend and remains the default.
Wire protocol: stream-json. Communication happens over stdin/stdout using newline-delimited JSON:
→ {"type": "user", "message": {"role": "user", "content": [...]}}
← {"type": "system", ...} // session metadata
← {"type": "assistant", ...} // streaming text chunks
← {"type": "result", ...} // final response with session ID
Each message is a self-contained JSON object. The backend reads lines from stdout, parses the type field, and translates to StreamEvent objects.
Process lifecycle. A single claude subprocess is spawned per user via _ensure_started(). The command includes flags for stream-json I/O, the selected model, the reasoning-effort level, and permission bypass (required for non-interactive use). If os_user is configured for the user, the process is spawned via sudo -H -u for OS-level isolation between users.
Save on shutdown. Before terminating, the backend sends a save prompt asking the agent to persist important context to memory. This preserves learned information across restarts. The other backends have no equivalent of this prompt; with semantic memory enabled, the per-message extraction pipeline covers durable fact persistence on every backend, so the save prompt is a claude-specific extra rather than a required feature.
Capabilities unique to Claude Code:
- Full multimodal support (images, files, structured content) forwarded verbatim
- Reasoning-effort tuning via the
--effortflag (CLAUDE_EFFORT_LEVEL) - Auto-compaction percentage override
- Save-to-memory on shutdown
- Subscription (Pro/Max) authentication; no API key required
CodexBackend (in codex.py) wraps OpenAI's Codex CLI via its app-server JSON-RPC protocol. It is OpenAI-only and authenticates through either a ChatGPT subscription or an API key.
Wire protocol: JSON-RPC 2.0 (app-server). The codex app-server uses its own thread/turn/item vocabulary:
→ {"method": "initialize", ...} // handshake
→ {"method": "initialized"} // notification
→ {"method": "thread/start", ...} // create the conversation thread
→ {"method": "turn/start", "params": {"threadId": ..., "input": [...]}}
← {"method": "item/agentMessage/delta", ...} // streaming text
← {"method": "item/completed", ...} // authoritative item text
← {"method": "turn/completed", ...} // terminal event per turn
The thread persists across messages; each message is a new turn. The model is pinned per turn, so /model switches take effect without restarting the thread.
Authentication. CODEX_AUTH_MODE=subscription (default) uses the ChatGPT login state in ~/.codex/auth.json; CODEX_AUTH_MODE=api_key uses OPENAI_API_KEY. With per-user os_user isolation, each OS account holds its own ~/.codex/auth.json.
Process lifecycle. A single codex app-server subprocess per user, spawned with approvalPolicy: never (no human in the loop to approve tool calls from Telegram) and full sandbox access (the surrounding per-user OS isolation is the security boundary). Set CODEX_BIN to pin the binary path; the per-user sudoers rule references the same path. Reasoning effort is tunable via CODEX_EFFORT_LEVEL (minimal/low/medium/high/xhigh, passed as a -c model_reasoning_effort config override on the spawn); when unset, no override is passed and each OS user's own ~/.codex/config.toml or the model default stays in charge.
Images. The app-server takes image input as local file paths, so Kai materializes attached images to temporary files for the duration of the turn and cleans them up afterward.
GooseBackend (in goose.py) wraps Block's Goose agent via the Agent Client Protocol (ACP). It enables access to any LLM provider that Goose supports.
Wire protocol: JSON-RPC 2.0 (ACP). Communication uses the ACP standard, which is JSON-RPC over stdin/stdout:
→ {"jsonrpc": "2.0", "method": "initialize", ...} // handshake
← {"jsonrpc": "2.0", "result": {...}, "id": 1} // handshake response
→ {"jsonrpc": "2.0", "method": "session/new", ...} // create session
← {"jsonrpc": "2.0", "result": {"sessionId": "..."}, ...}
→ {"jsonrpc": "2.0", "method": "session/prompt", ...} // send prompt
← {"jsonrpc": "2.0", "method": "session/update", ...} // streaming (notification)
← {"jsonrpc": "2.0", "result": {...}, "id": 3} // final response
The handshake (initialize + session/new) runs once when the process starts. Each prompt is a session/prompt request. Streaming updates arrive as notifications (no id field), and the final response is a result with a matching request ID.
Model translation. When the Goose provider is Anthropic, Kai's short model names are mapped to Anthropic's model IDs (e.g., "sonnet" becomes "claude-sonnet-4-6"), and full claude-* IDs pass through unchanged. For other providers, model names are passed through as-is, so you use the provider's native naming. Kai's deepseek provider key is translated to Goose's wire-level name (custom_deepseek) transparently.
Supported providers: anthropic, deepseek, google, ollama, openai, openrouter.
Process lifecycle. A single goose acp --with-builtin developer subprocess is spawned per user. The provider and model are configured via environment variables (GOOSE_PROVIDER, GOOSE_MODEL, and the provider's API key variable). When the user has an os_user in users.yaml, the spawn is wrapped in sudo -H -u <os_user> with the goose environment preserved through the sudo boundary - the same per-user OS isolation the other backends provide. Set GOOSE_BIN (the wizard prompts for it) so the sudoers rule and the spawned binary agree on one absolute path; see Multi-User Setup.
Current limitations:
- No save-on-shutdown. Goose has no equivalent of Claude Code's save prompt (see the note in the Claude section for why this rarely matters in practice).
- Capability-gated image input. Image blocks are forwarded when the agent advertises image support in the ACP handshake (current goose releases do); when it does not, they are dropped and the reply carries a notice that the model never saw the image. Other non-text content blocks are skipped.
-
DeepSeek thinking mode cannot complete tool calls. With the
deepseekprovider and a thinking-mode model (deepseek-v4-pro), plain-text multi-turn chat works, but any turn involving a tool call fails with HTTP 400 ("reasoning_content must be passed back to the API"). DeepSeek requires the model's reasoning content to be sent back on tool-call follow-up requests; goose does not implement that round-trip, and the upstream issue (goose#9200) was closed as not planned. Kai's chat sessions are agentic (the model calls tools constantly), so goose with DeepSeek thinking mode is not viable for chat. Note that a plain-text smoke test will pass on this combination; only tool-calling turns expose the failure.
OpenCodeBackend (in opencode.py) wraps OpenCode via the same ACP transport as Goose; both adapters share the AcpBackend base class. OpenCode's distinguishing feature is its provider breadth: it resolves models against its own registry of 75+ providers.
Protocol differences from Goose. OpenCode requires an integer protocolVersion in the handshake, and its tool calls emit server-initiated session/request_permission requests that block until answered; Kai auto-approves them in chat (tools must run) and auto-denies them in one-shot calls (review, triage, and extraction prompts must not execute tools).
Model selection. OpenCode model strings are full provider/model IDs (for example anthropic/claude-sonnet-4-6 or deepseek/deepseek-v4-pro), delivered to the subprocess via the OPENCODE_CONFIG_CONTENT environment variable. Kai validates the structural shape (two non-empty segments) and leaves the supported set to OpenCode, which resolves IDs against whatever the operator has authenticated.
Authentication. Operator-managed, outside Kai: opencode auth login writes credentials to ~/.local/share/opencode/auth.json. On installs with per-user os_user isolation, run the login as each target OS user so the auth file lands under that account's home.
Process lifecycle. A single opencode acp subprocess per user, with the same sudo -H -u isolation wrap as the other backends when os_user is set. Set OPENCODE_BIN to pin the binary path.
Current limitations: same ACP envelope as Goose (no save-on-shutdown; capability-gated image input - current OpenCode releases advertise image support).
All backends produce the same streaming interface for the bot to consume. The pattern is:
- The bot calls
pool.send(prompt, chat_id), which delegates to the backend'ssend()method. - The backend yields
StreamEventobjects as text accumulates. Each event containstext_so_far(the full response up to that point) anddone=False. - The bot creates a Telegram message on the first event, then edits it every 2 seconds with the latest accumulated text. This produces a live-typing effect.
- When the backend finishes, it yields a final
StreamEventwithdone=Trueand a completeAgentResponse.
Error handling follows the same path. If the backend times out or the process dies, the final StreamEvent has done=True with an AgentResponse where success=False and error describes what went wrong.
| Capability | Claude Code | Codex | Goose | OpenCode |
|---|---|---|---|---|
| Providers | Anthropic (subscription or API) | OpenAI (subscription or API) | Anthropic, DeepSeek, Google, Ollama, OpenAI, OpenRouter | Anthropic, DeepSeek, Google, Ollama, OpenAI, OpenRouter (and more via its own registry) |
| Wire protocol | stream-json | JSON-RPC 2.0 (app-server) | JSON-RPC 2.0 (ACP) | JSON-RPC 2.0 (ACP) |
| Multimodal input | Full (images, files) | Images via temp files | Images (capability-gated) | Images (capability-gated) |
OS user isolation (os_user) |
Yes | Yes | Yes | Yes |
| Session age recycling | Yes | Yes | Yes | Yes |
| One-shot agents (review, triage, memory extraction) | Yes | Yes | Yes | Yes |
| Model switch without restart | Restart per switch | Per-turn (no restart) | Restart per switch | Restart per switch |
| Reasoning-effort tuning | Yes (CLAUDE_EFFORT_LEVEL, default high) |
Yes (CODEX_EFFORT_LEVEL, opt-in) |
No | No |
| Auto-compaction override | Yes | No | No | No |
| Save-on-shutdown | Yes | No | No | No |
| Tool use | Claude Code built-in tools | Codex built-in tools | Goose developer extension | OpenCode built-in tools |
| Binary path pinning | CLAUDE_BIN |
CODEX_BIN |
GOOSE_BIN |
OPENCODE_BIN |
The Claude Code backend remains the most feature-rich; the others trade those extras for provider flexibility. Which matters more depends on your use case.
How /models and /model behave depends on the backend, because the set of valid models is knowable to different degrees:
| Backend |
/models UI |
What /model <id> accepts |
Invalid model surfaces as |
|---|---|---|---|
| claude | Curated keyboard (Opus / Sonnet / Haiku) | The aliases, plus any full claude-* ID (e.g. claude-opus-4-7 to pin a previous generation) |
Rejected at /model for non-claude-* strings; a bogus claude-* ID errors at the next message |
| codex | Curated keyboard (the Codex CLI's model list) | Only the curated list | Rejected at /model
|
| goose | Curated keyboard for anthropic / openai / google / deepseek; free text for openrouter / ollama | Curated list per provider, plus any claude-* ID on anthropic; any string on open-ended providers |
Rejected at /model for curated providers; open-ended or claude-* mistakes error at the next message |
| opencode | Free text always | Any provider/model-shaped ID |
Bare names (e.g. sonnet) rejected at /model; a structurally valid but unresolvable ID errors at the next message |
The split is deliberate: where Kai can know the valid set (claude aliases, the codex CLI list, the per-provider curated lists), invalid input is rejected immediately; where it cannot (OpenCode's registry, OpenRouter, Ollama, full ID pinning), Kai validates the shape and lets the CLI or provider produce the authoritative error on first use.
This section walks through setting up each backend from scratch. If you are running make config for the first time, the wizard handles most of this interactively. If you are editing .env or users.yaml by hand, the details below tell you exactly what to set and why.
Claude Code backend (default). Requires the Claude Code CLI (claude). Kai spawns it as a subprocess in stream-json mode. You need either an Anthropic subscription (Pro or Max plan) or API credits. The wizard prompts for CLAUDE_BIN to pin the binary path (the native installer's ~/.local/bin/claude or the Homebrew cask's /opt/homebrew/bin/claude); when unset, the service user's PATH resolves bare claude, so if claude works in the service user's shell, the backend works in Kai.
Codex backend. Requires the Codex CLI (codex). Authenticate via codex login (ChatGPT subscription) or set OPENAI_API_KEY with CODEX_AUTH_MODE=api_key. The wizard prompts for CODEX_BIN so the binary path is pinned for sudo-isolated installs.
Goose backend. Requires the Goose binary (goose), recent enough to support goose acp. You also need an API key for your chosen provider (except Ollama). The wizard prompts for GOOSE_BIN.
OpenCode backend. Requires the OpenCode binary (opencode). Authenticate via opencode auth login for each provider you intend to use. The wizard prompts for OPENCODE_BIN.
The .env file (or /etc/kai/env for protected installations) sets the system-wide defaults. Backend-related variables:
# Which backend to use globally: claude (default), codex, goose, or opencode.
DEFAULT_BACKEND=claude
# Required when the backend is goose or opencode (multi-provider backends).
# Valid values: anthropic, deepseek, google, ollama, openai, openrouter
DEFAULT_PROVIDER=openai
# API key for the provider. Set the one matching your provider.
# Ollama does not need an API key (it runs locally).
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
GOOGLE_API_KEY=...
OPENROUTER_API_KEY=sk-or-...
DEEPSEEK_API_KEY=sk-...
# Default model. Backend- and provider-dependent (see "Model selection
# by backend" above).
DEFAULT_MODEL=sonnet
# Codex authentication: subscription (default, via codex login) or api_key.
CODEX_AUTH_MODE=subscription
# Codex reasoning effort (minimal|low|medium|high|xhigh). Unset means
# no override: each OS user's own ~/.codex/config.toml or the model
# default decides.
#CODEX_EFFORT_LEVEL=high
# Agent binary paths. The wizard prompts for these on the matching
# backend; the chat backends and one-shot reasoners spawn these exact
# paths and the per-user sudoers rules pin them.
#CLAUDE_BIN=/opt/homebrew/bin/claude
#CODEX_BIN=/opt/homebrew/bin/codex
#OPENCODE_BIN=~/.local/bin/opencode
#GOOSE_BIN=/opt/homebrew/bin/gooseWhen DEFAULT_BACKEND=claude or codex, the provider is implied (anthropic and openai respectively) and DEFAULT_PROVIDER is not needed. When the backend is goose or opencode, set DEFAULT_PROVIDER and the corresponding API key.
Running make config (or sudo python -m kai config for protected installations) launches an interactive wizard that walks through every setting. For backends, the flow is:
-
Default backend - choose
claude,codex,goose, oropencode. - Provider (only for goose and opencode) - choose from the six supported providers.
- API key (only if the provider needs one) - paste your key. Ollama skips this step.
- Binary path (codex, goose, opencode) - confirm or override the detected binary location.
- Default model - for curated surfaces you pick from a list; for open-ended surfaces (opencode, openrouter, ollama) you type the model ID directly.
The wizard writes the result to .env (or /etc/kai/env). You can re-run it at any time to change settings.
When users.yaml exists, most settings become per-user. Each user entry can override the global backend and provider:
users:
# Alice uses the default Claude backend
- telegram_id: 123456789
name: alice
role: admin
model: opus
# Bob uses Codex (provider implied: openai)
- telegram_id: 987654321
name: bob
backend: codex
model: gpt-5.5
# Carol uses Goose with a local Ollama model
- telegram_id: 555555555
name: carol
backend: goose
provider: ollama
model: llama3.3:70b
# Dave uses OpenCode with DeepSeek
- telegram_id: 444444444
name: dave
backend: opencode
provider: deepseek
model: deepseek/deepseek-v4-proPer-user backend fields:
| Field | Type | Default | Description |
|---|---|---|---|
backend |
string | global default |
claude, codex, goose, or opencode. Overrides DEFAULT_BACKEND for this user. |
provider |
string | global default | Provider for multi-provider backends (goose, opencode). Ignored for claude and codex, whose provider is implied. |
model |
string | global default | Default model. Must be valid for the user's effective backend and provider. |
os_user |
string | none | OS account for per-user subprocess isolation (all backends). See Multi-User Setup. |
This means you can run a mixed deployment: some users on Claude, others on Codex, Goose, or OpenCode with different providers, all served by the same Kai instance.
Users can also change their model at runtime via /model or /settings model in Telegram. The runtime choice is stored in the database and takes precedence over the YAML default until cleared.
Workspaces can override model, timeout, environment variables, and system prompts. They do not have backend or provider overrides - the backend is determined by the user, not the workspace. A workspace model override is applied only when it is valid for the active user's backend; an override that names a model from a different backend's surface is skipped with a logged warning.
workspaces:
# Heavy lifting gets Opus and a longer timeout
- path: ~/projects/complex-app
claude:
model: opus
timeout: 300
# Docs workspace uses Haiku (fast, cheap)
- path: ~/projects/docs
claude:
model: haiku
# Custom environment and system prompt
- path: ~/projects/api-server
claude:
env:
DATABASE_URL: "postgres://localhost/myapp"
system_prompt: |
This is a FastAPI application. Run tests with pytest.Workspace model overrides sit below user overrides in the precedence chain. If a user has set their model via /model, that choice wins. If they have not, the workspace default applies.
Which models you can select depends on your effective backend and provider.
Claude backend / goose-on-anthropic (curated aliases plus full-ID passthrough):
| Model ID | Display name | Notes |
|---|---|---|
opus |
Opus | Most capable; resolves to the current Opus SKU |
sonnet |
Sonnet | Balanced (default) |
haiku |
Haiku | Fastest, lowest cost |
claude-* (full ID) |
- | Any full Anthropic model ID, e.g. claude-opus-4-7 to pin a previous generation |
Codex backend (the Codex CLI's own curated list): gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex, gpt-5.3-codex-spark, gpt-5.2.
Goose-on-openai (curated): gpt-5.5-pro, gpt-5.5, gpt-5.4-pro, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano.
Goose-on-google (curated): gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite.
Goose-on-deepseek (curated): deepseek-v4-pro, deepseek-v4-flash.
OpenCode (open-ended): full provider/model IDs resolved by OpenCode itself, e.g. anthropic/claude-sonnet-4-6, openai/gpt-5.5, deepseek/deepseek-v4-flash, openrouter/anthropic/claude-haiku-4-5.
OpenRouter and Ollama (open-ended, via goose): any model ID the service supports. There is no curated list and no default; specify the model explicitly.
When using the Goose backend, Kai deploys an extension configuration file to ~/.config/goose/config.yaml. This file controls which Goose extensions are loaded in ACP mode. The default configuration enables only the developer extension (file editing, shell commands), which mirrors the toolset available in the other backends.
The template lives at templates/config/goose-config.yaml in the repository. During make install it is copied to the service user's home, and to the home of each goose-backed os_user in users.yaml (owned by that account), so sudo-isolated sessions read the same configuration. You can add additional Goose extensions by editing this file before installing, or by modifying the deployed copies directly.
When multiple configuration sources define the same setting, this is the resolution order (highest priority first):
-
Database override - set via
/model,/settings model,/settings timeout, etc. in Telegram -
Per-user YAML - the
model,timeoutfields inusers.yaml -
Workspace config - the per-workspace overrides in
workspaces.yaml -
Global default -
DEFAULT_MODEL,DEFAULT_TIMEOUT, etc. in.env - Per-backend or per-provider default - "sonnet" for claude, "gpt-5.5" for codex, the provider default for goose
For backend and provider selection specifically:
-
Per-user YAML -
backendandproviderinusers.yaml -
Global default -
DEFAULT_BACKENDandDEFAULT_PROVIDERin.env
Backend and provider cannot be changed at runtime via Telegram commands. They are admin-controlled settings that require editing users.yaml or .env and restarting the service.
Scenario: Switch from Claude to Goose with OpenAI globally.
Edit .env:
DEFAULT_BACKEND=goose
DEFAULT_PROVIDER=openai
OPENAI_API_KEY=sk-your-key-here
DEFAULT_MODEL=gpt-5.5Restart Kai. All users will now route through Goose with OpenAI.
Scenario: Keep Claude as default but put one user on Goose.
In users.yaml, add backend fields to that user's entry:
- telegram_id: 987654321
name: bob
backend: goose
provider: google
model: gemini-2.5-flashSet the API key in .env:
GOOGLE_API_KEY=your-google-keyRestart Kai. Bob routes through Goose/Google. Everyone else stays on Claude.
Scenario: Put one user on Codex with a ChatGPT subscription.
Run codex login as the user's os_user (or the service user on a non-isolated install), then in users.yaml:
- telegram_id: 987654321
name: bob
backend: codex
model: gpt-5.5Restart Kai. The wizard's CODEX_BIN value keeps the sudoers rule and the spawned binary in agreement on isolated installs.
Scenario: Run a local model via Ollama for development.
Edit .env:
DEFAULT_BACKEND=goose
DEFAULT_PROVIDER=ollama
DEFAULT_MODEL=llama3.3:70bNo API key needed. Ollama must be running locally. Restart Kai.
Each provider that Kai supports has its own strengths, limitations, and cost structure. This section profiles each one to help you decide which to use and when.
Access methods: Claude Code CLI (subscription), or goose/opencode with the anthropic provider (API).
Anthropic's Claude models are the default and the most battle-tested backend for Kai. The current Opus generation leads most agentic coding benchmarks, excelling at multi-step reasoning, large codebase understanding, and following complex instructions across long autonomous workflows. Sonnet offers a strong balance of capability and speed. Haiku is the lightweight option for simple tasks.
Via Claude Code CLI (subscription): The subscription path is economical for consistent usage. Pro ($20/month) and Max ($100-$200/month) plans include Claude Code access with rolling usage windows. No API key needed - the CLI handles authentication. The tradeoffs are the rate limits and platform risks discussed in the introduction.
Via goose or opencode (API): Direct API access uses pay-per-token pricing with no rolling windows. Rates as of April 2026:
| Model | Input | Output | Context |
|---|---|---|---|
| Opus | $5.00/M tokens | $25.00/M tokens | 200K |
| Sonnet | $3.00/M tokens | $15.00/M tokens | 200K |
| Haiku | $1.00/M tokens | $5.00/M tokens | 200K |
Batch processing discounts (50%) and prompt caching discounts (90% on cache reads) are available through the API but not through Kai's integration, which uses real-time streaming.
API access avoids the subscription's rate limit windows but can get expensive fast. An always-on assistant running Opus for interactive conversations, PR reviews, and scheduled jobs can easily exceed $500/month in API costs. Sonnet is the more practical default for API usage.
When to use Anthropic: When code quality and complex reasoning matter most. Claude remains the strongest general-purpose coding agent. The subscription path is the best value if you stay within the rate limits; the API path is the escape hatch when you need guaranteed availability.
Access methods: Codex CLI (ChatGPT subscription or API key), or goose/opencode with the openai provider (API).
OpenAI's GPT models are the broadest ecosystem in AI. GPT-5.5 is competitive with Claude on general-purpose tasks and leads several computer-use and web automation benchmarks. It is less specialized than Claude for deep multi-file code reasoning but handles polyglot projects and broad knowledge tasks well.
The Codex CLI path mirrors the Claude Code subscription model: an agentic CLI authenticated through a consumer plan (ChatGPT), with its own curated model surface. The goose/opencode path uses the plain API with the standard model names.
Practical considerations: OpenAI's API rate limits are generally more generous than Anthropic's subscription windows, though they vary by tier and usage history. Pricing is competitive at the mid-range; frontier capability costs roughly the same as Claude Opus via API.
When to use OpenAI: As a primary alternative when Claude is having a bad day, or as a dedicated backend for a specific user. The Codex CLI is the natural choice if you already pay for ChatGPT; the API path suits bursty or metered workloads.
Access methods: goose or opencode with the google provider.
Google's Gemini models bring massive context windows and strong multimodal reasoning. Gemini 2.5 Pro supports a 1M token context window, far exceeding Claude and GPT's 200K, and is the most capable of the stable Gemini family; the 2.5 Flash variants cover speed- and cost-sensitive workloads.
Practical considerations: Gemini scores lower than Claude and GPT on coding-specific benchmarks like SWE-bench but excels at reasoning and multimodal tasks. Google offers a generous free tier for low-volume API usage, which makes it an interesting option for scheduled jobs or secondary tasks that do not need frontier coding ability.
When to use Google: When you need a massive context window (analyzing entire codebases or long documents in a single pass), when multimodal input is important, or as a cost-effective backend for non-coding tasks.
Access methods: goose or opencode with the deepseek provider.
DeepSeek's V4 family offers strong reasoning at aggressive prices. Kai's curated surface is deepseek-v4-pro (reasoning-heavy work) and deepseek-v4-flash (high-volume, latency-sensitive work), both with 1M context. The legacy deepseek-chat and deepseek-reasoner aliases are deprecated upstream and are not offered.
On the goose side, Kai translates its deepseek provider key to Goose's wire-level provider name automatically; you configure provider: deepseek and never see the difference. Be aware that thinking-mode models cannot complete tool-calling turns through goose; see the limitations list in the Goose backend section.
When to use DeepSeek: When cost-per-token is the constraint and you still want capable reasoning, or as the budget leg of a mixed deployment.
Access methods: goose or opencode with the openrouter provider.
OpenRouter is not a model provider. It is an aggregator that routes requests to dozens of providers and models through a single API key and a unified interface. You can access Claude, GPT, Gemini, Llama, Mistral, and many others without maintaining separate accounts with each provider.
Key characteristics:
- Single API key for all models. No need to manage per-provider credentials.
- Open-ended model selection. Kai treats OpenRouter as an open-ended provider, meaning any model ID that OpenRouter supports is valid. There is no curated list - you type the model ID directly.
- Per-request pricing. OpenRouter adds a small markup (typically 0-5% depending on the model) on top of the upstream provider's rate. Pricing is transparent and per-model.
- Automatic fallback. Some OpenRouter configurations can route to alternative providers if the primary is down, though this is managed on their side, not in Kai.
Practical considerations: OpenRouter is the most flexible option but adds a layer of indirection. Latency may be slightly higher than direct provider access. Model availability depends on OpenRouter's upstream agreements, which can change. You need to know the exact model IDs you want to use (e.g., anthropic/claude-sonnet-4-6).
When to use OpenRouter: When you want access to many models without managing multiple API keys, when you want to experiment with different models quickly, or when you want a single billing point for mixed-provider usage.
Access methods: goose or opencode with the ollama provider.
Ollama runs open-source models locally on your own hardware. There are no API keys, no per-token costs, no rate limits, and no data leaving your machine. The tradeoff is that you need hardware capable of running the models, and open-source models are significantly less capable than frontier models for agentic coding tasks.
Key characteristics:
- Zero cost per token. After hardware investment, usage is free.
- Complete privacy. Nothing leaves your machine. No API calls, no telemetry, no data retention policies to worry about.
- Open-ended model selection. Like OpenRouter, Ollama is treated as an open-ended provider. Any model available in the Ollama registry is valid.
- Hardware requirements. Capable models (70B+ parameters) need substantial RAM and benefit from GPU acceleration. A Mac mini with 16GB can run smaller models (7B-13B) but will struggle with larger ones. The quality gap between a 13B local model and a frontier model is significant for complex coding tasks.
Practical considerations: Local models are best suited for tasks where privacy is the primary concern, where you need guaranteed availability regardless of internet connectivity, or where the tasks are simple enough that frontier capability is not needed. Using Ollama as a fallback for Kai's scheduled jobs (weather checks, simple reminders) while running Claude or GPT for interactive conversations is a practical hybrid approach.
When to use Ollama: When privacy is non-negotiable, when you want a zero-cost fallback for simple tasks, or when you are running Kai in an environment without reliable internet access. Not recommended as the primary backend for complex coding tasks unless you have high-end hardware and are comfortable with the quality gap.
There is no single best provider. The right choice depends on what you value most.
If code quality is the priority: Anthropic via Claude Code CLI (subscription) or API. Claude remains the strongest coding agent available.
If availability matters more than peak quality: OpenAI (via Codex or goose/opencode) or Google. Both have reliable APIs with reasonable rate limits. Either can serve as a fallback when Claude is degraded or rate-limited.
If you want maximum flexibility: OpenCode or OpenRouter. One configuration surface, many models, easy experimentation.
If cost is the constraint: Ollama for zero marginal cost, DeepSeek or Google's free tier for cheap API capability, or Haiku / GPT-5.4 Nano for cheap frontier-adjacent capability.
If privacy is the requirement: Ollama. Nothing else comes close for complete data isolation.
The hybrid approach is often the most practical: Claude as the daily driver with another backend configured for specific users. Kai's per-user backend routing makes this straightforward. Alice runs Claude. Bob runs Codex. Carol runs a local Llama through Goose. All three talk to the same Kai instance through Telegram, and none of them need to know or care what the others are using.