Skip to content

feat: add a litellm proxy feature - #25

Open
ashwin153 wants to merge 2 commits into
mainfrom
claude/lithos-sandbox-support-f0h2yq
Open

ashwin153 wants to merge 2 commits into
mainfrom
claude/lithos-sandbox-support-f0h2yq

Conversation

@ashwin153

@ashwin153 ashwin153 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a litellm feature that installs the LiteLLM proxy on a private, uv-managed Python runtime under /usr/local/share/litellm, linked as /usr/local/bin/litellm.

The sandbox runner will drive this proxy to bridge Codex (Responses API only) and Claude Code (Anthropic Messages API only) onto Chat Completions-only upstreams, LithosAI first. See the companion PRs in orianna-ai/devcontainer (adds the feature to the image) and orianna-ai/orianna (the litellm provider on run_claude / run_codex).

What the feature does

  • Fetches a pinned uv release (0.12.15) as the wheel PyPI publishes for the image's architecture, verifies it against that wheel's sha256 embedded in the script, and uses it only from a staging directory that is removed afterwards.
  • Installs a managed Python (pythonVersion, default 3.13) and litellm[proxy] (version, default the reviewed 1.101.0; latest is an explicit opt-in) into the shared prefix. Nothing lands in a home directory and the image's own Python is untouched.
  • Root-owned, read-only to the remote user, same shape as the grok and antigravity features.
  • Nothing is authenticated at build time; the proxy reads keys from its runtime configuration.

Testing

  • Ran install.sh in a clean Ubuntu container for the default and for pythonVersion=3.12: the launcher resolves inside the prefix, its shebang points at the private interpreter of the requested version, litellm --version reports 1.101.0, and the proxy answers /health/liveliness about 6s after start with a wildcard config.
  • Scenarios: ubuntu (defaults; asserts the reviewed default version, prefix layout, private interpreter, no home leftovers, read-only install, liveliness), ubuntu_latest, and ubuntu_python (asserts the launcher runs on the requested 3.12 runtime).
  • pre-commit passes (shellcheck, shfmt, JSON formatting).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mr8awhwwWhJpkEKKHkEZpq

Installs the LiteLLM proxy on a private, uv-managed Python runtime under
/usr/local/share/litellm, linked as /usr/local/bin/litellm. The sandbox
runner drives it to bridge Codex (Responses API) and Claude Code (Messages
API) onto Chat Completions-only upstreams such as LithosAI.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mr8awhwwWhJpkEKKHkEZpq
Comment thread src/litellm/install.sh Outdated
Comment thread src/litellm/devcontainer-feature.json Outdated
Comment thread test/litellm/scenarios.json
The build tool is a fixed uv release fetched as the wheel PyPI publishes
for the image's architecture and checked against that wheel's digest
before it runs, instead of piping a mutable installer script into sh.
The package default moves from latest to a reviewed 1.101.0, with latest
kept as an explicit opt-in. A scenario now exercises a non-default
pythonVersion and asserts the launcher runs on the requested interpreter.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mr8awhwwWhJpkEKKHkEZpq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants