diff --git a/src/litellm/NOTES.md b/src/litellm/NOTES.md new file mode 100644 index 0000000..74b6328 --- /dev/null +++ b/src/litellm/NOTES.md @@ -0,0 +1,58 @@ +## Requirements + +A Debian or Ubuntu base image with `curl`, `unzip` and `sha256sum`, all of which `common-utils` +provides. The proxy is a Python package, but it does not use the image's Python: a pinned `uv` +release is fetched from PyPI into a staging directory and verified against the digest PyPI +publishes for it, installs a private interpreter and the package into the shared prefix, and +leaves with the staging directory. The image's own Python, if any, is neither read nor written. +It is a large install (a little under 700 MB with the interpreter), which lands as one layer in +whatever image installs it. + +```json +"features": { + "ghcr.io/orianna-ai/devcontainer-features/litellm:1": {} +} +``` + +## Where it installs + +Everything lands under `/usr/local/share/litellm`: the managed interpreter in `python/`, the +tool environment in `tools/litellm/`, and the launcher in `bin/litellm`, which is symlinked to +`/usr/local/bin/litellm`. The same shared-prefix shape the other features use: one root-owned +copy, readable and executable by every user, writable by none of them. + +`uv tool install` would otherwise put the interpreter under `~/.local/share/uv` and the launcher +under `~/.local/bin`, so an install run as root during a build puts both under `/root` — +reachable while building, unreadable to the remote user afterwards. The feature points each at +the shared prefix and stages `HOME` and the download cache, so nothing is left in a home +directory either. + +## What it is for + +A local translation proxy: Codex speaks only the OpenAI Responses API and Claude Code only the +Anthropic Messages API, and an upstream that serves only Chat Completions (LithosAI, for one) is +reachable from either through `litellm` bridging the wire protocol. The feature installs the +proxy; configuring and starting it, with the upstream's credential, is up to whatever drives +the CLIs. + +## Authentication + +Nothing is authenticated at build time. The install reaches only PyPI and the managed Python +downloads; it reads no credential from the environment it inherits, so none can be baked into an +image layer. At run time the proxy reads whatever keys its configuration names. + +## Version pinning + +`version` defaults to a reviewed release rather than to `latest`, and takes an exact `X.Y.Z`, +which freezes the package into the image digest. In March 2026 two litellm releases (1.82.7 and +1.82.8) were briefly replaced on PyPI by a credential stealer; a pinned, known-good default is +what keeps an ordinary image rebuild from picking up the next such release on its own. `latest` +is accepted as an explicit opt-in and resolves the newest release on PyPI at *image build* time — +still fixed for the life of the image, but moving on every rebuild. + +`uv` itself is pinned the same way: a fixed version, fetched as the wheel PyPI publishes for the +image's architecture and checked against that wheel's digest before it runs. Bumping it means +updating the version and both digests in `install.sh` together. + +`pythonVersion` picks the private interpreter. The package requires Python below 3.15; the +default stays on a release its wheels are well exercised on rather than the newest. diff --git a/src/litellm/README.md b/src/litellm/README.md new file mode 100644 index 0000000..19551da --- /dev/null +++ b/src/litellm/README.md @@ -0,0 +1,82 @@ +# LiteLLM proxy (litellm) + +Installs the LiteLLM proxy on a private Python runtime, independent of any Python in the image. + +## Example Usage + +```json +"features": { + "ghcr.io/orianna-ai/devcontainer-features/litellm:1": {} +} +``` + +## Options + +| Options Id | Description | Type | Default Value | +|-----|-----|-----|-----| +| pythonVersion | Version of the private Python runtime that runs the proxy, as X.Y or X.Y.Z. | string | 3.13 | +| version | Version of the litellm package to install, as X.Y.Z. The default is a reviewed release; latest opts into the newest release on PyPI at build time. | string | 1.101.0 | + +## Requirements + +A Debian or Ubuntu base image with `curl`, `unzip` and `sha256sum`, all of which `common-utils` +provides. The proxy is a Python package, but it does not use the image's Python: a pinned `uv` +release is fetched from PyPI into a staging directory and verified against the digest PyPI +publishes for it, installs a private interpreter and the package into the shared prefix, and +leaves with the staging directory. The image's own Python, if any, is neither read nor written. +It is a large install (a little under 700 MB with the interpreter), which lands as one layer in +whatever image installs it. + +```json +"features": { + "ghcr.io/orianna-ai/devcontainer-features/litellm:1": {} +} +``` + +## Where it installs + +Everything lands under `/usr/local/share/litellm`: the managed interpreter in `python/`, the +tool environment in `tools/litellm/`, and the launcher in `bin/litellm`, which is symlinked to +`/usr/local/bin/litellm`. The same shared-prefix shape the other features use: one root-owned +copy, readable and executable by every user, writable by none of them. + +`uv tool install` would otherwise put the interpreter under `~/.local/share/uv` and the launcher +under `~/.local/bin`, so an install run as root during a build puts both under `/root` — +reachable while building, unreadable to the remote user afterwards. The feature points each at +the shared prefix and stages `HOME` and the download cache, so nothing is left in a home +directory either. + +## What it is for + +A local translation proxy: Codex speaks only the OpenAI Responses API and Claude Code only the +Anthropic Messages API, and an upstream that serves only Chat Completions (LithosAI, for one) is +reachable from either through `litellm` bridging the wire protocol. The feature installs the +proxy; configuring and starting it, with the upstream's credential, is up to whatever drives +the CLIs. + +## Authentication + +Nothing is authenticated at build time. The install reaches only PyPI and the managed Python +downloads; it reads no credential from the environment it inherits, so none can be baked into an +image layer. At run time the proxy reads whatever keys its configuration names. + +## Version pinning + +`version` defaults to a reviewed release rather than to `latest`, and takes an exact `X.Y.Z`, +which freezes the package into the image digest. In March 2026 two litellm releases (1.82.7 and +1.82.8) were briefly replaced on PyPI by a credential stealer; a pinned, known-good default is +what keeps an ordinary image rebuild from picking up the next such release on its own. `latest` +is accepted as an explicit opt-in and resolves the newest release on PyPI at *image build* time — +still fixed for the life of the image, but moving on every rebuild. + +`uv` itself is pinned the same way: a fixed version, fetched as the wheel PyPI publishes for the +image's architecture and checked against that wheel's digest before it runs. Bumping it means +updating the version and both digests in `install.sh` together. + +`pythonVersion` picks the private interpreter. The package requires Python below 3.15; the +default stays on a release its wheels are well exercised on rather than the newest. + + +--- + +_Note: This file was auto-generated from the [devcontainer-feature.json](devcontainer-feature.json). Add additional notes to a `NOTES.md`._ diff --git a/src/litellm/devcontainer-feature.json b/src/litellm/devcontainer-feature.json new file mode 100644 index 0000000..509bd08 --- /dev/null +++ b/src/litellm/devcontainer-feature.json @@ -0,0 +1,21 @@ +{ + "description": "Installs the LiteLLM proxy on a private Python runtime, independent of any Python in the image.", + "id": "litellm", + "installsAfter": [ + "ghcr.io/devcontainers/features/common-utils" + ], + "name": "LiteLLM proxy", + "options": { + "pythonVersion": { + "default": "3.13", + "description": "Version of the private Python runtime that runs the proxy, as X.Y or X.Y.Z.", + "type": "string" + }, + "version": { + "default": "1.101.0", + "description": "Version of the litellm package to install, as X.Y.Z. The default is a reviewed release; latest opts into the newest release on PyPI at build time.", + "type": "string" + } + }, + "version": "1.0.0" +} diff --git a/src/litellm/install.sh b/src/litellm/install.sh new file mode 100755 index 0000000..30976ac --- /dev/null +++ b/src/litellm/install.sh @@ -0,0 +1,73 @@ +#!/bin/bash +set -euo pipefail + +VERSION="${VERSION:-1.101.0}" +PYTHON_VERSION="${PYTHONVERSION:-3.13}" +INSTALL_PATH="/usr/local/share/litellm" + +# uv is only a build tool here: a pinned release, verified against the digest PyPI publishes for +# it, staged for the install and gone with the staging directory. Bump the version and both +# digests together (https://pypi.org/pypi/uv//json lists them). +UV_VERSION="0.12.15" +UV_X86_64_WHEEL="https://files.pythonhosted.org/packages/1e/fd/432451d732917c49152a291de3ef171aa6b0f1a22d39780fb2c1f085ca4c/uv-0.12.15-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl" +UV_X86_64_SHA256="aee9802f46bae436bd91751bb33ddeb379ef1596b5c19df193219d545d244b60" +UV_AARCH64_WHEEL="https://files.pythonhosted.org/packages/e9/3a/52e6f0c159d133b03890b5c54823bdc10163ce6f010515f649e5804b5bba/uv-0.12.15-py3-none-manylinux_2_28_aarch64.whl" +UV_AARCH64_SHA256="013e3a5774fb2cc036a9422edc51c8c5273cca011ec04c2817249b8a4a0b0f02" + +for tool in curl dpkg sha256sum unzip; do + if ! command -v "${tool}" >/dev/null 2>&1; then + echo "${tool} was not found; it is needed to fetch and unpack the pinned uv release" >&2 + exit 1 + fi +done + +case "$(dpkg --print-architecture)" in +amd64) + wheel="${UV_X86_64_WHEEL}" + sha256="${UV_X86_64_SHA256}" + ;; +arm64) + wheel="${UV_AARCH64_WHEEL}" + sha256="${UV_AARCH64_SHA256}" + ;; +*) + echo "unsupported architecture $(dpkg --print-architecture); uv ships linux builds for amd64 and arm64 only" >&2 + exit 1 + ;; +esac + +staging="$(mktemp -d)" +trap 'rm -rf "${staging}"' EXIT + +curl -fsSL "${wheel}" -o "${staging}/uv.whl" +echo "${sha256} ${staging}/uv.whl" | sha256sum -c - >/dev/null + +# The wheel carries the binary under its scripts directory; nothing else in it is needed. +unzip -q -j "${staging}/uv.whl" "uv-${UV_VERSION}.data/scripts/uv" -d "${staging}/uv" +chmod 0755 "${staging}/uv/uv" + +if [ "${VERSION}" = "latest" ]; then + requirement="litellm[proxy]" +else + requirement="litellm[proxy]==${VERSION}" +fi + +# Everything the proxy needs at run time — the interpreter, the tool venv, the launcher — lands +# under the shared prefix. HOME and the cache are staged so nothing is left in a home directory. +env HOME="${staging}" \ + UV_CACHE_DIR="${staging}/cache" \ + UV_PYTHON_INSTALL_DIR="${INSTALL_PATH}/python" \ + UV_PYTHON_PREFERENCE=only-managed \ + UV_TOOL_DIR="${INSTALL_PATH}/tools" \ + UV_TOOL_BIN_DIR="${INSTALL_PATH}/bin" \ + "${staging}/uv/uv" tool install --python "${PYTHON_VERSION}" "${requirement}" + +if [ ! -x "${INSTALL_PATH}/bin/litellm" ]; then + echo "uv left no runnable launcher at ${INSTALL_PATH}/bin/litellm" >&2 + exit 1 +fi + +ln -sfn "${INSTALL_PATH}/bin/litellm" /usr/local/bin/litellm + +chown -R root:root "${INSTALL_PATH}" +chmod -R a+rX,go-w "${INSTALL_PATH}" diff --git a/test/litellm/scenarios.json b/test/litellm/scenarios.json new file mode 100644 index 0000000..dd4a42d --- /dev/null +++ b/test/litellm/scenarios.json @@ -0,0 +1,30 @@ +{ + "ubuntu": { + "features": { + "ghcr.io/devcontainers/features/common-utils:2": {}, + "litellm": {} + }, + "image": "mcr.microsoft.com/devcontainers/base:ubuntu26.04", + "user": "vscode" + }, + "ubuntu_latest": { + "features": { + "ghcr.io/devcontainers/features/common-utils:2": {}, + "litellm": { + "version": "latest" + } + }, + "image": "mcr.microsoft.com/devcontainers/base:ubuntu26.04", + "user": "vscode" + }, + "ubuntu_python": { + "features": { + "ghcr.io/devcontainers/features/common-utils:2": {}, + "litellm": { + "pythonVersion": "3.12" + } + }, + "image": "mcr.microsoft.com/devcontainers/base:ubuntu26.04", + "user": "vscode" + } +} diff --git a/test/litellm/ubuntu.sh b/test/litellm/ubuntu.sh new file mode 100755 index 0000000..4f42aa8 --- /dev/null +++ b/test/litellm/ubuntu.sh @@ -0,0 +1,84 @@ +#!/bin/bash +set -e + +# shellcheck source=/dev/null +source \ + dev-container-features-test-lib + +INSTALL_PATH=/usr/local/share/litellm +# Matches the feature's default; bump both together when the reviewed release moves. +DEFAULT_VERSION=1.101.0 + +# Guards uv's own layout, which leaves the interpreter and launcher in a home directory. +resolves_inside_the_shared_prefix() { + resolved="$(readlink -f "$(command -v litellm)")" || return 1 + test -x "${resolved}" || return 1 + case "${resolved}" in + "${INSTALL_PATH}"/*) return 0 ;; + *) return 1 ;; + esac +} + +runs_on_the_private_interpreter() { + interpreter="$(head -n 1 "${INSTALL_PATH}/bin/litellm" | sed 's/^#!//')" || return 1 + case "$(readlink -f "${interpreter}")" in + "${INSTALL_PATH}"/python/*) return 0 ;; + *) return 1 ;; + esac +} + +leaves_nothing_in_the_home_directory() { + test ! -e "${HOME}/.local/bin/litellm" && + test ! -e "${HOME}/.local/share/uv" && + test ! -e "${HOME}/.cache/uv" +} + +installs_the_reviewed_default_version() { + litellm --version | grep -qF "${DEFAULT_VERSION}" +} + +not_writable_by_remote_user() { + ! test -w /usr/local/bin/litellm && + ! test -w "${INSTALL_PATH}/bin/litellm" +} + +# The proxy's whole job is answering on a loopback port; a build that imports but cannot serve +# is not an install. +serves_its_liveliness_probe() { + config="$(mktemp)" + cat >"${config}" <<'YAML' +model_list: + - model_name: "responses/*" + litellm_params: + model: "openai/*" + api_base: http://127.0.0.1:9/v1 + api_key: unused + use_chat_completions_api: true +general_settings: + master_key: sk-test +YAML + LITELLM_LOCAL_MODEL_COST_MAP=True litellm --config "${config}" --host 127.0.0.1 --port 4123 --telemetry False >/tmp/litellm.log 2>&1 & + pid=$! + ready=1 + for _ in $(seq 1 120); do + if curl -sf http://127.0.0.1:4123/health/liveliness >/dev/null; then + ready=0 + break + fi + sleep 1 + done + kill "${pid}" 2>/dev/null || true + wait "${pid}" 2>/dev/null || true + test "${ready}" -eq 0 || cat /tmp/litellm.log >&2 + return "${ready}" +} + +check 'check if litellm exists' bash -c "command -v litellm" +check 'check if litellm runs' bash -c "litellm --version" +check 'check if litellm resolves inside the shared prefix' resolves_inside_the_shared_prefix +check 'check if litellm runs on the private interpreter' runs_on_the_private_interpreter +check 'check if litellm installed the reviewed default version' installs_the_reviewed_default_version +check 'check if the installer left nothing in the home directory' leaves_nothing_in_the_home_directory +check 'check if the shared install is read-only to the remote user' not_writable_by_remote_user +check 'check if litellm serves its liveliness probe' serves_its_liveliness_probe +reportResults diff --git a/test/litellm/ubuntu_latest.sh b/test/litellm/ubuntu_latest.sh new file mode 100755 index 0000000..1e49bd6 --- /dev/null +++ b/test/litellm/ubuntu_latest.sh @@ -0,0 +1,10 @@ +#!/bin/bash +set -e + +# shellcheck source=/dev/null +source \ + dev-container-features-test-lib + +check 'check if litellm exists' bash -c "command -v litellm" +check 'check if litellm runs' bash -c "litellm --version" +reportResults diff --git a/test/litellm/ubuntu_python.sh b/test/litellm/ubuntu_python.sh new file mode 100755 index 0000000..52094b7 --- /dev/null +++ b/test/litellm/ubuntu_python.sh @@ -0,0 +1,30 @@ +#!/bin/bash +set -e + +# shellcheck source=/dev/null +source \ + dev-container-features-test-lib + +INSTALL_PATH=/usr/local/share/litellm +# Matches scenarios.json; the option is an X.Y here, so the runtime is any patch of it. +REQUESTED_PYTHON=3.12 + +# Guards the option-to-runtime mapping: the launcher must run on the interpreter that was asked +# for, not on the default or on anything the image already had. +runs_on_the_requested_interpreter() { + interpreter="$(head -n 1 "${INSTALL_PATH}/bin/litellm" | sed 's/^#!//')" || return 1 + case "$(readlink -f "${interpreter}")" in + "${INSTALL_PATH}"/python/cpython-"${REQUESTED_PYTHON}".*) return 0 ;; + *) return 1 ;; + esac +} + +reports_the_requested_python() { + "$(head -n 1 "${INSTALL_PATH}/bin/litellm" | sed 's/^#!//')" --version | grep -qF "Python ${REQUESTED_PYTHON}." +} + +check 'check if litellm exists' bash -c "command -v litellm" +check 'check if litellm runs' bash -c "litellm --version" +check 'check if litellm runs on the requested interpreter' runs_on_the_requested_interpreter +check 'check if the requested interpreter reports its version' reports_the_requested_python +reportResults