From 1419aeee81b7e0d069b9152940d28c4740dd8995 Mon Sep 17 00:00:00 2001 From: "Paul S. Schweigert" Date: Fri, 14 Aug 2026 16:12:30 -0400 Subject: [PATCH 01/12] docs: expand LiteLLM backend documentation Signed-off-by: Paul S. Schweigert --- .../docs/how-to/backends-and-configuration.md | 9 +- docs/docs/integrations/litellm.md | 321 ++++++++++++++++++ docs/sidebars.ts | 1 + 3 files changed, 329 insertions(+), 2 deletions(-) create mode 100644 docs/docs/integrations/litellm.md diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index 150754ba0..3eeca09a3 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -32,7 +32,7 @@ The following table shows all available backends, their class names, import path | ------- | ----- | ------ | --------------- | ----------------- | | [Ollama](../integrations/ollama) | `OllamaModelBackend` | `mellea.backends.ollama` | base | `backend_name="ollama"` | | [OpenAI](../integrations/openai) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | -| [LiteLLM](../integrations/bedrock) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | +| [LiteLLM](../integrations/litellm) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | | [Hugging Face](../integrations/huggingface) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | | [WatsonX](../integrations/watsonx) | `WatsonxAIBackend` | `mellea.backends.watsonx` | `mellea[watsonx]` | `backend_name="watsonx"` (deprecated) | @@ -112,7 +112,8 @@ m = MelleaSession( > See the [LiteLLM docs](https://docs.litellm.ai/) for your provider's setup. LiteLLM provides unified access to 100+ providers — Anthropic, AWS Bedrock, Azure, -and more: +IBM WatsonX, and more. Set the provider's credentials in the environment and pass a +`/` model string: ```python # Requires: mellea[litellm] @@ -128,6 +129,10 @@ print(str(result)) # Output will vary — LLM responses depend on model and temperature. ``` +See the [LiteLLM integration](../integrations/litellm) for direct-mode vs proxy usage, +self-hosted proxy setup, per-provider configuration examples, an environment variable +reference, and the WatsonX migration path. + ## Hugging Face backend > **Backend note:** Requires `pip install "mellea[hf]"`. Models are downloaded from diff --git a/docs/docs/integrations/litellm.md b/docs/docs/integrations/litellm.md new file mode 100644 index 000000000..9d9cf4052 --- /dev/null +++ b/docs/docs/integrations/litellm.md @@ -0,0 +1,321 @@ +--- +title: "LiteLLM" +description: "Reach 100+ LLM providers through Mellea's LiteLLMBackend — direct calls, a self-hosted proxy, and the recommended migration path off the deprecated WatsonX backend." +sidebar_label: "LiteLLM" +# diataxis: how-to +--- + +The [`LiteLLMBackend`](../reference/glossary#litellm--litellmbackend) gives Mellea a +single code path to 100+ model providers — Anthropic, AWS Bedrock, Azure OpenAI, IBM +WatsonX, Google Vertex AI, and more — by delegating provider auth and request +translation to [LiteLLM](https://docs.litellm.ai/). You switch providers by changing +the `model_id` prefix; your Mellea code stays the same. + +**Prerequisites:** `pip install 'mellea[litellm]'` and provider credentials set as +environment variables (each provider is listed below). + +## When to use LiteLLM + +- **Multi-provider access.** One backend reaches most hosted providers, so you can + compare models or fail over between vendors without rewriting session code. +- **WatsonX migration.** The native `WatsonxAIBackend` is deprecated since v0.4; + LiteLLM (or the OpenAI backend) is the recommended replacement. See + [Migrating from the WatsonX backend](#migrating-from-the-watsonx-backend). +- **Central governance.** Paired with a self-hosted proxy, LiteLLM centralizes API + keys, routing, rate limits, and cost tracking across teams. + +For providers that already have a dedicated Mellea page, prefer it for provider-specific +detail: [AWS Bedrock](./bedrock.md) and [Vertex AI](./vertex-ai.md) both use this +backend under the hood. + +## Direct mode vs the LiteLLM Proxy + +There are two ways to use LiteLLM. **Direct mode** calls the provider straight from +your process. The **LiteLLM Proxy** is a standalone server you (or your platform team) +run; Mellea talks to the proxy, and the proxy talks to the providers. + +| | Direct mode | LiteLLM Proxy | +| --- | --- | --- | +| Setup | `pip install 'mellea[litellm]'` only | Run a separate proxy server | +| Credentials | Provider keys live in each app's environment | Keys live on the proxy; apps hold only a proxy key | +| Routing / fallback / rate limits | Per-app | Centralized on the proxy | +| Cost & usage tracking | Per-app | Centralized on the proxy | +| Best for | Single apps, local dev, quick provider comparison | Shared infrastructure, many apps, governed key management | + +## Direct mode + +The quickest path is [`start_session()`](../reference/glossary#melleasession) with +`backend_name="litellm"`. The `model_id` is a LiteLLM model string of the form +`/`: + +```python +# Requires: mellea[litellm] +# Returns: str +import mellea + +m = mellea.start_session( + backend_name="litellm", + model_id="anthropic/claude-sonnet-4-20250514", +) +result = m.chat("Give me three facts about the Amazon rainforest.") +print(str(result)) +# Output will vary — LLM responses depend on model and temperature. +``` + +> **Note:** For cloud providers, leave `base_url` unset (the default). LiteLLM infers +> the correct endpoint from the `model_id` prefix. Only set `base_url` when you target +> a proxy or a local server (see [Self-hosted LiteLLM Proxy](#self-hosted-litellm-proxy)). + +For full control, construct the [`Backend`](../reference/glossary#backend) directly and +pass it to [`MelleaSession`](../reference/glossary#melleasession): + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend(model_id="anthropic/claude-sonnet-4-20250514"), +) +``` + +## Provider configuration examples + +Set the provider's credentials in the environment, then pass the matching `model_id` +prefix. The [environment variable reference](#environment-variable-reference) below +lists every provider in one table. + +### Anthropic + +```bash +export ANTHROPIC_API_KEY=your-api-key-here +``` + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend(model_id="anthropic/claude-sonnet-4-20250514"), +) +``` + +### Azure OpenAI + +The `model_id` is `azure/` — your Azure deployment name, not a +base model name: + +```bash +export AZURE_API_KEY=your-api-key-here +export AZURE_API_BASE=https://your-resource.openai.azure.com +export AZURE_API_VERSION=2024-02-15-preview +``` + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend(model_id="azure/my-gpt-4o-deployment"), +) +``` + +### IBM WatsonX + +LiteLLM reaches WatsonX with the `watsonx/` prefix. Note the API-key variable is +`WATSONX_APIKEY` (no underscore before `KEY`) — different from the native backend's +`WATSONX_API_KEY`: + +```bash +export WATSONX_URL=https://us-south.ml.cloud.ibm.com +export WATSONX_APIKEY=your-api-key-here +export WATSONX_PROJECT_ID=your-project-id +``` + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend(model_id="watsonx/ibm/granite-3-3-8b-instruct"), +) +``` + +### AWS Bedrock and Google Vertex AI + +Both use `LiteLLMBackend` and have dedicated pages with credential setup and model-string +tables: + +- **Bedrock** — `bedrock/converse/`. See [AWS Bedrock](./bedrock.md). +- **Vertex AI** — `vertex_ai/`. See [Vertex AI](./vertex-ai.md). + +## Self-hosted LiteLLM Proxy + +The [LiteLLM Proxy](https://docs.litellm.ai/docs/simple_proxy) is a server that holds +your provider keys centrally and exposes them behind a single proxy key. Define your +models in a `config.yaml`: + +```yaml +model_list: + - model_name: my-model + litellm_params: + model: anthropic/claude-sonnet-4-20250514 + api_key: os.environ/ANTHROPIC_API_KEY +``` + +Start the proxy (defaults to port 4000): + +```bash +pip install 'litellm[proxy]' +litellm --config config.yaml +``` + +Point Mellea at the proxy with the `litellm_proxy/` prefix and the proxy URL as +`base_url` (forwarded to LiteLLM as `api_base`). Authenticate with the proxy's key via +`LITELLM_PROXY_API_KEY`: + +```bash +export LITELLM_PROXY_API_KEY=sk-... +``` + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend( + model_id="litellm_proxy/my-model", + base_url="http://localhost:4000", + ), +) +``` + +`my-model` is the `model_name` you defined in the proxy's `config.yaml` — the app never +sees which provider or model backs it. To pass the proxy key explicitly instead of using +the environment variable, add it to `model_options`: + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend( + model_id="litellm_proxy/my-model", + base_url="http://localhost:4000", + model_options={"api_key": "sk-..."}, + ), +) +``` + +## Migrating from the WatsonX backend + +The native `WatsonxAIBackend` is deprecated since v0.4. To move an existing WatsonX +session onto LiteLLM, swap the backend and prefix the model with `watsonx/`. + +Before (deprecated): + +```python +# Requires: mellea[watsonx] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.watsonx import WatsonxAIBackend + +m = MelleaSession( + WatsonxAIBackend(model_id="ibm/granite-3-3-8b-instruct"), +) +``` + +After (LiteLLM): + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend(model_id="watsonx/ibm/granite-3-3-8b-instruct"), +) +``` + +> **Warning:** LiteLLM reads the WatsonX API key from `WATSONX_APIKEY`, whereas the +> native backend read `WATSONX_API_KEY`. Rename the variable when you migrate, or the +> new backend will not find your credentials. `WATSONX_URL` and `WATSONX_PROJECT_ID` +> are unchanged. + +## Environment variable reference + +Set these before creating the session. LiteLLM reads them automatically based on the +`model_id` prefix. + +| Provider | `model_id` prefix | Environment variables | +| --- | --- | --- | +| Anthropic | `anthropic/` | `ANTHROPIC_API_KEY` | +| Azure OpenAI | `azure/` | `AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION` | +| AWS Bedrock | `bedrock/converse/` | `AWS_BEARER_TOKEN_BEDROCK` (or standard AWS credentials) | +| IBM WatsonX | `watsonx/` | `WATSONX_URL`, `WATSONX_APIKEY`, `WATSONX_PROJECT_ID` | +| Google Vertex AI | `vertex_ai/` | `VERTEXAI_PROJECT`, `VERTEXAI_LOCATION` (see [Vertex AI](./vertex-ai.md)) | +| LiteLLM Proxy | `litellm_proxy/` | `LITELLM_PROXY_API_KEY`, `LITELLM_PROXY_API_BASE` (or pass `base_url`) | + +See the [LiteLLM providers documentation](https://docs.litellm.ai/docs/providers) for +the full list and any provider-specific variables. + +## Model options + +Pass generation parameters with [`ModelOption`](../reference/glossary#modeloption), the +same as any other backend. Options set at construction apply to all calls; options +passed to `instruct()` or `chat()` apply to that call only and take precedence: + +```python +# Requires: mellea[litellm] +# Returns: MelleaSession +from mellea import MelleaSession +from mellea.backends import ModelOption +from mellea.backends.litellm import LiteLLMBackend + +m = MelleaSession( + LiteLLMBackend( + model_id="anthropic/claude-sonnet-4-20250514", + model_options={ModelOption.TEMPERATURE: 0.2, ModelOption.MAX_NEW_TOKENS: 512}, + ), +) +``` + +See [Configure Model Options](../how-to/configure-model-options.md) for the full list of +`ModelOption` keys. + +## Troubleshooting + +**`ImportError: The LiteLLM backend requires extra dependencies`:** + +```bash +pip install 'mellea[litellm]' +``` + +**"litellm allows for unknown / non-openai input params" or "litellm may drop the +following openai keys" warnings:** LiteLLM supports different parameters per provider. +Mellea logs which model options are unrecognized or may be dropped for the current +model, and passes the request through anyway. These warnings are informational — the +unsupported OpenAI parameters are dropped automatically (`drop_params=True`), and there +are occasional false positives. Remove the flagged options if a call misbehaves. + +**"There is a known bug with litellm. This generation call may fail" warning:** This +appears for WatsonX-over-LiteLLM when Mellea detects generation calls running across +multiple asyncio event loops. Run only synchronous Mellea functions, or run your async +Mellea code from a single `asyncio.run()` call. + +--- + +**See also:** [Backends and Configuration](../how-to/backends-and-configuration.md) | +[AWS Bedrock](./bedrock.md) | [Vertex AI](./vertex-ai.md) | [IBM WatsonX](./watsonx.md) diff --git a/docs/sidebars.ts b/docs/sidebars.ts index 0c6db0ba1..fd02bfa1d 100644 --- a/docs/sidebars.ts +++ b/docs/sidebars.ts @@ -79,6 +79,7 @@ const sidebars: SidebarsConfig = { 'integrations/ollama', 'integrations/huggingface', 'integrations/openai', + 'integrations/litellm', 'integrations/vertex-ai', 'integrations/bedrock', 'integrations/watsonx', From a69e2e74813e1a7c1002e717e5ccc012dd75e433 Mon Sep 17 00:00:00 2001 From: "Paul S. Schweigert" Date: Fri, 14 Aug 2026 16:31:36 -0400 Subject: [PATCH 02/12] style Signed-off-by: Paul S. Schweigert --- docs/docs/integrations/litellm.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/docs/integrations/litellm.md b/docs/docs/integrations/litellm.md index 9d9cf4052..5564ec8ec 100644 --- a/docs/docs/integrations/litellm.md +++ b/docs/docs/integrations/litellm.md @@ -1,13 +1,13 @@ --- title: "LiteLLM" -description: "Reach 100+ LLM providers through Mellea's LiteLLMBackend — direct calls, a self-hosted proxy, and the recommended migration path off the deprecated WatsonX backend." +description: "Reach 100+ LLM providers through Mellea's LiteLLMBackend: direct calls, a self-hosted proxy, and the recommended migration path off the deprecated WatsonX backend." sidebar_label: "LiteLLM" # diataxis: how-to --- The [`LiteLLMBackend`](../reference/glossary#litellm--litellmbackend) gives Mellea a -single code path to 100+ model providers — Anthropic, AWS Bedrock, Azure OpenAI, IBM -WatsonX, Google Vertex AI, and more — by delegating provider auth and request +single code path to 100+ model providers (Anthropic, AWS Bedrock, Azure OpenAI, IBM +WatsonX, Google Vertex AI, and more) by delegating provider auth and request translation to [LiteLLM](https://docs.litellm.ai/). You switch providers by changing the `model_id` prefix; your Mellea code stays the same. @@ -105,8 +105,8 @@ m = MelleaSession( ### Azure OpenAI -The `model_id` is `azure/` — your Azure deployment name, not a -base model name: +The `model_id` is `azure/`, your Azure deployment name rather +than a base model name: ```bash export AZURE_API_KEY=your-api-key-here @@ -128,7 +128,7 @@ m = MelleaSession( ### IBM WatsonX LiteLLM reaches WatsonX with the `watsonx/` prefix. Note the API-key variable is -`WATSONX_APIKEY` (no underscore before `KEY`) — different from the native backend's +`WATSONX_APIKEY` (no underscore before `KEY`), which differs from the native backend's `WATSONX_API_KEY`: ```bash @@ -153,8 +153,8 @@ m = MelleaSession( Both use `LiteLLMBackend` and have dedicated pages with credential setup and model-string tables: -- **Bedrock** — `bedrock/converse/`. See [AWS Bedrock](./bedrock.md). -- **Vertex AI** — `vertex_ai/`. See [Vertex AI](./vertex-ai.md). +- **Bedrock**: `bedrock/converse/`. See [AWS Bedrock](./bedrock.md). +- **Vertex AI**: `vertex_ai/`. See [Vertex AI](./vertex-ai.md). ## Self-hosted LiteLLM Proxy @@ -199,7 +199,7 @@ m = MelleaSession( ) ``` -`my-model` is the `model_name` you defined in the proxy's `config.yaml` — the app never +`my-model` is the `model_name` you defined in the proxy's `config.yaml`; the app never sees which provider or model backs it. To pass the proxy key explicitly instead of using the environment variable, add it to `model_options`: @@ -306,7 +306,7 @@ pip install 'mellea[litellm]' **"litellm allows for unknown / non-openai input params" or "litellm may drop the following openai keys" warnings:** LiteLLM supports different parameters per provider. Mellea logs which model options are unrecognized or may be dropped for the current -model, and passes the request through anyway. These warnings are informational — the +model, and passes the request through anyway. These warnings are informational: the unsupported OpenAI parameters are dropped automatically (`drop_params=True`), and there are occasional false positives. Remove the flagged options if a call misbehaves. From a30850e1d2a3cd4027b9c5a030c9c10797c22f15 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:29:17 -0400 Subject: [PATCH 03/12] Update docs/docs/how-to/backends-and-configuration.md Signed-off-by: Angelo Danducci --- docs/docs/how-to/backends-and-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index 3eeca09a3..d02865abc 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -129,7 +129,7 @@ print(str(result)) # Output will vary — LLM responses depend on model and temperature. ``` -See the [LiteLLM integration](../integrations/litellm) for direct-mode vs proxy usage, +See the [LiteLLM integration](../integrations/litellm.md) for direct-mode vs proxy usage, self-hosted proxy setup, per-provider configuration examples, an environment variable reference, and the WatsonX migration path. From 99b629fd6d5dde3f000ab7a23cd5527dc68090e3 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:29:29 -0400 Subject: [PATCH 04/12] Update docs/docs/how-to/backends-and-configuration.md Signed-off-by: Angelo Danducci --- docs/docs/how-to/backends-and-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index d02865abc..eb2d1a0de 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -32,7 +32,7 @@ The following table shows all available backends, their class names, import path | ------- | ----- | ------ | --------------- | ----------------- | | [Ollama](../integrations/ollama) | `OllamaModelBackend` | `mellea.backends.ollama` | base | `backend_name="ollama"` | | [OpenAI](../integrations/openai) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | -| [LiteLLM](../integrations/litellm) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | +| [LiteLLM](../integrations/litellm.md) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | | [Hugging Face](../integrations/huggingface) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | | [WatsonX](../integrations/watsonx) | `WatsonxAIBackend` | `mellea.backends.watsonx` | `mellea[watsonx]` | `backend_name="watsonx"` (deprecated) | From e86af4381b590a8cd7f3764fb247a055f5439921 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:29:36 -0400 Subject: [PATCH 05/12] Update docs/docs/how-to/backends-and-configuration.md Signed-off-by: Angelo Danducci --- docs/docs/how-to/backends-and-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index eb2d1a0de..79e93ce1a 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -31,7 +31,7 @@ The following table shows all available backends, their class names, import path | Backend | Class | Import | Required extras | `start_session()` | | ------- | ----- | ------ | --------------- | ----------------- | | [Ollama](../integrations/ollama) | `OllamaModelBackend` | `mellea.backends.ollama` | base | `backend_name="ollama"` | -| [OpenAI](../integrations/openai) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | +| [OpenAI](../integrations/openai.md) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | | [LiteLLM](../integrations/litellm.md) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | | [Hugging Face](../integrations/huggingface) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | | [WatsonX](../integrations/watsonx) | `WatsonxAIBackend` | `mellea.backends.watsonx` | `mellea[watsonx]` | `backend_name="watsonx"` (deprecated) | From 9a541ba490b0cd57abfcfdc31f58a3199de20de3 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:29:43 -0400 Subject: [PATCH 06/12] Update docs/docs/how-to/backends-and-configuration.md Signed-off-by: Angelo Danducci --- docs/docs/how-to/backends-and-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index 79e93ce1a..375c67e68 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -30,7 +30,7 @@ The following table shows all available backends, their class names, import path | Backend | Class | Import | Required extras | `start_session()` | | ------- | ----- | ------ | --------------- | ----------------- | -| [Ollama](../integrations/ollama) | `OllamaModelBackend` | `mellea.backends.ollama` | base | `backend_name="ollama"` | +| [Ollama](../integrations/ollama.md) | `OllamaModelBackend` | `mellea.backends.ollama` | base | `backend_name="ollama"` | | [OpenAI](../integrations/openai.md) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | | [LiteLLM](../integrations/litellm.md) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | | [Hugging Face](../integrations/huggingface) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | From a5734d25128baae9575abe739360963bb9cffc8b Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:29:52 -0400 Subject: [PATCH 07/12] Update docs/docs/how-to/backends-and-configuration.md Signed-off-by: Angelo Danducci --- docs/docs/how-to/backends-and-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index 375c67e68..db1078606 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -33,7 +33,7 @@ The following table shows all available backends, their class names, import path | [Ollama](../integrations/ollama.md) | `OllamaModelBackend` | `mellea.backends.ollama` | base | `backend_name="ollama"` | | [OpenAI](../integrations/openai.md) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | | [LiteLLM](../integrations/litellm.md) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | -| [Hugging Face](../integrations/huggingface) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | +| [Hugging Face](../integrations/huggingface.md) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | | [WatsonX](../integrations/watsonx) | `WatsonxAIBackend` | `mellea.backends.watsonx` | `mellea[watsonx]` | `backend_name="watsonx"` (deprecated) | > **Note:** Vertex AI uses the LiteLLM backend with appropriate model IDs. See the [Vertex AI integration](../integrations/vertex-ai) for details. For detailed setup instructions, click the backend name in the table above. From 8a5e0c695ea1d126670f862d7111713d6097b042 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:29:59 -0400 Subject: [PATCH 08/12] Update docs/docs/how-to/backends-and-configuration.md Signed-off-by: Angelo Danducci --- docs/docs/how-to/backends-and-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/how-to/backends-and-configuration.md b/docs/docs/how-to/backends-and-configuration.md index db1078606..adceda481 100644 --- a/docs/docs/how-to/backends-and-configuration.md +++ b/docs/docs/how-to/backends-and-configuration.md @@ -34,7 +34,7 @@ The following table shows all available backends, their class names, import path | [OpenAI](../integrations/openai.md) | `OpenAIBackend` | `mellea.backends.openai` | base | `backend_name="openai"` | | [LiteLLM](../integrations/litellm.md) | `LiteLLMBackend` | `mellea.backends.litellm` | `mellea[litellm]` | `backend_name="litellm"` | | [Hugging Face](../integrations/huggingface.md) | `LocalHFBackend` | `mellea.backends.huggingface` | `mellea[hf]` | `backend_name="hf"` | -| [WatsonX](../integrations/watsonx) | `WatsonxAIBackend` | `mellea.backends.watsonx` | `mellea[watsonx]` | `backend_name="watsonx"` (deprecated) | +| [WatsonX](../integrations/watsonx.md) | `WatsonxAIBackend` | `mellea.backends.watsonx` | `mellea[watsonx]` | `backend_name="watsonx"` (deprecated) | > **Note:** Vertex AI uses the LiteLLM backend with appropriate model IDs. See the [Vertex AI integration](../integrations/vertex-ai) for details. For detailed setup instructions, click the backend name in the table above. From 8e171d49aaa1df7833e6e305d19bc87467651f63 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:30:29 -0400 Subject: [PATCH 09/12] Update docs/docs/integrations/litellm.md Signed-off-by: Angelo Danducci --- docs/docs/integrations/litellm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/integrations/litellm.md b/docs/docs/integrations/litellm.md index 5564ec8ec..2f25d6ebd 100644 --- a/docs/docs/integrations/litellm.md +++ b/docs/docs/integrations/litellm.md @@ -5,7 +5,7 @@ sidebar_label: "LiteLLM" # diataxis: how-to --- -The [`LiteLLMBackend`](../reference/glossary#litellm--litellmbackend) gives Mellea a +The [`LiteLLMBackend`](../reference/glossary.md#litellm--litellmbackend) gives Mellea a single code path to 100+ model providers (Anthropic, AWS Bedrock, Azure OpenAI, IBM WatsonX, Google Vertex AI, and more) by delegating provider auth and request translation to [LiteLLM](https://docs.litellm.ai/). You switch providers by changing From 61b2e03a7843abcc6c29ee1b1e72fad735763238 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:30:37 -0400 Subject: [PATCH 10/12] Update docs/docs/integrations/litellm.md Signed-off-by: Angelo Danducci --- docs/docs/integrations/litellm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/integrations/litellm.md b/docs/docs/integrations/litellm.md index 2f25d6ebd..5e3c7e8d2 100644 --- a/docs/docs/integrations/litellm.md +++ b/docs/docs/integrations/litellm.md @@ -44,7 +44,7 @@ run; Mellea talks to the proxy, and the proxy talks to the providers. ## Direct mode -The quickest path is [`start_session()`](../reference/glossary#melleasession) with +The quickest path is [`start_session()`](../reference/glossary.md#melleasession) with `backend_name="litellm"`. The `model_id` is a LiteLLM model string of the form `/`: From 47fe8568b1d5390914d1c344c4a35fc8324d96a8 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:30:44 -0400 Subject: [PATCH 11/12] Update docs/docs/integrations/litellm.md Signed-off-by: Angelo Danducci --- docs/docs/integrations/litellm.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/integrations/litellm.md b/docs/docs/integrations/litellm.md index 5e3c7e8d2..df075376c 100644 --- a/docs/docs/integrations/litellm.md +++ b/docs/docs/integrations/litellm.md @@ -66,8 +66,8 @@ print(str(result)) > the correct endpoint from the `model_id` prefix. Only set `base_url` when you target > a proxy or a local server (see [Self-hosted LiteLLM Proxy](#self-hosted-litellm-proxy)). -For full control, construct the [`Backend`](../reference/glossary#backend) directly and -pass it to [`MelleaSession`](../reference/glossary#melleasession): +For full control, construct the [`Backend`](../reference/glossary.md#backend) directly and +pass it to [`MelleaSession`](../reference/glossary.md#melleasession): ```python # Requires: mellea[litellm] From b9f37e74abd706155bbbf129195e994e20c2df09 Mon Sep 17 00:00:00 2001 From: Angelo Danducci Date: Mon, 17 Aug 2026 17:30:53 -0400 Subject: [PATCH 12/12] Update docs/docs/integrations/litellm.md Signed-off-by: Angelo Danducci --- docs/docs/integrations/litellm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/integrations/litellm.md b/docs/docs/integrations/litellm.md index df075376c..215628774 100644 --- a/docs/docs/integrations/litellm.md +++ b/docs/docs/integrations/litellm.md @@ -273,7 +273,7 @@ the full list and any provider-specific variables. ## Model options -Pass generation parameters with [`ModelOption`](../reference/glossary#modeloption), the +Pass generation parameters with [`ModelOption`](../reference/glossary.md#modeloption), the same as any other backend. Options set at construction apply to all calls; options passed to `instruct()` or `chat()` apply to that call only and take precedence: