Skip to content

feat(llm): complete validation and credential plumbing for custom providers - #128

Open
dglezz1 wants to merge 1 commit into
google:mainfrom
dglezz1:feat/custom-llm-providers
Open

dglezz1 wants to merge 1 commit into
google:mainfrom
dglezz1:feat/custom-llm-providers

Conversation

@dglezz1

@dglezz1 dglezz1 commented Sep 18, 2026

Copy link
Copy Markdown

What

Artemis already routed provider: "custom" / "ollama" / "vllm" to ChatOpenAI with a configurable api_base, but the surrounding plumbing had two gaps:

  1. LLM.validate_provider had no branch for those providers, so a node configured with provider: "custom" could fail later at runtime with a less actionable error.
  2. Settings.get_api_key / set_api_key did not handle custom / ollama / vllm, leaving the admin UI without a way to set the key and the env normalization loop without a slot for a dedicated CUSTOM_LLM_API_KEY.

This PR closes both gaps and makes the configuration discoverable in .env.example.

Why

  • Many teams run Artemis against an OpenAI-compatible gateway (MiniMax, OpenRouter forks, internal proxies) but had no first-class way to declare the endpoint + key.
  • Local Ollama / vLLM users currently have to abuse OPENAI_BASE_URL; this adds dedicated env vars while keeping the existing path working.
  • The validator gap produces a confusing failure mode that we want caught at config-load time, not during the first LLM call.

Changes

  • artemis/config/llm.pyvalidate_provider now branches on custom / ollama / vllm. Custom requires a base URL (any of OPENAI_BASE_URL, CUSTOM_LLM_BASE_URL, or per-node api_base) and an API key. Local ollama / vllm tolerate a missing API key.
  • artemis/config/settings.py — adds CUSTOM_LLM_API_KEY / CUSTOM_LLM_BASE_URL / CUSTOM_LLM_MODEL with backwards-compatible fallback to OPENAI_API_KEY / OPENAI_BASE_URL. get_api_key and set_api_key now route the new providers through the dedicated fields. CUSTOM_LLM_API_KEY joins the placeholder-filter loop.
  • artemis/config/constants.py — adds ENV_CUSTOM_LLM_API_KEY / ENV_CUSTOM_LLM_BASE_URL / ENV_CUSTOM_LLM_MODEL.
  • .env.example — adds a Custom / Local OpenAI-Compatible Providers section with MiniMax, Ollama, and vLLM recipes.
  • tests/unit/test_custom_llm_providers.py — 8 new tests covering: missing base URL, missing API key (custom only), fallback to OPENAI_API_KEY, dedicated CUSTOM_LLM_* env vars, ollama/vllm tolerance for missing key, and Settings round-trips.

Verification

$ uv run pytest tests/unit/test_custom_llm_providers.py -v
============================= 8 passed, 1 warning in 0.12s ==============================

Existing test_config_package.py failures (7) are pre-existing in main (verified with git stash); they assume GOOGLE_API_KEY is set in the test environment. This PR does not introduce or fix them.

End-to-end smoke test against a hosted OpenAI-compatible endpoint (MiniMax, https://api.minimax.io/v1) using CUSTOM_LLM_* env vars returned a valid chat.completions response with model MiniMax-M3.

Backwards compatibility

  • Existing configs with provider: "openai" and OPENAI_BASE_URL continue to work unchanged.
  • provider: "custom" / "ollama" / "vllm" previously failed silently at runtime; the new validator now fails fast at config load with an actionable message.
  • .env stays gitignored (root .gitignore:105).

Out of scope

  • Auth header customization beyond Authorization: Bearer (some providers need x-api-key, etc.) — happy to follow up if needed.
  • Streaming-format normalization — OpenAI-compatible endpoints already stream SSE fine via LangChain.

…viders

The router already routed provider "custom" / "ollama" / "vllm" to
ChatOpenAI with a configurable api_base (router.py:369), but the
surrounding plumbing had two gaps:

  1. LLM.validate_provider had no branch for those providers, so a
     node configured with `provider: "custom"` could fail later in
     runtime with a less actionable error.
  2. Settings.get_api_key / set_api_key did not handle custom /
     ollama / vllm, leaving the admin UI without a way to set the key
     and leaving the env normalization loop without a slot for a
     dedicated CUSTOM_LLM_API_KEY.

This change:

* Adds LLM.validate_provider branches for custom / ollama / vllm.
  Custom endpoints require both a base URL (any of OPENAI_BASE_URL,
  CUSTOM_LLM_BASE_URL, or per-node api_base) and an API key. Local
  Ollama / vLLM servers tolerate a missing API key.
* Adds CUSTOM_LLM_API_KEY / CUSTOM_LLM_BASE_URL / CUSTOM_LLM_MODEL
  fields on Settings, with backwards-compatible fallback to
  OPENAI_API_KEY / OPENAI_BASE_URL when unset.
* Wires get_api_key and set_api_key for the new providers and
  registers CUSTOM_LLM_API_KEY in the placeholder-filter loop.
* Adds ENV_CUSTOM_LLM_* constants for env-driven provisioning.
* Documents the new env vars and provider configuration in
  .env.example with MiniMax, Ollama, and vLLM recipes.
* Adds tests/unit/test_custom_llm_providers.py covering all eight
  paths: missing base URL, missing API key (custom only), fallback
  to OPENAI_API_KEY, dedicated CUSTOM_LLM_* env vars, ollama/vllm
  tolerance for missing key, and Settings round-trips.

The .env file remains gitignored (root .gitignore:105).
@google-cla

google-cla Bot commented Sep 18, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant