[prodmanpd]

What does this PR do?

Adds LiteLLM as a first class chat provider for the AI SRE agent, registered in the eino model builder registry alongside openai, deepseek, qwen, ollama, claude, and gemini. Set agent.ai.provider: litellm to route detect and analyze through a LiteLLM proxy, one OpenAI compatible endpoint that fans out to 100+ upstream providers, reusing the already vendored eino-ext/components/model/openai with no new dependency.

Changes:

• pkg/agent/ai/eino/provider.go: buildLiteLLMChatModel plus the "litellm" registry entry.
• pkg/agent/ai/eino/provider_test.go: registry, exported validator, and egress tests for litellm.
• pkg/config/agent.go, pkg/agent/aisettings.go, pkg/agent/ai/eino/holder.go: provider list doc comments.
• config/config.yaml, pkg/config/default_config.yaml: provider comment plus a litellm usage note.
• src/agent/configuration.md: provider table row plus a LiteLLM gateway section.

Why?

This adds gateway support so operators can centralise keys, spend caps, caching, and fallbacks in one proxy while Versus keeps a single stable config. The proxy speaks the OpenAI chat completions wire format, so it slots into the registry as a real discrete builder rather than a generic base url toggle, which means SupportedProviders() (the enterprise write boundary validator) and the runtime fail fast path both cover it for free. Prior art: #160 proposed an OpenAI compatible base_url field naming Gemini, LiteLLM, and OpenRouter; the AI layer then moved to the Eino framework with discrete first class builders, and this PR follows that direction with a real buildLiteLLMChatModel.

How to test

Sample config:

agent:
  ai:
    enable: true
    provider: litellm
    api_key: ${LITELLM_VIRTUAL_KEY}   # proxy virtual key (Bearer)
    model: gpt-5                       # a LiteLLM model alias from your proxy config
    temperature: 0.2
    max_tokens: 1024

model is the alias defined in your proxy config (not a raw provider id); the endpoint defaults to a local sidecar at http://localhost:4000/v1.

Unit tests, go test -race ./pkg/agent/ai/eino/...:

--- PASS: TestChatModel_RegistryBuildsAllProviders/litellm
--- PASS: TestSupportedProvidersExported
--- PASS: TestChatModel_LiteLLM_EgressBearerAndJSONMode
ok  github.com/VersusControl/versus-incident/pkg/agent/ai/eino

The egress test spins up an httptest OpenAI compatible server and asserts that the virtual key rides the Bearer Authorization header, that JSON mode sends response_format:{type:json_object}, and that the reply round trips through detect.ParseFinding.

Live end to end through a real LiteLLM proxy. NewToolCallingChatModel(provider=litellm) builds buildLiteLLMChatModel, which calls the OpenAI SDK, which calls the LiteLLM proxy, which calls the upstream:

LIVE litellm response via http://localhost:4000/v1 (model=azure/gpt-4o-mini): "LITELLM_OK"
--- PASS: TestLive_LiteLLM (1.05s)

Proxy side confirmation of the same call (Azure OpenAI upstream through the proxy):

content: LITELLM_OK
model: azure/gpt-4o-mini
usage: {'completion_tokens': 6, 'prompt_tokens': 16, 'total_tokens': 22, ...}

Bug caught and fixed during the deep dive: the proxy path shares the OpenAI SDK, which validates the beta limitation client side, so for reasoning families (gpt-5.*, o series) an explicit temperature is rejected before the request leaves. Naively forwarding the configured temperature broke a gpt-5 alias. The builder now reuses the same isFixedSamplingModel omission as buildOpenAIChatModel, so an operator aliasing gpt-5 through the proxy does not have to set temperature: -1 by hand. The egress test asserts that temperature is absent from the wire for a gpt-5 alias.

Build, vet, and format are clean: go build ./..., go vet ./pkg/agent/ai/eino/... ./pkg/config/..., and gofmt -l. go test -race ./pkg/... is green for every package that builds locally, including pkg/agent/ai/eino, pkg/agent/ai/detect, pkg/agent/ai/analyze, pkg/config, and pkg/core. pkg/controllers and pkg/routes only fail to build locally because they //go:embed ui/dist, which needs the frontend build and is untouched by this PR.

Type of change

• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (config / API / default behavior)
• Documentation only
• Refactor (no functional change)
• CI / build / chore

Checklist

• go test ./... passes locally
• go vet ./... is clean
• Code is gofmt'd
• Added or updated tests for the change
• Updated user-facing docs under src/ if behavior changes
• Updated ROADMAP.md if this closes a roadmap item
• No secrets, tokens, or webhook URLs introduced in source / YAML
• No new third-party dependencies (or justified in the description)

[hoalongnatsu]

Thanks for the PR @prodmanpd; we'll review it

[hoalongnatsu]

Hi @prodmanpd

Blocker

B1 — The proxy endpoint is not configurable; LiteLLM is locked to localhost

buildLiteLLMChatModel defaults BaseURL to http://localhost:4000/v1 and
otherwise uses req.baseURL — but that field is a test-only seam
(pkg/agent/ai/eino/provider.go:37 — "test-only Options.BaseURL; "" uses the
provider default"), and there is no base_url mapstructure field anywhere in
AgentAIConfig. An operator therefore cannot point this provider at any proxy
other than a local sidecar.

That undercuts the main reason teams adopt LiteLLM: a shared, centralised
gateway (https://litellm.internal.corp) holding the keys, spend limits and
fallbacks. The ollama/qwen precedent does not apply — their defaults are the
canonical endpoints for those backends, whereas a LiteLLM address is inherently
deployment-specific.

The documentation compounds it by omission. src/agent/configuration.md says
"The endpoint defaults to a local proxy at http://localhost:4000/v1 (the
common sidecar deployment)", which reads as though a knob exists.

Ask: add an operator-settable base_url (env AGENT_AI_BASE_URL) wired
through AgentAIConfig → chatModelRequest, or — if that is out of scope for
this PR — state the limitation explicitly in the docs and the config comments.
The knob would also benefit self-hosted OpenAI-compatible backends (vLLM,
LocalAI).

---

Should-fix

S1 — UI provider list not updated, breaking a stated invariant

ui/src/lib/api.ts declares the AIProvider union and AI_PROVIDERS array with
a comment stating it "mirrors the OSS chat-model registry
(eino.SupportedProviders)". litellm is missing from both.

The enterprise runtime override derives its supported set from
eino.SupportedProviders(), so the server already accepts litellm — but the
AI-settings dropdown cannot offer it. Net effect: LiteLLM is YAML-only, with no
runtime provider switch, and the documented mirror invariant is now false.

HEADER_AUTH_PROVIDERS correctly needs no change — LiteLLM authenticates with a
Bearer virtual key, so the per-org runtime key override applies as-is.

S2 — Embeddings gap for an operator who standardises on LiteLLM

embedderBuilders (pkg/agent/ai/eino/embedder.go:43) is {openai, ollama, gemini}. Setting provider: litellm and using runbook RAG fails hard with
"unsupported ai provider "litellm" for embeddings".

Unlike deepseek/claude — deliberately unwired because they have no embedding
component — LiteLLM does proxy OpenAI-compatible /v1/embeddings, so this is
an avoidable gap. It is trivially wired by reusing buildOpenAIEmbedder with the
same endpoint default. At minimum, document that RAG needs a different embedding
provider.

[liuzhao1225]

I'm concerned about the supply-chain boundary of this LiteLLM integration. The upstream incident report confirms compromised PyPI releases 1.82.7/1.82.8 in March 2026; those releases were removed.

This reviewed change uses a proxy-compatible integration; it does not itself add a Python LiteLLM package installation. The deployment questions are who operates the proxy, which verified server version runs there, and whether credentials remain isolated from other providers. Please document those trust boundaries for users.

This is one of 49 observed LiteLLM integration PRs from the same account, whose author acknowledged the cross-project effort. The pattern prompted this review; it does not establish malicious intent or connect the author to the incident. The consolidated questions and corrections are in YouDub #130; this PR's review snapshot preserves the revision and scope.