[andrewklatzke]

Summary

• add @launchdarkly/ai-litellm-messages using the LiteLLM proxy's OpenAI-compatible API
• add @launchdarkly/ai-litellm-agents using the OpenAI Agents SDK over the same proxy
• infer LITELLM_BASE_URL and optional LITELLM_API_KEY, while allowing explicit overrides
• support tools, streaming, structured output, conversation history, native graphs, telemetry, and release configuration

Usage

import { litellmMessages } from '@launchdarkly/ai-litellm-messages';

const result = await litellmMessages(
  'launch-darkly-documentation-summarizer-messages',
  'How do AI Configs work?',
  { kind: 'user', key: 'user-123' },
);
console.log(result.response);

Set LITELLM_BASE_URL to the proxy's /v1 endpoint. LITELLM_API_KEY is optional for unauthenticated proxies.

Test plan

• LiteLLM agents tests (33 tests)
• LiteLLM messages tests (25 tests)
• yarn typecheck
• yarn build
• yarn code:check
• yarn lint:pkg
• LiteLLM messages and agents integration scenarios

The current main full test command also runs an unrelated Claude native-graph suite whose module mock is missing the newly added makeNodeTrackData export. The LiteLLM workspaces and all static/build checks pass.

Made with Cursor

[cursor]

Stale Bugbot comment from a previous run.

[jeffdupont]

Parity note alongside my review of python #102 (launchdarkly/python-ai-sdk#102) and the spec in launchdarkly/ai-sdks-monorepo#18. Tests pass at 093e771: yarn test exit 0 (litellm-agents 33, litellm-messages 26, and every other workspace), and yarn typecheck and yarn build both exit 0. The branch is 30 commits behind main (d32bede) and conflicts in six files: .release-please-manifest.json, .sdk_metadata.json, AGENTS.md, README.md, release-please-config.json and tsconfig.json. They look like list conflicts with the Vercel packages that landed on main. Every client export these packages import is still on main, but I haven't built the merged tree.

Things I'd like settled before merge:

1. Config parameters reach the proxy request. Since the Bugbot fix, api_key and base_url are stripped, but that's a denylist of two. I reproduced it with an injected client: api_base, user_config, extra_headers and metadata from model.parameters all show up in the chat.completions.create body. Whether the proxy acts on api_base / user_config depends on its client-side credential settings. I haven't tested that against a real proxy. In Python there's no proxy in the way, and the same pattern sends the customer's provider key to whatever host the config names (reproduced on python #102). I'd suggest one allowlist rule in both languages, written into the spec, as on #73 / python #107.
2. toLiteLLMAgents has no telemetry. It opens no span, sends no $ld:ai:graph:* events, and has no context option. Python's version emits both. The JS openai-agents native graph on main emits $ld:ai:graph:node, handoff_success, generation:success, invocation_success / failure, duration:total and total_tokens. Monorepo #18 says this adapter satisfies §2.2, and §2.2 requires those events.
3. setTracingDisabled(true) turns off OpenAI Agents tracing for the whole process as soon as a handler is created (handler.ts:408, native-graph.ts:131). A customer who also uses @openai/agents directly loses those traces when they add this package. The Runner's per-run tracingDisabled option (agents-core/dist/run.d.ts:98) limits it to our runs.
4. The public names and data don't match Python.
   • createLiteLLMAgentHandler here, create_litellm_agents_handler in Python. The other agent packages use the same name in both languages, and this one is about to freeze.
   • When the config has no provider, gen_ai.provider.name defaults to openai here and litellm in Python.
   • Content capture in the messages handler writes only the legacy gen_ai.prompt.N.* / gen_ai.completion.0.* attributes. Every other JS handler, and Python's litellm-messages, use setInputContentAttributes / setOutputContentAttributes.
   • Agent parameters: snake_case max_tokens / top_p are ignored here and honoured in Python.
5. Experimental. Same as on python #102: I'd keep both LiteLLM packages at 0.x, out of the 1.0 cut, and marked experimental until the two languages match. Monorepo #33 needs a line saying how a whole provider package counts as experimental.

Smaller notes:

• createLiteLLMAgentHandler() throws at construction when there's no base URL (handler.ts:55), but the messages handler throws at call time (handler.ts:197). It would help to pick one and write it into A.13.
• modelFor passes the parameter bag as the third constructor argument whenever the bag has a model key (handler.ts:79). That argument is OpenAIChatCompletionsModelOptions, whose only field is strictFeatureValidation (agents-openai/dist/openaiChatCompletionsModel.d.ts:8), so the branch has no useful effect.
• Streamed tool calls run one after another (handler.ts:443), but invoke runs them with Promise.all. Python runs both in parallel. That's harmless, just worth knowing.

[cursor]

Stale Bugbot comment from a previous run.

[andrewklatzke]

Addressed the review on d82847d, then merged main in 6f67463.

• Configs can only forward generation settings. api_base, extra_headers, metadata, and the other connection keys never reach the request. Agent settings map the allowlisted snake_case names onto the Agents SDK camelCase fields, with no providerData.
• Renamed createLiteLLMAgentsHandler / LiteLLMAgentsOptions (no alias; nothing has shipped). Missing base URL throws when the handler is created.
• Native graphs now open a launchdarkly.graph span and emit $ld:ai:graph:node, generation, handoff, and invocation events. Tracing is new Runner({ tracingDisabled: true }) per run.
• Content capture uses setInputContentAttributes / setOutputContentAttributes. Streamed tool calls run with Promise.all. Missing provider name defaults to litellm. Both READMEs say the packages stay on 0.x.

Left as-is: convenience wrappers still drop history, same as the other providers. The promotion register lives in the monorepo, not this repo.

[cursor]

Stale Bugbot comment from a previous run.

[jeffdupont]

Re-checked at 6f67463. Thanks, most of my 2 Oct points are fixed: parameter allowlist, per-run tracingDisabled, factory name, litellm provider default, content capture helpers, construct-time throw, and parallel streamed tools.

Still blocking:

1. Merge conflict with main in .release-please-manifest.json.
2. verbosity is dropped. litellm-agents/src/handler.ts:45 sets top-level modelSettings.verbosity, but the Agents SDK reads text.verbosity (see buildModelSettings in openai-agents/src/model-parameters.ts). This also skips that function's reasoning filtering and the text/truncation/contextManagement keys. Spec §2.x.2 says settings follow the openai-agents contract, so reuse or mirror buildModelSettings.
3. Graph span leaks on setup failure. In litellm-agents/src/native-graph.ts, the span opens at :116, but reverseTraverse and buildInput run before the try at :196. A missing tool handler leaves the span open and sends no invocation_failure. openai-agents wraps setup and run in one try/finally.
4. No CommonJS export. Both packages export only import. Other packages ship a require condition (e.g. openai-agents/package.json), so CJS consumers get ERR_PACKAGE_PATH_NOT_EXPORTED.
5. New graph telemetry is untested. native-graph.test.ts only gained on = vi.fn().

Worth settling with python-ai-sdk#102: graph-level events use makeNodeTrackData(root), while main's openai-agents uses makeGraphTrackData and doesn't call setLdSpanAttributes. The agents allowlists also differ: Python allows metadata/include_usage/top_logprobs, JS doesn't. The spec in ai-sdks-monorepo#18 still describes the old denylist and should record whichever list wins.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c26968a. Configure here.}

[cursor]

Graph always uses the root model

High Severity

The native-graph Runner modelProvider.getModel ignores its argument and always returns the root node's model. The Agents SDK resolves every agent through that provider, so handoff targets run as the root model instead of their own evaluated model.

 

^{Reviewed by Cursor Bugbot for commit c26968a. Configure here.}

[andrewklatzke]

I think this is a false positive. Every agent is constructed with its own Model instance (modelFor(client, node)), and the Runner only calls modelProvider.getModel when an agent's model is a string (#resolveModelForAgent in agents-core run.js). Handoff targets therefore run on their own model, and getModel is effectively unused. Leaving this open in case you'd like the provider removed.

[jeffdupont]

Checked this against the LiteLLM spec in monorepo #18. Exports, allowlists, tracing-off and the graph option forwarding all line up. LGTM, just merge #18 alongside so the api-surface lists exist on main.