[devin-ai-integration]

Summary

Adds two test-only files covering the chat-model wrappers' request building, retry/error handling, and cost tracking, with every SDK client mocked. No src/ changes.

• tests/llm/test_chat_api_unit.py (57 tests): message helpers, CheatMiniWoBLLM, all *ModelArgs.make_model (including the SelfHostedModelArgs backends), _extract_wait_time/handle_error, ChatModel init/pricing/__call__ (retries, OpenRouterError on missing usage, RetryError, n_samples, tracker), the OpenAI/OpenRouter/Azure/VLLM/HF subclasses, and AnthropicChatModel's own retry loop.
• tests/llm/test_litellm_api_unit.py (43 tests): LiteLLMModel._call_api param mapping (tools format conversion, tool_choice any→required, force_call_tool, reasoning_effort, both cache flags), _parse_response / thinking / tool-call extraction, full __call__ via TrackAPIPricingMixin, LiteLLMAPIMessageBuilder tool replay, LiteLLMModelArgs, and the module's __main__ smoke script (run via runpy with Study/ToolUseAgentArgs swapped for fakes).

How it stays hermetic: chat_api.OpenAI, anthropic.Anthropic, InferenceClient and AutoTokenizer are patched; tracking.get_pricing_*, litellm.get_model_info and tracking.completion_cost are stubbed; time.sleep is mocked; API-key env vars are cleared; and an autouse fixture saves and restores the thread-local tracking.TRACKER.instance. Fake responses use real openai.types.chat.ChatCompletion / litellm.ModelResponse objects that include usage.

Coverage (target modules)

Module	Before (whole suite on main)	After (these two files alone)
agentlab.llm.chat_api	40%	100%
agentlab.llm.litellm_api	24%	100%

100 tests: 95 pass, 5 are strict xfail. Each file runs in under 6s. Passes serially, with -n 4, and in random order (pytest-randomly).

Source bugs found (strict xfail, src/ left unchanged)

1. SelfHostedModelArgs.make_model() (huggingface backend) never passes the required base_model_name to HuggingFaceURLChatModel, so it raises TypeError.
2. When log_probs=True, ChatModel.__call__ reads choices[0].log_probs, but the OpenAI SDK field is logprobs, so it raises AttributeError.
3. AnthropicChatModel.__call__ calls tracking.TRACKER.instance(...) even when it is None, which is the state after any set_tracker() block exits. Every attempt then fails and gets retried with sleep(60). ChatModel guards this with an isinstance check.
4. LiteLLMModel._extract_thinking_content_from_response returns the literal string "None" as think for tool-call responses whose content is None.
5. LiteLLMModel(use_only_first_toolcall=True): _call_api calls msg.prepare_message() without the flag. The replayed assistant message therefore keeps every tool call but gets only one tool response (mismatched tool_call_ids).

Run

uv run pytest tests/llm/test_chat_api_unit.py tests/llm/test_litellm_api_unit.py -p no:cacheprovider -q -n 4
uv run --with pytest-cov pytest tests/llm/test_chat_api_unit.py tests/llm/test_litellm_api_unit.py -p no:cacheprovider -q --cov=agentlab.llm.chat_api --cov=agentlab.llm.litellm_api --cov-report=term-missing

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/12ad20f36bdd4cb394cfef967d670aba
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/12ad20f36bdd4cb394cfef967d670aba?variant=devin
Requested by: @danigrim

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.