[devin-ai-integration]

Summary

Test-only PR (child 1/11 of the unit-test fan-out). Adds three hermetic test files covering the LLM utility layer; no src/ or existing test changes.

• tests/llm/test_llm_utils_unit.py (124 tests): messages_to_dict (dict/str, langchain convert_message_to_dict path and role_map fallback via monkeypatched module globals), retry/retry_multiple (asserts the exact messages the fake chat receives on re-prompt), generic_call_api_with_retries (backoff delays, invalid-response/rate-limit/status-code table, exhaustion → RuntimeError, unexpected errors re-raised), OpenAI/Anthropic validity + retriable-vs-fatal status tables using real SDK exception classes, supports_tool_calling_for_openrouter (fake openai.Client), tokenizer helpers (fake tiktoken, cache_clear around each test), parsers, image→base64, BaseMessage/Discussion.
• tests/llm/test_prompt_templates.py (24 tests): format_message, construct_prompt, get_prompt_template matching/unknown model.
• tests/llm/test_huggingface_utils_unit.py (17 tests): _prepend_system_to_first_user (incl. column_remap), HFBaseChatModel.__init__ (mocked AutoTokenizer, GPT2 → prompt-template fallback) and __call__ (chat-template "roles must alternate" retry, server retry loop, n_samples, log_probs).

time.sleep is monkeypatched everywhere; no network, keys, or model downloads.

Coverage (target modules)

module	before (full suite on main)	after (new files only)	after (tests/llm/)
agentlab.llm.llm_utils	48%	95%	97%
agentlab.llm.prompt_templates	47%	100%	100%
agentlab.llm.huggingface_utils	22%	100%	100%

Remaining uncovered in llm_utils: import-time langchain detection (lines 25-35) and the __main__ block.

Tests: 165 (162 pass + 3 strict xfail), ~4s; pass serially, with -n 4, and under random order.

Bugs found (xfail strict, src/ not touched)

1. call_openai_api_with_retries: get_openai_status_code reads http_status, but openai>=1 exceptions expose status_code → openai.InternalServerError (5xx) is re-raised instead of retried.
2. call_anthropic_api_with_retries: anthropic.APIConnectionError is in the retriable error tuple but has no status_code, so transient network failures are re-raised immediately.
3. BaseMessage.merge(): collapses any single-element content via ["text"] → KeyError when the only element is an image.

Run

uv run pytest tests/llm/test_llm_utils_unit.py tests/llm/test_prompt_templates.py tests/llm/test_huggingface_utils_unit.py -p no:cacheprovider -q [-n 4]

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/9d3a89c26fdb4f2c94fe8b58e5a237aa
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/9d3a89c26fdb4f2c94fe8b58e5a237aa?variant=devin
Requested by: @danigrim

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)