[ipsorakis]

Summary

Test-only change. No production code is modified. Adds 16 test files (~200 tests) that run offline: LLM clients, API keys, tokenizers and input() are all faked, so no real model calls are made.

Coverage (pytest -n 8 -m 'not pricy' tests/ --cov=src/agentlab, MiniWoB served locally):

	Statements	Missed	Coverage
main	6465	3740	42%
this PR	6465	2633	59%

New test files, by area:

• llm: test_llm_utils_extra.py covers generic_call_api_with_retries backoff, retriable vs non-retriable status codes, the OpenAI/Anthropic wrappers, retry_multiple, messages_to_dict, BaseMessage/Discussion, and image encoding. test_chat_api_unit.py covers ChatModel retry/RetryError, pricing, Azure/OpenRouter/vLLM setup, AnthropicChatModel and CheatMiniWoBLLM. test_huggingface_utils_unit.py covers the chat-template fallback, retries and log_probs. test_litellm_api_unit.py covers how _call_api builds its params (tools, tool_choice, force_call_tool, cache control), tool-call parsing and LiteLLMAPIMessageBuilder. test_prompt_templates.py covers the prompt templates.
• analyze: test_episode_to_html.py, test_archive_studies.py, test_inspect_results_unit.py (summaries, global and flag reports, error-key mapping, staleness cache, get_study_summary and get_all_summaries).
• experiments: test_exp_utils.py (timeouts, add_dependencies, deprecated helpers), test_reproducibility_util_extra.py (_verify_report, assert_compatible, append_to_journal), test_study_utils.py.
• agents/benchmarks: agent_utils, tmlr_config, MostBasicAgent, DebugAgent, add_step_timing_to_env_info_decorator.

Things reviewers should know

• Existing flaky test, not caused by this PR: tests/agents/test_agent.py::test_llm_error_no_success sometimes fails when the full suite runs in parallel with -n 8 on a Devin VM, on main as well as on this branch. It passes on its own and it passes in CI here.
• Latent bug found, not fixed here: inspect_results.categorize_error and report_different_errors reference ERR_CLASS_MAP, which is never defined. Any row with a non-null err_msg raises NameError. The tests only cover the err_msg=None path.
• Tracker setup in test_chat_api_unit.py: AnthropicChatModel only checks hasattr(TRACKER, "instance"), so it crashes if an earlier test left TRACKER.instance = None in the same worker. Autouse fixtures give each test a fresh threading.local() and fake provider API keys.

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/43b4472bdcda4c1ebc075b23d0b7f601
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/43b4472bdcda4c1ebc075b23d0b7f601?variant=devin
Requested by: @ipsorakis

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

Enable auto-review to review your pull requests automatically.

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

🔍 Devin Review: 3 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)