[devin-ai-integration]

Summary

Adds the first tests for agentlab.agents.tool_use_agent.tool_use_agent in one new file, tests/agents/test_tool_use_agent_unit.py. Nothing in src/ or any existing test changes. The tests are hermetic: no network, browser, API keys or real sleeps.

How the tests are built

• FakeModel(payload: APIPayload) -> LLMOutput returns scripted LLMOutputs and records each payload. It also provides reset_stats and stats.stats_dict.
• FakeModelArgs(BaseModelArgs) returns that model plus a real OpenAIResponseAPIMessageBuilder or AnthropicAPIMessageBuilder.
• FakeActionSet.to_tool_description() stands in for the action set. Some tests use the real bgym.HighLevelActionSet(("coord",)) and OSWorldActionSet("computer_13") instead.
• Hint DBs are small CSVs written to tmp_path. requests.post, time.sleep and random.uniform are monkeypatched, and an autouse fixture resets the TRACKER thread-local.

What is covered (72 tests + 2 xfail)

• StructuredDiscussion.flatten: groups outside the last keep_last_n_obs are replaced by their summary (a group with no summary stays whole). Stale _cache_breakpoints are cleared and exactly one new one is set. Also covers auto group names, the empty/no-goal cases and keep_last_n_obs larger than the number of groups.
• Blocks:
  • Goal: system vs user role for each builder, and text/image/unknown content.
  • Obs and _format_tabs: error, screenshot, SoM, axtree, DOM and tabs parts, and replying "See Observation" to the previous tool calls.
  • GeneralHints and Summarizer: apply_init with high and low detail, and the LLM summary call that sets the group summary.
  • TaskHint:
    • Hint DB path: relative paths resolve next to the module, absolute paths are used as given.
    • Retrieval modes: direct (fnmatch), llm (topic choice, dedupe, bad answers), emb (loading the .npy file, top-n through mocked /encode and /similarity), plus an unknown mode.
    • _encode / _similarity retry with backoff, then raise.
• ToolUseAgentArgs: agent_name derivation, prepare/close passing through to the model args, make_agent falling back to DEFAULT_PROMPT_CONFIG, set_benchmark("osworld") turning on skip_preprocessing, and sanity checks on the predefined configs.
• ToolUseAgent:
  • Construction: default bgym action set for the openai and anthropic tool formats.
  • obs_preprocessor: skip path, page screenshot path, dom/axtree/SoM paths (mocked, plus one run of the real flatten_axtree_to_str).
  • get_action:
    • Payload flags and the system prompt in single- and multi-action mode.
    • Order of the goal and obs messages, and the chat_messages sent to xray (tool description first, tool message last).
    • Cost-tracker stats merging, including nested trackers.
    • A second step answering the first step's tool calls.
    • Summary + general hints + task hints, with the summary prepended to think.
    • keep_last_n_obs=1 replacing old observations with their summaries.

Coverage

Module	Before	After
agentlab.agents.tool_use_agent.tool_use_agent	0%	99% (396 stmts, 1 missed)

The one missed line is the pass body of the abstract Block.apply.

Source bugs found (tests marked xfail(strict=True), src/ not changed)

1. TaskHint.choose_hints_llm: if the LLM answers with valid JSON that is not an int, e.g. "1" (quoted), json.loads returns a str. The comparison hint_topic_idx < 0 then raises an uncaught TypeError instead of returning no hints (only JSONDecodeError is caught).
2. TaskHint.encode_hints: the embeddings path is f"{self.hint_db_rel_path}.embs.npy", which is resolved against the current working directory. _init, by contrast, resolves the CSV next to the module, so with the default relative hint_db.csv the embeddings next to that CSV are never found.

Run

uv run pytest tests/agents/test_tool_use_agent_unit.py -p no:cacheprovider -q          # 72 passed, 2 xfailed, ~3.5s
uv run pytest tests/agents/test_tool_use_agent_unit.py -p no:cacheprovider -q -n 4
uv run --with pytest-cov pytest tests/agents/test_tool_use_agent_unit.py -p no:cacheprovider -q \
  --cov=agentlab.agents.tool_use_agent --cov-report=term-missing

--cov=agentlab.agents.tool_use_agent.tool_use_agent (the submodule) crashes collection with RuntimeError: function '_has_torch_function' already has a docstring. That form makes coverage import the package before pytest does, so torch ends up imported twice. Pointing --cov at the package, as above, avoids it.

Also verified: random order (pytest-randomly with 3 seeds), black --check --line-length 100, and the full suite with -n 4. The full suite shows only the 8 known local failures from missing Chromium/MiniWoB.

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/4338d4551ea24ff19719ba133b6b9230
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/4338d4551ea24ff19719ba133b6b9230?variant=devin
Requested by: @danigrim

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

🔍 Devin Review: 2 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)