[devin-ai-integration]

Summary

Adds hermetic unit tests for the GenericAgent stack — no browser, no LLM, no network, no real sleeps. Only three new test files; nothing in src/ or existing tests is touched.

• tests/agents/test_dynamic_prompting_unit.py (57 tests): Flags copy/asdict/from_dict, visibility, Trunkater, fit_tokens (str / list / BaseMessage, shrink loop, max-iteration error), HTML/AXTree/Error/FocusedElement/Tabs, Observation + screenshots, ActionPrompt parsing and code-block fallback, Think, History, make_obs_preprocessor. Token counting is patched with a word count, so tiktoken never runs.
• tests/agents/test_generic_agent_unit.py (40 tests): GenericAgentArgs (name, set_benchmark per benchmark, legacy flags, set_reproducibility_mode, prepare/close), GenericAgent.get_action driven by a FakeChat callable (happy path, retries, busted retry, plan/memory/think history, fit_tokens delegation), _check_flag_constancy, _get_maxes, MainPrompt, plus Memory/Plan/Criticise parsing.
• tests/agents/test_reproducibility_agent_unit.py (30 tests): ReproChatModel (replay, <action>None</action> fallback, time.sleep patched), ReproAgent.get_action against synthetic exp dirs written with StepInfo.save_step_info in tmp_path (replay, step indexing, no-chat-messages error page, rebuilding a missing response from the recorded action), _format_messages, _make_backward_compatible, _make_diff/_diff_stats/_set_style, make_repro_agent and reproduce_study on a copy of tests/data/test_study/ with Study stubbed out.

Coverage (target modules)

module	before (whole suite on main)	after (these 3 files only)	goal
dynamic_prompting	80%	100%	≥92%
generic_agent.generic_agent	54%	100%	≥85%
generic_agent.generic_agent_prompt	86%	100%	≥95%
generic_agent.reproducibility_agent	0%	98% (only 73-74, the except AttributeError for CrossProd args, is missed)	≥75%

146 test cases (127 test functions, some parametrized): 143 pass and 3 are strict xfails. The run took about 5s serially and about 6s with -n 4.

Source bugs (strict xfail, not fixed)

1. Flags.from_dict ignores cls and always builds ObsFlags, so ActionFlags.from_dict(...) can't round-trip ActionFlags (test_action_flags_from_dict_returns_action_flags).
2. ReproChatModel.__call__ copies the Discussion shallowly (messages.copy() → Discussion.messages is shared), so retry() appends the replayed answer twice. Replays report n_retry=0.5, and chat_messages holds a duplicate assistant message (test_replay_does_not_duplicate_answer).
3. _diff_stats("", "") raises ZeroDivisionError (test_diff_stats_empty_strings).

Commands

F="tests/agents/test_dynamic_prompting_unit.py tests/agents/test_generic_agent_unit.py tests/agents/test_reproducibility_agent_unit.py"
uv run pytest $F -p no:cacheprovider -q          # 143 passed, 3 xfailed
uv run pytest $F -p no:cacheprovider -q -n 4     # 143 passed, 3 xfailed
uv run --with pytest-cov pytest $F -p no:cacheprovider -q --cov=agentlab.agents --cov-report=term-missing
uv run black --check --line-length 100 $F

Note: the module-name form (--cov=agentlab.agents.dynamic_prompting ...) fails during collection in this local env with RuntimeError: function '_has_torch_function' already has a docstring. That's torch being imported twice when coverage resolves the package before pytest does. It happens with the existing tests/agents/test_generic_prompt.py too, so it's an environment problem, not these tests. Pointing --cov at the parent package (agentlab.agents, above) avoids that and reports the same per-file lines.

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/3d43817633ad4dc08eb787cbe65914f1
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/3d43817633ad4dc08eb787cbe65914f1?variant=devin
Requested by: @danigrim

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.