[devin-ai-integration]

Summary

Adds hermetic unit tests (fake gym.Env + fake Agent, tmp_path, unittest.mock) for the experiment execution core. No browser, network, Ray, or real sleeps. Test files only; nothing under src/ or any existing test is touched.

The main test runs ExpArgs.run() end to end. It patches EnvArgs.make_env to return a FakeEnv, uses an AbstractAgentArgs subclass for the agent, then checks:

• summary_info.json, the step_*.pkl.gz pickles, screenshots, goal_object.pkl.gz and experiment.log
• chat forwarding and env.close()
• round-tripping the results through ExpResult

It also covers these error paths:

• the agent raises
• make_env raises
• agent_info["err_msg"] is promoted to the summary
• debug-mode re-raise
• KeyboardInterrupt → "Early termination??"
• cleanup failures inside the finally block are swallowed
• the TapeAgent / save_tape branch

The ray backend of run_experiments is tested by putting a fake agentlab.experiments.graph_execution_ray module into sys.modules. Real Ray is never started.

module	before (whole suite on main)	after (new files only)
agentlab.experiments.loop	60%	100%
agentlab.experiments.exp_utils	66%	100%
agentlab.experiments.launch_exp	77%	100%
agentlab.ui_assistant	58%	96% (only if __name__ == "__main__": main() is uncovered)

129 tests: 126 pass, 3 are strict xfails. Each file runs in under 3s locally, and they pass both serially and with -n 4.

Source bugs found (xfail strict, src/ not changed)

• loop.SEED_MAX = 2 ^ 32 is XOR and evaluates to 34, so ExpArgs.prepare() draws random task seeds from [0, 34) instead of [0, 2**32). Test: test_seed_max_is_two_to_the_32.
• ExpArgs._set_logger removes handlers from root.handlers while iterating over that same list, so every other StreamHandler survives. Test: test_set_logger_removes_all_previous_stream_handlers.
• In the run_experiments(parallel_backend="joblib") branch, exp_args_list is reassigned to the parallel subset. As a result the finally block never calls agent_args.close() on experiments flagged sequential. Test: test_run_experiments_joblib_closes_all_agents.

Files

• tests/experiments/test_loop_unit.py
• tests/experiments/test_exp_utils_unit.py
• tests/experiments/test_launch_exp_unit.py
• tests/test_ui_assistant_unit.py

Run

uv run pytest tests/experiments/test_loop_unit.py tests/experiments/test_exp_utils_unit.py tests/experiments/test_launch_exp_unit.py tests/test_ui_assistant_unit.py -p no:cacheprovider -q [-n 4]

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/d304966862d14d2a860b611410687343
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/d304966862d14d2a860b611410687343?variant=devin
Requested by: @danigrim

---

Devin Review

Status	Commit
⚪ Not started	—

Run Devin Review

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)