[devin-ai-integration]

Summary

Closes the code-scan coverage gap on study orchestration: make_study dispatch, Study.make_exp_args_list enumeration, and the Study.run relaunch loop were only exercised by skipped or pricy tests. New tests in tests/experiments/test_study.py cover them with CheatMiniWoBLLMArgs on miniwob_tiny_test, without launching any experiment or LLM call.

• dispatch: Study for 1+ agents with no parallel servers (and for a single agent even when parallel_servers is set), ParallelStudies for 2 agents + parallel_servers, SequentialStudies for 2 agents on webarena_tiny.
• enumeration: 2 agents × 4 env args yields 8 ExpArgs in agent-major order with order == 0..7.
• relaunch loop: a _FakeRunStudy subclass stubs _run/get_results/find_incomplete/save/set_reproducibility_info and replays a scripted (n_incomplete, n_error) sequence per trial, asserting the loop stops when nothing is incomplete, when errors stop decreasing, when >30% errored, and otherwise runs exactly n_relaunch times.

Writing the loop tests surfaced a bug: last_error_count was initialised to None and never updated, so the "last trial did not reduce the number of errors" break was dead code and a study whose error count plateaued kept retrying until n_relaunch. Fixed in Study.run:

             if n_incomplete == 0:
                 break
+            last_error_count = n_error

tests/experiments/test_study.py passes (8 passed, 1 skipped). The 3 failures in test_launch_exp.py::test_launch_system_* reproduce on unmodified main and are unrelated.

Devin-Org: engineering

Link to Devin session: https://app.devin.ai/sessions/4b25d84785bb4a338e6cd894c3e70752
Open in Devin Desktop: https://app.devin.ai/desktop/session/4b25d84785bb4a338e6cd894c3e70752?variant=devin

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring