Skip to content

Faster test suite: smaller slow tests, tasks tests in three CI shards - #11

Merged
GeophyAI merged 6 commits into
mainfrom
test/faster-suite
Oct 24, 2026
Merged

GeophyAI merged 6 commits into
mainfrom
test/faster-suite

Conversation

@GeophyAI

Copy link
Copy Markdown
Collaborator

The suite spent most of its time in about 240 tests that ran larger problems than their assertions need. These commits make them smaller and run tests/tasks in parallel on CI.

  • 25 slow test files shrunk: fewer epochs, steps, shots or stages, smaller grids, and setups shared within a file. Each change keeps what the test asserts and its tolerances; for the tests shrunk most, a deliberately broken copy of the code was confirmed to fail both the old and the new test, and each file was run several times to check for flakiness. The DDP/DD comparisons start the 1-rank reference and the multi-rank run side by side.
  • CI: tests/tasks runs as three shards of about equal length (tasks-plan, tasks-runner, tasks-rest); together they collect exactly the tests of tests/tasks, and a new test file lands in tasks-rest.
  • tests/conftest.py: SLURM_JOB_ID is removed for the session, so on a cluster a task test no longer waits on two scontrol calls per run for metadata it does not read.
  • eager_options defaults now come from sweep's EagerOptions (the eager step is compiled). The schema said use_compile: false while a task without an eager_options block ran sweep's compiled default; behaviour is unchanged, the documented default now matches it, and a test pins the two together.

Measured on the same hardware before and after (full tests/): CPU, 2 threads, cold compile cache 865 s -> 566 s; V100 1175 s -> 790 s. 0 failures on both.

🤖 Generated with Claude Code

@GeophyAI
GeophyAI merged commit 3695b18 into main Oct 24, 2026
9 checks passed
@GeophyAI
GeophyAI deleted the test/faster-suite branch October 25, 2026 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant