Split Rest CI into three Python shards and variables YAML

[MaxGhenis]

Rest reached its 90-minute timeout on #10024, #9913, #10139 and #10029 in October 2026. Recent historical totals were 66–90 minutes, almost all Python; Variables YAML took only 47–66 seconds. Split the existing nine isolated Python groups across two jobs in both PR and release CI, with Variables on shard 1.

The exact PR runner budget rises from 26 to 27. One extra runner buys roughly 45 minutes of headroom against the old Rest timeout. Shard 2 also supplies the legacy summary check, so no separate Variables or summary runner is needed. The two Rest jobs run sequentially: shard 2 waits for shard 1 to finish, then executes its tests and checks both results. This gives each shard about 44 minutes of headroom over the slowest historical run (45m36s against a 90-minute timeout); total Rest workflow latency remains similar to the original serial suite.

Job	Existing groups	Historical group-time estimate, before setup
Rest (Python, shard-1 + variables)	cached-arrays, core-heavy, policy-parameters, remaining-heavy, remaining; Variables YAML	33m05s–45m35s, including Variables
Rest (Python + variables), shard 2 and summary	heavy-a, heavy-b, core, policy	32m10s–45m36s

Estimates use GNU time artifacts from seven runs: 38322966221, 38251742355, 38329260767, 38326922385, 38336638621, 38289790999 and 38331138151. Comparing every partition of the nine whole groups and both Variables placements found this layout's maximum, 45m36s, to be the smallest. A literal 45m00s ceiling is not possible without dividing existing isolated groups; the requested approximately 45-minute target needs only two shards. Both jobs have a 90-minute timeout, leaving 44m24s beyond the slowest historical run for setup and variation. One canceled Variables report is empty; its estimate conservatively uses the largest observed Variables time and does not determine either shard's maximum.

Keep each group in its own sequential process, with four-thread caps, unchanged pytest arguments, unconditional timing/JUnit uploads, and unique artifact names. The local unsharded Make target keeps all groups in their original order. Quick Feedback's existing coverage generation and Codecov upload remain unchanged.

Preserve Python tests / Full Suite - Rest (Python + variables) as the second tested job. Its unconditional final check rejects failure, cancellation or skipping of either shard, including a canceled workflow that skips the second Python step. Failed steps remain job failures. Release Publish explicitly waits for both Rest jobs. The runner-budget guard checks exactly 27 runners, exact heavy-suite counts, the exact Python job set, and the unchanged reusable-workflow triggers.

The routing guard compares actual CI commands against independent broad pytest discovery, requiring every Python file and every existing group exactly once. A discoverable file assigned to no group fails the guard. Variables selection and reports remain unchanged and run exactly once. Separate Microsimulation/SPM processes and compatibility checks remain on their existing runner.

Validation: 32 local checks passed, each file run individually and sequentially with the prescribed Python environment and all four-thread caps. test_rest_python_groups.py: 23 passed in 224.73s, including exact file/group coverage and all 16 result pairs in each workflow. test_pr_runner_budget.py: 7 passed in 1.62s. test_make_test_matches_ci.py: 2 passed in 10.76s. Plugin autoload was disabled and --confcutdir=policyengine_us/tests/code_health --noconftest avoided country-model setup for tooling checks. make format, final Ruff formatting/lint checks, actionlint on both workflows, and git diff --check passed. No policy suites or microsimulations ran locally; current Linux shard timings will come from PR CI.

Hub microsimulation impact

Structurally zero: only CI workflows, Make routing, code-health guards and the changelog fragment change. No variable, parameter, reform, dataset or partner contract expectations change.

axiom: n/a: CI job routing