Run make test as CI's bounded batches instead of one process over the whole tree

[MaxGhenis]

Problem

make test, the documented way to run the full suite (CLAUDE.md, CONTRIBUTING.md), ran:

coverage run ... -m policyengine_core.scripts.policyengine_command test policyengine_us/tests/policy/ -c policyengine_us

That is every policy YAML file in one process. CI never runs that shape: .github/workflows/pr.yaml splits the YAML suites into policyengine_us/tests/test_batched.py subprocesses, sized to stay under about 8 GB each on 16 GB runners.

On 2026-10-02, three local processes of that shape ran in parallel, each over a third of tests/policy/baseline (about 1,500 files and 10,000 cases). Two of them reached 118 GB and 111 GB on a 128 GB Mac before they were killed.

One process with 457 baseline files (every tenth, 3,100 cases) on core 3.32.12 went from 1.4 GB to a 14.5 GB kill cap after 267 cases (17 minutes). Live simulations grew linearly: 50, 120, 320, 420 at cases 50, 100, 150, 250.

The causes are in policyengine-core's YAML runner and parameter nodes and are fixed in PolicyEngine/policyengine-core#569 and #570. This PR fixes the local entry point, so a full run is bounded on any core version.

Change

• make test now runs the Python suites as CI splits them (test-other-python-spm, test-other-python-rest, test-microsimulation, test-policy-contrib-python) and then test-yaml.

• test-yaml is new. It runs every YAML suite in pr.yaml, as Make prerequisites in sequence, so at most one batch is resident at a time; partners runs two small batches at once, as in CI.

• make test no longer writes coverage.xml; CI's coverage comes from the Quick Feedback job, which this does not touch.

• test_make_test_matches_ci.py (code health) runs make -n and fails if:

  • any Make target or test_batched.py command in pr.yaml is missing from make test (a sharded CI command counts as covered by its unsharded form);
  • anything under make test runs policyengine-core test directly.

  Both checks fail on the old Makefile and pass on this one.

• CLAUDE.md and CONTRIBUTING.md now give the safe local invocations:

  • one area: its make test-yaml-* target or test_batched.py ... --workers 1;
  • a few files: policyengine-core test;
  • never a whole tree or hundreds of files in one process;
  • one large suite at a time.

axiom: n/a: test tooling and docs only

Tests

• pytest policyengine_us/tests/code_health/test_make_test_matches_ci.py: passes; both tests fail on main's Makefile.
• make -n test: 37 test_batched.py invocations and no direct policyengine-core test.

🤖 Generated with Claude Code

[MaxGhenis]

Merged by the US + core hub on its merge gates. Audit:

• Head: 7910d46f389fe012ea276bf073b852a9b84fd1cc.
• Independent review:
  • r1 (~/reviews/us-hub/reviews-gen/us-9752/review-r1.md) approved 7ddc54d635, a main merge of the author's c3358da5a9. It raised one P2: make -j, or a -j in MAKEFLAGS, would run every suite at once, which is the memory blow-up make test exists to prevent.
  • The hub answered it in 7910d46f38: .NOTPARALLEL: serializes every target, and CONTRIBUTING.md now describes core Set maximum EITC age for 2021 to inf #569 and Add legislative references for EITC #570 as open drafts. Under GNU make 3.81, a two-target .NOTPARALLEL Makefile runs serially with -j2, and test_make_test_matches_ci.py passes.
  • r2 (review-r2.md) approved that delta. It traced not_parallel in make 3.81's job.c and the 4.x semantics, confirmed that no CI workflow passes -j or sets MAKEFLAGS, and found the recursive $(MAKE) loop stays serial.
• CI: every check passed at this head.
• Impact: local test tooling only, so no model output changes.
• Decisions: none. The P3 follow-ups are noted in r1: the guard reads only pr.yaml, and make -k test could be documented.