Stop the test shards' memory growing with every Simulation under sysmon coverage

[MaxGhenis]

Fixes #2330

Stacked on #2317 (which is stacked on #2269). The diff against ci-test-gc-freeze is this PR's change alone. Test code only.

Cause

Coverage's sys.monitoring core, which #2269 turns on with COVERAGE_CORE=sysmon, keeps every code object it has seen alive, so it can key its records by id(code) (SysMonitor.code_objects). policyengine-core compiles a system's ~1,110 variable files again for each new system, and every Simulation builds one. By the end of a CI shard, each pytest-xdist worker's coverage held 0.8 to 1.46 million code objects, and the shards used up to 1.8 GiB of swap. Details in #2330.

Change

policyengine_uk/tests/conftest.py:

1. Shared variable code. SourceFileLoader.get_code is wrapped for policyengine-core's per-system variable modules only (names <id>_<hash>_<file>).
   • Each variable file is compiled once per process, and every later system runs that code object.
   • Systems still get their own modules, classes, functions and globals.
   • The wrapper is installed in Cut the CPU the Python tests spend in garbage collection #2317's pytest_configure and removed in its pytest_unconfigure. So it also applies when a bare pytest finds the conftest during collection.
2. Amortized module release. release_after_new_systems() runs release_dead_variable_modules() once more than eight systems own modules beyond those alive at the last release.
   • It is called as each new system starts adding its variables (a wrapper on TaxBenefitSystem.add_variables_from_file) and after each test.
   • One property test had 165 systems' modules loaded at once. A process that keeps many systems alive (an lru_cache of systems) no longer scans the heap after every build and test.
3. Freed memory returned. After a test file's fixtures are torn down, glibc's malloc_trim(0) returns free heap memory; elsewhere this does nothing. The microsimulation worker had kept ~4 GiB of RSS after its simulations died, though locally only the 0.74 GiB cached dataset stayed referenced (tracemalloc).

test_latest_data_smoke.py stops keeping its unpinned dataset in _url_dataset_cache after its module. No other test uses that URL, and the cache entry is kept if the pinned dataset is set to the same URL.

The upstream fix for (1) is PolicyEngine/policyengine-core#608. Once it ships, the tests of the wrapper skip with a note to remove it, and the release (2) keeps working, because it no longer depends on the loader.

Measured on CI

Each run built one commit in every configuration at the same time. The 2 s runner sampler is from gc-measure/2317; a per-worker probe logs RSS after every system build and every test. The runner has 15.6 GiB.

Configuration	Run	Peak used, shards 1 / 2 / 3	Swap	pytest wall
Before (sysmon, code compiled per system)	38159894687	15.29 / 15.21 / 14.36 GiB	up to 1.84 GiB	859 / 1,224 / 730 s
(1)	38159894687	12.15 / 11.05 / 10.22 GiB	0	677 / 991 / 621 s
(1), an earlier per-build (2), and (3)	38165445604	8.24 / 8.00 / 8.19 GiB	0	776 / 952 / 637 s
The same on 2 shards	38165445604	8.11 / 8.09 GiB	0	1,308 / 1,100 s
This PR's final code (591f90f)	38173323996	8.70 / 7.56 / 8.71 GiB	0	702 / 877 / 628 s

• Outcomes. Within each run, every configuration had identical test outcomes. The final run had 3,784 passed and 3 skipped. That is every earlier test plus this PR's new tests, which ran on the Linux runners rather than skipping. test_an_exported_frame_loads_back_and_exports_itself, which failed on this base, was deselected in every run, so every figure covers the suite without it. It is fixed on main in 3b8f54b.
• A/B within run 38165445604.
  • Per-build release without (3): 10.40 / 10.12 / 10.00 GiB, 788 / 902 / 581 s.
  • With (3): 8.24 / 8.00 / 8.19 GiB.
• ctrace for comparison (no sharing, run 38159894687): 12.43 / 11.47 / 10.87 GiB, 1,162 / 1,412 / 1,007 s wall.
• Local CPU figure. A UK system build took 0.180 s of CPU instead of 0.289 s. That was measured with PYTHONDONTWRITEBYTECODE=1 and no __pycache__, so it compares against compiling every file for every system. With valid .pyc files the per-system saving is smaller; the CI wall times above are the measured effect.

Invariants (tests)

• test_variable_code_sharing.py:
  • Two systems have distinct variable classes, modules, formula functions and globals, and the same formula code objects.
  • The shared code for every loaded variable file equals compile() of its current source, and every variable file is covered.
  • Under sysmon coverage, a Simulation adds fewer than 100 code objects to coverage, against more than 1,000 with sharing turned off.
  • Once core shares code itself, these tests skip.
• test_variable_module_release.py:
  • A test that builds 11 systems runs a release and never has more than (live systems + 9) systems' modules loaded, whatever other modules keep alive. Without the release hook it reached live + 11.
  • A Hypothesis test models builds, deaths and releases with faked owner counts. A scan happens exactly when more than max_new owners are beyond the live ones, and at most max_new remain after.
• test_malloc_trim.py: it trims after a file's last test and not between tests of one file, and with glibc it finds malloc_trim.
• Each test fails with its change removed. Checked: the sharing and release hooks, the trim boundary, and an off-by-one in the release trigger.

Review

An independent Opus 5.5 review (~/reviews/uk-ci-test-memory-2026-10-11/review-2331-opus-*.md) went through three rounds:

1. It requested changes at 7e96e72. The per-system release was order-dependent, scanned per build, and was bypassed by core#608; there were smaller items too.
2. It approved 4a88e23, which fixed all 11 findings.
3. It approved 591f90f, which took its nits.
4. Its approval stood at 4ee6936, which re-cites the final run in comments. 4d472d9 rewords one comment as it asked, citing each run for its own figure. GPT-6.1 Sol attempts died on Codex transport failures.

Follow-ups (not in this PR)

• W2 owns the shard count and the pr_code_changes.yaml comment. It keeps four shards for wall time and is citing these measurements in Publish only the policyengine-uk release files whose checks passed #2306.
• W3's Report each pull request's effect on national aggregates on the certified dataset #2335 adds test_aggregate_impact_certified.py and moves the pin to 1.58.0. I'll re-measure once both land.
• A chip covers the unbounded lru_cache of systems in test_two_child_limit_age_exemption_properties.py, about 0.55 GiB per worker.

axiom: n/a: test infrastructure

🤖 Generated with Claude Code

[MaxGhenis]

Hand-off to the UK hub (session "Cut peak memory of pe-uk CI test shards").

• Head: 4d472d9. Base ci-test-gc-freeze (Cut the CPU the Python tests spend in garbage collection #2317 at 50ebd88), which is stacked on Split the CI Test job into shards and cut its runtime #2269.
• Merge order: Split the CI Test job into shards and cut its runtime #2269 → Cut the CPU the Python tests spend in garbage collection #2317 → this PR. When Cut the CPU the Python tests spend in garbage collection #2317 merges, GitHub retargets this PR to main and pr_code_changes CI runs for the first time; no checks run while the base isn't main.
• Review: an independent Opus 5.5 review approved 591f90f, after requesting changes at 7e96e72. Its approval stood at 4ee6936 (comment-only), and 4d472d9 rewords one comment as it asked. Files: ~/reviews/uk-ci-test-memory-2026-10-11/review-2331-opus-out.md, -delta-out.md, -final-out.md and -4ee-out.md. GPT-6.1 Sol attempts died on Codex transport failures.
• Passed: final-code CI measurement run 38173323996 (591f90f plus probe files): 3,784 passed, 3 skipped, and the new tests ran on Linux. Peaks were 8.70 / 7.56 / 8.71 GiB of 15.6 with no swap, against 15.29 / 15.21 / 14.36 GiB with up to 1.84 GiB of swap before. Local runs of the PR's test files passed under sysmon with xdist, and under a bare pytest.
• Pending: PR CI once retargeted to main.
• Coordination:
  • W3 (Report each pull request's effect on national aggregates on the certified dataset #2335, owns conftest.py) knows about the appended section and hooks; no clash.
  • W2 (Publish only the policyengine-uk release files whose checks passed #2306) keeps four shards for wall time and is citing these figures.
  • The Cut the CPU the Python tests spend in garbage collection #2317 owner knows this PR is stacked on their branch.
• Related: Compile each variable file once per process policyengine-core#608, the upstream code sharing, handed to the US + core hub. Once it ships, the tests of this PR's loader wrapper skip with a removal note.
• Follow-ups owed:
  • Re-measure on main once both this PR and Report each pull request's effect on national aggregates on the certified dataset #2335 have landed, and tell W2 if a shard goes back up.
  • Chip task_6af66c7f covers the lru_cache of systems in the two-child tests.
• Evidence and state: ~/reviews/uk-ci-test-memory-2026-10-11/ (STATE.md, run tables, local runs, scripts).

[MaxGhenis]

UK hub (2026-10-12 11:05Z): this PR's head 4d472d9c5 is merged unchanged into rollup B, #2380. The rollup now holds #2365, #2366, #2375, #2271, #2317 and #2331, and one CI run tests all six together on current main. Merging #2380 with a merge commit marks this PR merged. The hub cancelled this PR's own queued runs, which the rollup replaces. Please don't push to this branch now: a new commit would not be in the rollup, and this PR would stay open.