Skip to content

Rebalance Rest Python groups within the existing runner budget - #10102

Merged
MaxGhenis merged 2 commits into
mainfrom
rebalance-rest-core-group
Oct 12, 2026
Merged

MaxGhenis merged 2 commits into
mainfrom
rebalance-rest-core-group

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Rest's core process rose from 7.1 GB RSS in 38056548883 (base 75cdd80) to 13.3 GB in 38141803955 (#10058 head d692879 tested with main fdd7062). #10058 moved its capital-gains MTR file to heavy-a (8.1 GB, 9:55). Four main additions followed: SSI resource deeming (85.111s), WV senior/disability deduction (31.738s), Schedule A pre-tax premiums (3.813s), and ID health premiums (1.407s). Policy and remaining already measured 13.9 and 14.4 GB on the 16 GB runner.

Add three sequential pytest groups on the existing Rest runner, keeping each Python file in exactly one group. The workflow already runs the Makefile group list serially and uploads every group's reports through wildcards. No jobs, matrix entries, or concurrency are added. The routing guard now generates pass/failure/SIGKILL scenarios for the actual group count.

Destination Files moved, relative to policyengine_us/tests Measured case time Reason
core-heavy, in this order core/test_ssi_resource_deeming_invariants.py; test_federal_credit_limit_order.py; test_pinned_branch_credit_dependents.py 238.891s SSI retains a reformed model at import and constructs another in its fixture. Run it before federal/pinned tests warm the three bounded NY/AL cache entries.
policy-parameters policy/baseline/parameters/test_reform_backdating.py; test_future_dated_reform_uprating.py; test_uprating_extensions.py 143.460s Repeated full-model/reform-tree construction and a default Microsimulation only for parameter assertions. Route the directory so new parameter tests retain this boundary.
remaining-heavy test_niit_section_911_magi.py; test_ctc_foreign_earned_income_exclusion.py; test_ctc_foreign_earned_income_exclusion_property.py; unit/test_medicaid_abd_ssi_rate_standard_properties.py; test_capital_gain_distributions_loss_limit.py; test_schedule_d_filer_net_capital_gain.py 302.095s First four files retain three independent models (CTC files share one). Last two balance wall time with transient TX setups without warming NY/AL pins.

SSI is the only core extraction; the smaller additions stay there. Keep cached-arrays/heavy-a/heavy-b unchanged. Move the retained-model files and three long-running files to shorten remaining; separating NIIT/CTC/Medicaid from NY/AL pins avoids approaching 10 GB before transient allocations.

Group Before: measured RSS / wall in 38141803955 After: expected RSS / wall
cached-arrays 3.3 GB / 5:26 3.3 GB / 5:26
heavy-a 8.1 GB / 9:55 8.1 GB / 9:55, including pending #10058
heavy-b 8.5 GB / 7:31 8.5 GB / 7:31
core-heavy new about 8–10 GB / 5–6 min
core 13.3 GB / 11:51 about 7–10 GB / 10:26
policy-parameters new about 8–10 GB / 3–4 min
policy 13.9 GB / 8:46 about 8–10 GB / 6:22
remaining-heavy new about 8–10 GB / 6–7 min
remaining 14.4 GB / 22:18 about 10 GB / 14:42

These are planning estimates requiring Linux CI confirmation. JUnit measures each file's elapsed time; GNU time measures whole-process maximum RSS. Neither measures individual file memory contributions, so the 6.1 GB core rise cannot be assigned entirely to SSI. Source inspection identifies retained copies; the Makefile's roughly 1.45 GB per full parameter-tree copy gives remaining 14.4 − 3×1.45 ≈ 10.1 GB, around the requested 10 GB target, and core-heavy roughly 7.25 GB retained before transient allocations. GC, allocator reuse, and newer main/Core changes make actual peaks uncertain. RSS values follow the supplied convention: reported kbytes / 1,000,000. Wall estimates subtract moved case time; each new process adds roughly 1–2 minutes startup/collection. The longest group falls from 22:18 to an estimated 14:42. #10058 remains open, so its separate heavy-a move is anticipated and not duplicated here.

Follow-up if one file still dominates: SSI builds seven Simulations and duplicates a reformed policy. Reuse one read-only policy where isolation permits and collect released simulation graphs without dropping vectorized relationship coverage. The parameter-only Microsimulation could use the existing reference system if startup is not the intended contract; NIIT/CTC could release module caches at teardown. Measure isolated Linux RSS before those separate changes; this PR does not rewrite their tests.

Verify after the hub marks this draft ready (the PR workflow currently skips Python tests while draft), in this PR's existing Rest job: download rest-test-timings, inspect rest-python-core-resources.txt maximum RSS, then check every rest-python-*-resources.txt against the approximately 10 GB target and compare JUnit file times, including the new groups. Routing alone does not establish memory safety. No moved files, full suites, or microsimulations run locally.

Local validation: make format and three static guards (test_rest_python_groups.py, test_make_test_matches_ci.py, test_pr_runner_budget.py), one file at a time. All 21 checks passed: routing 12, Makefile/CI coverage 2, runner budget 7. Formatting and lint passed before each commit. Pytest discovery initially imported the full model; the rerun keeps the real package path but skips package initialization/conftests for these source/collection/command-only guards. The routing guard's own child collections use placeholders and import no model tests.

Hub microsimulation impact

No model change: test routing only.

axiom: n/a (test routing, no rule change)

@MaxGhenis
MaxGhenis marked this pull request as ready for review October 11, 2026 17:11
@MaxGhenis MaxGhenis closed this Oct 11, 2026
@MaxGhenis MaxGhenis reopened this Oct 11, 2026
@MaxGhenis MaxGhenis closed this Oct 11, 2026
@MaxGhenis MaxGhenis reopened this Oct 11, 2026
@MaxGhenis
MaxGhenis merged commit 4cb18d3 into main Oct 12, 2026
69 of 79 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

US + core hub merge audit, us#10102 at 19c10f929e (squash): rebalance the Rest test groups to keep each under about 10 GB.

  • Review: independent GPT-6.1 Sol review APPROVE at this head (routing is correct, and CI still runs every group).
  • Group memory at this head (CI run 38183213722, /usr/bin/time -v reports in the rest-test-timings artifact): core 7.4 GB (was 13.3 GB on a 16 GB runner), policy 9.6 GB, remaining-heavy 9.3 GB, heavy-b 8.1 GB, remaining 7.8 GB, and every other group under 6 GB. That was the condition for merging it.
  • Gates checked live at merge:
    • gh pr checks exits 0;
    • the latest Pull request run at the head concluded success;
    • MERGEABLE;
    • not a draft;
    • no reviewer's latest review requests changes;
    • the semantic-overlap guard;
    • head pinned.
  • Impact: none; this changes test routing only.
  • Axiom: n/a (test routing, no rule change).

@MaxGhenis
MaxGhenis deleted the rebalance-rest-core-group branch October 12, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant