Repository navigation
Land six reviewed pull requests together (rollup B) - #2380
Merged
Merged
Conversation
Extend UK and eligible Scottish rate limits by gross Gift Aid, remove the net-donation allowance, and adjust ANI once without changing taxable income. Preserve CGT and zero-gift behavior, with policy and property tests plus statutory documentation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the Gift Aid fix. The reform's net_income override read the income components only, so a supplied adjusted_net_income no longer set the taxable base: a married couple with supplied ANI of £50,000 and £0 paid no tax instead of £7,486. The override now returns a supplied ANI plus the grossed-up gift, as the baseline net_income does. Also from the review: - a helper test against each scale shape callers pass: a UK extra bracket at a distinct rate, and the three- and six-rate Scottish scales; - 2017-18 Scottish YAML cases on the three-rate scale; - a frozen positive-gift CGT case (unchanged from main); - corrected descriptions of the CGT test oracle and the maintenance loan income measure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each process that runs tests raises the first-generation GC threshold from 2,000 to 10,000 at start-up and calls gc.freeze() once the tests are collected, so collections stop walking the ~430,000 objects present after importing policyengine_uk. release_dead_variable_modules now also counts the frozen tax-benefit systems as live, since gc.get_objects() leaves frozen objects out; without that it would drop the import-time system's variable modules. Fixes #2316 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… exit Review findings on #2317: - Two test modules built a Simulation at import (council tax uprating's region-to-country map, the childcare test's minimum care hours). Their systems died during collection, so their variable modules were frozen, then released and never freed (about 19,000 objects per worker). Read the hours from the package's parameters and build the map on first use. - conftest.py records any variable-module owner whose system died before the freeze, and test_no_system_dies_before_the_freeze fails on one. - Set only the first-generation threshold and keep the interpreter's other thresholds (CPython 3.14.4 defaults to (2000, 10, 0)). - Undo the freeze and threshold at unconfigure, for in-process runs. - Correct the comment: about 1.6 million objects are frozen per worker, and on the shard whose runner memory isn't full the largest worker's peak rose from 5.3 to 5.7 GiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage's sys.monitoring core (COVERAGE_CORE=sysmon on CI) keeps every code object it has seen alive. policyengine-core compiles a system's variable files again for every new system, so each Simulation left about 2,500 code objects behind and the test shards grew to the runner's memory. The conftest now returns the code object compiled the first time for each variable file, so every system runs the same code with its own modules, classes and functions. Fixes #2330 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A property test on CI had 165 systems' variable modules loaded at once, because dead systems' modules were released only after each test, and Hypothesis builds hundreds of systems while it shrinks a failing example. The loader hook now also runs release_dead_variable_modules() when a new system starts loading its variables, so at most max_owners dead systems' modules are held. Also record the CI before/after numbers in the comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On CI the worker that ran the microsimulation tests kept about 4 GiB of RSS after their simulations were gone, though locally only the 0.74 GiB cached dataset stayed referenced. The conftest now calls glibc's malloc_trim(0) once a file's fixtures are torn down, and the latest-data smoke test removes its unpinned dataset from the simulation cache when its module ends, since no other test uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pytest_sessionstart never reaches a conftest that pytest finds only while collecting, as with a bare `pytest` from the repository root, so the hook was not installed there and the sharing and release tests failed. pytest_configure is also called for conftests registered later. The loader is restored in the existing pytest_unconfigure (a second definition would replace it). Also read coverage's tracers with getattr in the test helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opus review of 7e96e72: - The release ran from the SourceFileLoader hook, which policyengine-core#608 bypasses, so it would have stopped without notice. It now runs from TaxBenefitSystem.add_variables_from_file as each new system starts adding its variables. - With more than eight systems alive (an lru_cache of systems in a test module), every new system and every test scanned the heap and released nothing; a worker's later tests took 432 s instead of 219 s. The release now scans only once eight more systems own modules than were alive at the last scan, after tests as well as builds. - The release test assumed at most eight live systems and failed in a serial run; it now measures the live systems first. - Sharing tests skip, with a note, once policyengine-core shares the code; the note names the right function. - The malloc_trim test checks for glibc rather than Linux, malloc_trim gets its C signature, the loader restore leaves no copy on the subclass, the latest-data fixture keeps the cache when the latest URL is the pinned one, and comments say "about the same time" and "can build many more". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opus delta review of 4a88e23 (approved) left three nits: the comment was off by one (a scan needs more than max_new owners beyond the live ones); the release test could pass if builds reused dead systems' ids, so it now also asserts a release ran; and nothing pinned the amortization, so a Hypothesis test models builds, deaths and releases with faked owner counts and checks that a scan happens exactly when more than max_new owners are beyond the live ones and that at most max_new are left after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UC Regs 2013 reg. 55(4) treats statutory sick, maternity and paternity pay (and adoption, shared parental, parental bereavement and neonatal care pay, which the model has no variables for) as employed earnings, and reg. 55(5) deducts tax and primary Class 1 NI from "the amount of general earnings or benefits specified in paragraphs (2) to (4)". The model deducted the tax and NI on these payments from earned income but left the payments out, so Universal Credit rose by 15.4p per pound of sick or maternity pay above the thresholds. Adds YAML cases and Hypothesis properties: the award never rises with a claimant's statutory pay (two intended exceptions executed and labelled: reg. 32(2)(b) childcare work condition, ITA 2007 s. 55B(2)(b) Marriage Allowance cliff), sick and maternity pay count exactly as wages, earned income adds up, and a dependant's pay is not the claimants'. Fixes #2337 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ayment in government spending HOUSEHOLD_BENEFIT_VARIABLES listed scottish_child_payment, healthy_start_vouchers and two_child_limit_payment, but GOV_SPENDING_VARIABLES did not, so households received them at no cost to the government and gov_balance was overstated by the amount paid. two_child_limit_payment is zero at baseline (the regulations were not laid) but is paid when a reform turns it on. Adds a code-health test that every household benefit is in government spending or on an explicit, documented NOT_PAID_BY_GOVERNMENT list (empty today), YAML cases, and Hypothesis properties: gov_spending is the sum of its list, gov_balance = gov_tax - gov_spending, household benefits equal government spending on households, and a pound more of each payment is a pound more spending and a pound less balance. Fixes #2338 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 70851ab (GPT-6.1 Sol) found two blocking problems: - Statutory paternity pay was in no household income list, while statutory sick and maternity pay were. Once Universal Credit tapers it, household net income fell as paternity pay rose (UC -220 on 400 of pay, with the pay never counted). Add it next to maternity pay in household_benefits, both HBAI lists and gov_spending, and add a net income regression. - The monotonic property checked the award after the benefit cap. Reg. 82(1)(a) lifts the cap once earned income reaches the threshold, and statutory pay is earned income, so a correct cap would fail it. Check the award before the cap, and let the final award rise only where the family becomes exempt. Also: prove the Marriage Allowance cliff directly, qualify it for Scottish taxpayers, add a property that each payment counts once in household_benefits, add a YAML case where NI on paternity pay is deducted, and correct the comment and changelog: income tax falls on sick and maternity pay only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ncome) Conflicts in the three savings band variables and property_income_tax: keep Gift Aid's raised rate limits (ITA 2007 s. 414(2)(b)) and stack on #2352's non_savings_taxable_income, with property income taxed from non_savings_income_after_allowances. Two readers main added start from adjusted_net_income, which on this branch has the grossed-up gift deducted (s. 58). Both now read net_income: - non_savings_income_after_allowances, the base property income and savings stack on, would otherwise drop by the gift; - savings_allowance's band test now compares net income less allowances with the limits raised by the gift. That gives the same band as main's formula today, but stays right once adjusted net income gains the other s. 58 deductions (#2243). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ross rebuilds Fixes #2374, the low-severity notes from GPT-6.1 Sol's round-3 review of #2305: - add_legacy_overrides applies a batch to a copy and swaps it in, so a batch that fails applies none of its overrides. - Rebuilds (apply_parameter_assignments) and reprocessing (reset_parameters then process_parameters) add back the nodes a structural reform added to the processed parameters (graft_missing_nodes), so their overrides replay; process_parameters rolls back on failure. - The shared projected timelines are documented as shared. - The scenario docs say Scenario.apply edits the annualised values, and which steps run after the UC rebalancing and structural reforms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…raft scale bands Review of #2375 (GPT-6.1 Sol, review-hardening-out.md): - process_parameters processes a copy of the reloaded tree, so its rollback returns that tree unprocessed, not an object processing had already changed; the test now checks the contents and a retry. - add_legacy_overrides undoes a failed batch by restoring each target's values_list (update replaces it), keeping the tree's identity and costing O(batch) rather than a whole-tree clone. - graft_missing_nodes also appends scale bands a structural reform added and recurses into existing bands. - reset_parameters keeps only a processed tree, and keeps the first one across a second reset before processing. - Docstrings say grafted nodes are copies and come back even if a modifier deleted them after the reset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With #2352, savings and dividends sit above property income (ITA 2007 s. 16). Two YAML cases check that the s. 414(2)(b) extension applies on that order: £40,000 of property income with £20,000 of dividends (dividend tax £4,278.75 against £4,528.75) or £20,000 of savings (savings tax £5,646 against £5,846). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ilds keep Review round 2 of #2375 (GPT-6.1 Sol, APPROVE with two notes): - a failed override batch now restores the modified flags of its targets and their ancestors as well as their values, so core's cache eligibility is as it was; - the dated_reforms module docstring says rebuilds add back nodes a structural reform added and keep no other edits to existing processed parameters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unknown path is refused when targets are looked up, before anything changes, so the existing test never reached the rollback. A Reform.from_dict key reaches the overrides unparsed and fails after the first override has updated the rate; the new test checks the rate's values_list, its annual value, the recorded overrides and the modified flags on its path are all as before. It fails at 7819755 (flags left set) and passes here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The frozen expectations predated #2352 (savings and dividends above property income) and main's Scottish personal savings allowance on the UK bands, so the mixed-income rows and one allowance changed on main. Regenerated from main 3381500. With no gift, the branch matches main byte for byte on all 21 variables and the tax band. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A spy on apply_legacy_overrides records that the batch raised inside the overrides after the rate was updated, so the test cannot pass without running the rollback if Reform.from_dict keys are ever checked earlier. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…versal Credit earned income) at 0654c4c
…Child Limit Payment in government spending) at b0e8fb1
…form nodes across rebuilds) at d594634
Round-3 review nit. The design note listed two limitations the #2352 merge removed: savings and dividends now stack above property income, and the Personal Savings Allowance tests net income against the UK limits raised by the gift. It now describes that behaviour instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
Round-4 review nit: property income is taxed on top of non_savings_income_after_allowances less property after allowances, and savings and dividends on top of non_savings_taxable_income. Both derive from net_income; they differ only with Marriage Allowance (#1947). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Combined-tree tests (UK hub, 2026-10-12 10:20Z): at |
MaxGhenis
changed the title
This was referenced Oct 12, 2026
Collaborator
Author
|
Merged by the UK hub (audit). Rollup B, merged with a merge commit at the tested head
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands six reviewed pull requests with one CI run on current main. Each member's head commit is merged here unchanged, so merging this PR with a merge commit marks every member as merged. This PR adds no change of its own.
Why a rollup
PolicyEngine's Actions queue is starved: at 09:43Z on 2026-10-12, 18 jobs were running across the org and about 170 runs were queued. #2366 last ran its tests at 01:13Z, before rollup A (#2364) changed the government-spending list it edits, so it needs a re-test on current main anyway. Testing them together costs one run, not six, and tests the combination.
Members
0654c4c9bb0e8fb179gov_spending+£0.574bn, no household's income changes7819755de; an Opus 5.5 review of the delta APPROVE atd59463497; the hub's own Sol delta review of7819755de..d59463497is running3bf52c5c3(all four #2352 conflict resolutions checked), docs-only commits APPROVE atf5418da17and3ef3793bda8ff60fa0(run 4 recomputed from raw artifacts); head37cadb471changes only comments and a docstring, AST-identical toa8ff60fa0(hub check)591f90fd2, standing at4ee6936edand4d472d9c5(comment rewording only); its diff here is patch-identical to its reviewed diff against50ebd8877Each member's own description carries its mechanism, invariants, tests and real-run impact. #2365 and #2366 were each measured against main 3f174d0. The members' sessions say #2365 and #2366 are compatible in either order. #2271 was left out of rollup A because it conflicted with #2352; it has since merged main, including #2352. #2317 and #2331 must land together: #2317's own CI lost a shard's runner at 90% memory, and #2331 is the memory fix stacked on it.
Checks on the combined tree before pushing
uvx ruff format --check .anduvx ruff check .: clean.uvx ruff check --select F811,F821 policyengine_uk: clean, so no definition is silently replaced across members..github/check_fix_pr.py, run locally on this body and this branch's changes: see the Fix evidence check.uv sync --frozen --extra dev, policyengine-core 3.33.2):test_gift_aid.py,test_capital_gains_tax_components.py,test_household_benefits_in_government_spending.py,test_dated_reform_routes.py,test_gov_spending_properties.py,test_uc_statutory_pay_properties.py): 141 passed, 1 skipped (pytest -n 4);gov_spending.yaml2, all passing.COVERAGE_CORE=sysmonand--covas CI runs them: the members' Python files plustest_variable_module_release.pyandtest_variable_code_sharing.py, 152 passed and 1 skipped (-n 4); the two memory tests in a single process, 10 passed and 1 skipped.Reproducers
Each member's reproducers, one per line:
Reproducer: policyengine_uk/tests/policy/baseline/finance/benefit/family/universal_credit/income/uc_statutory_pay.yaml
Reproducer: policyengine_uk/tests/test_uc_statutory_pay_properties.py::test_award_never_rises_with_a_claimants_statutory_pay
Reproducer: policyengine_uk/tests/test_uc_statutory_pay_properties.py::test_each_payment_is_household_income_and_government_spending
Reproducer: policyengine_uk/tests/test_uc_statutory_pay_properties.py::test_statutory_pay_counts_as_wages_and_earned_income_adds_up
Reproducer: policyengine_uk/tests/test_uc_statutory_pay_properties.py::test_a_dependants_statutory_pay_is_not_the_claimants_earned_income
Reproducer: policyengine_uk/tests/test_uc_statutory_pay_properties.py::test_award_falls_along_a_path_of_rising_statutory_pay
Reproducer: policyengine_uk/tests/code_health/test_household_benefits_in_government_spending.py::test_every_household_benefit_is_government_spending_or_excluded
Reproducer: policyengine_uk/tests/code_health/test_household_benefits_in_government_spending.py::test_exclusions_are_household_benefits_outside_government_spending
Reproducer: policyengine_uk/tests/code_health/test_household_benefits_in_government_spending.py::test_the_check_reports_a_benefit_left_out_of_government_spending
Reproducer: policyengine_uk/tests/policy/baseline/gov/gov_spending.yaml::Scottish Child Payment and Healthy Start vouchers are government spending
Reproducer: policyengine_uk/tests/policy/baseline/gov/gov_spending.yaml::The Two Child Limit Payment is government spending when a reform turns it on
Reproducer: policyengine_uk/tests/test_gov_spending_properties.py::test_government_spending_counts_every_benefit_government_pays
Reproducer: policyengine_uk/tests/test_dated_reform_routes.py::test_a_batch_failing_after_an_update_restores_values_and_flags
Reproducer: policyengine_uk/tests/test_dated_reform_routes.py::test_a_rebuild_keeps_a_structural_reforms_parameter
Reproducer: policyengine_uk/tests/test_dated_reform_routes.py::test_reprocessing_keeps_a_structural_reforms_parameter
Reproducer: policyengine_uk/tests/test_dated_reform_routes.py::test_failed_reprocessing_leaves_the_system_as_it_was
Reproducer: policyengine_uk/tests/test_dated_reform_routes.py::test_a_rebuild_keeps_a_band_a_structural_reform_appended
Reproducer: policyengine_uk/tests/policy/baseline/gov/hmrc/income_tax/gift_aid.yaml::Gift Aid gives no income tax relief to a UK basic-rate earner
Reproducer: policyengine_uk/tests/test_variable_code_sharing.py::test_systems_share_code_but_not_modules_classes_or_functions
Reproducer: policyengine_uk/tests/test_variable_code_sharing.py::test_a_simulation_leaves_coverage_no_new_code_objects
Merge rule
Merge with a merge commit only, at the head CI tested, and only after the hub's #2375 delta review approves
d59463497. If a member's branch gains a commit after this point, that commit is not included and the member stays open.After it lands
Stop the test shards' memory growing with every Simulation under sysmon coverage #2331's owner re-measures shard memory on main once Report each pull request's effect on national aggregates on the certified dataset #2335 also lands, and tells W2 if a shard goes back up.
Charge income tax on statutory paternity pay #2368 (income tax on paternity pay) follows Count statutory sick, maternity and paternity pay as Universal Credit earned income #2365.
Use each rule's own Universal Credit earned income in the benefit cap, targeted childcare and local CTR #1999 reads the same UC earned-income variable: rerun its tests after a merge of main.
Relieve pension contributions to relevant earnings and charge the annual allowance excess once #2240 needs the follow-up in Give Gift Aid relief by raising the rate limits, not by deducting the donation #2271's body.
Report each pull request's effect on national aggregates on the certified dataset #2335 needs partition edits for Count Scottish Child Payment, Healthy Start and the Two Child Limit Payment in government spending #2366's three heads and a
statutory_paternity_payhead (comment).🤖 Generated with Claude Code