Repository navigation
Drop the clone override now that policyengine-core keeps each clone's input record - #2262
Merged
Merged
Conversation
…put record policyengine-core 3.32.27 contains PolicyEngine/policyengine-core#561, which fixes policyengine-core#559: Holder.delete_arrays forgets the record (Simulation._user_input_keys) of an input it deletes, including deletes made straight on the holder, and Simulation.clone gives each clone its own copy of the record and an empty _user_input_contexts. PE-UK's Simulation.clone and Simulation.delete_arrays overrides did the same by hand, so remove them and raise the policyengine-core floor from 3.32.12 to 3.32.27. Relock core 3.32.13 -> 3.32.27; the lock's own entry for this package catches up to 2.125.1. Keep the sweep in Simulation.calculate and SUPPLIED_INPUT_VARIABLES. Simulation.build_from_populations, which build_from_situation calls, replaces every holder but keeps the record (policyengine-core#605). A rebuilt simulation could then pass a carried-over trading loss off as supplied, which the sweep prevents. Its comment and the supplied_inputs docstring now name that case instead of #559. Tests: - a rebuild regression for Class 4 (fails with the sweep disabled); - a model-based property test: over random sequences of set_input, both kinds of delete, calculation, clones and rebuilds, a loss counts as supplied exactly while the last value set for it is kept. Two pinned examples make it fail on core 3.32.13 (one record shared by a clone) and with the sweep disabled (#605); - the missing-stored-array test now puts a stale record back itself, since core no longer leaves one after a holder delete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 2 found that the removed Simulation.delete_arrays override was not pure duplication of policyengine-core 3.32.27. Core's deletion forgets only the records of the values it removes. The override also dropped records that a population rebuild had already left with no stored value (policyengine-core#605), for any variable. Without it, deleting hours_worked for 2026 after a rebuild that dropped the 2026 input, then calculating 2026, made to_input_dict export the carried-over 2025 hours as a 2026 input (core 3.32.27 and 3.33.0). Restore the override, with its comment naming #605 instead of #559, and add test_input_records.py for that sequence (it fails without the override). The property test's step selector now reaches every simulation a run can make. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 3 (APPROVE) noted that Simulation.build_from_populations adopts whatever populations it is given: passed the existing ones, it keeps their holders. The holders are replaced because build_from_situation passes freshly instantiated populations. Word the supplied_inputs docstring that way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
retire-core-559-workaround
branch
from
October 10, 2026 16:52
173aa4a to
a25ce01
Compare
MaxGhenis
marked this pull request as ready for review
October 10, 2026 17:54
Collaborator
Author
|
Merged at head a25ce01 (merge commit d1afea8) with a merge commit, pinned by |
MaxGhenis
added a commit
that referenced
this pull request
Oct 10, 2026
Brings in #2262 (retire the policyengine-core 559 workaround). No file overlaps this branch's changes; no conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 11, 2026
…mulated-benefits-scoped-awards Conflict in simulation.py: main dropped the clone override because policyengine-core 3.32.27 copies the input record per clone (#2262). Core's clone now also links a cloned tax-benefit system to the clone, which was the only other thing this branch's override did, so the override goes. The apply_reform override (link while applying, restore after) stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 11, 2026
Brings in #2258 (120-minute Test job), #2180 (NICs thresholds indexed by September CPI after 2030-31), #2063 (salary sacrifice needs pay), #2038 (winter payments to the member a benefit is paid to), #2259, #2248, #2251, #2262, #2266, #2270, #2285, #2288, #2294 and #2021. Conflict: test_legacy_award_readers_properties.py imports. Kept both hypothesis.event (this branch's screened-out events) and hypothesis.example (#2294's found families). The rest of the file merged cleanly: #2294's without/silent/with-other comparison, the severe disability premium bound and the held benefit cap sit alongside this branch's work generators, screened reference and outside-member tests. Two comments now say the ESA and JSA readers also read the members' work. _legacy_award_payee.py merged cleanly: #2038's can_claim (Pension Credit) and this branch's own_report_is_paid are independent. The shared NI parameters already match main (#2058); #2180 changes only 2031-32 onward. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #2260. The rest waits on PolicyEngine/policyengine-core#605.
What this changes
policyengine-core 3.32.27 is the first release with PolicyEngine/policyengine-core#561. That PR fixes PolicyEngine/policyengine-core#559, where
Simulation._user_input_keys, the record ofset_inputvalues, drifted from storage. With #561:Simulation.clonegives each clone its own copy of the record and an empty_user_input_contexts.Holder.delete_arraysforgets the record of each value it deletes, including deletes made straight on the holder.This PR:
Simulation.cloneoverride, which copied the record and contexts by hand;policyengine-core>=3.32.12to>=3.32.27. The 3.32.26 tag is behind the Add total wealth #561 merge commit and 3.32.27 is ahead of it; PyPI serves 3.32.27. The lock moves core from 3.32.13 to 3.32.27, and the lock's own entry for this package catches up to 2.125.1;delete_arraysoverride and thecalculatesweep, and thesupplied_inputsmodule docstring, so they say what each now covers (below);What stays, and why
build_from_situationinstalls fresh populations, and so fresh holders, throughSimulation.build_from_populations, which keeps the record. That gap is filed as PolicyEngine/policyengine-core#605, reproduced there with country_template, and the issue has a further comment showing the export consequence. Two pieces of PE-UK bookkeeping still cover it:Simulation.calculatesweep andSUPPLIED_INPUT_VARIABLES. After a rebuild, a carried-over trading loss could pass as supplied, giving Class 4 profits of £20,000 and NIC of £445.80 instead of £30,000 and £1,045.80.Simulation.delete_arraysoverride. Core's deletion forgets only the records of values it removes. The override also drops records a rebuild left with no stored value, for any variable. Without it, rebuilding without the 2026hours_workedinput, deleting 2026 and calculating 2026 makesto_input_dictexport 2025's carried-over hours as a 2026 input (core 3.32.27 and 3.33.0).Both can go once core fixes #605. The independent reviews found these cases (see below); the first two versions of this PR had removed them.
Behaviour
No calculated value changes, and no existing test expectation changes. With the same core installed, the PE-UK code differs from main only by the missing
cloneoverride. Round 3 of the review compared clones with and without it on 3.32.27 and 3.33.0: plain clones, inputs set after cloning, nested branches throughget_branch, clones of branches, rebuilding a clone, and disk-backed inputs all came out the same. Two boundaries differ:set_inputis running on the original starts with an empty_user_input_contexts(core's behaviour); the removed override copied the original's. No PE-UKset_inputhandler clones a simulation. Round 1 showed the old copy could misrecord a later write on such a clone as an input._user_input_keysattribute at all keeps that absence; the override created an empty set. Normal PE-UK initialisation creates the record, so this arises only from a simulation built some other way (round 3 emulated it by deleting the attribute).Raising the floor also brings in core's own changes between 3.32.13 and 3.32.27; see core's changelog.
Invariants
supplied_input,supplied_input_periods) exactly while the last value set for it, byset_inputor a rebuild, has been neither deleted (Simulation.delete_arraysorHolder.delete_arrays) nor dropped by a rebuild. A value the engine carries over or calculates into a period never counts.to_input_dictexports only values that were supplied.Tests:
test_only_losses_set_and_kept_count_as_supplied(new) is a model-based Hypothesis property over random sequences ofset_input, both kinds of delete, calculation, clones and rebuilds. Each step can act on any simulation made so far, and every simulation is checked after every step. It runs 30 derandomized examples plus two pinned@examples. The pins make it fail on core 3.32.13 (a clone sharing one record) and with the sweep disabled (Generate takeup-informing variables before calibration, not after #605). Without the pins, 200 examples were needed to find the clone case on 3.32.13.test_class_4_does_not_count_a_loss_a_rebuild_drops(new) fails with the sweep disabled.test_input_records.py::test_deleting_after_a_rebuild_does_not_export_a_carried_over_input(new) fails without thedelete_arraysoverride.test_supplied_input_helpers_ignore_missing_stored_arraysnow puts a stale record back itself, since core no longer leaves one after a holder delete.Tests run
Locally. Evidence is in
~/reviews/core-user-input-keys-2026-10-02/uk_retire/. In the table, "related pytest" means the 23 test files that mentionentered_directly,_user_input_keys,clone,delete_arraysorsupplied_input, plustest_input_records.py. "Supplied-input YAML" means the bus journeys, bus fare spending, loss relief, Class 2 and Class 4 profits tests.policyengine-core test .../finance/benefit1,062 passed; supplied-input YAML 66 passedtest_class_4_counts_only_supplied_losses_on_plain_clonesfail, so the 3.32.27 floor is requireddelete_arraysoverride removed (mutant)test_input_records.pyfailsuv run --frozen ruff format --check policyengine_ukandruff check policyengine_uk: pass.uv lock --check: passes.make testand the whole tests folder were not run locally; CI runs them.Review
Independent review by GPT-6.1 Sol through Subfleet (
reviewtask,standardtier):delete_arraysoverride was still needed for rebuilt simulations), plus one precision note on the property test's selector. Fixed in 5f2ea22.build_from_populationsis fixed in a25ce01. Round 3 also reproduced every negative control above independently.Reports:
review-2260.md,review-2262-r2.md,review_r3.mdandreview_r4.mdin the evidence folder.axiom: n/a: engine bookkeeping only; no policy rule changes.
🤖 Generated with Claude Code