Repository navigation
Detect undeclared tax movement at any baseline level and count non-finite responses as movement - #2758
Detect undeclared tax movement at any baseline level and count non-finite responses as movement#2758MaxGhenis wants to merge 4 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The response allowance grows with the response, so an infinite response got an infinite allowance and passed all three movement checks, and NaN compared false with any allowance. One predicate, _moves, now applies the rule in the initial screen and both held checks, with the held slope's doubled allowance passed at its call. Tests pin the 2% and 4% boundaries on each side, any non-finite response failing every path, and agreement between the paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Explicit Hypothesis examples and two table rows make the boundary tests fail deterministically when the comparison becomes non-strict or drops the response-proportional term. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles. Checks failing on both base and head (1)
Counted-once checkEach of 91 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
Fixes #2606
What and why
The aggregate checker in
policyengine_uk/utils/aggregate_impact/guard.pyjudged movement in an undeclared tax head against a level-based allowance: £0.02 plus 2e-6 of the head's baseline. Once a household's head reached about £490,000, a £1 one-for-one response fit inside that allowance and passed.This PR moves undeclared tax movement onto the response allowance that declared readers already use,
0.02 × |expected| + 2e-6 × |response|. Hereexpectedcounts the pounds the checker added in the household: £1 for each perturbed unit. A new predicate,_moves(response, expected, allowances=1), holds the rule in one place. The initial screen injudge_totaland the first held check instill_breakscall it with one allowance. The held-slope check inbreaks_on_slopepassesallowances=2at its call, because it compares two points._movesalso counts any non-finite response as movement. The response allowance grows with the response, so an infinite response would get an infinite allowance, and NaN compares false against any allowance.Invariants
These hold for every household, response
rand additione ≥ 0:r, the initial screen and the first held check report movement exactly when|r| > 0.02|e| + 2e-6|r|. The held slope reports it exactly when|r|exceeds twice that allowance.+inf,-infand NaN.rand-rget the same verdict on every path.e > 0, a one-for-one response (r = ±e) always moves.Hypothesis properties in
test_aggregate_impact_guard.pytest invariants 1 to 7 over levels from £0 to £1bn, additions from £0 to £1m and responses up to £1tn. One of them tests invariant 5 as a differential between the two implementations of the first-point rule, the screen injudge_totalandstill_breaks. A table test pins each side of each boundary. It includesexpected = 1, response = 0.03, which the initial and first held checks reject and the held slope accepts. Two tests run the complete guard on a small hand-built model. One runs a £1 containment at levels of £0, £490,000 and £1bn; the other runs infinite, negative-infinite and NaN responses.Compatibility
A
counted_once:<component>check that passes on main can now fail. That happens when an undeclared tax head's response exceeds the response allowance, fits within main's level allowance, and still moves with the other heads held. This covers containment, partial responses and negative responses.A check that fails on main can now pass, because the allowance now grows with the pounds added, as the issue asks. With £2 added, a £0.03 response and a zero baseline, main fails and this branch passes.
In a household where the checker added nothing, any non-zero response now counts as movement. Main allowed £0.02 plus the level term there.
NaN changes from main. Main lets a NaN response pass all three movement checks, because
nan > xis false; this branch fails it. Main already fails+infand-inf, and so does this branch.Two NaN heads. When two undeclared heads both turn NaN, main reports a pass and this branch stops with Core's
ValueError, raised as it sets the held heads as inputs. #2759 covers that error and the four other checks that still pass NaN.Independent heads with zero response still pass.
A limit this branch shares with main. Core stores float variables as
float32, which can't hold a £1 change at a level of £16,777,216 or above. Both guards pass such a change. The complete-guard test at £1bn uses a float64 fake model, so it doesn't cover that storage.My local runs used no survey data. The aggregate report runs one guard on both sides, so the test plan compares it with reports from main's guard.
No file overlaps with #2719 or with open #2700, #2727 and #2735; #2731 and #2234 have merged. The branch merges cleanly into main at
03349c9d5, and its diff applies cleanly onto #2719's head (c086c4917). With this diff on each, the guard file passes 63 of 63 and the partition subset below passes 4. The branch changes no existing test's expected value.Test plan
ruff format --checkandruff checkpass on both changed Python files.ruff check --select F811,F821 policyengine_ukpasses across 1,895 Python files, andgit diff --checkpasses.changelog.d/2606.fixed.md.056e80fd8e91) where the reports on Derive economic assumption index horizon from the growth series #2756 and Correct programme agencies and national coverage #2757, which ran main's guard, fail none. The two groups of reports list the same mediated responses, so no check changes status. The check covers 2026-27 only, on main as here.No run used survey data: I unset
HUGGING_FACE_TOKENandPOLICYENGINE_UK_DEFAULT_DATASETfor each one.policyengine_uk/tests/aggregate_impact/, file by file: 330 passed, 9 skipped.test_aggregate_impact_certified.pyskips all 9 of its tests because they need the survey dataset. The other counts:childcare_provenance42,compare133,data_mode4,dataset13,guard63,measure44,partition11,reforms20.test_aggregate_impact_guard.pyalone: 63 passed.policyengine_uk/tests/code_health/test_aggregate_impact_workflow.py: 17 passed.policyengine_uk/tests/test_fiscal_partition.py -k 'no_head_reaches_another_head_of_its_total_undeclared or declared_reads_occur': 4 passed, 39 deselected.judge_totalcalls across spending, declared-reader, own-head and model-total cases, 600 held-check calls for containment and rises, and 45 complete-guard runs. Run directly, both guards fail+infand-inf. NaN passes on main and fails here.Mutation testing. I applied each mutant to a copy of the final guard and ran the final test file against it. The tests kill all 15.
>=in place of>wherefrom the first held checkwherefrom the held slopeReproducer
Reproducer: policyengine_uk/tests/aggregate_impact/test_aggregate_impact_guard.py::test_undeclared_tax_containment_fails_at_any_baseline; policyengine_uk/tests/aggregate_impact/test_aggregate_impact_guard.py::test_a_non_finite_undeclared_tax_response_fails
axiom: n/a: aggregate checker tooling, with no policy change
🤖 Generated with Claude Code