Skip to content

Certify policyengine-uk-data 1.58.0 with policyengine-uk 2.125.1 and core 3.33.0 - #555

Merged
MaxGhenis merged 4 commits into
mainfrom
certify-uk-data-1.58.0
Oct 10, 2026
Merged

MaxGhenis merged 4 commits into
mainfrom
certify-uk-data-1.58.0

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #554

Certified versions

Version Basis
UK data policyengine-uk-data 1.58.0, enhanced_frs_2024_25 (sha256 5ba399ea…b960b6) release manifest at HF commit fdeba385
UK model policyengine-uk 2.125.1 legacy_compatible_model_package: the publisher's claim in that manifest
Core policyengine-core 3.33.0 the same claim (compatible_core_packages); also used by the US bundle
US policyengine-us 2.2.1, populace-us-2024-spm-20260915 and the ACS-local regional release from #552 unchanged; core 3.33.0's US effect is below

Before this PR, the bundle (6.2.5) certified 1.56.16 with policyengine-uk 2.102.3 and core 3.32.10.

This is the release Max ruled on in d778 (2026-10-05): pin the latest policyengine-uk and core with the uk-data batch release, and certify that bundle. Merging releases it as policyengine 6.2.6, because main is at 6.2.5 and a changed fragment bumps the patch.

  • Versions, checked on live PyPI at 09:49Z on 2026-10-10: policyengine-uk 2.125.1 (uploaded 00:21Z) and policyengine-core 3.33.0 (09:43Z). I will re-check before merging.
  • #1927 is included. It has been in policyengine-uk since 2.123.6. 1.58.0 was built with 2.122.2 and core 3.32.13, so the newer pair is covered by a publisher compatibility claim, as Bump policyengine-uk to 2.102.3 for the FYE 2025 absolute poverty line #536 did for 1.56.16.
  • Max's precondition holds in code. In 2.125.1, Housing Benefit's assessable capital, applicable income and tariff income all read in_receipt_of_guarantee_credit. In 2.122.2 they read guarantee_credit > 0, which is entitlement.

Certification

  • Publisher claims. These commits change only releases/1.58.0/release_manifest.json. Tag 1.58.0 still points at 47bbb58d, and the build-time claims (2.122.2, 3.32.13) stay.
    • c30504aa appended policyengine-uk==2.123.4 and policyengine-core==3.32.21 (this PR's first round).
    • fdeba385 appends policyengine-uk==2.125.1 and policyengine-core==3.33.0.
    • uk-data's own validate_release_manifest (at 4cbedbec) passes on the new manifest.
    • uk-data's finalized-release guard reads the manifest at the tag, so later uploads are unaffected.
  • Command. scripts/bundle.py certify-data --country uk --data-producer populace --manifest-uri hf://model/policyengine/policyengine-uk-data-private@fdeba385…/releases/1.58.0/release_manifest.json --model-version 2.125.1.
    • The gate accepts 2.125.1 on the claim and still refuses an unclaimed version (2.125.0).
  • Core pin. packages.policyengine-core in the bundle manifest moves to 3.33.0, and bundle.py generate regenerates the four extras from it, as in Update PolicyEngine Core to 3.32.10 #540.
  • Regenerated.
    • uv.lock: uv 0.12.14 with --upgrade-package for the two packages only, 13 lines against main.
    • Both TRO sidecars.
    • docs/households.md's bundle line.
    • The test constants that pin the certified UK release.

Evidence

policyengine_uk.Microsimulation runs on the 1.58.0 artifact, in clean environments, for 2024, 2025, 2026 and 2030. "A" is policyengine-uk 2.122.2 with core 3.32.13, the build pair. The claims below cover these datasets and years; they are not proofs for every input.

Check Result
Core only: 3.32.13 → 3.33.0 under uk 2.122.2 Every total, caseload and poverty rate is bit-identical in all four years
Finiteness, uk 2.125.1 / core 3.33.0 Every aggregate is finite
Unrecognised inputs, A vs the certified pair The 20 input columns the model does not define are identical
Largest move in any poverty rate, A → certified pair 0.76pp (2024); 0.51pp (2026)

Model moves, A → certified pair. Each one is traced to a policyengine-uk 2.122.3–2.125.1 changelog entry:

  • Housing Benefit: −7.2% in 2026 (£9.14bn → £8.48bn; 1.303m → 1.232m benefit units). Bisecting releases 2.123.5, 2.123.6, 2.123.7, 2.124.0 and 2.125.0 puts the whole move at 2.123.6. Its Housing Benefit entry is #1927, the passport keyed on receipt. Pensioner absolute AHC poverty rises from 13.4% to 13.9% in 2026.
  • Income tax: +0.20% in 2026 (+£0.65bn). The cause is 2.123.0's property allowance fix: the allowance no longer comes off profits already net of expenses, so property allowance deductions fall from £2.5bn to £0.02bn.
  • Guarantee Credit: +0.23% in 2026, from 2.123.0's minimum-guarantee eligibility fix.
  • Employer NI: −0.02%, from 2.123.2's £5,000 secondary threshold.
  • High Value Council Tax Surcharge: −2.3% in 2030, from 2.123.1 (owners only).
  • Child Benefit: −0.18% in 2025 and −0.01% in 2030. In two benefit units, the property allowance fix raises the higher earner's adjusted net income from £79,576 to £80,576, past the end of the HICBC taper. With opt_out_charge_share at 1, an opted-out claimant stays opted out.

Core 3.33.0 and the US bundle. Core changes the US results, so I checked them. These are policyengine-us 2.2.1 runs on the certified populace_us_2024 for 2026:

  • The change. 7 of 19 US aggregates move between core 3.32.10 and 3.33.0. Refundable CTC falls by $0.105bn (−0.26%), income tax rises by the same amount, and household net income falls by $0.105bn. State income tax rises by $22k and the SPM poverty rate by under 1e-8pp. The CTC total and SNAP, SSI, EITC, TANF, Social Security, WIC, school meals and the ACA PTC are unchanged. Bisecting 3.32.25, 3.32.28, 3.32.29 and 3.33.0, every 3.32.x run is bit-identical to 3.32.10. The move is in 3.33.0: 69 tax units (61k weighted) switch from itemising to the standard deduction.
  • It is a determinism fix. I ran the same 3.32.29 simulation twice, asking for variables in a different order. Federal income tax differed by more than one cent in 194 tax units (170k weighted, $0.16bn in total; 196 differ at all, the other two by under $0.001), because values were left stale between the itemising and non-itemising branches. Under 3.33.0, the same two runs agree in every unit. That is the class of bug 3.33.0's changelog fixes ("branch input changes prevent caching stale calculations").
  • What I did not run. I did not run the ACS-local regional dataset. It is 9.8 GB, and it runs the same policyengine-us code on different rows.

What changes for users

These figures use policyengine.py's own API (ensure_datasets, Simulation, calculate_uk_poverty_rates and _by_age), for 2026. The left column is policyengine 6.2.5 from PyPI. The right column is this PR's head. Both include #557 and use the bundle's certified default dataset. Each total and overall rate equals the direct Microsimulation run exactly in both columns.

Poverty rate, 2026-27 6.2.5 (1.56.16, uk 2.102.3) This PR (1.58.0, uk 2.125.1)
Absolute BHC, everyone (the headline) 14.4% 15.0%
Absolute AHC, everyone 18.7% 20.3%
Relative BHC, everyone 19.0% 16.5%
Relative AHC, everyone 23.3% 21.1%
Absolute BHC, children (child) 16.3% 20.0%
Absolute AHC, children 23.7% 29.2%
Absolute BHC, 65 and over (senior) 13.5% 15.0%
Absolute AHC, 65 and over 11.9% 13.9%
Relative BHC, 65 and over 21.2% 16.8%
Total, 2026-27 (£bn) 6.2.5 This PR
Universal Credit 79.3 83.5
Housing Benefit 13.7 8.5
Child Benefit 17.7 16.9
Pension Credit 7.0 7.1
State Pension 133.7 128.3
Income tax 313.5 320.5
Household net income 1,764.7 1,710.8

Model versus data. These are direct-Microsimulation totals for 2026. The model step moves child absolute poverty by 0.2pp or less, and any poverty rate by at most 1.1pp. The data step accounts for +3.7pp (BHC) and +5.7pp (AHC) on child absolute poverty, and −2.5pp on relative BHC overall. The model step runs policyengine-uk 2.102.3 → 2.125.1 on the 1.56.16 artifact; the data step runs 1.56.16 → 1.58.0 on 2.125.1.

Total Model step Data step Combined
income_tax +0.1% +2.1% +2.3%
national_insurance +0.2% +2.7% +2.9%
universal_credit +5.4% −0.1% +5.3%
child_benefit −2.4% −2.1% −4.5%
pension_credit +2.3% −1.4% +0.9%
state_pension −2.7% −1.4% −4.0%
housing_benefit −1.8% −37.0% −38.1%
council_tax_reduction −0.4% −7.5% −7.9%
household_net_income −0.1% −3.0% −3.1%
hbai_household_net_income −0.0% −2.7% −2.8%

guarantee_credit is left out of these tables. In 2.102.3 it had no defined_for = "is_pension_credit_eligible", so its total there (£113bn) is not the same quantity. pension_credit is comparable.

Why absolute child poverty rises: the old data understated it. These figures are 2026-27 on policyengine-uk 2.125.1 / core 3.33.0, with outputs retained.

  • When it happens. Every release from 1.56.16 to 1.57.4 gives 16.3–16.7% for children, absolute BHC. The whole rise, to 20.0%, happens at 1.57.4 → 1.58.0.
  • Where it sits. Poor children in SPI-synthetic households go from 0.05m to 1.20m, out of 4.25m children in those rows. Poor children in FRS rows fall from 2.49m to 1.85m.
  • The cause is Add the weighted-module design note #529's fix. Before Add the weighted-module design note #529, SPI-synthetic rows gave employment income to 99.7% of their children and to 46% of out-of-work adults (Add the weighted-module design note #529's own table). For the 1,686 SPI households with children that become poor, mean household employment income falls from £79.4k to £12.5k, with both weighted by 1.58.0 household weights. 43% of them now have none.
  • Take-up is not the cause. If every benefit unit claimed Universal Credit, 1.13m of those 1.19m children would still be poor.
  • Checked against HBAI. In FYE 2025 the absolute line is 60% of HBAI's own median, £719.48 a week. The model's FYE 2025 median is £779.82 on 1.57.4 (+8.4%) and £734.73 on 1.58.0 (+2.1%). HBAI FYE 2025 reports 21% of children BHC, 27% AHC and 16% of everyone BHC. HBAI counts dependent children; these tables use ages 0–17.
FYE 2025 (model year 2024) 1.57.4 1.58.0 HBAI
Children, absolute BHC 18.2% 21.8% 21%
Children, absolute AHC 24.9% 29.8% 27%
Everyone, absolute BHC 15.4% 15.7% 16%

Child AHC now overshoots HBAI by 2.8pp. One hypothesis, still to be tested in uk-data, is Universal Credit's capital test.

  • What was measured. This is a 2026 BHC diagnostic, not a decomposition of the FYE 2025 AHC gap. For 34% of poor children in FRS rows (44% in SPI rows), their own benefit unit has UC-assessable capital above £16,000, and none of those units receives UC.
  • Why this may happen. The dataset does not supply benefit-unit capital (benunit_reported_capital, added in policyengine-uk 2.124.0), so UC falls back to a household wealth proxy.
  • Not yet shown. That this proxy accounts for the overshoot. That needs a counterfactual run once uk-data supplies the input.
  • Relative poverty falls because the lower, closer-to-HBAI median lowers the relative line.

Household snapshots, against main. On the certified pair these equal what this PR's 2.123.4 round produced.

  • Employer NI falls by £1.20 per employee in the three earning households: 2.123.2's statutory £5,000 secondary threshold, instead of 52 × £96.
  • In uk_couple_two_kids, the three-year-old gets the extended 30-hour childcare entitlement (£7,104.38) instead of the universal 15 hours (£3,552.19). Household benefits rise by £3,552.19 and net income by £3,552.18. This was bisected to policyengine-uk 2.114.2, which rewrote the extended entitlement's qualifying-child and income tests (SI 2022/1134 regs 13, 14(3)(c) and 18). Both parents work and each earns under £100,000.
  • The model surface grows to about 3,200 parameters and 1,100 variables.
  • personal_rent is now a formula variable with no defined_for, so test_uk_model_exposes_selected_variable_metadata checks child_benefit's defined_for (would_claim_child_benefit) instead.

Invariants checked

This PR changes pins and certification metadata, not logic. These properties were executed on the datasets and years named above:

  • Core bit-identity on UK data. Core 3.32.13 → 3.33.0 under policyengine-uk 2.122.2 gives identical outputs. It is a differential across two core versions.
  • Determinism under query order. Two query orders give identical US federal income tax in all 79,729 tax units under core 3.33.0. They do not under 3.32.29 (194 units differ by more than one cent).
  • Finiteness. Every aggregate is finite.
  • Inputs. The set of unrecognised inputs is unchanged.
  • Wrapper agrees with the model. policyengine.py's API equals direct policyengine_uk on every reported UK total and overall rate, in both 6.2.5 and this PR.
  • The certification gate refuses unclaimed model versions.

Review history

r1 was GPT-6.1 Sol at cdad33ae, and it requested changes. Each finding is addressed:

  • P1, the pin was not the latest. It now pins 2.125.1 / 3.33.0, the latest at 09:49Z on 10/10, on a new claim with new evidence.
  • P2, the build-time git sha names uk-data's commit. This is a publisher defect, fixed upstream in policyengine-uk 2.124.0 (build_metadata no longer reads an enclosing repo). 1.58.0's manifest still carries the old value, which the bundle copies faithfully. The next uk-data release built with ≥ 2.124.0 corrects it.
  • P3, mixed weighting. All earnings figures now use 1.58.0 household weights.
  • P4, household-level capital. The capital diagnostic is now per benefit unit.
  • P5, overclaimed scope. The scope is stated, and the median and bisect outputs are retained.
  • P6, rounding and year labels. Corrected.

r2 was GPT-6.1 Sol at 57fdcc59: APPROVE WITH NITS. Its three P2/P3 nits are fixed in this body and title: the capital diagnostic is now labelled a 2026 BHC hypothesis, the determinism count states its one-cent tolerance, and the title names the certified pair. Review text: ~/reviews/pepy-uk-data-1.58.0-cert/review_r2.md.

Checks run locally

  • make format and make lint pass.
  • bundle.py check --published-spm --include-tros --strict-tros, release_lock.py and uv lock --check (uv 0.12.14) pass.
  • check_release_credentials.py fails locally by design: it accepts only a read-role token, and mine is fine-grained. CI's secret is read-role.
  • At the merged head, the bundle, certification, release-manifest, region, model, metadata, snapshot and release-lock tests give 244 passed. One snapshot was refreshed: the model surface variable bucket went from 10 to 11.
  • CI runs the full suite on Python 3.11–3.14.

Evidence and scripts are in ~/reviews/pepy-uk-data-1.58.0-cert/ (out/, out/r2/).

axiom: n/a: a wrapper pin and data certification; no policy is encoded here.

🤖 Generated with Claude Code

MaxGhenis and others added 3 commits October 7, 2026 16:52
…core 3.32.21

Certify enhanced_frs_2024_25 from policyengine-uk-data 1.58.0 as the UK
default, on the publisher's compatibility claim added to the 1.58.0 release
manifest at HF commit c30504aa. Pin the latest policyengine-uk (2.123.4) and
policyengine-core (3.32.21), regenerate the extras, lock and TRO sidecars,
and move the tests that pin the certified UK release.

Fixes #554

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Employer NI falls by GBP 1.20 per employee: policyengine-uk 2.123.2 stores
  the Class 1 secondary threshold as the statutory GBP 5,000 a year rather
  than 52 x GBP 96.
- The couple's three-year-old now gets the extended 30-hour childcare
  entitlement (GBP 7,104.38) instead of the universal 15 hours
  (GBP 3,552.19). Bisected to policyengine-uk 2.114.2, which rewrote the
  extended entitlement's qualifying-child and income tests (SI 2022/1134).
- The model surface grows to about 3,200 parameters and 1,000 variables.
- personal_rent became a formula variable with no defined_for, so the
  metadata test checks child_benefit's defined_for instead.

Fixes #554

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the bundle manifest and both TRO sidecars by regenerating them on
main's: re-run certify-data against the 1.58.0 manifest at HF c30504aa,
re-apply the policyengine-core 3.32.21 package pin, then bundle.py generate
(with TROs). The UK entry keeps main's 6.2.2 stamps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Pin policyengine-uk ≥ 2.123.6, not 2.123.4. The certified bundle needs PolicyEngine/policyengine-uk#1927 ("Key the Housing Benefit Guarantee Credit passport on receipt"). It merged on 2026-10-08 at 12:07Z as bd13f3d1, and it ships in 2.123.6 and 2.123.7 but not 2.123.4.

On earlier releases the HB passport keys on Guarantee Credit entitlement, so pensioners who don't claim Pension Credit still get passported to maximum Housing Benefit. In the uk-triple-lock model-v2 pilot, that overstates the Burnham plan's HB offset by about £3.3bn on an affected path. The review on PolicyEngine/uk-triple-lock#24 makes #1927 a hard precondition for the certified bundle the rebuild uses (d778), and the rebuild will refuse a bundle whose HB passport doesn't read in_receipt_of_guarantee_credit.

So please certify 2.123.7, or 2.123.6, with whatever core it needs, and extend the 1.58.0 manifest's publisher claim to cover it. If that claim can't be extended without a new data build, say so here.

@MaxGhenis

MaxGhenis commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Heads-up for the UK tables in this PR body: the policyengine.py-API columns ("6.2.1" and "This PR") carry a wrapper bug, #556, fixed in #557. A UK year-file run took the projected year as policyengine-uk's data year, so the State Pension followed CPI instead of the triple lock. The direct-Microsimulation "model versus data" table is unaffected.

The same pepy_poverty.py on this branch's head with #557 applied (uk 2.123.4, data 1.58.0, 2026). These now equal the direct policyengine-uk run on every aggregate and overall poverty rate:

This PR body With #557
State Pension, £bn 124.4 128.3
Pension Credit, £bn 7.4 7.1
Income tax, £bn 319.9 320.5
Housing Benefit, £bn 9.2 9.1
Household net income, £bn 1,709 1,711
Absolute BHC, everyone 15.2% 15.0%
Absolute AHC, everyone 20.3% 20.2%
Relative BHC, everyone 16.6% 16.6%
Relative AHC, everyone 21.1% 21.0%
Absolute BHC, 65 and over 15.7% 15.0%
Relative BHC, 65 and over 17.5% 16.8%
Absolute BHC / AHC, children 20.0% / 29.2% 20.0% / 29.2%

The 6.2.1 column moves too: State Pension £127.5bn → £133.7bn, absolute BHC everyone 14.7% → 14.4%, and 65+ absolute BHC 15.2% → 13.5%. #557 has the full table and the record-level checks.

Reforms are affected too, which matters for the triple-lock rebuild. Take a State Pension rate reform (new £260/wk, basic £200/wk from 2026) through Simulation(policy=...):

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

#557 is merged into main (e080ce00), so this branch can merge main and pick up the fix. The UK tables in this PR body should then come from a re-run of pepy_poverty.py on the merged head. The corrected figures in my comment above came from this branch's cdad33a with #557 cherry-picked. Any triple-lock or State Pension rate run on this bundle needs the fix: without it, the reform above was understated by about 3% on this branch.

…e 3.33.0

- Merge main, which brings #557 (UK year files keep their survey data year).
- Re-certify enhanced_frs_2024_25 from policyengine-uk-data 1.58.0 for
  policyengine-uk 2.125.1 on the publisher claim at HF fdeba385. 2.125.1
  carries policyengine-uk#1927 (from 2.123.6): the Housing Benefit
  Guarantee Credit passport now keys on receipt.
- Pin policyengine-core 3.33.0, the latest. UK results are bit-identical to
  core 3.32.13. US national results change slightly because 3.33.0 removes
  query-order dependence that 3.32.x had in branch calculations.
- Resolve the bundle manifest and both TROs by regenerating them on main's;
  relock only the two packages with uv 0.12.14.
- The UK model surface grows to about 1,100 variables. All four UK household
  snapshots are unchanged.

Fixes #554

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

All three are addressed at 57fdcc59:

  1. #1927. The branch now pins policyengine-uk 2.125.1, which includes #1927 (from 2.123.6), with policyengine-core 3.33.0. Both were the latest on PyPI at 09:49Z on 10/10.
    • In 2.125.1, Housing Benefit's assessable capital, applicable income and tariff income all read in_receipt_of_guarantee_credit. In 2.122.2 they read guarantee_credit > 0.
    • The 1.58.0 publisher claim extends to the new pair without a new data build, at HF fdeba385.
    • On the 1.58.0 artifact, #1927 cuts Housing Benefit by 7.2% in 2026 (£9.14bn → £8.48bn; −71k benefit units). Bisecting releases puts the whole move at 2.123.6.
  2. and 3. Anchor UK year-file runs on the survey year so the State Pension follows the triple lock #557. Main (6.2.5) is merged. The UK tables in the body come from pepy_poverty.py re-run on this head and on policyengine 6.2.5. Both columns now equal the direct policyengine_uk run on every reported total and overall rate.

The body also documents a US effect. Core 3.33.0 moves refundable CTC by −$0.105bn on the certified US data. It is a determinism fix: under 3.32.29, federal income tax for 194 tax units depended on query order, and under 3.33.0 it does not. The independent review (round 2, GPT-6.1 Sol) is running on this head.

@MaxGhenis MaxGhenis changed the title Certify policyengine-uk-data 1.58.0 with policyengine-uk 2.123.4 and core 3.32.21 Certify policyengine-uk-data 1.58.0 with policyengine-uk 2.125.1 and core 3.33.0 Oct 10, 2026
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 10, 2026 10:58
@MaxGhenis
MaxGhenis merged commit 37d25c4 into main Oct 10, 2026
13 checks passed
@MaxGhenis
MaxGhenis deleted the certify-uk-data-1.58.0 branch October 10, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Certify policyengine-uk-data 1.58.0 with the latest policyengine-uk and core

1 participant