Skip to content

Certify policyengine-uk 2.130.1 with policyengine-uk-data 1.58.1 - #617

Merged
MaxGhenis merged 2 commits into
mainfrom
w4/certify-uk-2.130.1
Oct 15, 2026
Merged

MaxGhenis merged 2 commits into
mainfrom
w4/certify-uk-2.130.1

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #616

Certified versions

Version Basis
UK data policyengine-uk-data 1.58.1, enhanced_frs_2024_25 (sha256 056e80fd…ce8563784) release manifest at HF commit 89be6b73 (manifest sha256 545797bf…)
UK model policyengine-uk 2.130.1 legacy_compatible_model_package: the publisher's claim in that manifest (==2.122.2, ==2.130.1)
Core policyengine-core 3.33.0 unchanged; the claim also adds ==3.33.0
US policyengine-us 2.2.1 and its datasets unchanged

Before this PR, main served uk-data 1.58.0 with policyengine-uk 2.126.12 (#585).

Why now: the UK Autumn Budget is on 28 October, with no model freeze (cos d1152), so the latest qualified pair is what runs on the day.

Certification

  • The claim. HF commit 89be6b73, parent 2526edbb, changes only releases/1.58.1/release_manifest.json. It appends policyengine-uk ==2.130.1 and policyengine-core ==3.33.0. The UK hub wrote it after reviewing the evidence, and checked that the uploaded file matches the reviewed claim byte for byte. uk-data's own validate_release_manifest accepts the result.
  • The command. scripts/bundle.py certify-data --country uk --data-producer populace --manifest-uri hf://model/policyengine/policyengine-uk-data-private@89be6b73…/releases/1.58.1/release_manifest.json --model-version 2.130.1, run in an environment synced from this branch's lockfile. The gate refuses an unclaimed version: 2.127.0 exits 2 with "matches neither the build-time model (2.122.2) nor any publisher compatibility claim".
  • Regenerated together:
    • the bundle manifest;
    • both TRO sidecars (bundle.py generate --include-tros with the HF token; bundle.py check --include-tros passes);
    • the three policyengine-uk pins in pyproject.toml;
    • uv.lock (uv 0.12.13, --upgrade-package policyengine-uk only): 12 lines, and its 2.130.1 hashes match PyPI's;
    • the test constants that pin the certified pair.

Evidence

The full evidence is in ~/reviews/w4-cert-2.130.1/EVIDENCE.md, which the UK hub reviewed (~/reviews/uk-hub/reviews/claim-2.130.1/review-evidence.md). Every figure is from full policyengine_uk.Microsimulation runs. The aggregates, the reform scores and the UC counts use the whole 1.58.1 file. The 19 contract population checks use its seeded 1,000-household subsample, so a failure outside that sample could escape them. The data-change comparisons also use the 1.58.0 file and the UK hub's seeded builds. Only aggregates and counts are reported.

  • Contract. W1's UK Budget reform contract 1.0.0 holds on 2.130.1 with 1.58.1: 673 pass, 0 fail, 0 error (6 pending on d1344, 79 unsupported). All 19 population checks pass. This uses the runner with Give contract assertions unique ids and declare full-run population coverage #612 and Check the UC allowance response on the award before deductions #614 (see below).

  • Finiteness and reforms. Every aggregate is finite in 2024, 2025, 2026 and 2030. A 21% basic rate from 6 April 2026 raises £6.66bn of income tax in 2026-27 (£6.67bn on the current pair).

  • Target fit. 78.3% of 644 national targets are within 10%, against 73.6% for the current pair. This is the policyengine-model UK audit session's measurement, using uk-data 1.58.1's own create_target_matrix.

  • Most of the move is the data. In 2025, household net income falls £10.60bn: −£10.36bn from the data and −£0.25bn from the model. The data move is the release's code effect plus unseeded rebuild variation in each published build. The UK hub's seeded comparison puts the code effect at −£5.57bn, and the rest is noise in the two builds (policyengine-uk-data#507).

  • The model moves are each attributed to a merge, by running every first-parent merge from 2.127.0 to 2.130.1 and splitting rollup #2380 into its members:

    • #2365: UC −0.27% (2024).
    • #2366: gov_spending +£0.54bn.
    • #2271: income tax +£0.34bn. 1.58.1 holds 2,251 Gift Aid records, so #2271's "no Gift Aid" held only for the 2023-24 file it measured.
    • #2119: ESA −1.26%, HB −0.3%.
    • #2047: senior relative poverty −0.14pp.
    • #2031: JSA £2.0m → £1.4m.

    2.126.12 and 2.127.0 agree to 1.6e-8 relative, and the released 2.130.1 wheel equals its source build.

Universal Credit and the contract's allowance check

  • The old check failed on 2.130.1. It required that no universal_credit payment fall when every UC standard allowance rises 10%. On 2.130.1, one unit in the subsample fails: a capped award whose deductions rise.

  • 2.126.12's pass was an artefact. 2.126.12 applies no UC deductions below 1e6 benefit-unit weight, so it ran the subsample with deductions off; #1905 removed that gate.

  • The law allows a payment to fall this way (this establishes that the model's behaviour is lawful in kind, not that the observed unit's deduction amounts are right; see #2410):

    • the cap counts UC before deductions and holds a capped award flat (SI 2013/376 regs 80(2), 81);
    • deductions are then taken from the award (SI 2013/380 Sch 6);
    • the UK hub's independent legal check returned AGREE_WITH_CHANGES.
  • Check the UC allowance response on the award before deductions #614 restates the check:

    • no award before deductions falls by more than a penny;
    • each payment moves by its award's change less its deductions' change, to the penny.

    Both versions hold on the full file (0 award falls, 0 accounting breaks). How much deductions rise belongs to policyengine-uk#2410.

Known gaps, unchanged by this pair

  • Northern Ireland: NI households have no constituency or local-authority codes (policyengine-uk-data#580), so those runs raise "no households".
  • Engine skew: the data is calibrated with policyengine-uk 2.122.2 and served with a later model (policyengine-uk-data#581).
  • git_sha: the manifest's build.built_with_model_package.git_sha records uk-data's commit, not the model's.

Order and gates

This PR's contract evidence uses #612 and #614 (W1's runner: assertion ids, and the UC check on the award). Neither changes what this PR certifies. #614 holds for its own independent review.

Independent review (GPT-6.1 Sol): APPROVE (~/reviews/w4-budget-ci-2026-10-11/pr617-round1-sol.md).

  • It reproduced the certification scope, pins, lock, hashes and append-only claim, and that the gate accepts 2.130.1 and refuses 2.127.0.
  • Regeneration reproduces the manifest, pins and TROs byte for byte.
  • Every numeric claim reproduces from the raw evidence, and an independent full-file UC rerun matches exactly.
  • 149 targeted tests passed.
  • Its three low documentation findings are fixed above.

Full suite on 480644f, with the HF token: 1 failed, 1684 passed, 9 skipped. The one failure was the projection-horizon pin, fixed in 36b4af0 (policyengine-uk 2.130.0 projects to 2032), which now passes.

axiom: n/a: data-release certification; no policy encoding changes

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 14, 2026 11:55
scripts/bundle.py certify-data on the publisher's claim at HF commit
89be6b73 (releases/1.58.1/release_manifest.json, sha256 545797bf...),
with the manifest, both TRO sidecars, the policyengine-uk pins, uv.lock
(uv 0.12.13, --upgrade-package policyengine-uk only) and the test
constants that pin the certified pair regenerated together. The gate
refuses an unclaimed version (2.127.0 exits 2).

Fixes #616

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.130.0 projects single-year datasets through 2032
(policyengine-uk#2376), so the projected input policyengine.py builds for
a 2024 year file now runs to 2032. Years through 2030 are projected as
before. Found by the full suite on this branch (1 failed, 1684 passed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 15, 2026 02:30
@MaxGhenis
MaxGhenis merged commit 8f476d6 into main Oct 15, 2026
12 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged at 36b4af0 on the standard gates. This certifies policyengine-uk 2.130.1 with policyengine-uk-data 1.58.1 as the UK default pair, on the publisher's claim at HF 89be6b73.

  • Checks: all 12 passed on the head.
  • Independent review (GPT-6.1 Sol): APPROVE. Its three low documentation findings are fixed in the body.
  • Evidence: reviewed by the UK hub (~/reviews/uk-hub/reviews/claim-2.130.1/review-evidence.md).
  • Data release: Max approved it as d1360.

Next: the release, sim-api's update, and the production check.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Traceability note. The qualification evidence for this pair ran W1's contract with the restated UC allowance check from #614, which is not yet on main.

#614 lands as soon as its round-3 review agrees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Certify policyengine-uk 2.130.1 with policyengine-uk-data 1.58.1

1 participant