Repository navigation
Replay every OBR-costed fiscal event, June 2010 to Autumn Budget 2025 #156
Description
Extends the Autumn Budget 2025 port (#136) to every fiscal event in the OBR's Policy measures database since the June 2010 Budget: 32 events, from the uk_obr harvest (PMD November 2025 vintage, tax and spending measures by head and fiscal year).
Why now. The 28 October Budget is the next event. The official costings are the only key for national figures that PolicyEngine never touched, and replaying past events tests the model on the kinds of measures Budgets actually contain: reliefs, freezes, mid-year dates and benefit interactions. Synthetic drills have not reached those yet.
Recipe (the #136 pattern, per event):
- An event registry: every measure, with OBR costings by head and year. Each measure carries one construction:
pe_reform_delta,pe_baseline_modifier/reversal_on_certified_world, a package, ornot_expressiblewith ape_gapreason. - Counterparts for each expressible measure on the certified bundle, one artifact per measure and year.
- Staging under the ingest contract.
- A descriptive comparison: bins and named divergence axes, with no pass/fail (the Scorecard epistemics rule).
Two engine pins. Run on the current certified bundle (populace-uk-2023, policyengine-uk 2.89.2). Run again on the next certified pair (uk-data 1.58.0 with policyengine-uk 2.123.x, policyengine.py#555) once it is certified. The difference between the two runs shows what the recent fixes changed.
Plan.
- A pilot lane builds the generalised event pipeline and runs Autumn Budget 2024, Spring Budget 2024 and Autumn Budget 2017. Autumn Budget 2017 tests whether the bundle can simulate scoring years from 2017-18.
- An inventory lane classifies every measure in all 32 events for household-model scope and expressibility.
- Per-event lanes then fan out on the pilot's recipe.
- Not-expressible measures, ranked by size, become policyengine-uk capability issues.
Comparisons stay descriptive throughout (bins, explained share, named axes).
Activity
MaxGhenis commentedon Oct 9, 2026
Plan revised after Max's point: a faithful replay needs historical policy rules, per-year Microcosm populations and vintage calibration targets (the OBR's own forecast at each event). That is a large project, so the work is in two tracks.
pe_gaprows for what the engine cannot express #136. The vintage difference is tagged as a named divergence axis, not fixed. Branchreplay/pilot, lane20261009-095118-replay-track1-recent.replay/inventory, lane20261009-095118-replay-track2-scoping.Add the Autumn Budget 2024 results and updates left uncommitted when …
Regenerate the Autumn Budget 2024 comparison and summary from the ado…
Complete the four remaining recent-event replays on the certified bun…
Scope the vintage-faithful replay of 32 OBR-costed fiscal events (#156,…
Fix the PR review findings and rerun three events (#156)
Apply ruff format and relabel completed replays (#156)
Fix the review findings on the replay scoping (#156)
Add certified-bundle replay pipeline for recent OBR fiscal events (#157)
Pin the FRS vintage to each event's publication date and keep collect…
Scope the vintage-faithful replay of 32 OBR-costed fiscal events (#158)
MaxGhenis commentedon Oct 10, 2026
Status, 10 October.
Pass --bundle through the staging CLI's identity check
Read the Modal receipt's bundle identity from its request
Guard both protected trees for every replay output path
6 remaining items
Replay the five track-1 events on the policyengine 6.2.6 bundle
Show an intermediate bundle and a readable engine comparison
Record paired runs between bundles as hash-bound documents
List each driver's evidence in the engine comparison
Compare the 2.89.2 and 6.2.6 replays, with authored attribution
Document the engine comparison layout, paired runs and timings
Close the three gaps the delta review found
Register the policyengine 6.2.6 certified UK bundle
Rebuild the five track-1 event registries for the 6.2.6 bundle
Replay the five track-1 events on the policyengine 6.2.6 bundle
Show an intermediate bundle and a readable engine comparison
Record paired runs between bundles as hash-bound documents
List each driver's evidence in the engine comparison
Compare the 2.89.2 and 6.2.6 replays, with authored attribution
Document the engine comparison layout, paired runs and timings