Skip to content

Add certified-bundle replay pipeline for recent OBR fiscal events - #157

Merged
MaxGhenis merged 25 commits into
mainfrom
replay/pilot
Oct 10, 2026
Merged

MaxGhenis merged 25 commits into
mainfrom
replay/pilot

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Refs #156 (track 1). Replays recent OBR-costed fiscal events on the certified bundle (populace-uk-2023, policyengine-uk 2.89.2), generalising the Autumn Budget 2025 port (#136).

What's here

  • An event-parametrised pipeline: build_uk_event_registry, compute_uk_event, stage_uk_event, compare_uk_event, the Modal UK adapter modal_uk_event and adopt_uk_event.
  • Registries and descriptive comparisons for Spring Budget 2023, Autumn Statement 2023, Spring Budget 2024, Autumn Budget 2024 and Spring Statement 2025.
  • docs/uk_replay/RECIPE.md and YEARS.md.
  • Every source head and FY cell is accounted once. Measures by class: 0 expressible, 16 partial (13 plus LHA reset, UC childcare caps and Class 2 abolition), and the rest not expressible or outside household scope. The not-expressible measures split into 84 construction_pending and 76 model or data gaps.
  • 165 computed source rows. Comparisons use descriptive bins and named axes (population vintage, behavioural, baseline vintage, CY-proxies-FY, head scope). There is no pass/fail.

Review

  • GPT-6.1 Sol, first review: NEEDS-FIXES.
    • B1: an inert UC health construction.
    • B2: reform dates and the fiscal-year conversion.
    • B3: staging hid inert constructions.
    • Also M1–M3 and L1.
  • All fixed in 133613e. Reruns of Spring Statement 2025, Spring Budget 2023 and Autumn Statement 2023.
  • Re-review: MERGE-SAFE at 133613e. 35549bf adds only ruff format and the reviewer's suggested label ("Registered construction replay complete").

Divergence causes (from the review)

Measure Cause
Annual Allowance £60k Model bug, policyengine-uk#2237
SDLT surcharge +2pp Model bug, policyengine-uk#2238
UC standard allowance Model bug, policyengine-uk#2239, plus the CPI caveat
HICBC Fixed on main after the certified pin
Employee NICs (~1.25×) Certified tax base 15–20% above the OBR forecast
CGT Pooled-gains construction (upward bias), static vs behavioural
Winter Fuel Payment Baseline vintage
Employer NICs income tax head Wage incidence

Invariants

  • Source rows and signed/absolute £ in = classified and staged out, for every event.
  • Positive = gain to the Exchequer. Reversals are -(reversal − certified).
  • Head effects sum to measure totals.
  • Byte-identical reruns.
  • AB2025 fixtures unchanged.
  • An inert in-force construction blocks completion.

Tests: 176 replay and 324 extended tests (fix lane); 128 comparison, compute and registry tests after the label change.

The PR does not touch the app or the DB build, so merging publishes nothing.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-scorecard Ready Ready Preview Oct 10, 2026 4:13am UTC

Request Review

…the lane stopped (#156)

The track 1 lane hit its wall-clock limit with these files in the worktree and not yet
pushed: completed Autumn Budget 2024 artifacts (private school VAT, the SDLT additional
dwelling surcharge, the Winter Fuel means test) and the files updated alongside them.
Committed as found; not yet reviewed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pted Modal run (#156)

33 source rows now have computed counterparts across the five partial measures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dle (#156)

Autumn Statement 2023, Spring Budget 2024, Spring Statement 2025 and Spring Budget 2023
ran on Modal, passed the adoption validation and are staged and compared. All five recent
events now have numerical comparisons: 130 computed source rows across 13 partial measures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

MaxGhenis commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Independent review (GPT-6.1 Sol, read-only): NEEDS-FIXES. Full text: ~/PolicyEngine/_worktrees/replay-briefs/track1-review-out.md on the main session's machine. Summary:

Blocking

  • B1. The UC health element construction (Spring Statement 2025) is inert. The uc_reform.py modifier fixes uc_LCWRA_element before the registry reform applies, but the measure is reported as computed.
  • B2. Dict reforms skip the 30 April fiscal-year conversion, contrary to YEARS.md and RECIPE.md. As a result the Annual Allowance package's FY2023-24 was lost.
  • B3. Staging labels identical in-force worlds as a benign note, which hid B1 and B2.

Other findings

  • M1: the UC standard allowance counterfactual uses a CPI path that differs from the legislated amount's.
  • M2: the "not expressible" count mixes 87 construction_pending with 76 genuine gaps.
  • M3: the divergence queue lacks the engine diagnoses.
  • L1: notes.

Divergence causes found

  • Model bugs, now filed:
    • policyengine-uk#2237: the Annual Allowance excess is denied relief and also charged.
    • policyengine-uk#2238: SDLT counts the whole other-property stock as an additional purchase.
    • policyengine-uk#2239: the UC standard allowance omits the 2027-29 uplifts.
  • HICBC: fixed on main after the certified pin.
  • Employee NICs: the certified tax base is 15-20% above the OBR forecast; uk-data#537 is not the cause.
  • CGT: construction scope and behaviour.
  • Winter Fuel Payment: baseline vintage.
  • Employer NICs income tax head: wage incidence.

A fix lane is running: 20261009-215541-replay-track1-fixes. It also adds the three measures the review found expressible (LHA reset, UC childcare cap, Class 2 abolition).

Review (GPT-6.1 Sol): B1 the UC health element construction was inert, now refreshed after
the reform; B2 reform dates and the fiscal-year conversion, with docs corrected and the
Annual Allowance computed from 2023-24; B3 inert constructions now block numerical
completion; M1 UC counterfactual on the 3.8% CPI, naming policyengine-uk#2239; M2 the gap
split shown everywhere; M3 the divergence diagnoses with policyengine-uk#2237 and #2238;
L1 notes. Adds LHA reset, UC childcare caps and Class 2 abolition constructions. Reran
Spring Statement 2025, Spring Budget 2023 and Autumn Statement 2023 on Modal: 165 computed
rows (was 130). 176 replay and 324 extended tests pass; accounting identities hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's ruff format check flagged four files. The re-review (MERGE-SAFE at 133613e) suggested
'Registered construction replay complete' over 'Numerical replay complete', since the
replays cover registered partial constructions only; summaries regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 10, 2026 04:20
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merge audit: GPT-6.1 Sol review NEEDS-FIXES at 2c231b4 (B1-B3, M1-M3, L1); fixes in 133613e; re-review MERGE-SAFE at 133613e; 35549bf adds only ruff format and the reviewer's suggested status label. Checks green at 35549bf; MERGEABLE; no app or DB-build changes, so nothing is published. Squash-merged under Max's merge-on-gates rule.

This branch was successfully deployed

1 active deployment
Preview — 35549bff Deployed Oct 10, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant