Repository navigation
Solve Pension Credit take-up over entitled benefit units - #510
Merged
Merged
Conversation
Replace the OBR table 4.9 Housing Benefit target, which is GB DWP-funded spending but was compared with UK-wide modelled Housing Benefit, with DWP Spring 2026 targets for Housing Benefit spending and claims over Pension Credit qualifying age in Great Britain. Targets can now name the countries they cover. The figures under Pension Credit qualifying age are kept but not calibrated: the model pays working-age Housing Benefit to too few records for a target to be met without concentrating it on a handful of them. Stop drawing would_claim_uc for benefit units whose adults have all reached State Pension age, keeping every other unit's draw unchanged. Require policyengine-uk 2.102.5, which lets pension-age families make new Housing Benefit claims (PolicyEngine/policyengine-uk#1901). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
State in the release note that modelled GB Housing Benefit falls to about £8.0bn. Mark the mixed-age classification as an assumption, date the benefit-group equality from 2024-25, and quote the calibrated build's working-age claims. Add a test that the calibrated columns build on the dataset, since the loss matrix skips a target whose column raises, and note that would_claim_uc uses survey-year State Pension age. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…flag The create_frs smoke test's fake simulation did not answer the variables the would_claim_uc rule now reads (person_benunit_id, is_adult, is_SP_age), which failed CI with KeyError: 'person_benunit_id'. Pick the claimant-and-partner flag the way the locked policyengine-uk does for Housing Benefit's pension-age route: is_claimant_or_partner where the release defines it, otherwise is_adult as in 2.102.5. Both the would_claim_uc rule and the age-group targets use it. Under 2.102.5 the base FRS is identical to the previous commit's in every column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
pc-takeup-over-eligible
branch
from
October 1, 2026 14:09
ae43f1f to
41a07c9
Compare
This was referenced Oct 1, 2026
21 of 52 tasks
MaxGhenis
force-pushed
the
pc-takeup-over-eligible
branch
from
October 2, 2026 13:36
f09497f to
e386209
Compare
MaxGhenis
added a commit
that referenced
this pull request
Oct 3, 2026
…513) - TOTCAPB4 (DWP's below-average-resources measure since 2019/20) in place of TOTCAPB3, which stays as the fallback for earlier survey years. On #510 build D plus the column: the same 87.7k non-entitled reporters as with TOTCAPB3, GB entitled units 2,080.7k vs 2,083.5k, gate claims 1,290.0k vs 1,291.8k. The docstring now calls it an approximation (financial assets only; no Sch V disregard or reg 19 valuation). - uprating_factors.csv and uprating_growth_factors.csv rows recomputed from policyengine-uk 2.107.0's per-capita GDP index (2024 1.204, 2025 1.256), not copied from the stale savings row, so calibration and runtime grow the capital alike; the test now checks the rows against the model. - policyengine-core 3.32.13 in uv.lock (3.32.12 leaks memory per simulation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…accommodation Max ruled on d703 that working-age Housing Benefit should be calibrated too. policyengine-uk pays no Housing Benefit for specified (supported) or temporary accommodation, so the working-age awards it can pay are general-needs ones. DWP's working-age line includes both kinds, and DWP splits accommodation type only across all ages. The calibrated working-age target is therefore DWP's working-age line less all of its supported and temporary accommodation: £584.8m and 107k claims in 2025-26, a lower bound on working-age general-needs Housing Benefit. It exists for 2024-25 and 2025-26 only; from 2026-27 the all-age accommodation lines exceed the whole working-age line. Seeded calibrations of one saved 2025 input compared the options. Against the full working-age line, uncapped calibration loaded it onto about three effective records. Capping household weights at 20-40 times their prior reached 54-60% of DWP's spending and raised income-related ESA claimants to 1.6-2.0 times DWP's count. Against the net figure, the calibration meets it (1.04x) and no other national target crosses the 10% line compared with calibrating pension-age figures only. Tests: DWP's accommodation lines reconcile with its age split in every year; the net figure equals all-age general needs less pension-age Housing Benefit, the same quantity computed the other way; a property test of the netting; the calibrated working-age column equals the full line's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
pc-takeup-over-eligible
branch
from
October 4, 2026 18:39
e386209 to
9eaa2fb
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
pc-takeup-over-eligible
branch
from
October 4, 2026 18:40
9eaa2fb to
7c4972b
Compare
The loss matrix fills a year a target doesn't list with the latest earlier value within three years (_resolve_value). That carried the 2025-26 net working-age Housing Benefit targets (£584.8m, 107k claims) into 2026-27 to 2028-29, where DWP's lines give a negative net figure. Targets gain a carry_forward flag (default True, so nothing else changes), and the two net targets set it to False. The build calibrates 2025 only, so the dataset is unchanged; any loss matrix built for 2026-28 now leaves the net targets out. Also reword the docstring: policyengine-uk has no supported or temporary accommodation rules, but the FRS can't identify those residents, so a reported award there is modelled under the general-needs rules and treating the model's working-age HB as general needs is an approximation. The unmodelled supported and temporary accommodation HB (£5.2bn, all ages) is recorded as a limitation (d946). "Within 12%" becomes "about 12% below". Tests: the net targets resolve in 2024-25 and 2025-26 only while the other HB targets still carry forward; a property test of _resolve_value for any values, year and flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
pc-takeup-over-eligible
branch
from
October 5, 2026 16:29
7c4972b to
ba97fb1
Compare
…escribe the year resolver exactly The changelog and a test comment still said the model pays no supported or temporary accommodation Housing Benefit. It has no rules for it, and the FRS can't identify those residents, so treating its working-age HB as general needs is an approximation. The resolver docstring and the carry_forward comment now describe _resolve_value as it works: the nearest listed year, the earlier on a tie, used only if it is earlier and at most three years away. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The FRS build drew would_claim_pc against an unweighted count of every benefit unit, so entitled non-reporters claimed at about the take-up rate on top of every reporter. PolicyEngine/uk-triple-lock's coverage run (data/results.json, Enhanced FRS 2024-25 on policyengine-uk 2.90.2) put 2026-27 Pension Credit claims at 1.82m UK-wide, against DWP's 1.31m for Great Britain. After the imputations, compute Pension Credit entitlement and solve the claim probability that makes weighted take-up among entitled units equal DWP's FYE 2024 caseload take-up (62%). Reporters claim. Every non-reporter, entitled or not, is drawn at that probability, so units a reform makes entitled claim at the same rate. Replace the OBR Pension Credit target with DWP Spring 2026 Great Britain spending and caseload targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aseline The take-up step returns weighted aggregates for the calibration year: the solved probability, entitled units and reporters, and the Great Britain reporter baseline before any fill, which the build's summary prints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Pension Credit reports on SPI-synthetic households are imputed by the second-stage QRF (imputations/frs_only.py) from their SPI incomes, not observed claims. In build D, 154 such benefit units reported Pension Credit and were all anchored as claimants. They are now drawn at the solved probability like any non-reporter. Most SPI-synthetic households have zero survey weight when the probability is solved. The capital-gains band donors (imputations/capital_gains.py) are copies of sampled households re-weighted to HMRC taxpayer counts, so a copy of an SPI-synthetic household keeps the flag and has positive weight. The solved probability still does not move in build D', because none of its SPI-synthetic reporters is entitled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DWP's 62% caseload take-up covers Great Britain, but the solve counted Northern Ireland's entitled units and reporters too, so NI's reporter mix moved GB take-up away from 62%. Solve over GB entitled units and draw every non-reporter, NI included, at that probability. A property test checks that NI composition cannot change the GB probability or any GB flag. Correct the docstrings: the fill is zero when entitled GB reporters already make up the rate of entitled GB units, which is separate from calibration against DWP's absolute targets. OBR's Pension Credit line is the same GB series as DWP's, not identical figures. The summary now reports GB entitled units. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The built-dataset test asserted that every Pension Credit reporter claims, which no longer holds for SPI-synthetic units, whose reports are imputed and no longer anchor take-up. It now checks survey reporters. The module docstring and changelog now say the SPI reports are predicted by the second-stage imputation from SPI incomes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… take-up Until now every non-reporter claimed at the solved fill probability (about 13%), including units a reform newly entitles. That probability is a residual: the share of entitled non-reporters that tops FRS reporters up to DWP's 62%. Units with no entitlement in the calibration year are now drawn at DWP's caseload take-up for Savings Credit-only awards, 37% in FYE 2024 (table PC1; 42% in FYE 2023), the group with the smallest awards (£9 a week, against £87 and 69% take-up for Guarantee Credit). Reporters and the solved fill for entitled non-reporters are unchanged, so is the solved probability, and every entitled unit's flag. The rate is a new take-up parameter, pension_credit_newly_entitled. The generic assign_takeup_over_eligible helper, which applied one probability to every non-reporter, is removed; pension_credit_takeup_flags applies the two rates. Tests: reporters claim, entitled non-reporters draw at the solved probability and the rest at the new rate, and the new rate changes neither the probability nor any entitled unit's flag (property test); weighted take-up among entitled units still meets 62% and the rest claim at 37% over 200,000 units; Northern Ireland follows the same split; the parameter reads 37% for 2025 and 42% for 2022. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
pc-takeup-over-eligible
branch
from
October 5, 2026 18:15
ba97fb1 to
6565630
Compare
Contributor
Author
|
Hand-off to the UK hub (owner session e913bdc1, 2026-10-05).
|
MaxGhenis
added a commit
that referenced
this pull request
Oct 6, 2026
Conflicts resolved per runbook A4 step 10: datasets/frs.py keeps #514's assign_reported_takeup(...) calls for would_claim_pc and would_claim_uc, keeps #490/#510's comments, and ANDs #490's ~derive_all_claimants_over_state_pension_age(...) term onto the would_claim_uc draw (draw count unchanged); .gitignore takes the union of both sides; uv.lock reset to main's (b45c373), relocked in R. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833). |
MaxGhenis
added a commit
that referenced
this pull request
Oct 7, 2026
…ount (F1) test_full_target_set_available_offline (from #536) asserted at least 30 OBR targets offline. The release combines four member PRs that each replace OBR targets with better sources, which no single PR's CI saw together: #490 swaps obr/housing_benefit for DWP Housing Benefit targets, #510 swaps obr/pension_credit for DWP Pension Credit spend and caseload, #533 swaps the two OBR salary-sacrifice NI relief targets for HMRC's, and #530 merges the two OBR UC targets into obr/universal_credit. That leaves 29 OBR targets (34 on main), so release CI failed with 29 >= 30. The test now checks what it was for: offline, the committed workbooks give exactly the OBR targets that the same workbooks give when served, plus the receipts and NICs names. A broken fallback still fails it. Release-only fix for the 10/8 uk-data release (d833). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #490: this PR's branch is #490's branch plus six commits. It targets
mainso that CI runs, since this repo's CI only runs on PRs intomain. Until #490 merges, the diff here includes #490's commits; merge #490 first.Results (real builds, FY2025-26, Great Britain)
Production builds (
PE_UK_DATA_OA_CLONES=1, 512 epochs) with identical trained imputation models:Earlier build D, at 381e398, solved over UK units and anchored SPI-synthetic reporters. Its figures differed from D′ by under 1%.
d798 implemented (10/5). Max ruled 37%: units not entitled in the calibration year claim Pension Credit at DWP's Savings Credit-only caseload take-up, with reporters and the solved fill for entitled non-reporters unchanged. Commit "Draw units not entitled in the calibration year at 37% Pension Credit take-up" builds it. The rate is a new take-up parameter,
pension_credit_newly_entitled: 37% from FYE 2024 and 42% for FYE 2023, from DWP's table PC1.Build G2 is this PR with that rule, at ba97fb1 on #490 9f18a3d. The head is now 6565630, on #490 f4d8f95; #490's later commits change only docs and comments. Both sides are measured on G2's own weights, with the old flags rebuilt from the same seeded draws. Draws and flags align, with no violations across 61,212 benefit units.
Rebased on 10/4 onto #490's working-age change (#490 at 33092bf; this PR at 7c4972b). #490 now also calibrates working-age Housing Benefit, to DWP's working-age line less supported and temporary accommodation. Build G is this PR at 7c4972b, compared with F, which is #490 at 304baad:
The D′ tables below therefore describe this PR before d798, which changes only the draw for units not entitled in 2025 (see "d798 implemented" for G2). The commits were replayed unchanged, except that 2371630's message was reworded (now 334b327) for the round-3 review's note on capital-gains donors.
Acceptance gate: reporter baseline before any fill (PolicyEngine/uk-triple-lock#14)
The take-up step's aggregates for 2025, on survey weights, before calibration:
After calibration (dataset simulated for 2025)
would_claim_pcPaired reform: minimum guarantee +10% from 2025, same seeded flags in both runs
Under the rule in these builds, newly entitled non-reporters claimed at about the solved fill probability, which is low because reporters supply most of the 62%. d798 has since changed that to 37% (see "d798 implemented" above).
Take-up before and after calibration (d798)
The fill probability is solved on survey weights after the imputations, before cloning and calibration. It is the residual share of entitled non-reporters that tops FRS reporters up to DWP's 62%. Take-up among GB units entitled in 2025:
The before-calibration rows use the post-imputation dataset rebuilt on 10/4 from #490's code. D′'s own run solved 12.9% over 1,368k units.
Other outputs (2025)
E's pensioner-poverty change mixes the data change with policyengine-uk changes (State Pension age by date of birth, DWP rates, mixed-age couples), so these two builds can't separate them.
The targeted suite passes on D′'s dataset at e386209: 70 tests, including the take-up, HB target, registry, aggregate and reform-impact checks. The full suite passed on build G at 7c4972b: 657 passed, 2 skipped, 1 xfailed. The full suite passed on build G2 at 68e740c, which is 6565630 apart from #490's comment-only f4d8f95: 660 passed, 2 skipped, 1 xfailed.
Invariants and tests
draw < p, and units not entitled whendraw < 0.37. The newly entitled rate changes neither p nor any entitled unit's flag. Property test.pension_credit.would_claim_pc, and GB claims and spending land within 0.5-2x of DWP.axiom: n/a: data/calibration-only change
🤖 Generated with Claude Code