Repository navigation
Use the FRS benefit-unit capital for Pension Credit's capital test - #513
Merged
Merged
Conversation
TOTCAPB3 (DWP's derived benefit-unit total of savings and investments) for every FRS benefit unit; -1 where missing or negative. Pension Credit counts the claimant's and partner's capital (SPCA 2002 s.5); the WAS household imputation gives FRS Pension Credit reporters a weighted median of 74,300 pounds against the survey's own 300 for those the model then finds not entitled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
frs-pc-reported-capital
branch
from
October 1, 2026 19:46
c7fa5e1 to
0b45992
Compare
21 of 52 tasks
…K 2.107.0 - SPI-synthetic copies get pension_credit_reported_capital = -1: the FRS donor's capital does not belong with SPI-imputed incomes (24 of 152 SPI PC reporters were entitled in build E only because of it). - uprating_factors.csv and uprating_growth_factors.csv get the column with the savings row (policyengine-uk uprates it with the same per-capita GDP index), so uprate_dataset keeps the two in step. - policyengine-uk >= 2.107.0, the first release defining the variable (#2018); core 3.32.12 comes with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
October 2, 2026 14:13
…513) - TOTCAPB4 (DWP's below-average-resources measure since 2019/20) in place of TOTCAPB3, which stays as the fallback for earlier survey years. On #510 build D plus the column: the same 87.7k non-entitled reporters as with TOTCAPB3, GB entitled units 2,080.7k vs 2,083.5k, gate claims 1,290.0k vs 1,291.8k. The docstring now calls it an approximation (financial assets only; no Sch V disregard or reg 19 valuation). - uprating_factors.csv and uprating_growth_factors.csv rows recomputed from policyengine-uk 2.107.0's per-capita GDP index (2024 1.204, 2025 1.256), not copied from the stale savings row, so calibration and runtime grow the capital alike; the test now checks the rows against the model. - policyengine-core 3.32.13 in uv.lock (3.32.12 leaks memory per simulation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uk 2.107) policyengine-uk 2.107 (#1896) presumes a couple married when the dataset does not say, and its couple is the claimant and partner rather than any two adults: a member under 20 and 16+ years younger than the claimant is presumed to be their child. The family-structure test compared is_married with the adult count, which the lock bump to 2.107.0 made fail; it now compares with the claimant-and-partner count. family_type still counts is_adult, so that check is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With pandas nullable Float64 columns, a missing TOTCAPB4 (pd.NA) made the validity mask NA and wiped a valid TOTCAPB3. Convert each column to plain float64 first so the mask has no missing entries. The FRS reader uses plain NumPy dtypes, so this release's input is unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 5, 2026
Contributor
Author
|
Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on
main, independent of #490 and #510. It needs policyengine-uk 2.107.0, the first release definingpension_credit_reported_capital(#2018), and this PR bumps the floor and lock to it. Like every uk-data merge, it lands with the batched release on Max's go (d833); the every-unit vs reporters-only choice is d800.Why
About a third of FRS Pension Credit reporters get no modelled entitlement, and most of them fail on deemed income from imputed capital. Measured on #510's build D (2024-25 survey, FY2025-26, GB), original FRS households at survey weights, policyengine-uk main:
For those 415k, the household wealth imputation and the survey's own benefit-unit capital (
TOTCAPB3) disagree:TOTCAPB3The WAS imputation predicts household wealth from income, composition, tenure and region, and knows nothing about means-tested receipt. It also carries pension wealth in
corporate_wealth(#501 removes that).TOTCAPB3(used for this comparison) and its successorTOTCAPB4are DWP's derived benefit-unit totals of savings and investments; DWP's below-average-resources statistics useTOTCAPB4from 2019/20. In 2023-24 DWP also assigned Pension Credit to eligible FRS non-reporters using eligibility "sourced from survey data, combined with administrative data" (FRS background information and methodology, section 8.3).What changes
create_frswritespension_credit_reported_capital= FRSTOTCAPB4for every benefit unit. Earlier survey years fall back toTOTCAPB3, and -1 is written where both are missing or negative (none are in 2024-25). policyengine-uk (#2018) then uses it in place of the household proxy in Pension Credit assessable capital.TOTCAPB3withTOTCAPB4"from when it became available in 2019/20" (BAR FYE2023, changes since last publication).impute_income) get -1, so they fall back to the household proxy. Their incomes are SPI-imputed, and the FRS donor's capital does not belong with them. In build E's saved dataset there are 96 SPI-synthetic PC reporter units: 14 are entitled with the donor's capital and 3 with it cleared (calibrated weight 4.2k vs 2.1k). Capital-gains copies keep their FRS incomes, so they keep the value.uprating_factors.csvanduprating_growth_factors.csvget the column, with rows computed from policyengine-uk 2.107.0's per-capita GDP index (2024 1.204, 2025 1.256). The committedsavingsrow (1.190, 1.223) is stale against that model. Copying it would have grown the capital differently in calibration than at runtime, enough to move a unit across the £10,000 deemed-income disregard. A test checks the rows against the model.pyproject.tomlanduv.lock, with policyengine-core 3.32.13 (which cuts per-simulation memory overhead against 3.32.12).savingsand the other wealth columns, so wealth analyses still use the WAS imputation.TOTCAPB4andTOTCAPB3cover financial assets only. Second homes and land, which Pension Credit also counts, are not in them, and no Sch V disregard or reg 19 valuation is applied.Choice for review: every benefit unit (this PR) or reporters only.
Measured (real runs; build D plus this column, before any rebuild)
These use #510's own solver at 62% take-up, solved over GB entitled units as in #510 f09497f, original FRS households at survey weights, GB, FY2025-26. "All PRs" = policyengine-uk #2018, #2019, #1899, #1940 and #1925.
So with these changes calibration no longer has to inflate reporters to reach DWP's caseload. Build D took GB reporters from 1,199k (survey weights) to 1,736k.
Totals on build D's calibrated weights overshoot, because those weights already inflate reporters (GB Pension Credit is £6.24bn on main and £8.17bn with #2018 and every unit filled, without recalibration). On the same static basis (2025-26, calibrated weights, main against #2018 with every unit filled), other outputs that read the Pension Credit assessment also move:
Build E, below, gives the rebuilt figures.
Rebuild: build E vs D′ (real production builds, 1 OA clone, 512 epochs; GB, FY2025-26)
The #510 session ran both builds:
Pensioner poverty rises in E. Relative AHC goes from 17.5% to 18.4% (all-person AHC 23.6% to 23.7%). I split it by running both datasets under both policyengine-uk trees (2025, calibrated weights):
Interpretation (inferred; the 2×2 separates code from data-and-weights together, not calibration weights alone): the rise comes from calibration no longer inflating reporters. In D′ the +41% reporter upweight stood in for entitlement the model missed, and it crowded out entitled pensioners who don't claim. In E those people get their weight back. Pensioners in entitled, non-claiming benefit units rise from 0.70m to 1.29m, and those of them in AHC poverty from 0.16m to 0.32m. Take-up among the entitled falls to 60%, against DWP's 62%. The poverty figure moves away from HBAI, so it is called out here for review.
Max's questions on d800
~/reviews/uk-pc-reporter-gap-2026-10-01/d800/D800_ANSWER.md(local) answers them. The recommendation is unchanged: FRS financial capital for every benefit unit.At household level the WAS imputation barely tracks the FRS's own capital. The rank correlation between FRS
TOTCAPB4and WAS-imputed savings is 0.29 across households and 0.17 among means-tested reporters. 26% of households the FRS puts under £6k are imputed over £16k, yet national totals agree (£1.50trn vs £1.57trn).How the US builds combine sources:
Follow-ups:
Remaining gap (all PRs, every unit filled; survey weights, 88k)
Most of this is survey reporting that the model should not chase.
Tests
tests/test_pension_credit_reported_capital.py:TOTCAPB4is carried over unchanged. A validTOTCAPB3is the fallback whereTOTCAPB4is absent or invalid, including with nullable (pd.NA) columns.create_policyengine_uprating_factors_tablederives from the locked policyengine-uk, for 2020-2034.tests/test_policybench_transfer.py: the family-structure test now comparesis_marriedwith the claimant-and-partner count, because policyengine-uk 2.107 (#1896) defines a couple that way. Itsfamily_typecheck is unchanged.Before merge
policyengine-uk#2018 released; bump the lock: done (2.107.0).Integration build E: done; see the section above.axiom: n/a: data change
🤖 Generated with Claude Code