Calibrate GB universal credit to one OBR total; reach the UC payment top band

[MaxGhenis]

Summary

Two calibration-target fixes, both data-release changes for the d833 batch. Draft; do not merge outside the batch.

1. OBR universal credit. obr/universal_credit_outside_cap asked for £12.9bn of nearly all UC. Its estimate was +460% in the seeded main build and +455% to +460% in the ten seeded IID builds of Weight FRS BRMA draws by census private-rented households #516 from 2-3 October. obr/universal_credit_in_cap compared £66.4bn with all UK UC (+12%). They are replaced by one target, obr/universal_credit. It is the sum of EFO table 4.9's two UC rows (£79.29bn in 2025-26), counted over Great Britain only.
2. DWP UC payment distribution. The four dwp/uc_payment_dist/*_nan_to_nan targets (1.4k to 83k households) had a column of zeros. They are now reachable as *_30_000_to_inf, and the bands meet without gaps.

1. Universal credit inside and outside the welfare cap

What OBR's split is. The welfare cap is the Charter for Budget Responsibility's limit on welfare spending, not the household benefit cap. It excludes the State Pension and the payments most sensitive to the economic cycle: JSA, the Housing Benefit paid with it, and their UC equivalent (NAO, Managing the welfare cap, 2016). The sources agree on what that UC equivalent is:

• DWP's benefit expenditure and caseload tables (Spring Forecast 2026) give UC outside the cap as £12,875.55m in 2025-26, the figure in table 4.9 (£12.876bn), with a caseload of 1,323k. Their Notes ("Unemployment benefits table") say this UC "is the total expenditure outside the welfare cap directed to those in the Intensive Work Search group, the closest UC equivalent to JSA spend".
• OBR EFO March 2026, paragraph 4.24, footnote 20: "The unemployed caseload here refers to the universal credit caseload in the 'intensive work search' labour market regime."
• DWP's UC statistics methodology defines Stat-Xplore's "Searching for work" regime as Intensive Work Search. That is the all-work-related-requirements group (Welfare Reform Act 2012 s.22) with earnings below the Administrative Earnings Threshold, plus gainfully self-employed claimants in a start-up period. The threshold is set in UC Regulations 2013 reg 99(6): monthly earnings from employment of 18 hours a week at the National Living Wage for one person, and 29 combined hours for a couple, since May 2024 (SI 2024/529 and 2024/536).

The old column was UC for households whose benefit_cap_reduction is zero, which is a different concept.

Why one total rather than a corrected split. policyengine-uk 2.93.0, which this repo pins, has no UC conditionality regime. Its LCW/LCWRA input, uc_limited_capability_for_WRA, is is_disabled_for_benefits. This repo sets that from reported AA, DLA, PIP, SDA, incapacity benefit, IIDB, AFCS or ESA receipt, not from a work capability assessment. I built proxies of the Intensive Work Search group from the legal tests and FRS employment status, and scored them on seeded builds against DWP's GB figures (£12.876bn, 1,323k):

• Legal tests: below State Pension age, no LCW, not a carer, not the responsible carer of a child under 3, employment income below the AET (individual and couple), not under the minimum income floor.
• Health stand-in: FRS self-reported long- or short-term sickness is used in place of LCW.
• Unverified assumption: a benefit unit with a health claimant is counted inside the cap, so a health claimant takes the whole unit's UC with them.

Proxy (2025-26, GB benefit units on UC)	Build	Outside-cap UC	vs DWP	Caseload	vs DWP
Legal tests only	main	£25.40bn	+97%	2,479k	+87%
Legal tests + self-reported health	main	£12.25bn	−5%	1,309k	−1%
Same	#526	£16.30bn	+27%	1,665k	+26%
ILO-unemployed only	main	£3.78bn	−71%	372k	−72%

The close fit on main is partly luck. Main maps FRS EMPSTATI 11 ("other inactive") to LONG_TERM_DISABLED; #526 corrects that. With the correction, about 356k "other inactive" UC benefit units leave the health group and the proxy is 27% over. Other reasonable choices, such as how to treat students or couples where only one partner is in the group, move it between −26% and +42%.

Calibrating to a split this fragile would shift weight between kinds of UC household in a way no output reveals. The total, by contrast, is exactly OBR's figure. A chip tracks restoring the split once policyengine-uk models the work-related groups (#2081's uc_work_related_group_apart_from_earnings) and this repo supplies their inputs (#525, #526, #527).

Great Britain only. Both UC rows sit under table 4.9's "DWP social security", which DWP's tables label "DWP Social Security (GB)". Northern Ireland's UC is in the separate "NI social security" rows. The target therefore counts GB households only. In the seeded main build that excludes £1.65bn of the model's £74.4bn UC in 2025-26. This uses #490's countries field and restrict_to_countries, copied byte for byte, so the two PRs merge cleanly in either order (the review checked both). Other DWP targets have the same GB-versus-UK mismatch; a chip covers them.

Parsing. The two rows are now located by section across the whole sheet: one "Universal credit" row either side of "Welfare spending outside the welfare cap". The code no longer takes the first and second matches. No UC target is produced, rather than a partial total, in any of these cases:

• either row is missing;
• a section has two "Universal credit" rows;
• the two rows cover different years.

2. UC payment distribution

The Stat-Xplore extract (storage/uc_national_payment_dist.xlsx, May 2025) ends with an open band, "£2500.01 or over". parse_band only understood "X to Y", so that band got NaN bounds and uc >= nan matched nothing.

• Open top band. "or over" now parses to (£30,000, ∞) a year.
• Invalid bands rejected. An unrecognised band, a NaN, infinite or negative lower bound, or an inverted band now raises.
• No double counting. The parser rejects overlapping bands. Stat-Xplore's "£1500.01 or over" was the top band for months up to August 2022 and spans the finer bands added since; it is suppressed in the extract, and would double count if a new extract filled it in.
• No gaps. Bands are now (lower, upper] in whole pence, with lower set to the previous band's top.

Why the gaps mattered. The old test was uc >= min × 12 & uc < max × 12. It dropped awards of exactly £100.00, £200.00, … a month, and anything in the 12p gaps between bands. It also dropped most penny awards. policyengine-uk leaves 1p a month payable after deductions (SI 2013/380 Sch 6), and 44 records (136k benefit units in the seeded main build) sit there. Subtracting the capped deduction in float32 leaves about £0.1199951 a year rather than £0.12, so the old £0.12 lower bound excluded 40 of them (134k benefit units) and kept 4. Stat-Xplore's "Monthly Award Amount (payment bands)" metadata says the award is "the total amount of Universal Credit due ... after any deductions", which is what universal_credit is. So a penny award belongs in "£0.01 to £100.00".

Scored against the corrected bands, the main build is too high in that bottom band: singles +71%, lone parents +61%, couples with children +273%. This PR's build fits them (see Impact). A middle commit briefly reverted to the old bands on the wrong premise that Stat-Xplore's awards are before deductions; the third commit restores the gapless bands. The target definitions at the head are identical to the first commit's (all 116 affected targets compared).

Target names are unchanged apart from the four top bands. The top bands rest on few records (2025): 5 singles, 34 lone parents and 32 couples with children have UC above £30,000 a year. No couple without children does, so that target (1.4k households) stays at zero whatever the weights.

Invariants and tests

• Conservation (UC total). The UC column sums UC over benefit units in GB households, each once; Northern Ireland households are zero; the column is non-negative. Hypothesis property test, plus a test through create_target_matrix itself with a fake microsimulation.
• Band membership (payment bands). Each target's column counts exactly the benefit units whose award an independent interval search places in that band, so no award is counted twice, moved to a neighbouring band, or lost. Positive awards are always in some band, and zero awards never are. This is checked for float64 and float32 awards, on edges and one ulp either side. Hypothesis property test.
• Band layout. Within each family type, the bands run from 0 to ∞ and each starts where the previous one ends. The band counts add up to the extract's Total less "No payment" (within 10 households, since Stat-Xplore perturbs cells for disclosure control).
• Parsing (UC total). The UC target equals row 18 plus row 43 of the committed workbook in every year from 2024 to 2030. Rows found by section anywhere in the sheet. Missing, duplicate or year-mismatched rows give no target.
• Mutation checks. Each of these fails at least one test:
  • removing the country filter from create_target_matrix;
  • dropping countries=GREAT_BRITAIN;
  • band tests of >= lower & < upper (even keeping zero awards out);
  • the top band parsing to NaN again.

Impact (seeded production builds)

Origin/main b45c373 and this PR were built with the same seeded_build.py, seed 0, and identical cached downloads (no network fetches). Main's two repeat builds were output-identical. Settings: 512 epochs, with PE_UK_DATA_OA_CLONES=1 (a production build defaults to 10 OA clones), the setting the other batch PRs' seeded builds use.

Builds compared:

• This PR: commit aa60452. The head (974fa8c) has the same 116 affected targets; its own seed-0 build is identical in content (every column of the person, benefit unit and household tables, weights included; the HDF5 file hashes differ, as main's two repeat builds' did).
• Main: origin/main b45c373.

Both are scored on this PR's target definitions, at 2025, with the build's own model (policyengine-uk 2.93.0).

Fit to the national targets

Target	Target	main	this PR
obr/universal_credit (GB)	£79.29bn	£72.80bn (−8.2%)	£76.74bn (−3.2%)
…SINGLE_annual_payment_30_000_to_inf	3.52k	0.90k (−74%)	3.55k (+1%)
…LONE_PARENT_annual_payment_30_000_to_inf	82.91k	23.56k (−72%)	86.87k (+5%)
…COUPLE_NO_CHILDREN_annual_payment_30_000_to_inf	1.42k	0	0 (no record can reach it)
…COUPLE_WITH_CHILDREN_annual_payment_30_000_to_inf	61.73k	20.63k (−67%)	64.46k (+4%)
…_annual_payment_0_to_1_200: singles / lone parents / couples with children		+71% / +61% / +273%	+4% / +4% / +2%
Other 631 national targets: mean loss term		0.0454	0.0440
Other 631 national targets: share within 10%		84.5%	84.9%
Other 110 UC targets (dwp/uc/, uc_payment_dist): mean loss term		0.0280	0.0203
Other 110 UC targets: share within 10%		92.7%	95.5%

Among the remaining 627 targets, three move most:

• ons/multi_family_households: +1,071% → +1,063%. The model misses this one tenfold either way.
• UC couples with children: +14.9% → +10.5%.
• hmrc/cgt_taxpayers: +52.5% → +49.2%.

Every other target's error moves by 2.1 points or less.

Microsimulation (policyengine-uk main 84f5ad43a, the same datasets)

Aggregate	2025 main	2025 this PR	2026 main	2026 this PR
UC, UK	£75.87bn	£79.80bn (+£3.93bn)	£80.00bn	£83.91bn (+£3.91bn)
UC, GB	£74.14bn	£77.98bn (+£3.84bn)	£78.24bn	£82.07bn (+£3.83bn)
Benefit units on UC, GB	6.32m	6.32m	6.30m	6.32m
Benefit units with UC over £30,000 a year, GB	55k	161k	155k	249k
Benefit-capped households	139k	137k	277k	271k
Household benefits, UK	£546.97bn	£551.33bn (+£4.36bn)	£564.28bn	£568.64bn (+£4.36bn)
Income tax, UK	£295.22bn	£294.95bn (−£0.26bn)	£315.42bn	£315.14bn (−£0.28bn)
Household net income, UK	£1,709.19bn	£1,712.70bn (+£3.51bn)	£1,761.87bn	£1,765.36bn (+£3.49bn)
People in poverty, BHC / AHC	10.81m / 14.11m	10.60m / 13.91m	10.33m / 13.14m	10.07m / 12.96m
Children in poverty, BHC / AHC	2.90m / 4.21m	2.80m / 4.11m	2.62m / 3.66m	2.50m / 3.57m

Housing Benefit, Pension Credit and Child Benefit move by £0.03bn or less, and households and people by 0.01m or less.

What drives this. The calibration no longer pulls UC towards the old outside-the-cap target, and it now reaches the top payment band. Weight therefore moves to higher-award UC households rather than to more claimants: the number of GB benefit units on UC is broadly stable (−1.5k in 2025, +20k in 2026). That raises UC by about £3.9bn and lowers poverty by about 0.2m people (0.1m children).

Band edges. A build of the middle commit (4a1b1a6) has the OBR fix and the open top band but the old band edges. It gives UC +£3.62bn and AHC poverty −0.18m in 2025, so the OBR fix and the open top band together account for most of the change. Restoring gapless edges adds about £0.31bn of UC. No build separates the OBR fix from the open top band.

Scripts, logs and per-target tables are in ~/reviews/uc-welfare-cap-2026-10-02 (fit.py, metrics.py, compare.py, impact/).

Batch notes

• The Hypothesis dev-dependency hunks (pyproject and uv.lock) are copied from Map FRS EMPSTATI 11 to OTHER_INACTIVE, not LONG_TERM_DISABLED #526. Stop SPI-synthetic rows carrying benefit claims nobody observed #514, Supply is_claimant_or_partner from the FRS adult table #524 and Draw SPI incomes within earnings groups set by FRS employment status #529 declare hypothesis>=6.168.3 instead of hypothesis; every one of them locks Hypothesis 6.168.3.
• The countries mechanism is Calibrate Housing Benefit to DWP's GB figures by age group #490's, byte for byte.
• Interacts with Map FRS EMPSTATI 11 to OTHER_INACTIVE, not LONG_TERM_DISABLED #526: it changes FRS employment status, which this PR does not read.
• Independent review: GPT-6.1 Sol approved the first commit (aa60452). Its six findings are addressed in 974fa8c. A delta review by GPT-6.1 Sol approved 974fa8c: 20 tests pass, 11 mutations are caught, and the 116 target definitions and the two builds' contents are identical. Its five wording findings are corrected in this description.
• axiom: n/a: calibration targets in the data repo, no policy rule.

🤖 Generated with Claude Code

[MaxGhenis]

Hand-off to the UK hub (session "OBR UC welfare cap targets")

• Head: 974fa8c. Out of draft, MERGEABLE, all 4 checks pass.
• Independent review (GPT-6.1 Sol):
  • Round 1 approved aa60452 with six minor findings. All are fixed in 974fa8c.
  • The delta review approved 974fa8c. 20 tests pass and 11 mutations are caught. The 116 target definitions match aa60452, and the 974fa8c build is identical in content to the aa60452 build behind the impact numbers.
  • Its five findings were all about the description and are corrected there. No code change was needed.
• Pending: nothing on this PR. uk-data lands only in the d833 batch on Max's go, so do not merge it on its own.
• Dependencies: none. The countries hunks match Calibrate Housing Benefit to DWP's GB figures by age group #490 byte for byte, and the Hypothesis hunks match Map FRS EMPSTATI 11 to OTHER_INACTIVE, not LONG_TERM_DISABLED #526; both merge cleanly in either order.
• Follow-ups (chips filed):
  • restrict the other GB-only DWP targets to Great Britain;
  • restore the inside/outside welfare-cap split once policyengine-uk #2081 is in this repo's pin.

[MaxGhenis]

Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833).