UC caseload runs ~40% below administrative counts, capping aggregate reform estimates #452

Activity

1. vahid-ahmadi

   added a commit that references this issue on Aug 12, 2026

   Add Universal Credit deductions with Fair Repayment Rate cap (#1815)

   ...

   c9af1d9
2. vahid-ahmadi commented on Aug 13, 2026

   

   vahid-ahmadi

   on Aug 13, 2026

   CollaboratorAuthor

   More actions

   Root cause found: take-up is applied to ALL benefit units, not to entitled ones

   This is a hard ceiling that no amount of reweighting can lift, and it explains the caseload gap directly.

   datasets/frs.py:1417-1421 assigns UC claim status via assign_takeup_with_reported_anchors, with parameters/take_up/universal_credit.yaml = 0.55. Inside that helper (utils/takeup.py:52):

   target_count = int(rate * len(draws))

   len(draws) is every benefit unit, not the entitled ones. The module's own docstring (takeup.py:5-8) states the opposite intent:

   the remaining random fill should hit the target aggregate takeup rate across the non-reporting eligibles

   Implementation and documentation disagree. With ~55% of all benefit units flagged and FRS reporters forced in, effective take-up among entitled non-reporters lands near 45-47%, against DWP's published UC take-up of ~85% by caseload.

   Because universal_credit_pre_benefit_cap carries defined_for = "would_claim_uc", the set of possible UC claimants is fixed before calibration runs. Reweighting cannot create claimants — it can only inflate the weight of those that exist, by 44%, which immediately breaks the population, age, region and household-type targets.

   The targets are fine — it is the microdata that cannot reach them

   Worth correcting my framing in this issue: I wrote that UC caseload should be added to the calibration targets. It already is, consistently, from a current DWP vintage:

   • targets/sources/dwp.py:165-183, four family types → 6,096,900 households
   • targets/sources/dwp.py:144-162, by number of children → 2,998,111, reconciling exactly with the with-children family types
   • 104 dwp/uc_payment_dist/* targets → 6,095,654
   • constituency targets (utils/uc_data.py, May 2025 provisional) → 6,095,661; LA targets → 6,095,645

   Both halves of the loss agree on 6.096m. I should also correct the administrative figure I used: this issue quotes "~7.2m administrative households", but the repo's own targets say 6.096m, and that is the number the model is actually being asked to hit. The model's 4.22m against 6.096m is the sharper statement of the problem, and it changes any caseload scaling factor from 1.72x to 1.44x.

   Suggested fix

   1. Pass an eligibility mask to assign_takeup_with_reported_anchors and interpret rate as take-up among the entitled, matching the docstring.
   2. Set the parameter from DWP's published UC take-up (~0.85). The 0.55 value dates from a 2015 entry and predates the reported-anchor rework.
   3. Add dwp/uc/total_claimant_households = 6,096,900 as an explicit national target — 6.096m currently exists only as a sum of splits.
   4. Add obr/universal_credit_average_award (£13,006 = £79.29bn / 6.0969m) so caseload and spend cannot compensate for one another.

   Knock-on: this likely drives the benefit cap overshoot too

   The OBR UC spend targets (£79.29bn) are in the loss and easy to satisfy by inflating awards. Delivering that from 4.22m benefit units needs an average award of £18.8k against a target-implied £13.0k — a factor of 1.44, exactly 6.096/4.22. Benefit cap incidence is convex in the award, so a uniform 1.44x inflation pushes a disproportionate share over the threshold and inflates the excess super-proportionately. That predicts the observed pattern in PolicyEngine/policyengine-uk#1818 precisely: count 2.0x, reduction 3.1x.

   This is an inference, not a measurement. It is settled in one step by reading obr/universal_credit_in_cap estimate-vs-target from constituency_calibration_log.csv, which CI already uploads as an artifact on every push (.github/workflows/push.yaml:63-73) — no rebuild required.
3. MaxGhenis commented on Aug 17, 2026

   

   MaxGhenis

   on Aug 17, 2026

   Contributor

   More actions

   Diagnosis complete — the answer to the issue's three-way question ("take-up modelling, calibration targets, or FRS under-reporting") is: mostly a release whose weights never bound the caseload targets its own source registers, plus a smaller target-level layer. Not FRS under-reporting: the sample supports 6.4m UC benefit units under calibrated weights.

   Measurements

   Weighted benefit units with universal_credit > 0 by model year (policyengine-uk 2.90, annual basis), across cached and current releases of enhanced_frs_2023_24.h5:

   build	2024	2025	2026	UC spend 2025
   v1.40.x era (e00e9c87, what #1815's validation and the current policyengine.py bundle pin)	4.18m	4.19m	4.22m	£51.3bn
   v1.56.6 (19 June)	6.38m	6.40m	6.43m	£77.6bn
   v1.56.14 (latest, 26 July)	6.38m	6.40m	6.43m	£77.6bn
   administrative / OBR	—	~7.2m households (Stat-Xplore, point in time)	—	£75.8bn (OBR)

   Deductions aggregates re-measured on v1.56.14 at 2025: 2.95m deducting benefit units (DWP 3.3m, −11%) and £1.84bn deducted per year (DWP ~£2.0bn, −8%), incidence 46.1% (DWP 46–47%). The "~40% low" caveat in #1815's validation docs and the working paper is a property of the stale pin, not the current data.

   Mechanism, in three layers

   1. The v1.40.x build did not bind the caseload targets its source registers (~30 of the 40 points). At tag 1.40.3 the registry already emits the DWP family-type claimant count targets at 2025 (1.40.3:policyengine_uk_data/targets/sources/dwp.py:102-116) and the UC payment-distribution targets (:231-241), yet the shipped weights measure 4.2m and £51bn — far below both those targets (~6.1m) and the OBR spend target. By v1.56.6 the same anchors bind (6.40m, £77.6bn). Consumers that pin ≥1.56.6 recover the difference with no code change.

   2. The targets themselves top out ~11% below Stat-Xplore (the residual layer). The family-type counts sum to 6.096m, and the undercount adjustment intended to lift them is a no-op as written:

   undercount_relative = 1.27921 / sum(_UC_BY_FAMILY_TYPE.values())  # = 1.27921/6095.637 → ×1.0002

   (main:policyengine_uk_data/targets/sources/dwp.py:171; identical at 1.40.3:...dwp.py:108.) If the numerator is read as millions against a sum in thousands, the intended factor is ×1.2099, landing on 7.375m — right at the administrative count. As written, a perfect calibration to current targets cannot exceed ~6.1m plus slack. One-line fix, but it forces an explicit basis decision (next point).

   3. The OBR UC count targets are dead code. main:policyengine_uk_data/targets/build_loss_matrix.py:396-403 has a compute branch for obr/universal_credit_jobseekers_count / non_jobseekers_count, but no source module emits them at main (obr.py emits UC spend rows only). A November 2025 calibration log shows those targets binding then (~6.32m achieved against 1.6m + 4.7m) — they were dropped somewhere in the registry rework, and by 1.40.3 were gone.

   Definitional notes for the fix

   • The model measure here is annual any-receipt, which should exceed DWP's point-in-time monthly counts — so the gaps above are conservatively stated.
   • The OBR count basis (6.3m, forecast-year average) and Stat-Xplore (~7.2m households, point in time) are different truths. Fixing the undercount expression means choosing which one the count targets encode, explicitly.

   Suggested actions

   1. Re-pin consumers to ≥1.56.6 (the policyengine.py UK bundle currently certifies the v1.40.x-era artifact; the deductions validation docs and working paper caveat can then be updated from measurements, per the table above).
   2. Fix the undercount_relative expression and state the intended count basis in the target descriptions.
   3. Delete the dead OBR count compute branch, or re-emit those targets from obr.py if the OBR basis is wanted alongside the DWP one.

   Worktree with the code trail: analysis ran against origin/main (ebf733c) and tag 1.40.3; measurement scripts and outputs are local (aggregates only, per data-protection rules).
4. MaxGhenis commented on Aug 17, 2026

   

   MaxGhenis

   on Aug 17, 2026

   Contributor

   More actions

   Two corrections and one extension to the diagnosis above, after checking the consumer paths:

   1. policyengine-uk has no default dataset to re-pin — Microsimulation() requires an explicit dataset or POLICYENGINE_UK_DEFAULT_DATASET. So "re-pin consumers" means: anyone passing enhanced_frs_2023_24.h5 explicitly should use ≥1.56.6 (the #1815 validation docs and the working paper will be updated on that basis), and the certified policyengine.py path is a separate story (next point).

   2. The certified policyengine.py UK default is Microcosm's populace_uk_2023, and it is worse on UC than any enhanced FRS build: 3.26m benefit units / £36.9bn at 2025 on the current certification candidate (dd68c73), 2.17m / £24.0bn on the prior certified default (72aeefc) — its calibration surface does not bind UC at all. Filed with full measurements as Certified UK artifact carries less than half of admin UC caseload (3.26m vs ~6.7m; £36.9bn vs £75.8bn) microcosm#701.

   3. The targets fix for this repo (inert undercount_relative removed, explicit dwp/uc/households totals derived from the deductions statistics Table 1 — 2025 ≈ 6.7m, 2026 ≈ 7.2m — and the dead OBR count branch removed) is in flight as a PR referencing this issue.
5. MaxGhenis

   added a commit that references this issue on Aug 17, 2026

   Fix Universal Credit caseload targets (#458)

   ...

   cbd5ae1
6. MaxGhenis commented on Aug 20, 2026

   

   MaxGhenis

   on Aug 20, 2026

   Contributor

   More actions

   Post-merge measurement of #458's effect, with one correction and one residual:

   Correction first: enhanced_frs_2023_24.h5 is frozen (byte-identical across releases since June — it's the immutable baseline microcosm pins as its parity reference). The live build line is enhanced_frs_2024_25.h5, which CI genuinely rebuilds and recalibrates each release (512 epochs, verified in the 1.56.16 run log). Consumers still pinning 2023-24 will never see target fixes.

   Before/after on the live 2024-25 artifact (weighted benefit units, universal_credit > 0, 2025):

   	v1.56.14 (pre)	v1.56.16 (post)	admin/OBR
   UC benefit units	6.33m	6.35m	~6.7m (2025 avg)
   UC spend	£74.7bn	£75.3bn	£75.8bn
   deducting benefit units	2.70m	3.02m	~3.1m (mid-2025)
   deducted per year	£1.68bn	£1.86bn	~£2.0bn
   deduction incidence	42.7%	47.6%	46–47%

   Spend is essentially on target and the deductions aggregates landed within ~5–8% of DWP. The caseload residual (6.35m vs 6.7m) has a specific cause: the four family-type claimant targets still carry their stale published values summing 6.096m, so the optimiser compromises between 6.096m and the new 6.7m total. The follow-up flagged in #458 closes it: refresh _UC_BY_FAMILY_TYPE to a current Stat-Xplore month (values should sum ≈6.9–7.2m), at which point the two families pull together instead of against each other. The 2026 value (7.2m) only matters if a 2026-year calibration run exists; the artifact's weights look 2025-calibrated with uprated out-years.
7. MaxGhenis commented on Aug 23, 2026

   

   MaxGhenis

   on Aug 23, 2026

   Contributor

   More actions

   Re-measured on the live line after the microcosm retarget (microcosm#733 pins enhanced_frs_2024_25.h5 @ v1.56.14 as its parity reference; microcosm#735 compiles the DWP caseload as a calibration target). The aggregate gap is now small; the composition is wrong in both directions, and that is the defect that survives. Mechanisms are read from code at pinned commits; numbers are aggregates from local runs (policyengine-uk 2.91.0 @ c93e1a05, year 2025). Aggregate measurement scripts and outputs (no microdata): gist.

   Where the gap stands

   Benefit units with universal_credit > 0 at 2025, Great Britain only (the DWP facts are GB), against Apr–Dec 2025 means of the monthly facts chronicle now carries (chronicle#184 for the caseload, chronicle#188 for the elements and splits; childcare is the published Jan–Aug 2025 series):

   series	DWP avg	v1.56.14	gap	v1.56.16	gap
   households (deductions-derived; the microcosm#735 target = 6,758,889)	6.76m	6.18m	−8.6%	6.20m	−8.3%
   housing entitlement	4.30m	4.86m	+12.8%	4.87m	+13.1%
   LCWRA entitlement	2.42m	2.09m	−13.6%	2.10m	−13.0%
   carer entitlement	1.13m	0.56m	−50.5%	0.64m	−43.5%
   childcare element (Jan–Aug 2025)	0.18m	0.45m	+147%	0.47m	+156%
   single, no children	3.45m	2.99m	−13.1%	2.97m	−13.9%
   single, with children	2.23m	2.15m	−3.5%	2.13m	−4.1%
   couple, no children	0.28m	0.22m	−22.5%	0.23m	−20.9%
   couple, with children	0.90m	0.82m	−9.1%	0.87m	−3.2%
   4 children	0.17m	0.13m	−24%	0.16m	−11%

   Two basis notes. The model measure is an annual award, the admin series are monthly point-in-time counts; I have not quantified the difference. The UK artifact carries 0.15m NI benefit units on UC, so the #735 target (GB facts) will bind a UK column; on a UK basis the v1.56.14 total is 6.33m. Element counts are element > 0 among benefit units on UC; the childcare comparator is "households receiving the element", the other three are Stat-Xplore entitlement counts.

   Mechanisms

   1. Take-up is drawn for 55% of all benefit units. utils/takeup.py:50-59 sets target_count = int(rate * len(draws)) over every benefit unit, anchors FRS reporters, and fills the rest; the UC call at datasets/frs.py:1422-1426 passes the rate from take_up/universal_credit.yaml (0.55, dated 2015) and no eligibility mask. The draw mechanism is unchanged since the 13 August comment (#458 touched targets only; #462's benunit sort, the artifact note below, changed where stored flags land, not the rate, denominator or anchoring). On the v1.56.14 artifact, would_claim_uc is True for 33,709 of 61,223 benefit units (55.06%; int(0.55 × 61,223) = 33,672). microcosm's contract mirrors it: take_up_contract.json:63-79 freezes UC at 0.55 "for parity with the incumbent" (decision U8, follow-up = this issue) with an "unweighted full-population target denominator" (take_up_contract.json:5, stochastic_assignment.py:64-83).

   2. The capital test counts imputed pension wealth. is_uc_eligible requires uc_assessable_capital ≤ limit (is_uc_eligible.py:11-15; limit £16,000, capital/limit.yaml); the countable sources are savings, other residential property, non-residential property and corporate_wealth (capital/sources.yaml:34-39; the same file's description notes that other Schedule 10 disregards within these asset classes "are not yet split out separately"). Both builds construct corporate_wealth as WAS total pensions minus DB pensions plus shares, ISAs and trusts (uk-data imputations/wealth.py:145-155; microcosm was_wealth.py:255-264), and the variable's own docstring says it includes private pensions. Schedule 10 para 10 disregards the value of pension rights. Effect on v1.56.14: 6,447 benefit-unit records carry universal_credit_reported > 0 (4,437 source-FRS responses and 2,010 SPI-synthetic clones whose reported fields a second-stage QRF wrote; 5.70m weighted UK-wide, 5.54m on the GB subset = 82% of the admin average); 2,983 of them (46%) carry assessable capital above £16k, and 96% of those exceed it on corporate_wealth alone (mean imputed corporate wealth among over-limit records £250k, unweighted). uc_reported_capital, the override the model provides, is unset for every record. Net: the model keeps 3.91m of the 5.70m attached to positive universal_credit_reported, drops 1.80m (0.88m on the take-up draw, 0.80m on capital, 0.12m on the taper), and back-fills 2.42m from records with zero universal_credit_reported. Those zero-field records carry 38% of the modelled caseload; 31% of the positive-field weight is excluded.

   3. Element incidence is formula-driven and none of the four series above has a direct target. Childcare pays the lesser of the cap and coverage_rate × childcare_expenses when at least one adult is in work, every adult is in work, and at least one child is eligible (uc_childcare_element.py:10-17, uc_childcare_work_condition.py:11-18, uc_maximum_childcare_element_amount.py:10-15), and lands at 2.5× the households DWP records as receiving it. The carer element is positive when the benefit unit contains someone with positive modelled Carer's Allowance or CSP, and either award follows the care-hours condition or reported CA (uc_carer_element.py:4-14 → benunit_has_carer.py:10-11 → num_carers.py:10-11 → is_carer_for_benefits.py:10-11 → input/care.py:28-41 → carers_allowance.py:11-21, carer_support_payment.py:19-28); it lands at half the admin count. LCWRA proxies the dataset's is_disabled_for_benefits flag (uc_limited_capability_for_WRA.py:11-12), which uk-data sets from the sum of eleven reported disability-benefit amounts (datasets/disability_benefits.py:36-48, :185-187). Family, child-count and spend targets move these series only indirectly through shared weights. The chronicle#188 element facts are not in the feed microcosm#735 pinned (regenerated before #188 merged; its family and child references resolve to the May-2025 package values #188 replaced).

   4. Weights carry the rest. 3,615 records carry the 6.33m caseload; on-UC records average three times the mean weight and the top 10% of them hold 66% of the caseload (v1.56.14; 61% on v1.56.16).

   One artifact note. #462 (merged 18 August) found the 2024-25 raw benunit table unsorted, so stored benunit-level flags were misaligned against the model's benefit-unit order before the fix. v1.56.14 predates it: only 55% of records with positive universal_credit_reported carry would_claim_uc there, 88% on v1.56.16 (the remaining 12% is consistent with SPI-synthetic rows whose reported values were written after the draw). microcosm#733's parity reference is the pre-fix artifact.

   What microcosm#735 changes

   It compiles the deductions-derived caseload as the calendar-year average of nine monthly facts (the microcosm#731 ruling): 6,758,889, +0.9% vs the incumbent's 6.7m. All 30 dwp.uc.* ids resolve alongside the two OBR UC spend ids — households, four family types, five child counts, four payment-band ids (100 rows), Scotland child-under-1, 15 two-child-limit rows, 126 rows in all (microcosm#738 described the pre-#735 state of one active). No calibration run is recorded on it yet (no committed artifact; microcosm#623 open; PR #729: "No dataset was built"), so the table above is the current state. What a run can and cannot do follows from the mechanisms: weights cannot create records that the draw in (1) or the capital test in (2) has excluded, the four element series in (3) have no direct target, and the nine family/child references each resolve to one 2025-05 fact (the May-2025 packages chronicle#188 has since replaced); the four family types sum to 6.58m against the 6.76m average (microcosm#736 item 9). The composition after a run is something to measure, not assume.

   Suggested next steps

   1. Split pension rights out of countable UC capital — a separate build column for private pension wealth and a corporate_wealth without it — keeping shares, ISAs and trusts countable. A record-level counterfactual on v1.56.16 (incumbent weights unchanged) bounds the effect: zeroing corporate_wealth in the capital test takes the GB caseload from 6.20m to 6.69m (−8% → −1% vs the admin average), brings 1,764 records back on to UC, and moves the family splits toward the admin shares (single no children −14% → −5%); anchoring reporters on top adds 0.07m. Draft PRs now exist for both halves (microcosm#750 → the WAS-stage PR; Add private_pension_wealth as an explicit capital disregard; corporate_wealth keeps the share-like holdings policyengine-uk#1824 for the model-side disregard): on the WAS donor the folded component is £3.66tn of the £4.32tn old corporate_wealth mass and is not DB-free (retained DB rights and DB pensions in payment sit inside it), so the column is private_pension_wealth; re-imputing the wealth block with the microcosm stage at a fixed seed, the split takes the UC caseload from 5.19m to 6.27m benefit units (+1.07m) and eligible benefit units from 9.8m to 16.3m, weights unchanged. A uk-data mirror patch is held pending the last-release decision.
   2. Take-up: decide the denominator (entitled or reporting benefit units) and the rate basis, then re-adjudicate U8 in microcosm and pass an eligibility mask in uk-data. Neither pinned take-up file cites a published counterpart for 0.55 over all benefit units. On the post-Sort the benunit table by benunit_id in create_frs #462 artifact anchoring alone is worth only 0.02m, so this is about the denominator and the rate, and it comes after (1).
   3. Elements: add the four series as direct validation outputs first (chronicle#188 facts → contract ids per the #727 precedent → references). Regenerating the feed needs care: Add benefit cap calibration targets to loss function #188 renamed the children/family concept from dwp.uc_households to dwp.uc_benefit_units, so the nine family/children selectors would match nothing, and the households selector (dwp.uc_benefit_units, no dimension pin) would also see the 117 new monthly split facts. Formula changes, childcare especially, come after the administrative concepts are pinned.
   4. Re-pin the parity reference to a post-Sort the benunit table by benunit_id in create_frs #462 build (v1.56.16 or later) before microcosm#686's control build.
   5. Keep the per-release ask from this issue: the table above, per release, in the validation output.

   Basis: DWP facts are Great Britain and benefit-unit grain; the deductions-derived total (6.76m) and the Stat-Xplore family-type total (6.86m) are different series; all model numbers are on the incumbent's artifacts — no microcosm-built 2024-25 spine is measured here (#686 pending).