You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue tracks defects that PolicyEngine/policyengine-uk-data PRs and issues fix or report and that microcosm still has. Each review records the last uk-data item it covered. The next review starts at the following number, updates this body, and posts a comment that lists what changed.
Main pass 12:09–12:40 BST (11:09–11:40 UTC), covering uk-data items up to and including uk-data#516.
Addendum 14:10–14:25 BST (13:10–13:25 UTC) for uk-data#517–#519 and for the microcosm items opened in between.
Last uk-data item covered:uk-data#519, an issue filed 2026-10-02 12:55 UTC. The last PR covered is uk-data#518. The next review starts at uk-data#520.
Scope: the 32 items filed from uk-data#486 (29 September) to uk-data#519: 17 PRs and 15 issues. The older related issue uk-data#467 is included. Out of scope: uk-data#488 and #489 (LAD lookups, closed unmerged).
uk-data state:
main is at b45c373 (25 September), and none of the items had merged. The last merged code is uk-data#480 (4 September).
Of the 17 PRs, only uk-data#490 is out of draft; the other 16 are drafts.
Weight context: the SPI channel holds 47.6% of microcosm's calibrated household weight (13.8m of 29.0m households), against 27.7% for uk-data's SPI half. An SPI-row defect therefore weighs about 1.7 times as much here.
How to update this issue
List uk-data items numbered above the last one covered: gh api "repos/PolicyEngine/policyengine-uk-data/issues?state=all&sort=created&direction=desc&per_page=30".
Check each new item against microcosm main and the open microcosm PRs, and add it to the right section below.
Re-check the merge state of the uk-data and policyengine-uk PRs named here.
Tick an item once microcosm main has the fix, and name the PR.
Move an engine item up once its policyengine-uk change is in the lock.
Update the audit record (date, time, last item covered, refs) and post a comment summarising the changes.
Fix in microcosm now: no policyengine-uk change needed
Where:hmrc_spi_income_spine redraws dividend_income for every FRS-channel person aged 16 or over from the SPI stage-1 forest, which predicts from age, gender and region only.
93% of SPI-channel 16–17-year-olds have earnings (£11.0bn), against 4% on FRS rows.
Dependants aged 16–19 in non-advanced education carry £12.7bn.
£0.95bn of it sits in UC-reporting benefit units.
Fix: exclude FRS dependent children (16–19, in non-advanced education, inside a parent's benefit unit) from the stage-1 recipients. Bundle it with the dividend fix so one rebuild covers both.
council_tax is the bill after council tax reduction.uk-data#496 (fixed there by uk-data#499).
uk_runtime/frs_council_tax.py:70-101 keeps CTANNUAL, which the FRS 2024-25 DV summary defines as "Annual CT amount after discounts/reduction".
A test pins CTANNUAL = 0 at £0.
The imputation pool includes recipients' net bills.
council_tax_rebate (CTREBAMT × 365.25/7, uk_runtime/frs_spine.py:805-807) already holds the add-back, but nothing adds it.
Measured, FRS channel:
20.6% of English CTR reporters have a £0 bill, and 25.6% a bill below their own CTR (Wales: 14.8% and 18.4%).
The add-back totals £0.78bn in England, £0.05bn in Wales and £0.06bn in Scotland.
obr.council_tax binds council_tax_less_benefit, so CTR comes off twice: about £0.5bn for English working-age reporters. The UK row ends at −10.4%.
Scotland:
microcosm nets sewerage, which uk-data main does not, but at the wrong rate: uk-data#499's status-discount check gives 0.81–0.89 here, against 0.75 with the gross DVs.
OBR table 4.9 benefit rows are measured over the UK. Follow-up to uk-data#490.
Fixed by UK: port the unblocked uk-data fixes tracked in #1095 #1100: each row binds Great Britain or England and Wales, and obr.pip binds pip + dla over England and Wales (ruling of 2 October). The pension-age HB spending row and the Chronicle stamp now have their own item below.
Where:
These rows in uk/uk_population_targets.json carry no country condition: obr.attendance_allowance, carers_allowance, pip, winter_fuel_allowance, pension_credit, esa, jobseekers_allowance, universal_credit, statutory_maternity_pay and housing_benefit.
dwp.esa_* lacks a GB filter.
DWP's lines cover GB. They cover England and Wales only for CA from September 2018, DLA/PIP/AA from April 2020 and Winter Fuel from April 2024.
Measured: share of each fitted UK column outside the publisher's scope: AA 10.8%, CA 12.3%, PIP 8.9%, ESA 7.5%, PC 4.7%, UC 2.3%.
Fix:
Add region household_conditions, as dwp.uc.* and dwp.hb.* already use.
Where:BENEFIT_CODES (uk_runtime/frs_spine.py:131-152) has no code 117/118 (Adult Disability Payment) or 121/122 (Child Disability Payment).
Measured:
Raw FRS: 215k ADP daily-living recipients (£0.98bn), 147k ADP mobility, 51k CDP care and 27k CDP mobility.
73% of Scottish daily-living recipients are dropped. PIP daily-living reporters per 1,000 residents: Scotland 24.6, England 66.0.
Fix:
Map ADP into the pip_* and CDP into the dla_* reported columns. Do it with the country-scope item, or about £1.4bn of Scottish spending lands in a UK-measured PIP column fitted to an England and Wales line.
Codes 69 and 70 are benefit recoveries; leave them unmapped.
SPI rows carry benefit claims nobody observed. Remainder of uk-data#514.
contributory ESA on earners: 3.7% of SPI reporters here, against 41% in uk-data.
Still open:
1.51m SPI benefit units claim UC only because their FRS donor reported it. The OR refresh in uk_runtime/uc_capital_coherence.py:154-158 never removes a donor's anchor.
receives_benefits_in_own_right (uk_runtime/frs_education.py:193-198) disagrees with the person's own reports for 3.04m SPI persons, 61k of them aged 16–19.
SPI rows report IS, WTC, CTC and income-based JSA at 3.75, 3.0, 2.6 and 9.1 times the FRS rate.
Local targets and predictors use the age-18 child.uk-data#486.
Fixed by UK: port the unblocked uk-data fixes tracked in #1095 #1100 for the constituency UC bands, the ETB predictors and the WAS donor. Still open: the LCFS variable names, and María's signature on the reworded uc_unit_vs_household_grain adjudication and census fence.
Already correct: the national targets, which use UC's child or qualifying young person and the ONS dependent child.
Constituency UC child bands use the age-18 num_children (uk_runtime/local_targets.py:282-289), bound under adjudication uc_unit_vs_household_grain. 2.77% of UC-reporting units fall in a different band. Fix in Align UK UC claim, child and period definitions #882's lane.
ETB VAT and services pair the engine's is_adult/is_child with ETB's own counts (uk_runtime/etb_vat.py:34-37, uk_runtime/etb_services.py:42-57). 2.3% of households mismatch.
Measured: with buy-to-let included, 91% of rent-receiving WAS households hold the matching property, against 25% now. This reaches the HB, PC and legacy capital tests.
Where:uc_capital_coherence redraws TOTCAPB4 only for post-fill UC reporters, by children and couple status (uk_runtime/uc_capital_coherence.py:141,217-239).
Measured: 2.28m SPI benefit units have interest and dividends that imply over £16k of capital at a 4% yield, but TOTCAPB4 of £16k or less. 105k of them are UC reporters.
Fix: condition the redraw on investment income, extend it beyond reporters, and add a per-release receipt of income held without a matching capital stock.
Port verified: it gives the same property income as uk-data#509's rule on all 34,966 FRS 2024-25 person records.
CI fails only test_no_incumbent_data_package_references_in_live_tree: the changelog fragment and the test docstring cite the incumbent package by name instead of uk-data#503/#509.
Against FYE 2025: the v21c scorecard sits inside HBAI's 95% confidence intervals for relative poverty (14.63% before housing costs, 19.31% after). Its median is 6.7% above HBAI's.
Fix (validation only, never bound):
re-base to FYE 2025;
report each statistic as inside or outside its interval;
source the tables through a PolicyEngine/chronicle package that records vintages, since DWP revises FYE 2022–25 in March 2027.
Where: one FRS 2024-25 access-fund award carries the calendar-month period code (ACCSSPD 5). Once annualised, it is eleven times the next largest FRS value, and it is almost certainly an annual or termly award. The detail is in the retired entry in uk/input_mass_reviewed_exclusions.json, as it stood before a0c678c21.
dwp.hb.amount_pension_age cannot compile. calendar_year_window needs both fiscal years from one series identity, but the DWP forecast-table facts in feed 825406f carry one measure_id per year (expenditure_2024, expenditure_2025).
Needs: a PolicyEngine/chronicle series-identity change, or a ruling to bind FY2025-26 alone. Raise it together with the request that Chronicle stamp table 4.9's DWP lines GB or England and Wales.
Depends on policyengine-uk model updates
microcosm locks policyengine-uk 2.100.0. Since 2.102.4, policyengine-uk requires a policyengine-core that the certified US default refuses, so no release from 2.102.4 on can be locked until #1086 is resolved. Each item names the policyengine-uk change it needs and that change's state on 2 October 2026.
Legacy-benefit reporters never move to UC.uk-data#492.
Needs:
a policyengine-uk input for claiming UC at legacy closure (uk-data#492's option b; not yet filed);
552.6k working-age benefit units report HB, tax credits, IS, income-based JSA or income-related ESA and have would_claim_uc False: 313.6k on FRS rows and 239.1k on SPI rows.
2.100.0 already closes tax credits, IS and income-based JSA, so those reporters lose the benefit today.
Income-related ESA is paid whatever the flag says.
Why the data alone can't fix it: anchoring in the data (option a) fails here, because HB and CTC require ~would_claim_uc in the 2025 calibration year.
Shared households:rent is the whole dwelling's HHRENT, placed on the household head (uk_runtime/frs_spine.py:829). About 685k calibrated benefit units in shared households (353k on FRS rows, 332k on SPI rows) have no rent of their own.
Sub-letting: SUBRENT sits in property_income. microcosm's own household subrent column is redrawn for SPI copies and read by nothing. Ruling needed: does SPI INCPROP already include rent-a-room receipts?
Boarders and lodgers: CVPAY, the rent they pay (£0.81bn), is not carried.
Fix: set pension_credit_reported_capital to frs_benunit_capital (TOTCAPB4, which equals TOTCAPB3 for 81% of PC reporters).
Smaller here than in uk-data, because microcosm already strips pension wealth from corporate_wealth. Deemed income from the engine's capital reaches the reported PC for 11.6% of FRS-row PC reporters, against 1.7% with FRS capital.
export is_hbai_dependent_child and the claimant-or-partner flag from the FRS roles;
update the UC take-up population in uk_runtime/frs_take_up.py:51-62, which mirrors the old is_uc_eligible. Its lockstep test pins the deprecated is_WA_adult, so the drift would pass silently.
Why: microcosm stores no birth year and holds ages fixed across model years. The model's default test (older member born 1954 or earlier, computed as model year minus age) therefore drifts one cohort a year.
Fix: set has_mixed_age_couple_pension_credit_saving at build time from survey-year facts.
Accounts dividends keyed on person_id, the keying half of uk-data#498: already correct.
Not applicable
uk-data#505: microcosm never simulates the SPI tape, and region UNKNOWN exists only on the donor table. One optional check: composite and address-abroad donors hold 18% of SPI dividends.
This issue tracks defects that PolicyEngine/policyengine-uk-data PRs and issues fix or report and that microcosm still has. Each review records the last uk-data item it covered. The next review starts at the following number, updates this body, and posts a comment that lists what changed.
Audit record
07bbb65). Nine items are ticked and two are added. No new uk-data items were reviewed, so the next review still starts at uk-data#520.3598c38.004b5c9and now atc78dff4.ebff841.af6bc17and Weight UK BRMA assignment by census private-rented households #1087 atec0d0ee.packages/microcosm-build/src/microcosm/build/, with line numbers at main3598c38.c5a1cba(30 September): stored 2024-25 columns, calibrated weights, policyengine-uk 2.100.0.How to update this issue
gh api "repos/PolicyEngine/policyengine-uk-data/issues?state=all&sort=created&direction=desc&per_page=30".Fix in microcosm now: no policyengine-uk change needed
hmrc_spi_income_spineredrawsdividend_incomefor every FRS-channel person aged 16 or over from the SPI stage-1 forest, which predicts from age, gender and region only.uk/spec/sources.yaml:1605-1609,uk_runtime/spi_income.py:641-672.uk_runtime/spi_spine.py:204and intest_uk_spi_income.py. Create the SPI income channel on the microcosm spine with the FRS-only second stage #683 adopted the redraw for parity with the incumbent.person_id(uk_runtime/frs_spine.py:538-571).uk_runtime/spi_income.py:74).council_taxis the bill after council tax reduction. uk-data#496 (fixed there by uk-data#499).council_taxis the bill before the reduction, and Scottish water and sewerage are netted at the gross cells (65% for recipients, ruling of 2 October). The charge paid follows the reduction scheme's combined maximum: the larger of the status discount and 35%. Still open: María's signatures on the two register texts in UK: port the unblocked uk-data fixes tracked in #1095 #1100's description, and the UK: supply the engine's local_authority input from the ladder's local_authority_code, plus council tax follow-ups from policyengine-uk#1855/#1864 and uk-data#482/#484 #953/WS-C deferred adjudications and the reconciliation layer #736 corrections below.uk_runtime/frs_council_tax.py:70-101keeps CTANNUAL, which the FRS 2024-25 DV summary defines as "Annual CT amount after discounts/reduction".council_tax_rebate(CTREBAMT × 365.25/7,uk_runtime/frs_spine.py:805-807) already holds the add-back, but nothing adds it.obr.council_taxbindscouncil_tax_less_benefit, so CTR comes off twice: about £0.5bn for English working-age reporters. The UK row ends at −10.4%.water_and_sewerage_chargesrun about 13–16% low. This is a residual of uk-data#467; Whole-spine parity for the UK: re-pinned incumbent, comparison instruments, signed register, and the E10 evidence (#686) #747 fixed the zeroing itself.obr.pipbindspip + dlaover England and Wales (ruling of 2 October). The pension-age HB spending row and the Chronicle stamp now have their own item below.uk/uk_population_targets.jsoncarry no country condition:obr.attendance_allowance,carers_allowance,pip,winter_fuel_allowance,pension_credit,esa,jobseekers_allowance,universal_credit,statutory_maternity_payandhousing_benefit.dwp.esa_*lacks a GB filter.household_conditions, asdwp.uc.*anddwp.hb.*already use.obr.attendance_allowancemoving from 0.0% to +2.6% against the new AA caseload rows.obr.pipbinds PIP alone against OBR's combined DLA and PIP line.BENEFIT_CODES(uk_runtime/frs_spine.py:131-152) has no code 117/118 (Adult Disability Payment) or 121/122 (Child Disability Payment).pip_*and CDP into thedla_*reported columns. Do it with the country-scope item, or about £1.4bn of Scottish spending lands in a UK-measured PIP column fitted to an England and Wales line.spi_benefit_coherencestage and its release-blocking gate. It also zeroes income-related ESA, and keeps contributory ESA.uc_reporter_redraw;uk_runtime/uc_capital_coherence.py:154-158never removes a donor's anchor.receives_benefits_in_own_right(uk_runtime/frs_education.py:193-198) disagrees with the person's own reports for 3.04m SPI persons, 61k of them aged 16–19.dwp.jsa_claimants(UK spine: JSA claimants 2.8× the DWP caseload — legacy-benefit reporting after the UC migration #869).uc_capital_coherence. It should:dwp.esa_*rows;receives_benefits_in_own_right;uc_age_eligible.uc_unit_vs_household_grainadjudication and census fence.num_children(uk_runtime/local_targets.py:282-289), bound under adjudicationuc_unit_vs_household_grain. 2.77% of UC-reporting units fall in a different band. Fix in Align UK UC claim, child and period definitions #882's lane.is_adult/is_childwith ETB's own counts (uk_runtime/etb_vat.py:34-37,uk_runtime/etb_services.py:42-57). 2.3% of households mismatch.NumCh18R8overlapsNumAdultR8in 8% of households (uk_runtime/was_wealth.py:36-57). Fix with Port UK compact source stages and 2024-25 refresh from uk-data #145's predictor revisit.other_residential_property_value. Part of uk-data#501 and uk-data#495.rental_incomeadded as a WAS predictor.uk_runtime/was_wealth.py:173mapsDVHseValR8only. Buy-to-let (DVBltValR8, £573bn in WAS round 8) is left out, and stays out in UK spine follow-ups: SPI donors funded, WAS coherence, residential split, Child Benefit, release-cut blockers (#1063) #1089.uc_capital_coherenceredraws TOTCAPB4 only for post-fill UC reporters, by children and couple status (uk_runtime/uc_capital_coherence.py:141,217-239).test_no_incumbent_data_package_references_in_live_tree: the changelog fragment and the test docstring cite the incumbent package by name instead ofuk-data#503/#509.uk_runtime/cgt_asset_type.py:752-754readsproperty_income > 0as evidence of residential property, so the 42 loss-making landlords lose that signal after UK FRS spine: stop counting lodgers' rent and property losses as property income; sub-letting rent for every tenure #1081.hbai_basiscompares engine year 2024 with HBAI FYE 2024. Period 2024 is FRS FY2024/25 (docs/uk-uc-paid-target-contract.md), which is HBAI FYE 2025.torch.Generator;uk_runtime/spi_band_donors.py:354-380).ACCSSPD5). Once annualised, it is eleven times the next largest FRS value, and it is almost certainly an annual or termly award. The detail is in the retired entry inuk/input_mass_reviewed_exclusions.json, as it stood beforea0c678c21.uk_runtime/frs_spine.py), so the column stops depending on where calibration lands.dwp.hb.amount_pension_agecannot compile.calendar_year_windowneeds both fiscal years from one series identity, but the DWP forecast-table facts in feed 825406f carry onemeasure_idper year (expenditure_2024,expenditure_2025).Depends on policyengine-uk model updates
microcosm locks policyengine-uk 2.100.0. Since 2.102.4, policyengine-uk requires a policyengine-core that the certified US default refuses, so no release from 2.102.4 on can be locked until #1086 is resolved. Each item names the policyengine-uk change it needs and that change's state on 2 October 2026.
would_claim_ucFalse: 313.6k on FRS rows and 239.1k on SPI rows.~would_claim_ucin the 2025 calibration year.rentis the whole dwelling's HHRENT, placed on the household head (uk_runtime/frs_spine.py:829). About 685k calibrated benefit units in shared households (353k on FRS rows, 332k on SPI rows) have no rent of their own.property_income. microcosm's own householdsubrentcolumn is redrawn for SPI copies and read by nothing. Ruling needed: does SPI INCPROP already include rent-a-room receipts?pension_credit_reported_capitaltofrs_benunit_capital(TOTCAPB4, which equals TOTCAPB3 for 81% of PC reporters).corporate_wealth. Deemed income from the engine's capital reaches the reported PC for 11.6% of FRS-row PC reporters, against 1.7% with FRS capital.uk_runtime/frs_disability.py:282-284uses the old definition. 0.55m of the 4.93m people it flags fail the tax-credit condition.directly_held_shares,unit_and_investment_trusts,cash_isaandstocks_and_shares_isa, which have no policyengine-uk PR yet.mortgage_debtisHMortGR8, which includes £234bn secured on other property. Choose between all secured debt and main residence only (TotMortR8, £1,014bn), so the debt is not counted twice.is_hbai_dependent_childand the claimant-or-partner flag from the FRS roles;uk_runtime/frs_take_up.py:51-62, which mirrors the oldis_uc_eligible. Its lockstep test pins the deprecatedis_WA_adult, so the drift would pass silently.has_mixed_age_couple_pension_credit_savingat build time from survey-year facts.MAIND ↔ is_marriedbridge is wrong-sided, because MAIND flags recipients only.Already handled in microcosm
These need no new work and are listed so later reviews skip them.
uk_runtime/rowwise_geography.py, and no caller turns it off.person_id, the keying half of uk-data#498: already correct.Not applicable