Stop SPI-synthetic rows carrying benefit claims nobody observed

[MaxGhenis]

This changes the published dataset. It ships only in the batched uk-data release on Max's go (d833); don't merge it on its own. No dataset was uploaded or released from this branch.

Ruling (Max, d821, 2026-10-05): approved into the batch with one change, made in deb16e9: council tax reduction (CTR) is no longer zeroed on SPI rows. SPI CTR stays the stage-two QRF draw, as on main. Zeroing it comes later, together with a household-level CTR imputation (alongside #499). Every other rule ships unchanged.

CTR is unchanged against main; its total still moves through reweighting. The impact figures below come from a rebuild of this head (deb16e9) measured against main (b45c373), both with #507's seed. On every record the two builds share, this head's SPI CTR reports are identical to main's, and at main's weights the CTR paid differs on fewer than 10 survey households. The 2025 CTR total is £2.09bn against main's £2.20bn (−£0.11bn, −5.0%), with £0.37bn on SPI rows against £0.49bn. That gap comes from reweighting, driven by the PR's other changes; no target constrains CTR itself. Zeroing SPI CTR moves to a later PR with a household-level CTR imputation (#499). See "Council tax reduction against main" below.

Problem

impute_income stacks a 10,000-household copy of the FRS (household_is_spi_synthetic), replaces its incomes with SPI draws, and calibration gives those rows weight. On the main build (b45c373, policyengine-uk 2.93.0), they hold 8.62m of 31.09m households (27.7%).

Their benefit columns come from two places:

• Second-stage QRF draws. impute_frs_only_variables fills every *_reported column in FRS_ONLY_PERSON_VARIABLES, drawing each person's reports from their age, gender, region and six incomes. It sees nothing of their benefit unit (partner, children, rent, capital), their health or their history.
• Copies of the FRS donor. create_frs builds would_claim_uc, would_claim_pc and would_claim_child_benefit (anchored on the donor's reports), receives_benefits_in_own_right and ssmg_reported before the SPI rows exist.

policyengine-uk 2.93.0 treats a positive report as an existing claim, in three ways:

• Continuing-award gate: housing_benefit_eligible, income_support_eligible, is_WTC_eligible, is_CTC_eligible, esa_income_eligible, jsa_income_eligible, sda and ssmg.
• Claim flag: would_claim_housing_benefit, would_claim_IS, would_claim_WTC, would_claim_CTC and would_claim_council_tax_reduction. Each is claims_all_entitled_benefits | report > 0. claims_all_entitled_benefits sums reports over the whole simulation, so it is False in any dataset run.
• Paid amount:
  • esa_income and jsa_income: the reported award less tariff income;
  • council_tax_benefit, where there is no scheme;
  • esa_contrib, jsa_contrib, incapacity_benefit, iidb, afcs and bsp, by adds.

What that does on the main build (b45c373)

Data file (2024-25), calibrated weights:

	FRS rows	SPI rows
Child Benefit reporters in a benefit unit with no child or 16–19 qualifying young person, by weight	suppressed (<10 records)	34%
ESA (contributory) reporters earning over ESA's permitted-work limit, by weight	suppressed (<10 records)	41%
UC / PC / CB reporters whose unit doesn't claim, weighted benefit units	0 / 0 / 0	462k / 29k / 106k
People whose receives_benefits_in_own_right disagrees with their own reports, weighted	0	2.48m

SPI report rates by weight, relative to FRS rows:

Benefit	SPI rate ÷ FRS rate
IIDB	6.2x
Bereavement support	4.6x
ESA (contributory)	3.6x
AFCS	2.4x
State pension	1.09x

Benefits paid on SPI rows, policyengine-uk 2.93.0, 2025:

Benefit	Total £bn	On SPI rows £bn	SPI share
Universal Credit	75.60	7.36	9.7%
ESA (contributory)	5.06	2.90	57%
Child Benefit	16.90	4.56	27%
Council tax reduction	2.21	0.49	22%
ESA (income-related)	2.15	0.18	8.2%
Pension Credit	6.21	0.16	2.6%
IIDB	0.82	0.53	65%
AFCS	2.40	1.49	62%
Bereavement support	0.59	0.32	53%
State pension	125.90	38.46	31%
PIP	27.70	5.89	21%

Change

apply_spi_donor_benefit_rules (imputations/frs_only.py) runs on the SPI rows after the QRF draw. It runs before the disability categories and flags are derived, because ESA, IIDB and AFCS feed is_disabled_for_benefits.

Reports zeroed on SPI rows:

Reports	Why
UC, Pension Credit, Housing Benefit, Income Support, WTC, CTC, income-related ESA and JSA, Sure Start Maternity Grant	Entitlement turns on the unit's joint means and make-up, which here come from the imputed incomes. UC and Pension Credit keep a claim route through the redrawn take-up flags (below). The others need a report in 2.93.0, so SPI rows no longer receive them. policyengine-uk 2.93.0 switches tax credits off from 6 April 2025, and IS and income-related JSA from 1 April 2026 (active.yaml). HB paid on SPI rows rests on fewer than 10 survey households, so its amount isn't shown. CTR would also need a report, so it is kept for now (see "Draws kept"). SSMG needs one of these awards and was a donor copy.
ESA and JSA (contributory), incapacity benefit, SDA	Paid only to people out of work or incapable of it, but the QRF gives them to earners. ESA and JSA caseloads are calibrated to DWP figures (620k and 71k), so FRS rows must carry them. FRS rows held 304k of the main build's 724k ESA (contributory) caseload.
Child Benefit	The model reads it only through the take-up flag, and the draw ignores the children.

Donor values restored: IIDB, AFCS and bereavement support. They follow from an injury, military service or a death, not income, and the QRF over-draws them (6.2x, 2.4x and 4.6x by weight). No target is set on them.

Draws kept:

• State pension: paid as reported over pension age. 99.5% of SPI people aged 67 or over report it, against 98.3% on FRS rows (records).
• Winter fuel payment: not read by the model.
• AA, DLA and PIP: drawn below the FRS rates, as the income gradient implies.
• Carer's allowance: the draws respect its earnings limit (no SPI reporter is above it), though policyengine-uk has no earnings test.
• Maternity allowance: kept as drawn, as on main. This PR does not change its treatment, and the pinned rule-set test now lists it explicitly among the reports kept as drawn.
• Council tax reduction (ruling d821): kept as drawn for this release. In policyengine-uk 2.93.0, would_claim_council_tax_reduction is claims_all_entitled_benefits | council_tax_benefit_reported > 0, so zeroing the report left SPI rows no way to claim CTR. It will be zeroed together with a household-level CTR imputation (Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499).

Flags rebuilt from the rows' own reports:

• would_claim_uc and would_claim_pc are redrawn with assign_reported_takeup, the helper create_frs now uses too. With no reports, every SPI unit draws at the take-up rate, because whether a synthetic family claims a means-tested benefit at its imputed income is unobserved. The draws use their own seed (101; create_frs uses 100) and don't depend on which columns exist.
• would_claim_child_benefit keeps the donor's flag. The award doesn't depend on the replaced incomes, and the donor's claim is for the same children.
• receives_benefits_in_own_right is rebuilt from the rows' own UC, JSA and ESA reports. That makes it False on every SPI row, so SPI 16–19-year-olds can count as qualifying young people.

The QRF chain is unchanged. Every column stays in it, so the kept draws are bit-identical, and a test checks this. The cost is that the kept values were drawn alongside values that are later zeroed or restored. create_frs keeps its draw order, so the base FRS doesn't change; the builds below confirm it table by table.

Other effects of zeroing that follow from this change:

• ESA and JSA (contributory) are taxable, so SPI rows' total income falls, and that feeds income tax and the HMRC income-band targets.
• Disability flags feed UC's LCWRA element.
• The 16–19 bursary flags and Scottish Child Payment eligibility read some of these reports.

Invariants (tested; Hypothesis for 2–6)

1. The three rule sets are pinned exactly, and so is the set of benefit reports kept as drawn (which now includes CTR). Changing a column's treatment is a deliberate edit.
2. Zeroed columns are zero, restored columns equal the donor's values, nothing else changes, and the input isn't mutated.
3. A unit with a member reporting UC or Pension Credit claims it, and receives_benefits_in_own_right matches the person's own reports. This is checked with those reports left unzeroed, so the test can fail.
4. The rules are deterministic and idempotent.
5. Draws don't depend on which other columns are present.
6. Ids are shuffled and gapped, so matching by position would fail.
7. With no reports, UC and PC are claimed at the take-up rate (40,000 units, ±1pp), and the Child Benefit flag is untouched.
8. Through impute_frs_only_variables:
   • kept draws are bit-identical with and without the rules;
   • restored columns equal the donor's values;
   • the disability flags match the final reports.
9. SPI CTR is the stage-two QRF draw, bit-identical to the draw before the rules run: not zeroed and not the donor's value (test_council_tax_reduction_keeps_the_stage_two_draw). The property test for invariant 2 now covers CTR as one of the columns that must pass through unchanged.

I broke the code in seven ways, and each failure was caught:

Code broken	Test that fails
Rules not applied	8
No donor restore	2, 8
Child Benefit flag redrawn	1, 7, 8
receives_benefits_in_own_right not rebuilt	3, 8
Anchors not rebuilt	3, 7
A draw skipped when a column is missing	5
A column dropped from a rule set	1

After the d821 change (deb16e9), three more breaks (patched in-process), each caught:

Code broken	Test that fails
CTR zeroed again	1, 9
CTR restored to the donor's value	1, 9
CTR taken out of the QRF chain (as #499 does)	1, 9

A one-off differential check ran origin/main's own impute_frs_only_variables and this head's on the same synthetic inputs (CTR drawn non-zero for some rows): SPI CTR and the other 14 kept columns are bit-identical.

Tests (no dataset build): at deb16e9 the five core files (test_spi_donor_benefit_rules, test_frs_only_imputation, test_reported_takeup_anchors, test_legacy_benefit_proxies, test_imputation_source_flags) give 36 passed. Every test file mentioning SPI, frs_only or council_tax_benefit (15 files, run on the same code just before a formatting-only change to one assertion) gives 107 passed, 5 skipped (they need built FRS datasets) and 1 xfailed (the existing test_hf_destinations guard).

test_frs_only_reported_values_correlate_with_training_pattern now checks the QRF's income gradient on carer's allowance, because UC is zeroed on these rows.

Measured with real builds

Two production-settings builds, measured with policyengine-uk 2.93.0 (the lock):

• Before (main): b45c373 + Seed the calibration weight dropout so builds are reproducible #507's seed, built as 039ea56.
• After (this head): deb16e9 + the same seed, built as d3156d8.

The Problem section's figures come from an earlier build of main without the seed, so they differ slightly from the before column here (for example, CTR is £2.21bn there and £2.20bn here).

Both used 512 calibration epochs, PE_UK_DATA_OA_CLONES=1 and #507's seeded calibration dropout (348dd49, applied locally, at its default seed of 0). Every target download was served from the same frozen cache (0 misses; the request logs are identical). The after build finished on 5 October with exit 0. Re-running the audit and headline scripts on the before build, in the environment used for after2, reproduces its earlier outputs exactly.

The CTR check also uses the earlier build of this branch: eb14fd0 + the same seed (ea6f3b5), which zeroed CTR.

Evidence: ~/reviews/uk-data-spi-reported-2026-10-01/ on the build host (Max's Mac). The h5 files are FRS-derived, so they stay local.

• Build provenance:
  • run_gated_build.sh builds the worktree's commit with PE_UK_DATA_OA_CLONES=1.
  • setup_build_worktree.sh applies the Seed the calibration weight dropout so builds are reproducible #507 seed commit 348dd49.
  • cached_build.py patches requests.get to serve http_cache/.
  • Per-build records: builds/{before,after2}.commit (exact heads and bases), .http (every request served from cache), .sha256, .started/.finished, and after2's gzipped calibration logs.
• Measurement scripts (in analysis/):
  • audit_spi_reports.py: benefits, reports and anchors;
  • headline.py: tax, income and poverty;
  • coherence2.py and report_rates.py;
  • compare_builds.py: record-level equality, reported as counts;
  • cells_and_decomposition.py: the disclosure counts and the split of each change;
  • spending_check.py: the public-spending parts of household benefits;
  • impact_tables.py: these tables.
• Aggregate outputs: analysis/*_after2.*, plus audit_after2_wafter.json and headline_after2_wafter.json (this head's data at the earlier build's weights) cells_and_decomposition.json and spending_check.json. All of these are aggregates only.

The base FRS is unchanged. All three tables are identical column by column (34,966 people, 18,850 benefit units, 16,288 households). The h5 checksums differ, but the table contents don't.

What else changes in the data. On the records the two builds share, the changes are the SPI rows' reports and flags, as intended, plus two knock-ons. Zeroing the taxable ESA and JSA (contributory) lowers some SPI rows' total income, and two later steps select on total income:

• Capital gains are drawn band by band from one seeded stream, so when a row moves band, the draws after it shift. Gains differ on 8,107 SPI and 13,298 FRS person records.
• CGT band donors are picked with total-income propensities, so 250 of the 270 donor households are different households.

Pension contributions also differ on fewer than 10 records each on SPI and FRS rows, and output-area codes differ on a few households. The rail, bus and road-fuel spending columns differ too, because the build fits them to fixed targets at each build's own weights (see below).

Coherence on SPI rows, after:

• 0 Child Benefit, UC, PC or ESA/JSA (contributory) reporters.
• 0 benefit units that report UC, PC or CB but don't claim it (before: 462k / 30k / 106k weighted). Units that don't report get a random flag by design, so only reporters are checked.
• 0 people whose receives_benefits_in_own_right disagrees with their reports (before: 2.48m weighted).
• IIDB, AFCS and bereavement support now appear on SPI rows at 0.90x, 0.96x and 1.06x the FRS rate (records).

Disclosure: every cell below was checked against the number of distinct survey households behind it. The enhanced FRS holds every household at least twice, as itself and as its capital gains clone. SPI copies, their clones and the CGT band donors repeat households again. Record counts therefore overstate survey records. Any cell resting on 1 to 9 survey households reads "suppressed (<10 records)", and so does any change computed from one.

Benefits, 2025:

Benefit	Before £bn	After £bn	Change £bn	SPI rows before → after £bn	Caseload before → after (k)
Universal Credit	75.67	75.41	−0.26 (−0.3%)	7.33 → 5.43	6,373 → 6,395
Pension Credit	6.20	6.20	+0.00	0.16 → 0.14	1,703 → 1,698
Housing Benefit	12.97	12.96	−0.00	suppressed (<10 records) → 0	1,560 → 1,567
Council tax reduction	2.20	2.09	−0.11 (−5.0%)	0.49 → 0.37	3,316 → 3,138
ESA (income-related)	2.15	2.11	−0.04 (−1.9%)	0.17 → 0	190 → 190
ESA (contributory)	5.08	5.19	+0.12 (+2.3%)	2.89 → 0	726 → 721 (DWP target 620)
JSA (contributory)	0.30	0.30	+0.00 (+0.6%)	suppressed (<10 records) → 0	69 → 69
Child Benefit	16.95	16.89	−0.06 (−0.4%)	4.57 → 4.26	8,400 → 8,369
State pension	125.82	125.82	−0.00	38.47 → 37.59	11,475 → 11,481
PIP	27.73	27.57	−0.16 (−0.6%)	5.87 → 3.32	3,651 → 3,625
DLA	8.26	8.65	+0.39 (+4.7%)	1.47 → 1.46	1,375 → 1,424
Attendance allowance	8.72	8.72	−0.00	1.35 → 1.35	1,667 → 1,661
Carer's allowance	4.67	4.67	+0.01 (+0.1%)	0.58 → 0.50	1,078 → 1,079
IIDB	0.83	0.61	−0.22 (−26.3%)	0.53 → 0.11	255 → 135
AFCS	2.40	0.95	−1.44 (−60.2%)	1.49 → 0.09	218 → 120
Bereavement support	0.59	0.80	+0.21 (+35.0%)	0.32 → 0.52	133 → 112

Other report-gated benefits, £bn: Income Support 0.38 → 0.38; WTC 0 → 0; CTC 0 → 0; income-related JSA suppressed (<10 records) → suppressed (<10 records); incapacity benefit 0 → 0; SDA suppressed (<10 records) → suppressed (<10 records); SSMG 0.03 → 0.02.

Benefits, 2026:

Benefit	Before £bn	After £bn	Change £bn	SPI rows before → after £bn	Caseload before → after (k)
Universal Credit	79.80	79.58	−0.22 (−0.3%)	7.66 → 5.69	6,352 → 6,385
Pension Credit	6.93	6.92	−0.00 (−0.1%)	0.18 → 0.16	1,712 → 1,710
Housing Benefit	13.43	13.44	+0.00	suppressed (<10 records) → 0	1,566 → 1,574
Council tax reduction	2.25	2.14	−0.11 (−4.9%)	0.49 → 0.37	3,329 → 3,151
ESA (income-related)	2.20	2.16	−0.04 (−2.0%)	0.17 → 0	190 → 191
ESA (contributory)	5.21	5.33	+0.12 (+2.3%)	2.97 → 0	729 → 724
JSA (contributory)	0.30	0.31	+0.00 (+0.6%)	suppressed (<10 records) → 0	70 → 69
Child Benefit	17.67	17.60	−0.07 (−0.4%)	4.77 → 4.44	8,432 → 8,401
State pension	133.44	133.40	−0.04	40.90 → 39.92	11,519 → 11,524
PIP	28.90	28.73	−0.17 (−0.6%)	6.12 → 3.46	3,665 → 3,638
DLA	8.61	9.01	+0.40 (+4.7%)	1.53 → 1.52	1,380 → 1,429
Attendance allowance	9.09	9.09	−0.00	1.41 → 1.41	1,673 → 1,668
Carer's allowance	4.86	4.87	+0.01 (+0.1%)	0.60 → 0.53	1,082 → 1,083
IIDB	0.85	0.63	−0.22 (−26.3%)	0.55 → 0.12	256 → 136
AFCS	2.46	0.98	−1.48 (−60.2%)	1.53 → 0.09	219 → 120
Bereavement support	0.61	0.82	+0.21 (+35.0%)	0.32 → 0.54	134 → 113

Other report-gated benefits, £bn: Income Support 0 → 0; WTC 0 → 0; CTC 0 → 0; income-related JSA 0 → 0; incapacity benefit 0 → 0; SDA suppressed (<10 records) → suppressed (<10 records); SSMG 0.03 → 0.02.

Headline:

	2025 before	2025 after	2025 change	2026 before	2026 after	2026 change
Income tax £bn	294.35	293.89	−0.47	314.53	314.02	−0.50
National Insurance £bn	51.70	51.76	+0.05	53.99	54.05	+0.06
Household benefits £bn	546.69	545.19	−1.50	563.91	562.37	−1.54
Household net income £bn	1,709.79	1,708.54	−1.25	1,762.40	1,761.14	−1.27
Relative poverty, BHC	18.42%	18.26%	−0.16pp	17.56%	17.30%	−0.26pp
Relative poverty, AHC	21.78%	22.11%	+0.33pp	20.55%	20.78%	+0.23pp
Child relative poverty, AHC	29.79%	30.62%	+0.83pp	26.48%	26.96%	+0.48pp
Pensioner relative poverty, AHC	15.30%	15.25%	−0.04pp	13.51%	13.72%	+0.21pp

Where each 2025 change comes from. This splits each change over the 52,596 households in both builds:

• "Data change at before's weights" applies this head's values at main's weights.
• "Reweighting and band donors" is the rest: this head's weights, plus the 250 donor households that differ.

Some rows are left out. Housing Benefit and JSA (contributory) are left out because their data change is the suppressed SPI-row amount. Household net income is left out because it includes capital gains, which were redrawn. 2026 splits the same way.

	Change £bn	Data change at before's weights	Reweighting and band donors
Universal Credit	−0.26	−3.42	+3.16
Pension Credit	+0.00	−0.02	+0.03
Council tax reduction	−0.11	suppressed (<10 records)	−0.11
ESA (income-related)	−0.04	−0.17	+0.13
ESA (contributory)	+0.12	−2.89	+3.01
Child Benefit	−0.06	0	−0.06
State pension	−0.00	0	−0.00
PIP	−0.16	0	−0.16
DLA	+0.39	0	+0.39
Attendance allowance	−0.00	0	−0.00
Carer's allowance	+0.01	0	+0.01
IIDB	−0.22	−0.41	+0.19
AFCS	−1.44	−1.39	−0.05
Bereavement support	+0.21	+0.09	+0.11
Income tax	−0.47	−0.47	+0.00

SPI rows' household weight fell from 8.63m to 8.17m (of 31.14m).

Council tax reduction against main

CTR £bn	Main	Earlier build (eb14fd0, CTR zeroed)	This head	This head at the earlier build's weights
2025, all rows	2.20	1.73	2.09	2.17
2025, SPI rows	0.49	0	0.37	0.44
2026, all rows	2.25	1.78	2.14	2.22
2026, SPI rows	0.49	0	0.37	0.44

• The carve-out is the only change since the earlier build.
  • At person level, the only column that differs is council_tax_benefit_reported, on 2,859 SPI people.
  • The benefit-unit tables are identical.
  • The household tables differ only in their weights and in the rail, bus and road-fuel spending columns. After the weight solve, the build fits those columns to fixed targets; bus fares are also spread by region and income quintile.
• At the earlier build's weights, this head adds £0.44bn of CTR, all of it on SPI rows (2025 and 2026).
  • Every other benefit in the tables, income tax and National Insurance come out exactly as in the earlier build.
  • Household benefits rise £0.473bn in 2025: £0.436bn of CTR plus £0.037bn of rail and bus subsidy spending. The subsidy part appears because this head's spending columns were scaled to its own weights; at its own weights each build's subsidy total is the same £25.15bn.
• Reweighting then takes CTR down £0.08bn (SPI rows −£0.07bn), in both years.
  • In 2025 it moves the other benefits in the tables, income tax and NI by small amounts: income tax +£0.08bn, DLA +£0.05bn, bereavement support +£0.02bn, Child Benefit +£0.01bn and PIP −£0.01bn. The rest move by less than £0.01bn.
  • Household net income falls £0.23bn through reweighting.
• Against main, the CTR data is back but the total is not.
  • SPI CTR reports are identical to main's on every shared record. The only differences in who reports CTR come from the reselected band donors.
  • At main's weights, the CTR paid differs on fewer than 10 survey households.
  • The 2025 total is still £0.11bn (5.0%) below main's, and SPI rows are £0.12bn (25%) below, both through reweighting. In the data file, the weighted number of SPI people reporting CTR fell from 1.37m to 1.10m (−20%).
  • No target constrains CTR. The constituency calibration sets the published weights, and it sees CTR only through OBR's net council tax receipts (UK, England, Scotland and Wales), which subtract it.

What to read into these:

• CTR is 5% below main, through reweighting (above).
• UC falls £0.26bn. The SPI rows' changes (zeroed reports and redrawn take-up flags) remove £3.42bn at main's weights. Reweighting puts back £3.16bn: £1.65bn on FRS rows and £1.51bn on SPI rows, which now claim UC through the redrawn flags. UC has OBR spending and DWP caseload targets.
• AFCS falls 60%. The QRF had put 62% of it on SPI rows. Whether £0.95bn is closer to the truth than £2.40bn hasn't been checked against MoD statistics here, and no target is set on it.
• Bereavement support rises 35%. Restoring donor values fixed the imputation: SPI rows report it at 1.06x the FRS rate by record. By weight, though, it's 3.8x, and no target constrains it. Of the +£0.21bn, +£0.09bn is the restored donor values at main's weights and +£0.11bn comes from reweighting.
• ESA (contributory) caseload stays about 100k above DWP's 620k in both builds (726k and 721k). FRS rows now carry all of it.
• DLA rises 4.7% though no DLA amount changes. Its data change at main's weights is zero, so the +£0.39bn is reweighting. No target is set on DLA.
• Child AHC poverty rises 0.83pp in 2025 (0.48pp in 2026). I haven't broken this down by benefit or by reweighting.

Interactions

• Calibrate Housing Benefit to DWP's GB figures by age group #490 adds a pension-age exclusion to would_claim_uc in create_frs. Whichever of Calibrate Housing Benefit to DWP's GB figures by age group #490 and Stop SPI-synthetic rows carrying benefit claims nobody observed #514 merges second must apply the same exclusion to the SPI redraw, or Calibrate Housing Benefit to DWP's GB figures by age group #490's test_built_dataset_has_no_pension_age_uc_claimants fails. This is noted on Calibrate Housing Benefit to DWP's GB figures by age group #490.
• Solve Pension Credit take-up over entitled benefit units #510 overwrites would_claim_pc for every unit after imputation, filling SPI rows like non-reporters. So it supersedes this PR's PC redraw, and its spi_synthetic_benunits exclusion becomes redundant.
• Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499 (SPI-synthetic rows carry imputed reported council tax reduction on every person #497) zeroes council_tax_benefit_reported on SPI rows by taking it out of the QRF chain. After ruling d821 this PR keeps CTR as drawn, so CTR zeroing lands with Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499. Both PRs define SPI_DONOR_ZEROED_PERSON_VARIABLES, so Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499 must be reconciled when it rebases onto this one, and test_rule_sets_are_pinned pins CTR as kept, so it fails until Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499 updates it. Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499's test asserting that the zeroed and imputed lists don't overlap contradicts this PR's design for the other columns, so whichever merges second drops that assertion. Both add hypothesis>=6.168.3 the same way.
• Seed the calibration weight dropout so builds are reproducible #507: both builds above include its seeded calibration dropout (applied locally, not in this branch) and the same cached target downloads, so the comparison isn't swamped by run-to-run noise.

Not in this PR

• Council tax reduction on SPI rows. Per ruling d821, this PR leaves CTR as drawn. Zeroing it, with a household-level CTR imputation that gives SPI rows a claim route, comes later alongside Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499.
• Donor work status. SPI rows keep the donor's employment_status, hours_worked and the ESA/JSA proxy flags built from them, which can contradict the imputed incomes.
• Other donor copies. Healthy Start vouchers and free school meals are still copied from the donor. ssmg_reported is annualised ×52 on a one-off grant; that's harmless, because only > 0 is read.

Reviews:

• Round 1 (Opus, 96672af): REQUEST_CHANGES, addressed in eb14fd0.
• Round 2 (eb14fd0): APPROVE. Its remaining comments are handled in 90377e0, which drops the small-cell comparator and adds tests for the take-up year and the stage-two UC redraw. Calibrate Housing Benefit to DWP's GB figures by age group #490's exclusion is an integration requirement and is recorded on Calibrate Housing Benefit to DWP's GB figures by age group #490.
• deb16e9 (ruling d821) moves CTR from zeroed to kept and adds its tests. It has not been re-reviewed yet.

axiom: n/a: dataset construction in uk-data; no policy rule encoded.

🤖 Generated with Claude Code

[MaxGhenis]

Hand-off to the UK hub (owner session c56de936 / e09285c9 is closing).

• Head: 90377e06575cd8f057ac3d449a8df25c875957a9. Ready for review, MERGEABLE, all 4 checks pass (gh pr checks exit 0).
• Reviews:
  • Opus r1: REQUEST_CHANGES at 96672af, all fixed in eb14fd0.
  • Opus / GPT-6.1 Sol r2: APPROVE at eb14fd0 (~/reviews/uk-data-spi-reported-2026-10-01/review/review-514-2.md). Its remaining comments are handled in 90377e0.
  • UK hub's batch review (Sol): APPROVE WITH NITS at 90377e0 (~/reviews/uk-hub/batch-reviews/ud514/review.md). Both nits were in the PR body and are fixed: an exact evidence path with provenance, and the take-up check reworded to positive reporters. The code is unchanged, so no delta review was needed.
• Impact: real seeded builds with frozen targets; see "Measured with real builds" above. Evidence: ~/reviews/uk-data-spi-reported-2026-10-01/.
• Pending:
  • Max's d821 (merge Stop SPI-synthetic rows carrying benefit claims nobody observed #514). It lands only in the d833 uk-data batch.
• Integration order:
  • Calibrate Housing Benefit to DWP's GB figures by age group #490: whichever merges second applies Calibrate Housing Benefit to DWP's GB figures by age group #490's pension-age UC exclusion to the SPI redraw in apply_spi_donor_benefit_rules (noted on Calibrate Housing Benefit to DWP's GB figures by age group #490).
  • Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499: drop its test asserting the zeroed and imputed lists don't overlap.
  • Solve Pension Credit take-up over entitled benefit units #510: supersedes this PR's PC redraw.
  • Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499 and this PR add hypothesis>=6.168.3 the same way; the second to merge relocks.
• Follow-ups (chips): task_9b0a9391 (CTR claim route for SPI rows) and task_d1e33c0c (donor work status on SPI rows).

[MaxGhenis]

Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833).