Skip to content

UK local file: support is counted on the FRS host, but SPI-drawn incomes carry no tape-record id #1287

Open

Description

@vahid-ahmadi

What's wrong

The disclosure layer (PolicyEngine/policyengine.py#605) counts the support behind a published number by source_household_id. In the UK build that id names the raw FRS household of the family (uk_runtime/national_sampling.py:135-141), which is right for FRS values. But for the highest incomes, the values that drive a reform's change come from another licensed survey, the SPI Public Use Tape (UKDS SN 9422, uk_runtime/spi_income.py:75-79), and the local file carries no id for the tape record behind them:

  • The SPI support channel and the income-band donor rows are copies of FRS households (household_support_channel, household_is_spi_income_band_donor; uk_runtime/graph.py:828-848) whose incomes are drawn from the tape. The band donors cover total income from £200,000, a band edge set in the code (uk_runtime/spi_band_donors.py:114-150).
  • Their lineage columns (source_household_id, household_source_id, the clone index) all point to the FRS host. Nothing records which tape record, or how many, sit behind a seat's investment income.

So a seat estimate for a measure on dividends, savings or property income can pass the 10-household floor on FRS hosts while resting on fewer tape records, or on the same tape values drawn into several hosts. I could not check from the code whether one tape record can be drawn into more than one host. #1278 already asks whether the SPI agreement sets the floor at 30 rather than 10; this is the counting question behind it.

Evidence

Code references above, on main at c28b453. No data was read.

Proposed fix

  1. Export, per person, a key for the tape record each SPI-drawn value came from (an opaque hash of the tape's record reference is enough; the file is private), and a household-level flag for rows whose income came from the tape.
  2. In policyengine.py, let the layer count support on more than one key and apply the floor to each (a follow-up there once the column exists).
  3. Until then, publish measures whose change sits mainly on SPI-drawn rows (dividend, savings and property income rate changes, and the top of the income distribution generally) by region or nationally only, and say so in the method note (UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131). The readiness run in UK local: rerun the 2025 Autumn Budget constituency analysis on the local candidate to find data gaps and per-seat support before Budget day #1241 measures, privately, how much of each reform's change sits on SPI-flagged rows.

Priority

Medium-high. It does not block the readiness run, but it decides whether investment-income measures can go to seat level on 28 October. A decision (export the key, or region-only for these measures) is needed by about 24 October.

Related: #1241, #1278, #1276, #1131, #1240.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions