UK local: rerun the 2025 Autumn Budget constituency analysis on the local candidate to find data gaps and per-seat support before Budget day

@vahid-ahmadi

@vahid-ahmadi, could you take this one?

Summary

We want to publish constituency results for the 28 October 2026 Budget. #1131 leaves open whether they come from Microcosm's local release or from the eFRS constituency weights, as in 2025. Before choosing, we need to know two things for the policies a Budget typically touches:

how the local candidate behaves at seat level;
how many survey households sit behind each estimate.

The quickest way to find out is to rerun last year's constituency analysis on the local candidate and on the eFRS, and compare them.

The mansion-tax check on the 11 October candidate shows why this matters:

owner-occupied homes above £2m in England come from a few dozen survey households;
all but 3 of the 543 English seats have fewer than 10 such households behind them;
the eFRS passes the record checks only because it reweights the same households into every seat.

See #1112 and the HVCTS umbrella issue (#1240). Other policies may have the same problem in milder forms, and we'd rather find out now than on the day.

What to run

Reforms.
- The 2025 Autumn Budget measures in the dashboards: two-child limit repeal, fuel duty freeze extension, rail fares freeze, threshold freeze extension, dividend, savings and property income tax +2pp, student loan repayment threshold freeze, salary sacrifice NIC cap.
- The combined package.
- The HVCTS at £2m and £1.5m.
- Years 2026-27 to 2030-31, or at least 2026-27 and 2029-30.
Code. Use the 2026 dashboard's pipeline: PolicyEngine/uk-autumn-budget-dashboard-2026, src/uk_budget_data/pipeline.py, ConstituencyCalculator. Last year's local-area dashboard (PolicyEngine/Autumn-budget-local-area) has the same reforms if that's easier.
Datasets.
- The local candidate: score this exact build: staged/uk-local-candidate-f100-s42-20261011T200435Z-108c1225 in policyengine/populace-uk-private.
  - File: microcosm_uk_2024_25_local.h5, sha256 77456e20bc70d8d9116e47559a44de1226e942b68a3a8ad84dc79f5ddfacfcab. The local run directory is uk-k25-h100000-ngfes-f0.5-u0.0216.
  - This is not the national dataset microcosm_uk_2024_25.h5 used in UK: main_residence_value stops at about £2.8m (about £3.0m in 2026), so there are no homes in the £3.5m+ surcharge bands #1112, and not the eFRS. If a newer candidate is frozen before you start, use it and record its release id and sha256 in your summary.
  - The pipeline reads a 650 × N parliamentary_constituency_weights.h5. Until Microcosm exports its seat assignment in that format (weight-matrix export, PR to be linked), a group-by on constituency_code_oa with household_weight gives the same seat totals.
  - policyengine-uk holds household_id as int32, and Microcosm's clone ids wrap. Join engine output to the H5 by row position, not by id.
- The eFRS (policyengine-uk-data 1.58.0) with its constituency weights, as the 2025 baseline.
Other constituency analyses. Add any other constituency analysis we published in 2025 or plan for this Budget, for example the two-child-limit comparison with official statistics mentioned in UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131.

What to measure

For each reform, year and seat (and local authority, if cheap):

Support:
- distinct survey households whose net income changes, behind each seat estimate;
- effective records among those households;
- the largest single household's share of the seat's total change.
Clones and copies of one survey household count once: use Microcosm's source_household_id. The eFRS doesn't export a source id, so use an attribute fingerprint for now.
Disclosure pass rate:
- how many seats pass the rules below;
- how many still pass after secondary suppression against the national and regional totals and against the package's other components.
Uncertainty: 90% intervals from resampling survey households with weights held fixed. Report:
- the median relative half-width;
- how many seats have an interval crossing zero.
Say clearly that this understates uncertainty, because calibration isn't re-run.
Comparison:
- rank correlation by seat between Microcosm and the eFRS for each reform;
- where they diverge most, as a count of seats and a region, not seat-level values;
- independent benchmarks where they exist, such as official two-child-limit statistics. HMRC income by constituency is a calibration target, so it isn't independent.
Data gaps: the reforms and subpopulations with thin support or implausible patterns, and the characteristic each depends on that isn't anchored locally. Examples are property values, investment income, salary sacrifice, student loans and fuel spending. Map each to an existing issue (UK: main_residence_value stops at about £2.8m (about £3.0m in 2026), so there are no homes in the £3.5m+ surcharge bands #1112, UK local build (K=25, 2026-10-06): data-quality defects on the target surface and the spine #1123, UK spine: composition-faithful SPI income draw — bridge predictors and rank-preserving replacement (with-children UC residual) #840, UK spine: housing benefit £2.7bn against OBR £12.1bn — pensioner and supported-accommodation HB missing after the UC migration #867, WS-C deferred adjudications and the reconciliation layer #736, UK: supply the engine's local_authority input from the ladder's local_authority_code, plus council tax follow-ups from policyengine-uk#1855/#1864 and uk-data#482/#484 #953) or file a new one.

Deliverables

Private: the full seat-level results, unsuppressed, kept in a private repository (for example PolicyEngine/uk-mansion-tax-microdata, or a new private Budget-analysis repo) or locally. Never in a public repository.
Public summary (a comment on this issue), aggregates only:
- for each reform and year, the count of seats passing and failing the checks;
- the median interval width;
- the Microcosm–eFRS rank correlation;
- the gap list with links.
A recommendation: which reforms can be published at seat level on Budget day, which only by region or nationally, and which data fixes would change that. This feeds the suppression rule still open in UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131.

Ideally by 20 October, so the findings can shape the frozen candidate build and the dashboard's suppression rules.

Disclosure control: please read before posting anything

Every output here derives from UKDS-licensed microdata (FRS, WAS, LCFS, SPI and others). The UKDS End User Licence (CD137, clause 8, with CD171 §5.2.1) binds every output. This issue and its comments are public, and so are the dashboard repos. Posting here, committing, or uploading CI logs or artifacts all count as publishing.

Aggregates only. Publish only weighted totals or means over at least 10 distinct survey households (clones and copies count once). Write "<10" for anything smaller.
No unit-record values. No maximum or minimum, no top-k values, no single household's income, value or weight, and nothing that shows where a distribution ends, for example that a value band is empty.
No dominant cells. Don't publish a cell where one survey household is more than half of the total.
No differencing. A new number combined with already-public numbers must not isolate a small group or narrow a unit record. Watch out for:
- two runs under different uprating factors or thresholds;
- a total minus published parts;
- a package minus its other components;
- shares that sum to one with one part hidden;
- before and after totals.
Support figures in bands. Publish counts of survey households per seat only as <10, 10–29 or 30+, never exact.
Seat-level values stay private. No seat tables, maps or screenshots in public threads. A map colour is a value: a public map must hatch withheld seats (as in the shareable mode of the PolicyEngine/uk-mansion-tax-microdata dashboard), not colour them.
CI. In a public repo, don't print unsuppressed outputs in workflow logs or upload them as artifacts.
Edit history. GitHub keeps edit history for issues and comments, so editing doesn't remove a value; delete and re-post instead.
Review. When unsure, ask before posting, and have a second person check anything new. analysis/disclosure.py in PolicyEngine/uk-mansion-tax-microdata has a validator for result tables.

Links

UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131 (constituency method note and the open suppression rule), UK local build (K=25, 2026-10-06): data-quality defects on the target surface and the spine #1123 (local build defects), UK: main_residence_value stops at about £2.8m (about £3.0m in 2026), so there are no homes in the £3.5m+ surcharge bands #1112 (high-value homes), the HVCTS umbrella issue (UK: data fixes to model the High Value Council Tax Surcharge (mansion tax) correctly, and the related housing and council-tax gaps #1240).
High-value housing facts for the UK local calibration: HM Land Registry Price Paid Data, UK HPI, ONS house price statistics by constituency, the National Statistics Postcode Lookup, and the OBR High Value Council Tax Surcharge costing chronicle#343 (Land Registry and house-price facts).
PolicyEngine/uk-autumn-budget-dashboard-2026, and the private PolicyEngine/uk-mansion-tax-microdata (mansion-tax check, disclosure tooling and hex-map prototype).

Status update on the readiness run. No survey data has been read yet, so there are no seat-level or support figures in this comment.

Done

Code readiness. Read the 2026 dashboard pipeline (pipeline.py, ConstituencyCalculator), the disclosure layer from Add disclosure control and uncertainty for UK geography outputs policyengine.py#605, and Microcosm's and the eFRS's lineage code, against what the comparison needs: loading the local file, seat assignment, weights, years 2026-27 to 2030-31, the nine 2025 measures, the package, and the HVCTS at £2m and £1.5m.
A private script and runbook for the comparison. It uses the dashboard's own reforms and simulation builder, reads constituency_code_oa, region_code_oa and source_household_id from the H5 by row position (never by household_id, UK clone stages can number identifiers past the 32-bit range policyengine-uk stores #1194), runs every reform, year and seat through the layer with the package and region nesting in one release, applies secondary suppression, and writes the audit and summaries to a private directory outside any repository. It has been checked on synthetic arrays only. It is not in any repository.
Four issues filed for what would stop seat results passing on Budget day:
- #24 constituency PR: what the pipeline needs to score the local file and the eFRS through the disclosure layer uk-autumn-budget-dashboard-2026#43: what the pipeline needs to score the local file and the eFRS through the layer (the concrete list for Certified populace-us persists 18 formula-owned variables eCPS doesn't (formula_owned_export gate never wired into the build) #24's constituency PR).
- Disclosure release: area codes that don't match skip the nesting and prior-public checks without an error policyengine.py#625: mismatched area codes (parent_of, prior_public, and three spellings of the UK regions) skip the nesting and prior-public checks without an error.
- eFRS source_household_id doesn't name the survey household: SPI copies and capital-gains clones count as separate households policyengine-uk-data#591: the eFRS source_household_id doesn't name the survey household, so SPI copies and capital-gains clones count separately or the layer refuses the file.
- UK local file: support is counted on the FRS host, but SPI-drawn incomes carry no tape-record id #1287: support on the local file is counted on the FRS host, but SPI-drawn incomes carry no tape-record id.

Blocked

The data comparison itself. Both the local candidate (policyengine/populace-uk-private, sha256 77456e20…) and the eFRS with its constituency weights are private, and this needs my own Hugging Face access, which isn't set up yet. I'll run the comparison once it is. If a newer candidate is frozen by then, the script records its id and sha256.

Ranked risks, before data

How thin a seat gets, without data:

Local file (one seat per row): a household's K copies (K = 25 in this run directory's name) are spread over the seats of its region. So the distinct survey households behind a seat for a measure scale with (survey households affected in the region × K) ÷ seats in the region, capped by K. Narrow measures fall below the floor of 10 quickly, and London and the South East, with many seats, thin fastest.
eFRS (650 × N matrix): every household in a nation carries weight in every seat, so the distinct count passes by construction. The effective sample (n_eff) and dominance rules are what bind.

From most to least urgent:

HVCTS at £2m and £1.5m. Already shown in UK: main_residence_value stops at about £2.8m (about £3.0m in 2026), so there are no homes in the £3.5m+ surcharge bands #1112 and UK: data fixes to model the High Value Council Tax Surcharge (mansion tax) correctly, and the related housing and council-tax gaps #1240: the £2m+ tail rests on a few dozen survey households and no property-value target anchors it locally. Seat level isn't viable on any current build; a lower threshold helps support only a little. Region or national only, labelled "owner-occupied main residences".
Dividend, savings and property income +2pp. Moderate incidence, but the change sits on the top of the income distribution, where incomes are drawn from the SPI tape. Dominance and the p% rule will bind, and support by tape record isn't countable yet (UK local file: support is counted on the FRS host, but SPI-drawn incomes carry no tape-record id #1287, UK SPI stage 1 keeps 4–17% of the tape's donors, and the QRF tail gate counts rows rather than draws #1276). Investment and rental income have no local target (WS-C deferred adjudications and the reconciliation layer #736 for rental income). Likely region only.
Two-child limit repeal. The lowest incidence of the benefit measures: households on UC with three or more children, concentrated in a minority of seats. It will be thin in many seats on the local file. But it is the one measure with an official seat-level series (DWP, UC claimants and the two-child limit, April 2025), so it is both the best independent check (UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131) and the first candidate for a modelled seat estimate if the direct one fails.
Salary sacrifice cap (2029-30 only). HMRC puts those sacrificing more than £2,000 at about 3.3 million of 7.7 million users (policy paper). The broad-base haircut gives every employee a small change, so the count floor passes while the signal sits on the capped minority. Salary sacrifice has no local anchor (UK pensions: calibrate State Pension, Pension Credit and pension contributions to the resident administrative facts (PolicyEngine/chronicle#305), pin the State Pension age boundary, fix the pension spine stages, and the engine pieces a State Pension uprating analysis needs #1069).
Student loan threshold freeze. Broad among younger graduates. But the Plan 2 flag captures current repayers, not the liable stock (UK spine: SLC plan-2 borrower stocks at a third of SLC's — the FRS flag captures repayers, not the liable stock #868), so borrowers just below the threshold, whom a freeze pulls in, are missing. Age is the only local anchor.
Fuel duty and rail fares freezes. Broad and safe on disclosure. But fuel and rail spending are imputed with no local target (UK consumption-tax levels: open items after #1167 #1207), so seat differences come only from who lives there. A validity risk, not a disclosure one.
Threshold freeze extension and the combined package. Broad, and income is a local target (HMRC SPI by constituency, so not an independent check). The package needs the package-against-components check in the same release (#24 constituency PR: what the pipeline needs to score the local file and the eFRS through the disclosure layer uk-autumn-budget-dashboard-2026#43).

Gaps between the asks and what landed (relevant to the Budget run)

Add disclosure control and uncertainty for UK geography outputs policyengine.py#605 against UK geography outputs: disclosure control and uncertainty for region, constituency and local-authority impacts policyengine.py#589. As agreed, it defers the modelled seat estimator, replicate weights and automatic secondary suppression. It documents that three households pooling what they know aren't protected. Secondary suppression therefore has to be a loop in the dashboard. That loop must read Finding.suggested as a component name for package findings and an area otherwise (#24 constituency PR: what the pipeline needs to score the local file and the eFRS through the disclosure layer uk-autumn-budget-dashboard-2026#43).
The floor. Whether the SPI agreement sets it at 30 rather than 10 is still open (UK local surface: remaining sub-national items (consolidation) #1278). The script takes --min-sources 30 to test both.
Check every public file for disclosure, and keep legacy constituency rows private uk-autumn-budget-dashboard-2026#30 against Regional and constituency results under disclosure control: noise, intervals and status-aware maps uk-autumn-budget-dashboard-2026#24. It is the first of that issue's three PRs. Its open review points are on its own thread. The regional and constituency PRs are where #24 constituency PR: what the pipeline needs to score the local file and the eFRS through the disclosure layer uk-autumn-budget-dashboard-2026#43 and Disclosure release: area codes that don't match skip the nesting and prior-public checks without an error policyengine.py#625 land.

🤖 Generated with Claude Code

@juaristi22

@juaristi22 the comparison has run. Everything below is an aggregate over seats; the seat-level audit stays private.

Setup

Local candidate uk-local-candidate-f100-s42-20261011T200435Z-108c1225 (sha matches the issue).
eFRS from policyengine-uk-data 1.58.1 with its constituency weights. Those only have a 2025 key, so the 2025 calibration is reused for every year.
policyengine 6.4.1, which already has the Health take-up wiring + certified-stack computability gate (real residuals from #604) #605 layer.
12 reforms, 2026-27 to 2030-31, 1,000 bootstrap replicates, noise off.
The £1.5m HVCTS band uses a placeholder charge of £2,500.

Verdict by reform

Reform	Seat level?
Threshold freeze extension (2028+)	Yes on both datasets, and they agree (rank correlation about 0.84)
Salary sacrifice cap (2029+)	Yes on both; agreement about 0.65
Combined package	Yes in 2029-30. On the local file it's weak in 2027-28, where 56% and 27% of seats are withheld.
Fuel duty freeze	Passes disclosure on both, but the two datasets barely agree (about 0.07). This is a validity problem: fuel spending has no local target.
Dividend, savings and property +2pp; student loan freeze; two-child limit; rail fares	Region only. The local file reports almost no seats. The eFRS reports more, but only because every household carries weight in every seat in its nation.
HVCTS at £2m and £1.5m	National only. The local file has no more distinct £2m+ survey households than the eFRS. Its few passing seats come from copies of the same households.

Share of affected seats that are reported / approximate / withheld (local ; eFRS), 2029-30

Reform	Local	eFRS
Threshold freeze	92 / 0 / 8	98 / 0 / 2
Salary sacrifice	71 / 0 / 29	96 / 0 / 4
Package	93 / 0 / 7	100 / 0 / 0
Fuel duty	68 / 8 / 24	93 / 1 / 6
Dividend +2pp	0 / 31 / 69	84 / 5 / 11
Savings +2pp	0 / 28 / 72	49 / 24 / 27
Property +2pp	0 / 34 / 66	60 / 34 / 6
Student loan freeze	0 / 14 / 86	61 / 25 / 14
Two-child limit	0 / 0 / 100	70 / 1 / 29
Rail fares	4 / 65 / 31	78 / 10 / 13
HVCTS £2m	0 / 1 / 99	0 / 1 / 99

Gaps found

The eFRS has a third stacked layer in 1.58.1, the CGT band donors. Without undoing all three offsets, survey ids fall back to a fingerprint. This adds to policyengine-uk-data#591, which Consumer-existence guard: every pool input must earn its imputation; drop deprecated marketplace leaf (#578 inc 2) #597 fixes.
Some recoverable_parent findings are left over. A region is withheld while all its seats are published, and the layer suggests nothing. There are a handful on the local file and up to a few dozen per year on the eFRS. Each costs about one more seat, but the dashboard needs a rule for choosing it (uk-autumn-budget-dashboard-2026#43).
The package knock-on is large. Checking the package against its components withholds thin components wherever the two-child limit is withheld. On the local file in 2026, that alone withholds most rail seats and over a quarter of fuel seats. So measures that pass on their own fail when released beside the package. It may be better to release the package and components in separate releases, or the package at region level only.
The two datasets rank seats very differently for most targeted measures. Rank correlations are near zero for two-child, fuel, rail, savings and property. Before choosing a source for UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131, this needs an external check. The DWP two-child figures by constituency are the obvious one, and that check hasn't been run yet.

Suggested next steps before 28 October

Decide UK: two-page note on how constituency impacts are estimated (draft for the Budget-day analysis) #1131 per measure: seats for the threshold freeze, salary sacrifice and the package; regions for the rest; national only for HVCTS.
Run the DWP two-child check by seat on both datasets.
Add the recoverable_parent rule and a decision on the package release structure to Make US source stages manifest-driven #43 / Track US geography and local-area source support #48.

🤖 Generated with Claude Code

@juaristi22

@juaristi22 an addendum to the comparison above, after finding 1 of the dashboard #48 review.

The question. Before policyengine-uk-data#597, an SPI copy's clones could carry the stacked id rather than the FRS original's. A check that only asks whether ids collapse clones then counts the copy as a separate survey household. The review said the readiness script has the same gap, and it was right about the script: its check would have accepted a stacked file.

The local file is not affected, so the results above stand. All checks were on the household table, aggregates only:

In Microcosm's spine, the SPI channel and CGT support copies keep the FRS household's source_household_id. Only the row id is remapped, and the channel is recorded in household_support_channel and the copy flags.
Every distinct source_household_id in the local file is a raw FRS household id: 100% fall in the set of unflagged eFRS 1.58.1 household ids. None carries a stacking offset.
The Consumer-existence guard: every pool input must earn its imputation; drop deprecated marketplace leaf (#578 inc 2) #597 orphan rule fails on this file, but for a different reason. About 41% of rows are copies, and for about 18% of distinct ids the draw holds only copies, not the FRS original. The local file samples households from the full population, so a copy can be drawn without its host. The id is still the host's, so it is still counted once.
Copies whose original is in the draw match it on age and sex composition, household size and region in 100% of rows. Pairing them with a random original instead matches in under 0.1%. Tenure differs for some SPI copies, as expected: the SPI housing step re-imputes it.

Rerun. The corrected id would be the same vector the run already used, so the 2029-30 local tier shares are unchanged and no verdict changes.

Fix to the script. The private readiness script now adds two checks to the "collapses clones" test. First, every source id must be an unflagged FRS household id. Second, copies must match their original. It passes on this file and refuses a test copy where the SPI copies are given stacked ids. The plain orphan rule would wrongly refuse a sampled file like this one.

🤖 Generated with Claude Code

Navigation Menu

Repository navigation

UK local: rerun the 2025 Autumn Budget constituency analysis on the local candidate to find data gaps and per-seat support before Budget day #1241

Summary

What to run

What to measure

Deliverables

Disclosure control: please read before posting anything

Links

Done

Blocked

Ranked risks, before data

Gaps between the asks and what landed (relevant to the Budget run)

Verdict by reform

Share of affected seats that are reported / approximate / withheld (local ; eFRS), 2029-30

Gaps found

Suggested next steps before 28 October

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

UK local: rerun the 2025 Autumn Budget constituency analysis on the local candidate to find data gaps and per-seat support before Budget day #1241

Description

Summary

What to run

What to measure

Deliverables

Disclosure control: please read before posting anything

Links

Activity

vahid-ahmadi commented on Oct 15, 2026

Done

Blocked

Ranked risks, before data

Gaps between the asks and what landed (relevant to the Budget run)

vahid-ahmadi commented on Oct 15, 2026

Verdict by reform

Share of affected seats that are reported / approximate / withheld (local ; eFRS), 2029-30

Gaps found

Suggested next steps before 28 October

vahid-ahmadi commented on Oct 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions