Repository navigation
Conversation
The two area lists carried each area's hex-map position in x and y columns. Nothing in the package reads them: every reader uses code, name and, for constituencies, country. Every other column, row and quoted name is unchanged. The new test checks each file's exact columns and that every row has as many fields as the header. It failed on both files before the columns were removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fit against the base build: pass
A drop of more than 2.0% fails. Notes
National targets in a worse error band on the head (8)
Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 4 failing on both sides, 0 fixed. Moves flagged for review: 138 (thresholds in Notes
Moves flagged for review
98 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (4)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #638
Draft: merge after the UK Budget on 28 October 2026.
axiom: n/a: data (area list columns only; no policy change)
Why
storage/constituencies_2024.csvandstorage/local_authorities_2021.csvcarried each area's position on a hex map inxandycolumns. Nothing in this package reads those columns, and the PolicyEngine app has its own hex-map layout files and does not read these lists. Removing the columns stops the lists carrying positions that nothing uses.What changes
storage/constituencies_2024.csv:code,name,x,y,region,countrybecomescode,name,region,country(650 rows).storage/local_authorities_2021.csv:code,x,y,namebecomescode,name(360 rows).tests/test_area_lists.py: each file has exactly the expected columns, and every row has as many fields as the header.changelog.d/area-lists-without-xy.removed.md. Merging it starts a data release, as for any fragment; the built datasets do not depend on these columns.Every other byte is unchanged. Checked against the original files:
,<x>,<y>put back after the field beforex, at the position where the fields before it end.csvand with pandas (all values as strings), the new files equal the originals withxandydropped.Who reads these files
In this repository, every reader uses
code,nameand, for constituencies,country: the constituency and local authority calibration and loss matrices (datasets/local_areas,calibration/matrix_builder.py), the long-format geography weights, the local H5 publisher, the target database load (db/etl.py), local Universal Credit matching (utils/uc_data.py), the validation notebooks and the tests. A search of the tests, docs, notebooks, tools, workflows and build scripts found no use ofxory.Outside this repository (checked in local checkouts of the other PolicyEngine repositories):
storagefolder and usescodeandname.policyengine-uk-data-privatestorage bucket only to find an area's row in a weight matrix, bycode. This repository's release uploads datasets and weights, not these two CSVs, so the bucket copies do not change.name.Older versions of .py and the Public API read
xandyfrom copies of these files on Hugging Face (policyengine/policyengine-uk-dataandpolicyengine/policyengine-uk-data-public) and in thepolicyengine-uk-data-privatestorage bucket. This repository's release does not write those copies, so this change leaves them as they are.Tests
Red-green, in one commit: the new test was written first and failed on both files while
xandywere present, then the columns were removed and it passed.uv run pytest policyengine_uk_data/tests/test_area_lists.py: 2 passed.test_la_council_tax_targets.py,test_la_loss_council_tax.py,test_local_uc_etl.py,test_local_uc_alignment.py,test_oa_region_assignment.py::TestRealCrosswalk): 58 passed, 14 skipped (they need the enhanced FRS dataset, which CI builds).ruff format --check .and the changelog fragment check pass.🤖 Generated with Claude Code