Default the BRMA from the household's region

[MaxGhenis]

Fixes #2020

What changes

brma gets a formula. A household with no BRMA input is placed in its region's default BRMA, REGION_DEFAULT_BRMA in variables/household/BRMA.py. Before this, it got Maidstone, whatever the region. Explicit inputs still win: policyengine-core only runs a formula for a period that has no input.

For a single 40-year-old private renter in 2025 (LHA category B), the annual LHA rate moves as below. The rates are the published ones that #2022 brought in, computed on this branch at bf27e88 with each BRMA set explicitly. Lothian's and Maidstone's one-bedroom rates are both £172.60 a week.

Region (no BRMA input)	Before (Maidstone)	After
London	£8,975.20	£15,503.80 (Inner South East London)
Scotland	£8,975.20	£8,975.20 (Lothian)
Wales	£8,975.20	£7,778.68 (Cardiff)
Northern Ireland	£8,975.20	£7,245.68 (Belfast)

Which BRMA each region gets

Each region gets its BRMA with the most private-rented households, counting only the part of the BRMA inside that region. The counts are in parameters/gov/dwp/LHA/brma_private_rented_households.csv.gz (936 rows; columns region, brma, bedrooms, households; the uncompressed CSV's sha256 is fd40dae019e5…c2ae1f, identical to policyengine-uk-data#516 at d3b5ba0). Region.UNKNOWN keeps Maidstone. The table is written out explicitly in code, and test_brma_region_default.py recomputes it from the CSV.

Region	Default BRMA	Private-rented households (share of region)	Runner-up (share)
North East	Tyneside	70,245 of 202,564 (34.7%)	Teesside (21.6%)
North West	Central Greater Manchester	111,363 of 606,926 (18.3%)	Greater Liverpool (12.0%)
Yorkshire and the Humber	Leeds	76,842 of 452,617 (17.0%)	Bradford & South Dales (11.3%)
East Midlands	Leicester	82,461 of 380,100 (21.7%)	Nottingham (18.2%)
West Midlands	Birmingham	107,825 of 435,828 (24.7%)	Black Country (17.7%)
East of England	Central Norfolk & Norwich	46,636 of 478,624 (9.7%)	South West Herts (8.2%)
London	Inner South East London	119,127 of 1,025,579 (11.6%)	Outer North London (9.6%)
South East	Southampton	50,400 of 730,770 (6.9%)	Brighton and Hove (6.5%)
South West	Bristol	74,829 of 482,150 (15.5%)	Bournemouth (10.7%)
Wales	Cardiff	35,780 of 228,601 (15.7%)	Swansea (8.3%)
Scotland	Lothian	62,457 of 323,001 (19.3%)	Greater Glasgow (15.9%)
Northern Ireland	Belfast	26,599 of 132,449 (20.1%)	Lough Neagh Upper (20.0%)

Sources. Private-rented households (private landlord or letting agency, plus other private rented) from four censuses, each mapped to BRMAs:

Census	Area unit	Mapped to BRMAs by
England and Wales 2021 (ONS TS054)	LSOA	Population-weighted centroids in the VOA (England) and Rent Officers Wales BRMA boundaries
Scotland 2022 (NRS)	Ward	Output-area household shares in the Scottish Government BRMA boundaries
Northern Ireland 2021 (NISRA)	Postcode district	NIHE postcode-district BRMAs, times Northern Ireland's private-rented share (OGL sources only)

• Northern Ireland's census has no bedrooms question, so its rows use bedrooms = all. The defaults sum over bedrooms, so this doesn't affect them.
• The build (download and build scripts with every input pinned by sha256) is in policyengine-uk-data#516, which fixes the imputation that issue policyengine-uk-data#515 describes.
• Two near ties:
  • South East: Southampton 6.9% against Brighton and Hove 6.5%.
  • Northern Ireland: Belfast leads Lough Neagh Upper by 79 households. Its split applies one Northern Ireland-wide private-rented share to every district, so it follows the distribution of all households. An earlier version split by postcode-level census counts under the LPS licence and had Belfast ahead by 7,242. That licence doesn't clearly allow publishing derived figures, so a312679 replaced those rows.

Why not the list of rents. The first version of this PR used the BRMA with the most entries in lha_list_of_rents.csv.gz. policyengine-uk-data#515 found that every Scottish, Welsh and Northern Irish list in that file is a copy of an English BRMA's list, so those entry counts are English sample sizes. I checked this here: all 480 of those (BRMA, category, year) lists match an English list from the same year, sorted rent for rent. That version had made Scotland's default Aberdeen and Shire. The census changes three defaults:

• Scotland: Aberdeen and Shire to Lothian;
• Yorkshire: Sheffield to Leeds;
• East of England: Peterborough to Central Norfolk & Norwich.

Inputs carry forward

Before this PR, a brma input for one year carried into later years through auto_carry_over_input_variables. policyengine-core only does that when a variable's formula returns nothing: see the if array is None: branch in Simulation._calculate. A variable with a formula would therefore drop an earlier input in later years. That would affect household calculations that set BRMA for one year, and microsimulation years after the dataset's last projected year (2030).

The formula restores this, reading only values already stored:

• A year's region is the last region known at or before it, or region's default (London) if there is none. That is how region's own inputs carry forward.
• The latest earlier BRMA is the latest year whose BRMA is readable on this branch.
• Carry or reset: the BRMA carries forward unless some region known after that year, up to this one, differs from the region then. In that case the household has moved and gets its new region's default.

brma never calculates region, or an earlier year's brma. Two earlier versions did, and the independent reviews caught both:

• Round 2: calculating region for an earlier year could cache London there, because core's carry-over gives region's default for a year once a later year is known.
• Round 3: calculating this year's region had the same effect on earlier years, through core's carry-over. For example, with region set for 2024 and nothing else, calculating brma for 2026 changed 2025's income tax.
• Round 3: a BRMA stored only on another branch was recalculated year by year until core's spiral limit gave Maidstone.

Gaps of any length work, because the latest earlier year is read directly.

Test pins that changed

Both moved only because the household sets no BRMA and is in London (set explicitly, or by default):

1. tests/policy/baseline/finance/benefit/family/LHA.yaml, "BRMA default value": the household sets no region, so it is in London.
   • brma: MAIDSTONE to INNER_SOUTH_EAST_LONDON.
   • BRMA_LHA_rate (2020, category C): 9,753.12 to 16,155.36. That is VOA's April 2020 two-bedroom rate for Inner South East London, £310.68 a week; Maidstone's is £187.56.
2. tests/policy/integration/entitledto_scenarios.yaml, EntitledTo Add contributory JSA #1 (single 30-year-old, London, private renter): the household gets the Universal Credit shared accommodation rate published from April 2024 and held for 2025-26. That is £650.00 a month for Inner South East London, against £444.83 for Maidstone.
   • universal_credit: 3,539.64 to 6,001.68.
   • household_net_income: 14,165.10 to 16,627.14.
   • Both rise by £2,462.04, which is (£650.00 − £444.83) × 12. These match the values the Use published LHA determinations and apply Sch 3B paras 3 and 3A #2022 session computed with brma: INNER_SOUTH_EAST_LONDON set explicitly.

New tests

• tests/policy/baseline/household/brma.yaml:
  • London, Scotland, Wales and Northern Ireland households without a BRMA.
  • Explicit inputs inside and outside the region win.
  • An unknown region keeps Maidstone.
  • All 13 regions in one multi-household case.
  • These pin brma only, not LHA rates, so they don't move when Use published LHA determinations and apply Sch 3B paras 3 and 3A #2022 changes the rates. Run against main, the region-default cases fail, because main gives MAIDSTONE.
• tests/test_brma_region_default.py:
  • The table equals the census CSV's per-region mode, each mode is a strict maximum, and the table covers every region but UNKNOWN. The CSV covers every BRMA in BRMAName.
  • For each of the 13 regions, the default gives the same BRMA_LHA_rate and uc_LHA_cap as setting that BRMA explicitly. This is a differential check between the formula path and the input path.
  • Explicit cases:
    • carry-forward, including when region was input only in an earlier year;
    • no carry-back;
    • moving region, including when the BRMA's own year has no known region;
    • leaving a region and returning, in either order;
    • the reviews' repros, as regression tests: calculating brma, with or without a BRMA input, leaves region and 2025 income tax unchanged; region carries forward until its next input; a BRMA stored only on another branch isn't carried (11 years, beyond core's spiral limit).
  • Two Hypothesis property tests (40 derandomised examples each), each over 1-4 households and up to three BRMA inputs, calculated in a random order:
    • regions input every year (moves and returns included);
    • regions input in only some years; this one also checks that every year's region is unchanged.

Invariants

The rule: in a year with a BRMA input, brma is that input. Otherwise, if there is an earlier input and the household's region has been the same in every year from the latest such input to this year, brma is that input. If neither holds, brma is the default for this year's region. Each year's region is the latest region input at or before it, or London if there is none.

1. The rule holds in any order of calculation. It holds for any households, any regions by year (moves and returns included) and any BRMA inputs. Like core's own input handling, it assumes every household has BRMA inputs in the same years (or none) and region inputs in the same years: core fills a household missing from a year's input with the default.
2. Calculating brma never changes region for any year, nor anything that depends on it.
3. The table is the census CSV's strict per-region mode, and the default gives the same LHA rates as inputting it.

Two Hypothesis property tests check invariants 1 and 2, with regions input every year or in only some years. Each runs 40 derandomised examples with a random order of calculation. Mutation checks:

• the any-order property test fails if the formula ignores region changes, drops carry-forward or carries from the earliest year instead of the latest;
• the table test fails if Scotland goes back to Aberdeen and Shire;
• each earlier formula fails the regression tests added after its review.

Downstream

• API and app. policyengine-api's add_yearly_variables (services/household_calculation_service.py) fills every variable a household leaves unset. Input-only variables get their default value; variables with a formula are sent as null and calculated. On main, every API household calculation therefore sent brma: MAIDSTONE explicitly. With this PR, isInputVariable for brma becomes false, so the API sends null and the formula sets the BRMA from the household's region. The app-v2 household builder renders basicInputs without checking that flag (libs/metadataUtils.ts, getBasicInputFields), so users can still choose a BRMA. The app's starter household sets neither region nor brma. It is in London by default, so its LHA rates move from Maidstone's to Inner South East London's.
• Exports. policyengine-core's to_input_dict() and to_input_dataframe() export only input-only variables by default, so they drop brma once it has a formula, as they already do for country and current_education. policyengine-uk's own UKSingleYearDataset.from_simulation uses simulation.input_variables and still keeps an input brma; checked on this branch.
• Microsimulation. Single-year datasets are extended to 2030 (extend_single_year_dataset) and set brma for every household in each year, so the formula only runs after 2030, where it carries the 2030 value forward. Checked: I ran main (7b9fc37) and this branch on the enhanced FRS 2024-25 (policyengine-uk-data 1.57.4), whose years run 2024 to 2030. I ran e77f3c7, 15eeb50, d1b0ade and 14eb69d of this branch. After merging main, I re-ran d0bebbc against main 2cbf20c (core 3.32.13): all 24 arrays are still bitwise identical. In 2025, 2030, 2031 and 2034 these arrays are bitwise identical to main: brma, BRMA_LHA_rate, LHA_cap, universal_credit, housing_benefit and household_net_income. In 2031 and 2034, brma comes from the formula's carry-forward and matches core's carry-over on main.
• microcosm. Its release gates accept formula-owned stored columns (release_input_coverage.py). The pinned efrs_parity_reference.json lists 13 formula-owned overrides; regenerating it with this release adds brma, and two tests that expect 13 will need updating.
• policyengine-uk-data. Not affected: it writes brma explicitly. Its BRMA imputation still samples the list of rents; policyengine-uk-data#515 tracks moving it to the same census table.

Interaction with #2022 (published LHA rates)

#2022 merged on 4 October, and it changed the same two pins. The merge with main, bf27e88, resolves both conflicts to Inner South East London's published rates, shown above. The new brma.yaml doesn't pin rates, and the rate differential test compares the default with an explicit input under whichever rates are in force, so neither needed re-pinning.

Known limits

• Partial inputs. In a situation with several households where only some set brma or region in a year, policyengine-core fills the others with the variable's default (Maidstone, London) for that year, and the formula doesn't run for them. A filled-in Maidstone then carries forward like an input. Main behaves the same way.
• Core carry-over, now fixed. Core used to return a variable's default when any later period was already known. For example, with region input for 2025, calculating 2040 before 2026 gave London for 2026, and brma, which reads the region inputs, could then disagree with region. Carry over only inputs, the latest at or before the requested period policyengine-core#562 fixed it in the core main now locks, 3.32.27. On this branch, 2026 now comes out Scotland in either order.
• Inputs set after construction. A brma set with simulation.set_input after the simulation is built is dropped on the marginal-tax-rate branches: marginal_tax_rate and its employer-cost and capital-gains variants delete every array that is neither in the construction-time simulation.input_variables nor input-only. A brma given in the situation or dataset is unaffected. Main already does the same for every formula-backed input: there, country set this way is Wales on the simulation but Scotland on the adult_1_pay_rise branch. A general fix is a separate change.

Commands run

• ruff format --check . and ruff check .: clean.
• CI make test on e77f3c7 (first version): YAML 1,378 passed; pytest, with the microsimulation tests on the real dataset, 397 passed, 1 skipped, 1 xfailed.
• On the merge with main (core 3.32.13): the three changed YAML files plus the two marginal-tax-rate YAML files gave 24 passed. pytest policyengine_uk/tests/test_brma_region_default.py gave 26 passed.
• On the merge bf27e88 (main 3173baa, core 3.32.27):
  • brma, LHA, EntitledTo and the marginal-tax-rate YAML files gave 20 passed;
  • test_brma_region_default.py gave 26 passed;
  • the microsimulation differential was last run at d0bebbc against main 2cbf20c. The shared Mac was over its memory limit, so I didn't re-run it here.

axiom: n/a: input default for household calculations (BRMA location), not a statutory rule

🤖 Generated with Claude Code

[MaxGhenis]

Merge audit: squash-merged at head bf27e88.

• CI: all checks passed on bf27e88 (run 38089405456).
• Reviews: an independent GPT-6.1 Sol review approved d0bebbc (round 5; rounds 2 to 4 requested changes, all addressed). bf27e88 only adds the merge with main, whose two conflicted test pins an independent Opus 5.5 review approved. It checked them against lha_published_rates.csv.gz and with explicit-BRMA probes.
• Not covered by CI: Pay pre-2026 UC health element claimants the legislated protected amount #2248 and Count a loss on one source as nil in Pension Credit income and earnings #2259 merged to main after that CI run started. A trial merge was clean, and their tests don't reference BRMA, LHA or rent.
• Squash rather than a merge commit: intermediate commits held Northern Ireland census rows whose licence didn't clearly allow publishing derived figures; a312679 replaced them with OGL-only rows.
• Follow-up: a brma set with set_input after construction is still dropped on the marginal-tax-rate branches, as main already does for every formula-backed input.