Skip to content

Stop counting rent paid by boarders and lodgers as their property income - #503

Merged
MaxGhenis merged 2 commits into
mainfrom
frs-cvpay-not-property-income
Oct 7, 2026
Merged

MaxGhenis merged 2 commits into
mainfrom
frs-cvpay-not-property-income

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

frs.py builds each person's property_income from three FRS amounts: SUBRENT, CVPAY and ROYYR1. CVPAY is not rent received. It is the rent a boarder or lodger pays the householder, and it sits on the boarder's or lodger's own adult record. So the build counted an expense as income for the person paying it.

This PR removes CVPAY from property_income and moves the calculation into a tested helper, frs_property_income.

Evidence that CVPAY is rent paid

Source: the documentation shipped with the FRS 2024-25 release this repo builds from (UK Data Service SN 9563).

  • Questionnaire (question instructions, "Rent within Conventional Households"). The block is asked about each person not related to the HRP in the second and later benefit units. ConvBL asks whether that person is a boarder, a lodger or neither. CvPay then asks "How much rent did [name] pay", for board and lodging, after deducting any state benefits to help with rent. CvPd is the period it covers.
  • Data dictionary (ADULT table). CVPAY is labelled as the amount of rent after state benefits for boarders and lodgers. The BENUNIT table's derived variables BOARDER and LODGER are labelled as rent paid by a boarder and by a lodger.
  • Derived-variable specification (FRS user guide, older release). BOARDER is the weekly amount a boarder benefit unit pays the householder. It is built from CVPAY, plus any housing benefit paid on top.
  • The data (raw FRS 2024-25 and 2023-24, aggregates). Every adult with CVPAY above zero is outside the HRP's benefit unit, is not the HRP, lives in a conventional household and is coded as a boarder or lodger. The weighted mean is about £110 a week in 2024-25.

The other inputs were checked against the same documents and are unchanged:

FRS variable Meaning Treatment
ROYYR1 Rent from other property in the last 12 months, before tax and after allowable expenses (question PropRent) Stays in property_income
ROYYR2, ROYYR3, ROYYR4 Royalties; income as a sleeping partner; overseas pension Stay in miscellaneous_income
SUBRENT Rent received for letting part of the home to someone outside the household (questions SubLet, SubRent) Stays in property_income for the HRP of owner households
TENTYP2 5 and 6 Owned with a mortgage (including part rent, part own); owned outright Unchanged

The FRS quality report says most monetary amounts are converted to weekly values, which matches the WEEKS_IN_YEAR annualisation here.

Invariants and tests

frs_property_income(person, household) is meant to have these properties. test_frs_property_income.py checks the first three on 200 seeded random sets of households with non-negative amounts, which is all the FRS records for these variables (Hypothesis is not a dependency of this repo):

  • CVPAY has no effect. Changing CVPAY never changes the result.
  • Conservation. Total property income equals WEEKS_IN_YEAR × (SUBRENT of owner households + all ROYYR1), so each amount is counted once.
  • Locality. Raising one person's ROYYR1 by £x raises that person's result by WEEKS_IN_YEAR × £x and nobody else's.
  • Non-negative. A negative amount is floored at zero (example test).

Five example tests cover a lodger's rent, rent from other property for a non-HRP adult, subletting rent going to the HRP, the floor at zero, and the repeated index labels that create_frs produces by stacking the adult and child tables. Putting CVPAY back into the helper fails 398 of the 605 tests.

Differential check. I built the base FRS on main (b45c373) and on this branch (db089ce; the later commit only adds tests) and compared every column of every table. Only property_income differs, on 43 records, and each of those falls to zero. The helper is otherwise the old inline code: the one behavioural difference is that a person whose household is missing from the household table now gets no SUBRENT where the old code raised KeyError.

Measured impact

All figures are aggregates. Simulations use policyengine-uk main (44240bd8).

Base FRS 2024-25 (survey weights, no imputation or calibration)

Stored values in the two base builds, before uprating: property_income falls from £16.98bn to £16.18bn (−£0.81bn, −4.8%). People with any property income fall from 2.01m to 1.87m (−138k, from 43 survey records). Summing CVPAY in the raw tables gives the same £0.81bn for 2024-25, and £1.23bn for 2023-24.

2026 main this branch change
Property income £18.44bn £17.57bn −£0.88bn
Taxable property income £16.54bn £15.80bn −£0.74bn
Income tax £297.27bn £297.14bn −£0.13bn

Means-tested benefits rise slightly for some of the people affected, and poverty rates barely move. Each of those changes rests on fewer than ten survey benefit units or households, so the figures are not shown.

Enhanced FRS (full rebuild)

I built main (b45c373) four times and this branch twice. All six builds used production settings (512 epochs, PE_UK_DATA_OA_CLONES=1), the same raw inputs, the same cached trained models (income, wealth, consumption, VAT, public services, salary sacrifice) and the repo's uv.lock (policyengine-uk 2.93.0).

A rebuild does not isolate the fix. Four things differ between a main build and a branch build:

  1. The fix. 43 FRS records lose their property income (86 rows once impute_capital_gains has stacked its copy of the dataset).
  2. Later stages that read income. Their outputs change for other records too. Comparing a main and a branch build row by row on the 52,576 households that are not capital gains donors:
    • Wealth. impute_wealth predicts wealth from household income, and uprate_property_by_region then rescales every household with a positive imputed home value in a region by one regional factor. Imputed property wealth differs for 21,722 households.
    • Stage-2 imputation. impute_frs_only_variables is fitted afresh in each build with property_income among its predictors. On the SPI-donor half, imputed benefit reports and pension contributions differ (for example reported Universal Credit on 878 person rows).
    • Capital gains and energy. Capital gains differ for 23,664 people and energy consumption for about 50,000 households. I have not traced those paths.
  3. A different draw of capital gains donor households. stack_cgt_band_donors picks 270 donor households with a seeded draw whose probabilities depend on each first adult's total income. Changing a few incomes changes the draw. All four main builds stack the same 270 households, and both branch builds stack another set. Only 58 of the 270 are common to both.
  4. Build-to-build variation. Two builds of the same commit are identical row by row except for household weights and the transport spending columns that are rescaled from them. I have not traced the cause. The second pair of builds (one main, one branch) also fell back to the stored Scottish council tax band distribution when its download failed.

So the table gives the range of the four main builds beside each branch build.

2026 main, range of 4 builds branch build 1 branch build 2
Property income £58.75bn to £59.39bn £59.30bn £58.64bn
Property income, FRS half £44.72bn to £45.23bn £45.10bn £44.40bn
People with property income 2.533m to 2.551m 2.546m 2.523m
Income tax £312.4bn to £313.2bn £313.9bn £312.4bn
Universal Credit £79.79bn to £79.92bn £80.03bn £79.94bn
Housing Benefit £13.95bn to £14.08bn £14.03bn £14.04bn
Pension Credit £6.91bn to £6.96bn £6.95bn £6.96bn
Council tax reduction £2.253bn to £2.262bn £2.188bn £2.175bn
Household net income £1,756.1bn to £1,759.0bn £1,763.2bn £1,758.5bn
Poverty BHC, all people 14.67% to 14.70% 14.70% 14.69%
Child poverty BHC 16.92% to 16.97% 16.66% 16.64%
Child poverty AHC 23.94% to 24.00% 23.83% 23.80%
Effective sample size, households 1,207 to 1,218 1,220 1,230
People in the SPI-donor half (weighted) 19.23m to 19.24m 19.10m 19.08m
  • Property income. The total barely moves, because calibration reweights to the HMRC property income targets. Build 1 is inside main's range and build 2 is £0.11bn below it.
  • Outside main's range in both branch builds: council tax reduction (about −£0.08bn), Universal Credit (about +£0.1bn), child poverty (BHC about −0.3pp, AHC about −0.15pp), the weight on the SPI-donor half (about −0.14m people) and the effective sample size (slightly higher). I cannot say how much of this comes from the fix itself and how much from the later stages and the different donor draw.
  • Everything else in the table has at least one branch build inside main's range.

Calibration targets

Each build's fit to the national targets, evaluated at the calibration year (2025) with get_loss_results:

main, range of 4 builds branch build 1 branch build 2
Mean absolute relative error, 636 targets 0.1736 to 0.1746 0.1749 0.1738
Targets within 10% 430 to 437 437 441
Mean absolute relative error, 26 HMRC property income targets 0.1625 to 0.1655 0.1693 0.1606
Property income targets within 10% 21 to 22 21 21
Property income, sum over the amount targets (target £55.78bn) £56.55bn to £57.17bn £57.11bn £56.47bn
People with property income, sum over the count targets (target 2.351m) 2.452m to 2.472m 2.472m 2.449m
  • Property income targets. The fit is about the same. Two of the 26 fall outside main's range in both branch builds, each by less than one percentage point of relative error.
  • Other targets. 228 of the 637 targets fall outside main's range in both branch builds. The largest moves are in the HMRC capital gains bands (for example hmrc/cgt_taxpayers_band_1000000: 12% to 14% over target on main, 8% under on the branch) and the DWP Universal Credit payment bands. A different donor draw would move the capital gains bands.

Not in this PR

  • The householder's side. This PR does not move the rent to the householder who receives it. The FRS asks the householder nothing about it (SUBRENT covers only lets to people outside the household). Summing CVPAY within the household would recover the part the boarder or lodger pays out of pocket, because CVPAY is net of state help with rent. How to count the receipt, and the matching housing cost, for tax, benefits and household income is a separate change.
  • Property losses. ROYYR1 is always recorded as a positive amount, with a separate profit-or-loss flag (RENTPROF). The build ignores the flag, so reported losses count as income: £0.32bn in the 2024-25 FRS, from 42 records. This predates the PR. Fixed in Stop counting FRS property losses as income; count sub-let rent for every tenure #509, which is stacked on this PR.
  • SUBRENT. The build counts SUBRENT only for owner households, although some renting households also report it. SUBRENT can also be before or after allowable expenses (SUBALLOW). Left as it is here; Stop counting FRS property losses as income; count sub-let rent for every tenure #509 removes the owner-only rule and documents SUBALLOW.

Merge note

This PR lands only in the single uk-data release batch that Max approves (d833), first in its stack (#503, then #506 and #509). Every merge to uk-data main builds and publishes a release, so nothing here merges on its own.

#498 edits the lines just above the property income block, so the two PRs conflict textually in frs.py. Whichever merges second keeps both one-line calls: frs_dividend_income(...) and frs_property_income(...).

axiom: n/a: data pipeline change (FRS variable mapping), no rule change.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits September 30, 2026 23:10
FRS CVPAY is the rent a boarder or lodger pays the householder, after
deducting any state benefits to help with rent (FRS question CvPay, asked
about each person not related to the HRP in the second and later benefit
units of a conventional household). frs.py added it to the payer's own
property_income. Move the property income calculation into a tested
helper, frs_property_income, that keeps SUBRENT and ROYYR1 and drops CVPAY.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #503: the seeded cases only draw non-negative amounts, so the
floor at zero was never active, and every fixture had a unique index
while create_frs stacks the adult and child tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Handing this PR to the UK hub. Its original session is gone, and the session that owns #509 (stacked on this one) took it over on 2026-10-02. It lands only in Max's uk-data release batch (d833), first in its stack (#503, then #506 and #509).

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833).

@MaxGhenis
MaxGhenis merged commit ee788ff into main Oct 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant