Skip to content

UK landlords: bind the SPI net property amounts, leave property_wealth to the engine, add receipts and finance costs - #1145

Draft
juaristi22 wants to merge 11 commits into
mainfrom
uk-property-income-1106
Draft

juaristi22 wants to merge 11 commits into
mainfrom
uk-property-income-1106

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1106. #1121 has merged, so this branch is rebased onto main (75167a688); the rebase brought in only a changelog sentence, #1121's receipts note and an equivalent rewrite of one weighted share in frs_uc_start_up_period. Draft: the modelling decisions below are held.

What changes

A. The SPI net property-income amounts bind unscaled (4d2dbc894). SPI Table 3.7 "net income from property" is landlords' profit after allowable expenses and before residential finance costs, the concept property_income holds (PolicyEngine/policyengine-uk#2172 documents it). The incumbent's x1.9 scaling set the Property Rental Income Statistics (PRIS) receipts, which are before expenses, against it. The signed exclusion is retired and the 13 rows bind at £31.78bn for 2025 (the incumbent fixture holds £55.78bn).

B. property_wealth stays with the engine (2364aad35). The WAS stage saved the survey's own total, which overrides policyengine-uk's sum of the three property components and is never uprated (the uk-data#543 defect). The release export drops it, and so does the measure resolver's scratch H5, inside #1115's _engine_scratch_frame. It is a reviewed export exclusion and an engine-derived coverage exclusion; the three components stay required. On the A9 candidate at 2025 this moves property_wealth from £7,469.5bn to £7,633.4bn and household land value from £4,595.0bn to £4,697.3bn (target £4,559.8bn).

C0. policyengine-uk 2.123.0 (db5c94fea), the release with PolicyEngine/policyengine-uk#2172's landlord inputs. Only policyengine-uk moves in the lock; policyengine-core stays 3.32.19 on the UK side and 3.32.5 on the US side.

C1. Finance costs on the SPI channel (47db70fc2). The tape's LLIR_RESTRICT_AMT_TOT (restricted land property finance cost) becomes property_finance_costs, drawn last in the first-stage SPI model so every earlier draw is unchanged, and uprated by obr.mortgage_interest. The donor is refused unless TAX_CRED carries the 20% relief (median ratio between 0.195 and 0.205).

C3. A property_components stage (9859249d5, walk rule a74802bbc) after regional_property_uprating.

  • FRS landlords report ROYYR1 after show card K6, which nets mortgage interest and capital. Their finance costs are drawn from the tape among its landlords with a profit after finance costs: person-keyed incidence by decile, then the amount from a regime-gated QRF on profit and age. The draws are added back to property_income (see the question for Max below).
  • Every landlord gets property_rental_income (receipts) from PRIS 2024-25 Table 13. Landlords are ranked by profit and walked into its eleven bands with the CGT Table 3 walk's rule, each band filled to its PRIS share of the spine's own landlords. Receipts are linear within a band and never below profit; the open top band is solved to the individuals' receipts total.
  • A release-blocking stage gate, uk_stage_property_components. The vendored PRIS resource has 44 rows (Tables 1, 2, 6, 8 and 13).

PRIS stays a diagnostic. 69ed06aff bound three PRIS targets (individuals' receipts, the eleven landlord counts by size of receipts, residential finance costs); 7cfab0932 reverts it, so active national references stay at 1,244. Every PRIS count or receipts target measured pushed the SPI property amounts out of their bound (below). The stage receipt reports the spine against PRIS instead.

Measurement

Licensed arms with a national calibration and local staging only, on the commits before the rebase. The scripts are in data/ukds/acceptance/1106-property-income/ and the receipts in experiments/1106-uk-property-income-receipts.md. The Chronicle feed is the 825406f artifact, rebuilt on 2026-10-08 with both digests matching the pin.

  • Control (UK: port the remaining uk-data fixes tracked in #1095, engine items included #1121's head, policyengine-uk 2.122.2): loss 0.00712, 98.3% of 1,167 targets within 10%, 7 of 7 calibrated-seam gates pass.
  • C3 (A, B, C0, C1 and the stage; no PRIS targets): loss 0.00714, 98.1% of 1,180 within 10%. The 13 SPI property amounts fit to 0.1% (48% off at design weights).
  • Engine quantities at 2025 (policyengine-uk 2.123.0, control against c3x):
    • The finance-cost relief is £1.90bn for 1.03m landlords, with £2.21bn of costs carried forward for 0.31m.
    • property_finance_costs total £11.71bn (PRIS: £12.13bn).
    • The property allowance deduction falls from £0.117bn to £0.062bn.
    • Housing Benefit −£21m, Council Tax Reduction −£26m, Pension Credit +£10m, Universal Credit +£3m.
    • income_tax takes 0.33 s against 0.23 s to compute.
  • The share-filled walk on the spine: every band, the lowest included, holds its share within rounding. At the C3 calibrated weights it gives £46.11bn of receipts against PRIS's £49.81bn (2024 values), and 1.19m, 0.90m and 0.35m landlords in the lowest three bands against 1.29m, 0.85m and 0.33m.
  • PRIS as targets, four arms. Each fits the PRIS rows it binds to within 3.3%, but each breaks the SPI fit:
    • All 13 rows with the count-filled walk: £20,000 to £30,000 of total income −17.8%.
    • The same without PRIS's lowest band: still −17.7%.
    • All 13 with the share-filled walk: £70,000 to £100,000 +48.2%.
    • The two totals alone: still +30.9%.
    • uk_weight_ratio fails in all four.
  • The final head (7b20ef73c, arm c6x): loss 0.00710, 98.1% of 1,180 within 10%, the SPI property amounts and counts within 0.2%. All 7 calibrated-seam gates and all 33 spine gates pass once the stale West Midlands deferral is dropped; the cell sits at +23.4%.
    • Against PRIS at the calibrated weights: receipts £45.72bn against £49.81bn (−8.2%), finance costs £10.59bn against £11.08bn (−4.4%), and 2.77m landlords against 2.85m. The bands from £40,000 run short, the open band by 39%.
    • At 2025: relief £1.89bn for 1.01m landlords; the property allowance deduction £0.142bn for 0.33m landlords (the control: £0.117bn for 0.22m).

Decisions held

  • The open top band's multiplier pins at 1, so landlords with £100,000 or more of receipts carry no deductible expenses. The bounded bands take their midpoints, and PRIS covers UK property only, while the SPI profit and the engine's receipts include overseas property.
  • Removing UK: port the remaining uk-data fixes tracked in #1095, engine items included #1121's West Midlands deferral. It is stale on this head (the cell sits at +23.4%, inside the 25% bound), so uk_target_fit fails on it until it goes.
  • For Max, the FRS add-back (decision 6). C3 adds FRS landlords' imputed finance costs back to property_income, so the engine relieves the interest once, as the s. 274A reduction; the capital part stays netted. This departs from the uk-data#509 rule of using FRS amounts as reported. The alternative is to leave FRS profit as reported and give FRS landlords no finance costs. We will change it before merge if Max disagrees.
  • Signatures (María): the input-mass parity entry for property_wealth, retiring the two stale property count exclusions, and the net-new landlord columns' signed difference.

Known residuals

Testing

  • The engine-free and engine-UK lanes, the coverage manifest --check, and the vendoring and pin checks: run on C3.
  • After the revert: the target, calibration, country-spec and stage tests (293 pass).
  • After the rebase: H2 spine parity, the stage and FRS spine tests (120 pass), tools/ci_test_plan.py verify, and ruff on every changed file.
  • The base is now main, so CI runs from this push on.

🤖 Generated with Claude Code

juaristi22 and others added 10 commits October 8, 2026 21:21
HMRC's Property Rental Income Statistics are receipts before allowable
expenses; SPI Table 3.7 net income from property is landlords' profit
after allowable expenses and before residential finance costs, the
concept property_income holds (policyengine-uk#2172 now documents it so).
The incumbent's x1.9 scaling (uk-data#311) set the first against the
second, so the 13 amount rows were signed out pending a ruling. María
ruled on 2026-10-05 to bind each concept on its own variable.

- The signed exclusion property_income_undercount_adjustment_missing is
  retired, the hmrc_spi family outcome becomes active_strict_band_pins
  with a rationale for the net concept, and the contract note replaces
  the x1.9 sentence. The uprating index stays obr.per_capita.gdp.
- Regenerated from feed 825406f: 1,244 active references on #1121's base (13 added,
  none changed), 15 signed out; membership candidates 1,253 -> 1,265.
- Both national compile-parity receipts gain the 13 rows. A property-
  specific reason precedes the generic SPI drift reason, which blamed
  the fixture's one-year projection: the 2025 values are £31.78bn
  against the fixture's £55.78bn (1.9 x £29.36bn).
- Tests: the reference counts, the inverted property test (13 rows on
  the per-capita GDP hold, no value operation), the membership outcome
  and signed-rationale lists; the income-anchors note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk defines property_wealth as residential_property_value
(main_residence_value + other_residential_property_value) plus
non_residential_property_value. The WAS stage saved the survey's own
total, which also counts owned land and the overseas and other-property
remainder; a persisted copy overrides the engine's sum, and the variable
has no uprating index, so the published aa31bdf6 year files hold it flat
from 2026 to 2030 while the sum grows 13% (the uk-data#543 defect).

- UK_RELEASE_EXPORT_DROPPED_COLUMNS gains household.property_wealth, and
  the measure resolver's scratch H5 applies the same drops in #1115's
  _engine_scratch_frame, beside the alias codes it already leaves out;
  engine_scratch_dropped_columns in the receipt names them. The spine
  keeps the column for the WAS stage's identity gate and donor checks.
- household.property_wealth is a reviewed export exclusion (constant and
  gates.json), and the coverage gate's two coverage halves read the frame
  the release writes, where the raw calibrated frame would report it as a
  stale exclusion; the build-state half keeps reading the spine (#1115).
- The coverage ledger gets a declared engine_derived_exclusions section,
  separate from the evidence-derived known gaps; the builder refuses
  drift, overlap with the gaps, and any column that is not a formula-
  owned override the reference persists. Manifest: 144 required, 1
  reviewed exclusion; the three components stay required.
- Tests: the scratch H5 and receipt, the export columns, the export-
  frame coverage binding, the household spine read for nonnegativity,
  the ledger and manifest, and an engine test that the engine's total is
  the component sum in 2024 and 2025 and that a persisted total would
  override it.

The gates.json change moves the release contract's UK gate-battery pins
(the full-manifest policy, manifest and fingerprint digests and the
release_cut part's two digests; microcosm-data contract.py and its test
mirror), re-derived from the live spec as the pin tests do.

The input-mass parity register entry (efrs-post-calibration) waits for
María's signature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aggregates only. A: the 13 bound SPI amount rows compile to GBP 31.78bn
for 2025 against the fixture's 55.78bn. B: on the A9 candidate at 2025
the engine-derived property_wealth is GBP 7,633.4bn against 7,469.5bn
persisted (+2.2%), household land value 4,697.3bn against 4,595.0bn
(target 4,559.8bn), land value 6,755.9bn against 6,653.6bn (target
7,117.8bn); the saved total sat below the uprated component sum for
45,232 of 63,941 households.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2.123.0 carries PolicyEngine/policyengine-uk#2172: property_rental_income,
property_finance_costs and property_finance_costs_brought_forward, the
s. 274A finance-cost reduction, and a property allowance that no longer
stacks on expenses already netted from profit. The lock's UK side moves
exactly to 2.123.0 (uv lock --upgrade-package policyengine-uk==2.123.0)
and nothing else moves: policyengine-core stays 3.32.19 on the UK side,
which 2.123.0 accepts, and 3.32.5 on the US side (#1086's per-country
pins). The UK extras of microcosm-build, microcosm-data and
microcosm-frame require >=2.123.0. The step from #1121's 2.122.2 also
brings #2160, #2096, #2097 and #2144.

Re-derived with no data change: the uprating pin stamp (values
unchanged), the concept coverage (288 inputs, the three property inputs
uncovered) and the mapping reviewed against 2.123.0 (the property_income
note quotes its new documentation, alignment still approximate; the
liquid-asset reason holds unchanged), the spec's declared dependency and
the H2 fixture's, the coverage manifest's source digests, and
APPROVED_UV_LOCK_SHA256.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SPI tape's LLIR_RESTRICT_AMT_TOT is the "restricted land property
finance cost (tax relief is included in TAX_CRED)" (SPI Public Use Tape
2022-23 documentation): the dwelling-related finance costs ITTOIA 2005
s. 272A no longer deducts and s. 274A relieves as a tax reduction. It
becomes the first-stage output property_finance_costs, drawn last so it
conditions on every income leaf and leaves every earlier draw where it
was; the band donors copy it with the record. It is uprated from 2022 by
the engine's own index for the variable, obr.mortgage_interest.

The donor is refused unless the median TAX_CRED / LLIR_RESTRICT_AMT_TOT
on rows with finance costs lies in [0.195, 0.205] (the 20% relief carried
in TAX_CRED); the stage receipt records the check. FRS-channel rows start
at zero until property_components imputes their landlords.

Lockstep: SPI required columns and source columns in the spec and its
projection, the stage-1 outputs, the stage outputs and non-negativity
list, the graph cells (after the first-stage outputs, the legacy column
order), the export allow-list and gates.json with the gate-battery
digests, the H2 fixture (synthetic LLIR and TAX_CRED on every third
landlord) and the coverage manifest's fixture digests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.123.0 reads two landlord inputs beside property_income:
property_rental_income (receipts before expenses, which decide between
the property allowance and the expenses route) and property_finance_costs
(the year's dwelling-related finance costs, relieved as the s. 274A tax
reduction). A person-grain property_components stage after
regional_property_uprating fills both.

Finance costs on the FRS concept. ROYYR1 is rent after show card K6,
which nets mortgage payments and loan interest. The stage prepares and
rebases the SPI 2022-23 tape exactly as hmrc_spi_income_spine does and
keeps its landlords with profit after restricted finance costs above
zero, the analog of an FRS landlord reporting a profit. Incidence is a
person-keyed uniform against the tape's share in the landlord's decile of
profit after finance costs; the amount is the regime-gated QRF fitted on
the analog landlords with costs, given that profit and age, drawn at a
second person-keyed quantile. The costs are added back to the FRS
landlord's property_income so the engine relieves the interest once; the
mortgage capital the card also nets stays netted. The add-back is
decision 6's recommendation, to be confirmed with Max before merge. The
SPI channel keeps the costs the income stage drew.

Receipts for every landlord, from PRIS 2024-25 vendored from the pinned
feed (hmrc_property_rental_income_facts.json: Tables 1, 2, 6, 8 and 13,
44 rows, typed by uk_runtime.hmrc_property_rental). Landlords on both
channels are ranked by profit and walked top band first into Table 13's
eleven bands at their individuals-basis counts with the CGT Table 3
walk's rule; receipts are linear within a bounded band, never below
profit, and the open top band takes max(100,000, k x profit) with k
solved to the individuals' Table 2 receipts. Receipts less profit are the
deductible expenses. Sub-letting rent keeps equal receipts and no
expenses until Rent a Room is released; landlords without profit carry no
receipts (a recorded residual).

Gate uk_stage_property_components (stage health, release-blocking, in
the spine scope): no receipts below profit, every band above the lowest
within twice the largest landlord weight of its target, and the top
band's multiplier below its cap.

Lockstep: the sources manifest and its projection, two schema branches
and the operation-kind allow-list; the graph roster, kernel registry and
spine tool; the H2 fixture (40 stages, synthetic PRIS facts sized to the
frame the stage reads) and the coverage manifest with its family
coverage contract; the vendoring register and the country package; the
export allow-list, gates.json, the stage-health parameter keys, the
release contract's gate mirrors and its gate-battery digests; the
stage-roster tests; the build-graph doc's stage count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1106)

Three PRIS 2026 targets (tax year 2024-25), per María's ruling of
2026-10-05 that the receipts bind on their own variable:

- hmrc.pris.rental_income.individuals: Table 2's individuals' rental
  income before expenses on property_rental_income, uprated by the
  engine's obr.per_capita.gdp (GBP 49.81bn -> 51.99bn for 2025).
- hmrc.pris.landlord_count_by_rental_income_band: Table 13's eleven
  landlord counts by size of rental income, restated from all tax
  entities to individuals by Table 1's share (2.85m / 2.88m) through
  scaled_by_ratio and held at their 2024-25 levels; banded on
  property_rental_income.
- hmrc.pris.residential_finance_costs: Table 8's residential finance
  costs on property_finance_costs, restated by Table 6's individuals'
  share of allowable expenses (30.03 / 34.75) and uprated by the engine's
  obr.mortgage_interest (GBP 12.82bn -> 12.13bn for 2025).

Table 13's lowest band publishes only its upper edge (up to GBP 10,000),
so the binding declares band_floor 0 and the shared band reader takes a
declared floor for a band that states only an upper edge; a band without
one is still skipped as unreadable.

obr.mortgage_interest joins UK_ENGINE_INDEX_PARAMETERS and the pins are
re-derived at 2.123.0. The contract records the three as new
declarations with no incumbent registry row and adds the PRIS category.
Regenerated: the national references and membership (1,244 -> 1,257
active; hmrc_pris family, 11 fan-out rows), the local references (the
category only) and the 2025 compile-parity receipt (13 ledger-only
rows; the 2023 receipt does not move). Pins moved: active reference
counts, contract target counts (392 -> 395), registry scope (341 ->
344), unmapped declarations (149 -> 152), the engine index count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

The walk filled each Table 13 band to PRIS's individuals-basis count.
Before calibration the spine carries 2.21m weighted landlords against
PRIS's 2.85m, so the ten bands from GBP 10,000 took 71% of the spine's
landlords where PRIS has 55%, and every profit rank landed in a higher
receipts band than PRIS's distribution implies. The national calibration
then scaled that up: on the C3 arm's calibrated weights, which bind no
receipts, the stage's receipts came to GBP 55.39bn against PRIS's
GBP 49.81bn, with 1.16m landlords at GBP 10,000 to 20,000 against 0.85m
and 0.71m in the lowest band against 1.29m. Binding the PRIS targets (the
C4 arm) then pulled weight across total-income bands: the SPI property
amount at GBP 20,000 to 30,000 of total income went to -17.8% and the
weight-ratio gate failed.

Each band now takes its share of the individuals-basis landlords times
the landlords' weight in the frame. The open top band's target mean is
unchanged (the scale cancels), so k is still solved to the individuals'
receipts per landlord. Evaluated offline on the same calibrated weights
(scripts/walk_variants.py in the measurement folder): receipts
GBP 46.11bn and band counts of 1.19m, 0.90m, 0.35m, 0.16m from the
lowest up, against PRIS's 1.29m, 0.85m, 0.33m, 0.15m; the misallocation
summed over the bands falls from about 1.12m landlords to 0.25m.

The receipt records each band's published count beside its target mass
and the share scale. Declared text moves with it (the operation's walk
and the stage notes in both manifests, the gate notes, the stage's
changelog), and so do the gate-battery digests, the H2 fixture's stage
payload and the coverage manifest's digests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit 82c744c. Receipts are a fixed function of profit
rank, so every PRIS count or receipts target measured (all 13 rows, the
13 rows without the lowest band, and the two totals alone, under both
walk rules) moved weight across the SPI's total-income bands: the SPI
property amounts left their bound and `uk_weight_ratio` failed. The
`property_components` stage still vendors PRIS 2024-25 and its receipt
reports the spine against it; the stage gate keeps the band shares.

Active national references return to 1,244. The arms are in
experiments/1106-uk-property-income-receipts.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
C3, C6, C7 and C8 of the receipts note: the stage on the licensed spine,
the calibration arms (control, C3, C3 without the stale West Midlands
deferral, C4, C4 without PRIS's lowest band, the share-filled walk with
all 13 PRIS rows and with the two totals only), the engine quantities at
2025, and the decision to keep PRIS a diagnostic. The stage's changelog
fragment says why PRIS binds no target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-property-income-1106 branch from 82c744c to 7b20ef7 Compare October 8, 2026 20:25
@juaristi22
juaristi22 changed the base branch from uk-data-ports-1095-2 to main October 8, 2026 20:25
@juaristi22 juaristi22 closed this Oct 8, 2026
@juaristi22 juaristi22 reopened this Oct 8, 2026
Arm c6x on 7b20ef7: the share-filled walk with PRIS a diagnostic,
spine and calibration built afresh. All 7 calibrated-seam gates and all
33 spine gates pass with #1121's stale West Midlands deferral dropped at
gate time; receipts at the calibrated weights against PRIS; the engine's
landlord quantities at 2025.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@MaxGhenis, we need two rulings on the FRS property rule from #1081 and uk-data#509 before this merges.

1. Adding imputed finance costs back to FRS landlords' profit. ROYYR1 is the rent after the items on show card K6, which include mortgage payments and loan interest. So FRS landlords' profit sits below the SPI concept that property_income now holds: profit before residential finance costs (PolicyEngine/policyengine-uk#2172). The property_components stage draws each FRS landlord's finance costs from the SPI 2022-23 tape and adds them back to property_income. The draw has two steps: whether a landlord has costs follows the tape's share in their decile of profit after finance costs, and the amount comes from a QRF on that profit and age. The capital part of the mortgage payments stays netted.

  • Size. On the licensed spine at design weights (2024 values), 32% of the 1.03m FRS landlords draw costs, £2.37bn in all. That raises the FRS channel's profit from £9.5bn to £11.9bn.
  • Income tax. As reported, the interest is in effect relieved at the landlord's marginal rate. With the add-back, income tax charges it and gives the 20% reduction of ITTOIA 2005 s. 274A instead.
  • Means tests. Housing Benefit, Income Support, tax credits, Council Tax Reduction and Pension Credit read property_income_after_finance_costs, so they still see the reported amount.
  • HBAI. HBAI net income reads property_income, so it rises by the interest, less the extra tax.
  • Fit. With the add-back, the 13 SPI property amounts calibrate to within 0.2%, and every gate passes once a stale deferral inherited from UK: port the remaining uk-data fixes tracked in #1095, engine items included #1121 is removed (experiments/1106-uk-property-income-receipts.md, section C8).
  • The alternative is to leave FRS profit as reported and give FRS landlords no finance costs. All finance costs would then come from the SPI channel: £7.78bn of the £10.15bn now drawn at design weights, against PRIS's £11.08bn for individuals. The SPI amounts bound on property_income would then lean on reweighting, as frs_property_income's docstring notes.

2. If the add-back stands, what about landlords who report a loss? Because K6 nets mortgage capital and interest, an FRS loss is often not a tax loss. Take £12,000 of rent, £2,000 of repairs and £11,000 of mortgage payments, £9,000 of them interest. That is reported as a £1,000 loss, but for tax the profit is £10,000, with £9,000 of finance costs. The stage draws finance costs only for FRS landlords reporting a profit, so landlords with a mortgage who report a loss stay at zero. The consistent extension would draw their interest the same way, add it to the reported loss, and keep the result if it is positive. That turns some reported losses into profits, against "a loss counts as zero". It affects at most 67 adults in the FRS 2024-25 (0.11m weighted). Should we extend the add-back to them, or keep their losses at zero?

If you disagree with either, we'll change the stage before merge and re-measure.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK: model landlord property income on the taxable (net of expenses) concept, not rent received

1 participant