Repository navigation
UK landlords: bind the SPI net property amounts, leave property_wealth to the engine, add receipts and finance costs - #1145
juaristi22 wants to merge 11 commits into
Conversation
HMRC's Property Rental Income Statistics are receipts before allowable expenses; SPI Table 3.7 net income from property is landlords' profit after allowable expenses and before residential finance costs, the concept property_income holds (policyengine-uk#2172 now documents it so). The incumbent's x1.9 scaling (uk-data#311) set the first against the second, so the 13 amount rows were signed out pending a ruling. María ruled on 2026-10-05 to bind each concept on its own variable. - The signed exclusion property_income_undercount_adjustment_missing is retired, the hmrc_spi family outcome becomes active_strict_band_pins with a rationale for the net concept, and the contract note replaces the x1.9 sentence. The uprating index stays obr.per_capita.gdp. - Regenerated from feed 825406f: 1,244 active references on #1121's base (13 added, none changed), 15 signed out; membership candidates 1,253 -> 1,265. - Both national compile-parity receipts gain the 13 rows. A property- specific reason precedes the generic SPI drift reason, which blamed the fixture's one-year projection: the 2025 values are £31.78bn against the fixture's £55.78bn (1.9 x £29.36bn). - Tests: the reference counts, the inverted property test (13 rows on the per-capita GDP hold, no value operation), the membership outcome and signed-rationale lists; the income-anchors note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk defines property_wealth as residential_property_value (main_residence_value + other_residential_property_value) plus non_residential_property_value. The WAS stage saved the survey's own total, which also counts owned land and the overseas and other-property remainder; a persisted copy overrides the engine's sum, and the variable has no uprating index, so the published aa31bdf6 year files hold it flat from 2026 to 2030 while the sum grows 13% (the uk-data#543 defect). - UK_RELEASE_EXPORT_DROPPED_COLUMNS gains household.property_wealth, and the measure resolver's scratch H5 applies the same drops in #1115's _engine_scratch_frame, beside the alias codes it already leaves out; engine_scratch_dropped_columns in the receipt names them. The spine keeps the column for the WAS stage's identity gate and donor checks. - household.property_wealth is a reviewed export exclusion (constant and gates.json), and the coverage gate's two coverage halves read the frame the release writes, where the raw calibrated frame would report it as a stale exclusion; the build-state half keeps reading the spine (#1115). - The coverage ledger gets a declared engine_derived_exclusions section, separate from the evidence-derived known gaps; the builder refuses drift, overlap with the gaps, and any column that is not a formula- owned override the reference persists. Manifest: 144 required, 1 reviewed exclusion; the three components stay required. - Tests: the scratch H5 and receipt, the export columns, the export- frame coverage binding, the household spine read for nonnegativity, the ledger and manifest, and an engine test that the engine's total is the component sum in 2024 and 2025 and that a persisted total would override it. The gates.json change moves the release contract's UK gate-battery pins (the full-manifest policy, manifest and fingerprint digests and the release_cut part's two digests; microcosm-data contract.py and its test mirror), re-derived from the live spec as the pin tests do. The input-mass parity register entry (efrs-post-calibration) waits for María's signature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aggregates only. A: the 13 bound SPI amount rows compile to GBP 31.78bn for 2025 against the fixture's 55.78bn. B: on the A9 candidate at 2025 the engine-derived property_wealth is GBP 7,633.4bn against 7,469.5bn persisted (+2.2%), household land value 4,697.3bn against 4,595.0bn (target 4,559.8bn), land value 6,755.9bn against 6,653.6bn (target 7,117.8bn); the saved total sat below the uprated component sum for 45,232 of 63,941 households. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2.123.0 carries PolicyEngine/policyengine-uk#2172: property_rental_income, property_finance_costs and property_finance_costs_brought_forward, the s. 274A finance-cost reduction, and a property allowance that no longer stacks on expenses already netted from profit. The lock's UK side moves exactly to 2.123.0 (uv lock --upgrade-package policyengine-uk==2.123.0) and nothing else moves: policyengine-core stays 3.32.19 on the UK side, which 2.123.0 accepts, and 3.32.5 on the US side (#1086's per-country pins). The UK extras of microcosm-build, microcosm-data and microcosm-frame require >=2.123.0. The step from #1121's 2.122.2 also brings #2160, #2096, #2097 and #2144. Re-derived with no data change: the uprating pin stamp (values unchanged), the concept coverage (288 inputs, the three property inputs uncovered) and the mapping reviewed against 2.123.0 (the property_income note quotes its new documentation, alignment still approximate; the liquid-asset reason holds unchanged), the spec's declared dependency and the H2 fixture's, the coverage manifest's source digests, and APPROVED_UV_LOCK_SHA256. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SPI tape's LLIR_RESTRICT_AMT_TOT is the "restricted land property finance cost (tax relief is included in TAX_CRED)" (SPI Public Use Tape 2022-23 documentation): the dwelling-related finance costs ITTOIA 2005 s. 272A no longer deducts and s. 274A relieves as a tax reduction. It becomes the first-stage output property_finance_costs, drawn last so it conditions on every income leaf and leaves every earlier draw where it was; the band donors copy it with the record. It is uprated from 2022 by the engine's own index for the variable, obr.mortgage_interest. The donor is refused unless the median TAX_CRED / LLIR_RESTRICT_AMT_TOT on rows with finance costs lies in [0.195, 0.205] (the 20% relief carried in TAX_CRED); the stage receipt records the check. FRS-channel rows start at zero until property_components imputes their landlords. Lockstep: SPI required columns and source columns in the spec and its projection, the stage-1 outputs, the stage outputs and non-negativity list, the graph cells (after the first-stage outputs, the legacy column order), the export allow-list and gates.json with the gate-battery digests, the H2 fixture (synthetic LLIR and TAX_CRED on every third landlord) and the coverage manifest's fixture digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.123.0 reads two landlord inputs beside property_income: property_rental_income (receipts before expenses, which decide between the property allowance and the expenses route) and property_finance_costs (the year's dwelling-related finance costs, relieved as the s. 274A tax reduction). A person-grain property_components stage after regional_property_uprating fills both. Finance costs on the FRS concept. ROYYR1 is rent after show card K6, which nets mortgage payments and loan interest. The stage prepares and rebases the SPI 2022-23 tape exactly as hmrc_spi_income_spine does and keeps its landlords with profit after restricted finance costs above zero, the analog of an FRS landlord reporting a profit. Incidence is a person-keyed uniform against the tape's share in the landlord's decile of profit after finance costs; the amount is the regime-gated QRF fitted on the analog landlords with costs, given that profit and age, drawn at a second person-keyed quantile. The costs are added back to the FRS landlord's property_income so the engine relieves the interest once; the mortgage capital the card also nets stays netted. The add-back is decision 6's recommendation, to be confirmed with Max before merge. The SPI channel keeps the costs the income stage drew. Receipts for every landlord, from PRIS 2024-25 vendored from the pinned feed (hmrc_property_rental_income_facts.json: Tables 1, 2, 6, 8 and 13, 44 rows, typed by uk_runtime.hmrc_property_rental). Landlords on both channels are ranked by profit and walked top band first into Table 13's eleven bands at their individuals-basis counts with the CGT Table 3 walk's rule; receipts are linear within a bounded band, never below profit, and the open top band takes max(100,000, k x profit) with k solved to the individuals' Table 2 receipts. Receipts less profit are the deductible expenses. Sub-letting rent keeps equal receipts and no expenses until Rent a Room is released; landlords without profit carry no receipts (a recorded residual). Gate uk_stage_property_components (stage health, release-blocking, in the spine scope): no receipts below profit, every band above the lowest within twice the largest landlord weight of its target, and the top band's multiplier below its cap. Lockstep: the sources manifest and its projection, two schema branches and the operation-kind allow-list; the graph roster, kernel registry and spine tool; the H2 fixture (40 stages, synthetic PRIS facts sized to the frame the stage reads) and the coverage manifest with its family coverage contract; the vendoring register and the country package; the export allow-list, gates.json, the stage-health parameter keys, the release contract's gate mirrors and its gate-battery digests; the stage-roster tests; the build-graph doc's stage count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1106) Three PRIS 2026 targets (tax year 2024-25), per María's ruling of 2026-10-05 that the receipts bind on their own variable: - hmrc.pris.rental_income.individuals: Table 2's individuals' rental income before expenses on property_rental_income, uprated by the engine's obr.per_capita.gdp (GBP 49.81bn -> 51.99bn for 2025). - hmrc.pris.landlord_count_by_rental_income_band: Table 13's eleven landlord counts by size of rental income, restated from all tax entities to individuals by Table 1's share (2.85m / 2.88m) through scaled_by_ratio and held at their 2024-25 levels; banded on property_rental_income. - hmrc.pris.residential_finance_costs: Table 8's residential finance costs on property_finance_costs, restated by Table 6's individuals' share of allowable expenses (30.03 / 34.75) and uprated by the engine's obr.mortgage_interest (GBP 12.82bn -> 12.13bn for 2025). Table 13's lowest band publishes only its upper edge (up to GBP 10,000), so the binding declares band_floor 0 and the shared band reader takes a declared floor for a band that states only an upper edge; a band without one is still skipped as unreadable. obr.mortgage_interest joins UK_ENGINE_INDEX_PARAMETERS and the pins are re-derived at 2.123.0. The contract records the three as new declarations with no incumbent registry row and adds the PRIS category. Regenerated: the national references and membership (1,244 -> 1,257 active; hmrc_pris family, 11 fan-out rows), the local references (the category only) and the 2025 compile-parity receipt (13 ledger-only rows; the 2023 receipt does not move). Pins moved: active reference counts, contract target counts (392 -> 395), registry scope (341 -> 344), unmapped declarations (149 -> 152), the engine index count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) The walk filled each Table 13 band to PRIS's individuals-basis count. Before calibration the spine carries 2.21m weighted landlords against PRIS's 2.85m, so the ten bands from GBP 10,000 took 71% of the spine's landlords where PRIS has 55%, and every profit rank landed in a higher receipts band than PRIS's distribution implies. The national calibration then scaled that up: on the C3 arm's calibrated weights, which bind no receipts, the stage's receipts came to GBP 55.39bn against PRIS's GBP 49.81bn, with 1.16m landlords at GBP 10,000 to 20,000 against 0.85m and 0.71m in the lowest band against 1.29m. Binding the PRIS targets (the C4 arm) then pulled weight across total-income bands: the SPI property amount at GBP 20,000 to 30,000 of total income went to -17.8% and the weight-ratio gate failed. Each band now takes its share of the individuals-basis landlords times the landlords' weight in the frame. The open top band's target mean is unchanged (the scale cancels), so k is still solved to the individuals' receipts per landlord. Evaluated offline on the same calibrated weights (scripts/walk_variants.py in the measurement folder): receipts GBP 46.11bn and band counts of 1.19m, 0.90m, 0.35m, 0.16m from the lowest up, against PRIS's 1.29m, 0.85m, 0.33m, 0.15m; the misallocation summed over the bands falls from about 1.12m landlords to 0.25m. The receipt records each band's published count beside its target mass and the share scale. Declared text moves with it (the operation's walk and the stage notes in both manifests, the gate notes, the stage's changelog), and so do the gate-battery digests, the H2 fixture's stage payload and the coverage manifest's digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit 82c744c. Receipts are a fixed function of profit rank, so every PRIS count or receipts target measured (all 13 rows, the 13 rows without the lowest band, and the two totals alone, under both walk rules) moved weight across the SPI's total-income bands: the SPI property amounts left their bound and `uk_weight_ratio` failed. The `property_components` stage still vendors PRIS 2024-25 and its receipt reports the spine against it; the stage gate keeps the band shares. Active national references return to 1,244. The arms are in experiments/1106-uk-property-income-receipts.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
C3, C6, C7 and C8 of the receipts note: the stage on the licensed spine, the calibration arms (control, C3, C3 without the stale West Midlands deferral, C4, C4 without PRIS's lowest band, the share-filled walk with all 13 PRIS rows and with the two totals only), the engine quantities at 2025, and the decision to keep PRIS a diagnostic. The stage's changelog fragment says why PRIS binds no target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
82c744c to
7b20ef7
Compare
Arm c6x on 7b20ef7: the share-filled walk with PRIS a diagnostic, spine and calibration built afresh. All 7 calibrated-seam gates and all 33 spine gates pass with #1121's stale West Midlands deferral dropped at gate time; receipts at the calibrated weights against PRIS; the engine's landlord quantities at 2025. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@MaxGhenis, we need two rulings on the FRS property rule from #1081 and uk-data#509 before this merges. 1. Adding imputed finance costs back to FRS landlords' profit. ROYYR1 is the rent after the items on show card K6, which include mortgage payments and loan interest. So FRS landlords' profit sits below the SPI concept that
2. If the add-back stands, what about landlords who report a loss? Because K6 nets mortgage capital and interest, an FRS loss is often not a tax loss. Take £12,000 of rent, £2,000 of repairs and £11,000 of mortgage payments, £9,000 of them interest. That is reported as a £1,000 loss, but for tax the profit is £10,000, with £9,000 of finance costs. The stage draws finance costs only for FRS landlords reporting a profit, so landlords with a mortgage who report a loss stay at zero. The consistent extension would draw their interest the same way, add it to the reported loss, and keep the result if it is positive. That turns some reported losses into profits, against "a loss counts as zero". It affects at most 67 adults in the FRS 2024-25 (0.11m weighted). Should we extend the add-back to them, or keep their losses at zero? If you disagree with either, we'll change the stage before merge and re-measure. |
Closes #1106. #1121 has merged, so this branch is rebased onto main (
75167a688); the rebase brought in only a changelog sentence, #1121's receipts note and an equivalent rewrite of one weighted share infrs_uc_start_up_period. Draft: the modelling decisions below are held.What changes
A. The SPI net property-income amounts bind unscaled (
4d2dbc894). SPI Table 3.7 "net income from property" is landlords' profit after allowable expenses and before residential finance costs, the conceptproperty_incomeholds (PolicyEngine/policyengine-uk#2172 documents it). The incumbent's x1.9 scaling set the Property Rental Income Statistics (PRIS) receipts, which are before expenses, against it. The signed exclusion is retired and the 13 rows bind at £31.78bn for 2025 (the incumbent fixture holds £55.78bn).B.
property_wealthstays with the engine (2364aad35). The WAS stage saved the survey's own total, which overrides policyengine-uk's sum of the three property components and is never uprated (the uk-data#543 defect). The release export drops it, and so does the measure resolver's scratch H5, inside #1115's_engine_scratch_frame. It is a reviewed export exclusion and an engine-derived coverage exclusion; the three components stay required. On the A9 candidate at 2025 this movesproperty_wealthfrom £7,469.5bn to £7,633.4bn and household land value from £4,595.0bn to £4,697.3bn (target £4,559.8bn).C0. policyengine-uk 2.123.0 (
db5c94fea), the release with PolicyEngine/policyengine-uk#2172's landlord inputs. Only policyengine-uk moves in the lock; policyengine-core stays 3.32.19 on the UK side and 3.32.5 on the US side.C1. Finance costs on the SPI channel (
47db70fc2). The tape'sLLIR_RESTRICT_AMT_TOT(restricted land property finance cost) becomesproperty_finance_costs, drawn last in the first-stage SPI model so every earlier draw is unchanged, and uprated byobr.mortgage_interest. The donor is refused unlessTAX_CREDcarries the 20% relief (median ratio between 0.195 and 0.205).C3. A
property_componentsstage (9859249d5, walk rulea74802bbc) afterregional_property_uprating.property_income(see the question for Max below).property_rental_income(receipts) from PRIS 2024-25 Table 13. Landlords are ranked by profit and walked into its eleven bands with the CGT Table 3 walk's rule, each band filled to its PRIS share of the spine's own landlords. Receipts are linear within a band and never below profit; the open top band is solved to the individuals' receipts total.uk_stage_property_components. The vendored PRIS resource has 44 rows (Tables 1, 2, 6, 8 and 13).PRIS stays a diagnostic.
69ed06affbound three PRIS targets (individuals' receipts, the eleven landlord counts by size of receipts, residential finance costs);7cfab0932reverts it, so active national references stay at 1,244. Every PRIS count or receipts target measured pushed the SPI property amounts out of their bound (below). The stage receipt reports the spine against PRIS instead.Measurement
Licensed arms with a national calibration and local staging only, on the commits before the rebase. The scripts are in
data/ukds/acceptance/1106-property-income/and the receipts inexperiments/1106-uk-property-income-receipts.md. The Chronicle feed is the 825406f artifact, rebuilt on 2026-10-08 with both digests matching the pin.uk_target_fitfails only because UK: port the remaining uk-data fixes tracked in #1095, engine items included #1121's West Midlands deferral is stale: that cell sits at +23.1%, inside the 25% bound. With the deferral dropped (c3x), 7 of 7 gates pass.c3x):property_finance_coststotal £11.71bn (PRIS: £12.13bn).income_taxtakes 0.33 s against 0.23 s to compute.uk_weight_ratiofails in all four.7b20ef73c, armc6x): loss 0.00710, 98.1% of 1,180 within 10%, the SPI property amounts and counts within 0.2%. All 7 calibrated-seam gates and all 33 spine gates pass once the stale West Midlands deferral is dropped; the cell sits at +23.4%.Decisions held
uk_target_fitfails on it until it goes.property_income, so the engine relieves the interest once, as the s. 274A reduction; the capital part stays netted. This departs from the uk-data#509 rule of using FRS amounts as reported. The alternative is to leave FRS profit as reported and give FRS landlords no finance costs. We will change it before merge if Max disagrees.property_wealth, retiring the two stale property count exclusions, and the net-new landlord columns' signed difference.Known residuals
property_finance_costs_brought_forwardis unset.Testing
--check, and the vendoring and pin checks: run on C3.tools/ci_test_plan.py verify, and ruff on every changed file.🤖 Generated with Claude Code