You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
UK spine follow-ups after #1045: SPI band donors as a mass-conserving support channel, WAS tenure coherence, the residential draw's bound and Table 7 composition, then a national build #1063
spi_income_band_donors (#1006, uk_runtime/spi_band_donors.py) copies 120 FRS households per reserved income band into the SPI channel and starts each at the band's published taxpayers over 120, about 2,992, 508, 167 and 83 per row. The copies are appended to the incoming weights and the stage records declared_factor=None, so about 450,000 households of mass are created rather than moved, and the graph declares the node's mass policy as free. The release-input coverage manifest has no family entry for the stage, so its mass record is never checked by reason. The draw uses numpy.random.default_rng(3) over person-sorted candidates rather than identity-keyed uniforms, so the identity-stability tool cannot reconstruct it from ids alone, and the stage requires at least 120 positive-propensity candidates per band rather than scaling its count on small frames.
#1045 worked through the same questions for the CGT band donors and ended up replacing them with a mass-conserving support split, because the Table 3 redraw reassigns every donor's value and the donors' only remaining role was row support. The principles that came out of it apply here: support rows conserve mass and declare it, carry light and equal weights within a band, and draw reproducibly from ids. The SPI donors are different in the one way that matters: they carry real tape income vectors that every later stage imputes from, so the channel design fits them directly. Donors carry equal weights within each band, chosen so no donor exceeds a declared maximum weight and every band keeps a declared minimum count; the mass they carry is reallocated from the households of the income band each donor is drawn from, proportionally, exactly as the SPI support channel takes its copies' mass from the FRS households of their region, so total mass and each band's mass are conserved and the record carries declared_factor=1.0 with a declared reason. The coverage manifest gains a family entry with mass_conserving semantics, the graph policy becomes declared, the seating uses stable_identity_uniforms on household ids with a declared seed and salt so the E8 receipt reconstructs it, and small frames scale the count and the mass with recorded scales, asserting the published band masses only when both are one. The mass_increasing_support semantics then loses its last user and can retire from the validator.
Related: #970 on demoted donors and the projection fence, since the same weight leverage applies to any support row calibration can move ten-fold.
Measured on the same build (arm ch3 at the #1045 head), the was_wealth debt segment (#685) puts £381bn of its £1,730bn of mortgage_debt on households whose tenure_type is not OWNED_WITH_MORTGAGE: 22% of the mass on 2,738 rows (1.4m households), £189bn of it on SPI-synthetic rows, against 10% in the WAS donor. The incidence climbs with income (15% of the top income decile of non-mortgaged households carries a mortgage, against 6% of the donor's outright owners and 5% of its private renters) and one in five of the outright-owner rows with a mortgage owes more than its main_residence_value. The June populace_uk_2023 release carried £740bn (48%), the figure Nikhil flagged. has_mortgage_tenure enters the segment as one soft predictor of the sign gate and the QRF, so income and the wealth columns drawn before it dominate the leaves; the FRS repayment flows are clean because spi_housing_shell zeroes them off mortgaged tenure. The recipient predictor is_renting is the engine formula, which omits RENT_FROM_HA (PolicyEngine/policyengine-uk lists RENT_PRIVATELY twice), while the donor flag is DVPriRntR8 == 1, private renters only: housing-association renters match owners and social renters and receive £300bn of main_residence_value (53% positive; donor renters 0%) and £348bn of property_wealth, and council renters match private renters (property wealth positive for 10.7% against 1.1% of the donor's social renters). policyengine-uk 2.101.0 added mortgage_debt and net_wealth, so the leak reaches net wealth on the next engine bump.
Two more spine defects from Vahid's review of #1045 (items 3 and 6) belong here rather than in issues of their own. hmrc_cgt_asset_type_spine realises the residential-property flag by a weighted systematic walk in ascending gain order against a logistic in log gains, and the residential_gains check of uk_stage_cgt_asset_type bounds the realised-minus-expected gains by the wider of 5 percent and maximum_gains_sigma × gains_bernoulli_sigma. Summation by parts gives the error as Σ B_i (g_i − g_{i+1}) + B_N g_N, with B the walk's running balance (bounded by one row's weight), so it is set by the balance at the top of the order times the largest gains: a handful of rows carrying billions. The sigma term is not an envelope for that walk (it assumes independent draws and shrinks when rows are split), the 5 percent term is what binds, and a configuration change that moves the balance across a different top row moves the realised gains by whole rows: arm ch2 (copy cap 80, commit 817e13f, reverted) failed at 10.29bn against a 5.38bn bound with one copy-clone (weight 20.8, GBP 571m of gains per person) carrying GBP 11.9bn of GBP 123bn at a residential probability of 0.009, while cap 60 passes at 0.08 of the bound. Baseline liability is insensitive (18/24 on both schedules from 2025-26); a reform that separates them is not. Separately, the asset-type stage restricts BADR and Investors' Relief claimants to the three eligible types and leans types towards gainers who show the stock they imply (#1045 c1 and c6); with BADR carrying about GBP 18bn, the realised HMRC Table 7 composition, fenced and diagnostic, drifts further from its targets than the control's (unlisted shares 66.6 percent of non-residential gains against 55.3 published; 62.1 on the control), and the restricted type fit is where to hold it.
Items this issue covers:
The channel design for spi_income_band_donors: equal light weights within a band under a declared maximum weight and minimum count, mass reallocated from the households of the income band each donor is drawn from, declared_factor=1.0 with a declared reason, graph policy declared.
A family entry for the stage in the release-input coverage manifest with mass_conserving semantics, so the terminal family gate checks its record.
Identity-keyed seating (stable_identity_uniforms with a declared seed and salt) so the E8 receipt reconstructs the draw from ids, and recorded count and mass scales on small frames, asserting the published band masses only when both are one.
Mortgage debt only on mortgaged tenure: the stage ends with mortgage_debt == 0 wherever tenure_type != OWNED_WITH_MORTGAGE, checked by a stage-health gate, the rule the repayment flows already follow. The path preserves the joint structure between the wealth columns rather than clipping after the draw: fit the debt segment stratified by tenure, so mortgaged recipients draw from mortgaged donors and the non-mortgaged pool's off-tenure mortgages (10% of donor mortgage mass) are the mass the rule drops, recorded in the receipt, or give the gate tenure flags with enough signal that the draw itself respects the tenure.
Tenure predictors on both sides of the WAS fit: replace is_renting (engine formula, omits RENT_FROM_HA; donor DVPriRntR8 == 1) with an owner-outright / mortgaged / private-rent / social-rent predictor built the same way on the donor (Ten1R8, DVPriRntR8) and on the recipient (tenure_type), so housing-association and council renters draw from social-renter donors, with a check that main_residence_value is zero off owner tenures. The engine formula's omission needs its own fix in PolicyEngine/policyengine-uk.
Residential draw: stratify the walk by gain band with a per-band offset, as the BADR half already does per Table 4.1 band, so the error is bounded band by band; replace the Bernoulli sigma in the gate's bound with the walk's deterministic per-band bound (the heaviest row's weight times the band's largest gain) and report the top-row concentration (share of residential gains on the ten heaviest placed rows) in the stage receipt; reproduce the ch2 failure at 817e13f with the relaxed gate as the regression case for the new bound.
Table 7 composition: bind it inside the restricted type fit (claimants' types drawn from the eligible three at Table 7's conditional shares among those three, non-claimants' types at the shares that restore the published marginal composition), report the composition by claimant status against Table 7 in the stage receipt, and keep the diagnostic fence until the fit holds it.
The issue closes on a national dataset built from the spine with all eight items in, passing every spine and terminal gate with the identity tool green, so the fixes are measured together rather than one licensed build at a time.
changed the title [-]Make the SPI income-band donors a mass-conserving support channel[/-][+]UK spine follow-ups after #1045: SPI band donors as a mass-conserving support channel, WAS tenure coherence, the residential draw's bound and Table 7 composition, then a national build[/+]on Sep 30, 2026
Status (2026-10-02): draft PR #1089 carries all eight items plus three fold-ins, stacked on #1084 (uk-pensions-1069); receipts in experiments/1063-uk-spine-followups.md.
What landed
Items 1, 2, 4: the SPI income band donors are a mass-conserving support channel. Each band seats max(120, ceil(published taxpayers / 250)) donors at equal weights, funded in proportion from the households of each donor's region; graph policy declared, no created mass, an identity-keyed seating the E8 receipt reruns from ids with the licensed tape.
Item 3: mass_increasing_support is retired from the coverage validator.
Items 5, 6: WAS wealth is coherent with tenure (no mortgage debt off mortgaged tenure, no main-residence value off owner tenures) and with its own totals; new gate uk_stage_was_wealth_coherence. The rule drops £127bn of the donor's £1,266bn mortgage mass, all other-property mortgages off tenure.
Items 7, 8: the residential-property flag is carried as weight by a new expansion stage, cgt_residential_split, after the incidence anchor. Each liable gainer's solved probability splits the household into arms at p·w and (1 − p)·w (2^k arms for k gainers), so the residential count and gains are identities on HMRC Table 8a in every gain band at design weights, with no draw and no seed; calibration decides between the arms as it does for the incidence clones. The asset-type stage types the arms and fits Table 7 once per claimant status with an error-diffusion walk (seed 554 retired). A per-band walk was tried first and replaced: its gains error in the open band was bounded by one stake several times the band's whole expectation.
Fold-ins: salary sacrifice predicted only for people who are paid; Child Benefit claims and opt-outs redrawn after the SPI chain against HMRC's August 2025 statistics (new stage child_benefit_take_up: 86.7% of eligible children claimed for against HMRC's 86.6%, 9.0% of claiming families opted out, 11.1m children in payment against 13.5m before).
Release cut: the certifier rehearsal on the 2026-09-30 build failed 5 of 20 gates; each is cleared at its source (export surface read without ids and with the weight, 43 declared outputs allow-listed, incapacity_benefit_reported dropped at the release boundary, FRS 2024-25's recoded TRAIN2 read, the LCFS donor floored at 0 with a receipt, three signed register entries). The two plan-2 measure exclusions are renewed to 2026-11-03.
Measured (licensed local arms, nothing released): the walk arm passed every spine gate with calibration loss 0.00853 against main's 0.00836, ESS 5,217 (5,198), max-to-median weight 1,115 (1,079; bound 1,151); the WAS arm passed every spine gate including the coherence gate. The arm on the residential split, the final national build, its certifier run and the identity tool are still to land.
Not in this PR: VAT is an engine question (PolicyEngine/policyengine-uk#1996). Two further engine issues are to be filed: is_renting omits housing-association tenure, and the Child Benefit calculation never reads child_benefit_opts_out.
spi_income_band_donors(#1006,uk_runtime/spi_band_donors.py) copies 120 FRS households per reserved income band into the SPI channel and starts each at the band's published taxpayers over 120, about 2,992, 508, 167 and 83 per row. The copies are appended to the incoming weights and the stage recordsdeclared_factor=None, so about 450,000 households of mass are created rather than moved, and the graph declares the node's mass policy asfree. The release-input coverage manifest has no family entry for the stage, so its mass record is never checked by reason. The draw usesnumpy.random.default_rng(3)over person-sorted candidates rather than identity-keyed uniforms, so the identity-stability tool cannot reconstruct it from ids alone, and the stage requires at least 120 positive-propensity candidates per band rather than scaling its count on small frames.#1045 worked through the same questions for the CGT band donors and ended up replacing them with a mass-conserving support split, because the Table 3 redraw reassigns every donor's value and the donors' only remaining role was row support. The principles that came out of it apply here: support rows conserve mass and declare it, carry light and equal weights within a band, and draw reproducibly from ids. The SPI donors are different in the one way that matters: they carry real tape income vectors that every later stage imputes from, so the channel design fits them directly. Donors carry equal weights within each band, chosen so no donor exceeds a declared maximum weight and every band keeps a declared minimum count; the mass they carry is reallocated from the households of the income band each donor is drawn from, proportionally, exactly as the SPI support channel takes its copies' mass from the FRS households of their region, so total mass and each band's mass are conserved and the record carries
declared_factor=1.0with a declared reason. The coverage manifest gains a family entry withmass_conservingsemantics, the graph policy becomesdeclared, the seating usesstable_identity_uniformson household ids with a declared seed and salt so the E8 receipt reconstructs it, and small frames scale the count and the mass with recorded scales, asserting the published band masses only when both are one. Themass_increasing_supportsemantics then loses its last user and can retire from the validator.Related: #970 on demoted donors and the projection fence, since the same weight leverage applies to any support row calibration can move ten-fold.
Measured on the same build (arm ch3 at the #1045 head), the
was_wealthdebt segment (#685) puts £381bn of its £1,730bn ofmortgage_debton households whosetenure_typeis notOWNED_WITH_MORTGAGE: 22% of the mass on 2,738 rows (1.4m households), £189bn of it on SPI-synthetic rows, against 10% in the WAS donor. The incidence climbs with income (15% of the top income decile of non-mortgaged households carries a mortgage, against 6% of the donor's outright owners and 5% of its private renters) and one in five of the outright-owner rows with a mortgage owes more than itsmain_residence_value. The Junepopulace_uk_2023release carried £740bn (48%), the figure Nikhil flagged.has_mortgage_tenureenters the segment as one soft predictor of the sign gate and the QRF, so income and the wealth columns drawn before it dominate the leaves; the FRS repayment flows are clean becausespi_housing_shellzeroes them off mortgaged tenure. The recipient predictoris_rentingis the engine formula, which omitsRENT_FROM_HA(PolicyEngine/policyengine-uk listsRENT_PRIVATELYtwice), while the donor flag isDVPriRntR8 == 1, private renters only: housing-association renters match owners and social renters and receive £300bn ofmain_residence_value(53% positive; donor renters 0%) and £348bn ofproperty_wealth, and council renters match private renters (property wealth positive for 10.7% against 1.1% of the donor's social renters). policyengine-uk 2.101.0 addedmortgage_debtandnet_wealth, so the leak reaches net wealth on the next engine bump.Two more spine defects from Vahid's review of #1045 (items 3 and 6) belong here rather than in issues of their own.
hmrc_cgt_asset_type_spinerealises the residential-property flag by a weighted systematic walk in ascending gain order against a logistic in log gains, and theresidential_gainscheck ofuk_stage_cgt_asset_typebounds the realised-minus-expected gains by the wider of 5 percent andmaximum_gains_sigma × gains_bernoulli_sigma. Summation by parts gives the error asΣ B_i (g_i − g_{i+1}) + B_N g_N, withBthe walk's running balance (bounded by one row's weight), so it is set by the balance at the top of the order times the largest gains: a handful of rows carrying billions. The sigma term is not an envelope for that walk (it assumes independent draws and shrinks when rows are split), the 5 percent term is what binds, and a configuration change that moves the balance across a different top row moves the realised gains by whole rows: armch2(copy cap 80, commit 817e13f, reverted) failed at 10.29bn against a 5.38bn bound with one copy-clone (weight 20.8, GBP 571m of gains per person) carrying GBP 11.9bn of GBP 123bn at a residential probability of 0.009, while cap 60 passes at 0.08 of the bound. Baseline liability is insensitive (18/24 on both schedules from 2025-26); a reform that separates them is not. Separately, the asset-type stage restricts BADR and Investors' Relief claimants to the three eligible types and leans types towards gainers who show the stock they imply (#1045 c1 and c6); with BADR carrying about GBP 18bn, the realised HMRC Table 7 composition, fenced and diagnostic, drifts further from its targets than the control's (unlisted shares 66.6 percent of non-residential gains against 55.3 published; 62.1 on the control), and the restricted type fit is where to hold it.Items this issue covers:
spi_income_band_donors: equal light weights within a band under a declared maximum weight and minimum count, mass reallocated from the households of the income band each donor is drawn from,declared_factor=1.0with a declared reason, graph policydeclared.mass_conservingsemantics, so the terminal family gate checks its record.mass_increasing_supportsemantics from the coverage validator and the manifest generator once no stage declares it (UK CGT: BADR-qualifying gains, the Table 3 income joint and wealth-conditioned gain carriers (#1014) #1045 removed its only user with the band-donor stack).stable_identity_uniformswith a declared seed and salt) so the E8 receipt reconstructs the draw from ids, and recorded count and mass scales on small frames, asserting the published band masses only when both are one.mortgage_debt == 0wherevertenure_type != OWNED_WITH_MORTGAGE, checked by a stage-health gate, the rule the repayment flows already follow. The path preserves the joint structure between the wealth columns rather than clipping after the draw: fit the debt segment stratified by tenure, so mortgaged recipients draw from mortgaged donors and the non-mortgaged pool's off-tenure mortgages (10% of donor mortgage mass) are the mass the rule drops, recorded in the receipt, or give the gate tenure flags with enough signal that the draw itself respects the tenure.is_renting(engine formula, omitsRENT_FROM_HA; donorDVPriRntR8 == 1) with an owner-outright / mortgaged / private-rent / social-rent predictor built the same way on the donor (Ten1R8,DVPriRntR8) and on the recipient (tenure_type), so housing-association and council renters draw from social-renter donors, with a check thatmain_residence_valueis zero off owner tenures. The engine formula's omission needs its own fix in PolicyEngine/policyengine-uk.ch2failure at 817e13f with the relaxed gate as the regression case for the new bound.The issue closes on a national dataset built from the spine with all eight items in, passing every spine and terminal gate with the identity tool green, so the fixes are measured together rather than one licensed build at a time.