Skip to content

UK spine follow-ups after #1045: SPI band donors as a mass-conserving support channel, WAS tenure coherence, the residential draw's bound and Table 7 composition, then a national build #1063

Description

@juaristi22

spi_income_band_donors (#1006, uk_runtime/spi_band_donors.py) copies 120 FRS households per reserved income band into the SPI channel and starts each at the band's published taxpayers over 120, about 2,992, 508, 167 and 83 per row. The copies are appended to the incoming weights and the stage records declared_factor=None, so about 450,000 households of mass are created rather than moved, and the graph declares the node's mass policy as free. The release-input coverage manifest has no family entry for the stage, so its mass record is never checked by reason. The draw uses numpy.random.default_rng(3) over person-sorted candidates rather than identity-keyed uniforms, so the identity-stability tool cannot reconstruct it from ids alone, and the stage requires at least 120 positive-propensity candidates per band rather than scaling its count on small frames.

#1045 worked through the same questions for the CGT band donors and ended up replacing them with a mass-conserving support split, because the Table 3 redraw reassigns every donor's value and the donors' only remaining role was row support. The principles that came out of it apply here: support rows conserve mass and declare it, carry light and equal weights within a band, and draw reproducibly from ids. The SPI donors are different in the one way that matters: they carry real tape income vectors that every later stage imputes from, so the channel design fits them directly. Donors carry equal weights within each band, chosen so no donor exceeds a declared maximum weight and every band keeps a declared minimum count; the mass they carry is reallocated from the households of the income band each donor is drawn from, proportionally, exactly as the SPI support channel takes its copies' mass from the FRS households of their region, so total mass and each band's mass are conserved and the record carries declared_factor=1.0 with a declared reason. The coverage manifest gains a family entry with mass_conserving semantics, the graph policy becomes declared, the seating uses stable_identity_uniforms on household ids with a declared seed and salt so the E8 receipt reconstructs it, and small frames scale the count and the mass with recorded scales, asserting the published band masses only when both are one. The mass_increasing_support semantics then loses its last user and can retire from the validator.

Related: #970 on demoted donors and the projection fence, since the same weight leverage applies to any support row calibration can move ten-fold.

Measured on the same build (arm ch3 at the #1045 head), the was_wealth debt segment (#685) puts £381bn of its £1,730bn of mortgage_debt on households whose tenure_type is not OWNED_WITH_MORTGAGE: 22% of the mass on 2,738 rows (1.4m households), £189bn of it on SPI-synthetic rows, against 10% in the WAS donor. The incidence climbs with income (15% of the top income decile of non-mortgaged households carries a mortgage, against 6% of the donor's outright owners and 5% of its private renters) and one in five of the outright-owner rows with a mortgage owes more than its main_residence_value. The June populace_uk_2023 release carried £740bn (48%), the figure Nikhil flagged. has_mortgage_tenure enters the segment as one soft predictor of the sign gate and the QRF, so income and the wealth columns drawn before it dominate the leaves; the FRS repayment flows are clean because spi_housing_shell zeroes them off mortgaged tenure. The recipient predictor is_renting is the engine formula, which omits RENT_FROM_HA (PolicyEngine/policyengine-uk lists RENT_PRIVATELY twice), while the donor flag is DVPriRntR8 == 1, private renters only: housing-association renters match owners and social renters and receive £300bn of main_residence_value (53% positive; donor renters 0%) and £348bn of property_wealth, and council renters match private renters (property wealth positive for 10.7% against 1.1% of the donor's social renters). policyengine-uk 2.101.0 added mortgage_debt and net_wealth, so the leak reaches net wealth on the next engine bump.

Two more spine defects from Vahid's review of #1045 (items 3 and 6) belong here rather than in issues of their own. hmrc_cgt_asset_type_spine realises the residential-property flag by a weighted systematic walk in ascending gain order against a logistic in log gains, and the residential_gains check of uk_stage_cgt_asset_type bounds the realised-minus-expected gains by the wider of 5 percent and maximum_gains_sigma × gains_bernoulli_sigma. Summation by parts gives the error as Σ B_i (g_i − g_{i+1}) + B_N g_N, with B the walk's running balance (bounded by one row's weight), so it is set by the balance at the top of the order times the largest gains: a handful of rows carrying billions. The sigma term is not an envelope for that walk (it assumes independent draws and shrinks when rows are split), the 5 percent term is what binds, and a configuration change that moves the balance across a different top row moves the realised gains by whole rows: arm ch2 (copy cap 80, commit 817e13f, reverted) failed at 10.29bn against a 5.38bn bound with one copy-clone (weight 20.8, GBP 571m of gains per person) carrying GBP 11.9bn of GBP 123bn at a residential probability of 0.009, while cap 60 passes at 0.08 of the bound. Baseline liability is insensitive (18/24 on both schedules from 2025-26); a reform that separates them is not. Separately, the asset-type stage restricts BADR and Investors' Relief claimants to the three eligible types and leans types towards gainers who show the stock they imply (#1045 c1 and c6); with BADR carrying about GBP 18bn, the realised HMRC Table 7 composition, fenced and diagnostic, drifts further from its targets than the control's (unlisted shares 66.6 percent of non-residential gains against 55.3 published; 62.1 on the control), and the restricted type fit is where to hold it.

Items this issue covers:

  1. The channel design for spi_income_band_donors: equal light weights within a band under a declared maximum weight and minimum count, mass reallocated from the households of the income band each donor is drawn from, declared_factor=1.0 with a declared reason, graph policy declared.
  2. A family entry for the stage in the release-input coverage manifest with mass_conserving semantics, so the terminal family gate checks its record.
  3. Retiring the mass_increasing_support semantics from the coverage validator and the manifest generator once no stage declares it (UK CGT: BADR-qualifying gains, the Table 3 income joint and wealth-conditioned gain carriers (#1014) #1045 removed its only user with the band-donor stack).
  4. Identity-keyed seating (stable_identity_uniforms with a declared seed and salt) so the E8 receipt reconstructs the draw from ids, and recorded count and mass scales on small frames, asserting the published band masses only when both are one.
  5. Mortgage debt only on mortgaged tenure: the stage ends with mortgage_debt == 0 wherever tenure_type != OWNED_WITH_MORTGAGE, checked by a stage-health gate, the rule the repayment flows already follow. The path preserves the joint structure between the wealth columns rather than clipping after the draw: fit the debt segment stratified by tenure, so mortgaged recipients draw from mortgaged donors and the non-mortgaged pool's off-tenure mortgages (10% of donor mortgage mass) are the mass the rule drops, recorded in the receipt, or give the gate tenure flags with enough signal that the draw itself respects the tenure.
  6. Tenure predictors on both sides of the WAS fit: replace is_renting (engine formula, omits RENT_FROM_HA; donor DVPriRntR8 == 1) with an owner-outright / mortgaged / private-rent / social-rent predictor built the same way on the donor (Ten1R8, DVPriRntR8) and on the recipient (tenure_type), so housing-association and council renters draw from social-renter donors, with a check that main_residence_value is zero off owner tenures. The engine formula's omission needs its own fix in PolicyEngine/policyengine-uk.
  7. Residential draw: stratify the walk by gain band with a per-band offset, as the BADR half already does per Table 4.1 band, so the error is bounded band by band; replace the Bernoulli sigma in the gate's bound with the walk's deterministic per-band bound (the heaviest row's weight times the band's largest gain) and report the top-row concentration (share of residential gains on the ten heaviest placed rows) in the stage receipt; reproduce the ch2 failure at 817e13f with the relaxed gate as the regression case for the new bound.
  8. Table 7 composition: bind it inside the restricted type fit (claimants' types drawn from the eligible three at Table 7's conditional shares among those three, non-claimants' types at the shares that restore the published marginal composition), report the composition by claimant status against Table 7 in the stage receipt, and keep the diagnostic fence until the fit holds it.

The issue closes on a national dataset built from the spine with all eight items in, passing every spine and terminal gate with the identity tool green, so the fixes are measured together rather than one licensed build at a time.

Activity

  1. changed the title [-]Make the SPI income-band donors a mass-conserving support channel[/-] [+]UK spine follow-ups after #1045: SPI band donors as a mass-conserving support channel, WAS tenure coherence, the residential draw's bound and Table 7 composition, then a national build[/+] on Sep 30, 2026
  2. juaristi22 commented on Oct 2, 2026

    @juaristi22
    CollaboratorAuthor

    Status (2026-10-02): draft PR #1089 carries all eight items plus three fold-ins, stacked on #1084 (uk-pensions-1069); receipts in experiments/1063-uk-spine-followups.md.

    What landed

    • Items 1, 2, 4: the SPI income band donors are a mass-conserving support channel. Each band seats max(120, ceil(published taxpayers / 250)) donors at equal weights, funded in proportion from the households of each donor's region; graph policy declared, no created mass, an identity-keyed seating the E8 receipt reruns from ids with the licensed tape.
    • Item 3: mass_increasing_support is retired from the coverage validator.
    • Items 5, 6: WAS wealth is coherent with tenure (no mortgage debt off mortgaged tenure, no main-residence value off owner tenures) and with its own totals; new gate uk_stage_was_wealth_coherence. The rule drops £127bn of the donor's £1,266bn mortgage mass, all other-property mortgages off tenure.
    • Items 7, 8: the residential-property flag is carried as weight by a new expansion stage, cgt_residential_split, after the incidence anchor. Each liable gainer's solved probability splits the household into arms at p·w and (1 − p)·w (2^k arms for k gainers), so the residential count and gains are identities on HMRC Table 8a in every gain band at design weights, with no draw and no seed; calibration decides between the arms as it does for the incidence clones. The asset-type stage types the arms and fits Table 7 once per claimant status with an error-diffusion walk (seed 554 retired). A per-band walk was tried first and replaced: its gains error in the open band was bounded by one stake several times the band's whole expectation.
    • Fold-ins: salary sacrifice predicted only for people who are paid; Child Benefit claims and opt-outs redrawn after the SPI chain against HMRC's August 2025 statistics (new stage child_benefit_take_up: 86.7% of eligible children claimed for against HMRC's 86.6%, 9.0% of claiming families opted out, 11.1m children in payment against 13.5m before).
    • Release cut: the certifier rehearsal on the 2026-09-30 build failed 5 of 20 gates; each is cleared at its source (export surface read without ids and with the weight, 43 declared outputs allow-listed, incapacity_benefit_reported dropped at the release boundary, FRS 2024-25's recoded TRAIN2 read, the LCFS donor floored at 0 with a receipt, three signed register entries). The two plan-2 measure exclusions are renewed to 2026-11-03.

    Measured (licensed local arms, nothing released): the walk arm passed every spine gate with calibration loss 0.00853 against main's 0.00836, ESS 5,217 (5,198), max-to-median weight 1,115 (1,079; bound 1,151); the WAS arm passed every spine gate including the coherence gate. The arm on the residential split, the final national build, its certifier run and the identity tool are still to land.

    Not in this PR: VAT is an engine question (PolicyEngine/policyengine-uk#1996). Two further engine issues are to be filed: is_renting omits housing-association tenure, and the Child Benefit calculation never reads child_benefit_opts_out.

  3. 1 remaining item

  4. added 2 commits that reference this issue on Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions