Repository navigation
Band edges derive from the exclusion-pruned register, silently widening surviving banded targets #792
Description
Found while executing the #757 exclusion pass (working packet: the 757-swap acceptance evidence, runs of 2026-08-26).
The measurement
Three seam runs on the same input (spine-g.h5, sha 02900abe…), same doctrine (1500 epochs, family_equal), same code, differing only in the measure-exclusion register:
| run | register | targets bound | result |
|---|---|---|---|
| rebind proof | 5 entries (salsac) | 403 | baseline |
| control | the same 5 entries, operator-supplied | 403 | all 403 initial estimates reproduce the rebind exactly (0 mismatches) |
| exclusion pass | 47 entries (the adjudicated #757 pass) | 361 | 12 of the 361 shared measures move, some by multiples |
The movers (initial estimates, design weights — the solve has not run yet):
| measure | 403-target run | 361-target run | shift |
|---|---|---|---|
dwp/uc_payment_dist/LONE_PARENT_annual_payment_9_600_to_10_800 |
33,497 | 320,780 | +858% |
dwp/uc_payment_dist/SINGLE_annual_payment_7_200_to_8_400 |
66,003 | 370,400 | +461% |
dwp/uc_payment_dist/SINGLE_annual_payment_12_000_to_13_200 |
43,020 | 185,620 | +332% |
hmrc/self_employment_income_income_band_40_000_to_50_000 |
£10.8bn | £25.7bn | +138% |
hmrc/self_employment_income_count_income_band_300_000_to_500_000 |
9,314 | 18,183 | +95% |
hmrc/state_pension_income_band_40_000_to_50_000 |
£9.3bn | £16.8bn | +81% |
dwp/uc_payment_dist/LONE_PARENT_annual_payment_20_400_to_21_600 |
20,439 | 33,506 | +64% |
hmrc/self_employment_income_income_band_500_000_to_1_000_000 |
£2.6bn | £3.8bn | +45% |
hmrc/private_pension_income_count_income_band_70_000_to_100_000 |
357,690 | 511,890 | +43% |
hmrc/property_income_count_income_band_300_000_to_500_000 |
21,021 | 24,451 | +16% |
hmrc/dividend_income_income_band_300_000_to_500_000 |
£3.2bn | £3.6bn | +13% |
dwp/uc_payment_dist/SINGLE_annual_payment_20_400_to_21_600 |
8,837 | 9,766 | +11% |
The other 349 shared measures are bit-stable, including every obr.*, ons.*, slc.*, voa.*, scotgov.* measure and the UC caseload counts.
Diagnosis
The mover set is confined to the two families whose measures pass through engine-stochastic components (income-band assignment interacting with engine-randomized variables; UC annual-payment banding). The materialized target roster determines which engine variables are computed and in what order, so a global RNG stream inside the engine (or its imputation defaults) is consumed differently when the roster changes — the measured value of target A depends on whether target B is materialized. The measure-resolution receipt already records the roster; nothing fences the cross-talk.
Why it blocks the certified cut
The #757 release-cut certification composes over uk_target_fit verdicts measured on this surface. With exclusion-set-dependent materialization, signing an exclusion moves other targets' measured values (three of the 361-run's beyond-25% failures — self_employment 40-50k +128%, state_pension 40-50k +118%, private_pension_count 70-100k +66% — are artifacts of this class, not fit regressions). The surface must be roster-invariant, or each measure's RNG must be isolated (per-variable seeding keyed by variable identity, never by computation order), before a certified line can bind on it.
Suggested fix directions
- Isolate engine-side randomness per variable (identity-keyed seeding, the same doctrine the spine's own draws follow), or
- Materialize the full compiled register always and prune only at binding time (restores roster-invariance at the cost of computing excluded measures), plus a regression fence: two materializations under different rosters must agree on shared measures.
Parent: #757 (WS-E of #145). Evidence: calibration_diagnostics_rebind.json / calibration_diagnostics_rng_control.json / calibration_diagnostics_055_exclusions.json in the 757-swap acceptance dir.
Activity
[-]UK measure materialization is exclusion-set-dependent: 12 target measures move when the register roster changes[/-][+]Band edges derive from the exclusion-pruned register, silently widening surviving banded targets[/+]juaristi22 commentedon Aug 27, 2026
Diagnosis correction — the RNG hypothesis is refuted; the mechanism is deterministic band-edge derivation. (Retitled accordingly.)
What rules RNG out: formula-time randomness is statically banned in policyengine-core (a bytecode fence at
Variableconstruction;random()raises); every UK stochastic draw is minted spine-side, identity-keyed (blake2b overseed:salt:id), and frozen into the H5 as an input column; and this issue's own control receipts already prove it —calibration_diagnostics_rebind.jsonvscalibration_diagnostics_rng_control.json(same roster, different code pins, same input sha) differ on 0 of 403 rows, and the 055-exclusions vs phase3 runs (payload-identical spines) differ on 0 of 361.The actual mechanism: banded measures take each band's upper edge from the lower edges of sibling specs in the live (already-pruned) registry —
_band_edges_by_groupiteratesregistry.specs(target_materialization.py), and_band_boundstakes the next surviving edge, else infinity. The exclusion register is applied before materialization (tools/calibrate_uk_national_dataset.py), so excluding a band silently widens its lower neighbour. The register was acting as a measure-definition input, not a row filter.Proven to FP exactness on the acceptance receipts: every 361-roster initial estimate equals its 403-roster value plus the 403-roster values of exactly the excluded sibling bands it absorbed (max residual 1.1e-5 on values up to £2.6e10 — e.g.
LONE_PARENT_annual_payment_9_600_to_10_800's upper edge went 10,800 → 20,400, absorbing 8 excluded bands: +857.6%). Replaying the edge logic predicts 21 widened bands; the 12 movers are those whose absorbed region carries mass, and the other 9 are bit-stable because it is empty. 12 + 9 = 21. The two mover families are the only banded families in the register, hence 349 bit-stable.These are not fit regressions: the four
uk_target_fitcells this issue blocks pass (or nearly) at published band widths —self_employment_income_income_band_40_000_to_50_000+7.2% at the 403-roster vs +128.2% at 361;state_pension40–50k +26.3% vs +118.1%;private_pension_income_count70–100k +11.6% vs +66.3%;self_employment_income_count300–500k −0.04% vs +39.6%.Of the issue's two fix directions: (1) per-variable seeding is a no-op — already the house doctrine, nothing on this path is stochastic; (2) is the fix, landed as PR #803: band edges derive from the compiled pre-exclusion register (
band_edge_registrythreaded through the kernel, the UK stage,run_uk_calibration, and the seam tool), the kernel refuses a banded spec absent from the supplied edge set, the seam refuses a pruned register without its compiled band-edge source (and one whose extras don't exactly reconstitute the exclusion receipt), and a roster-invariance regression fence asserts survivors are bit-identical under sibling exclusion — it fails on the pre-fix code by construction. The 12 movers reverting and the 4 cells reading at published widths is verified on the next licensed seam run.Derive band edges from the compiled register, never the pruned roster
Derive band edges from the compiled register, never the pruned roster
Disposition the #803 review: required band-edge register, refusing co…
Merge pull request #803 from PolicyEngine/792-band-edge-fix
juaristi22 commentedon Aug 28, 2026
Data claim verified on the post-#803 seam runs. The fix's prediction — the four blocked
uk_target_fitovershoots are band-edge artifacts that pass at published widths — is confirmed on spine-i, same register (47 exclusions), same doctrine (1500 epochs,family_equal), code differing by the #803 merge:terminal_gates_055_exclusions.json)hmrc/self_employment_income_income_band_40_000_to_50_000hmrc/state_pension_income_band_40_000_to_50_000hmrc/private_pension_income_count_income_band_70_000_to_100_000hmrc/self_employment_income_count_income_band_300_000_to_500_000The fifth hmrc cell in the blocked set,
hmrc/dividend_income_count_income_band_500_000_to_1_000_000(−36.7%, the excluded amount-sibling's count), also recovered to −2.2% — its undershoot was the same corruption from the other side.All four (and the fifth) hold on spine-j as well (v11: +0.1% / +18.2% / +10.4% / −0.1% / −2.3%). Receipts:
runs/uk-623-first-calibrated/spine-assessment-v10/and-v11/calibration diagnostics againstdata/ukds/acceptance/757-swap/terminal_gates_055_exclusions.json. With #810's register revision (the #807 lift), the entire hmrc SPI band surface is green — the residualuk_target_fitblockers are UC-side only.