Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
980db48
Level the energy stage's electricity to the kWh ONS household spendin…
juaristi22 Oct 28, 2026
e6c4e4e
Vendor the Gambling Commission's machine yield by category
juaristi22 Oct 28, 2026
e4a5a43
Re-pin the gate digests and coverage manifest for the energy level
juaristi22 Oct 28, 2026
84e1a4b
Bind SPI pension income by area, reconciled to the national pension b…
juaristi22 Oct 28, 2026
3e4f820
Merge PR A (microcosm#1419) into PR C
juaristi22 Oct 28, 2026
3cafca8
Give the annual-measure test's stub engine the total pension
juaristi22 Oct 28, 2026
e8bc36b
Regenerate the H2 spine fixture for the energy level on the merged tree
juaristi22 Oct 28, 2026
e3614ce
Level gas to the kWh ONS household spending buys too
juaristi22 Oct 28, 2026
4778f53
Bind the ONS electricity and gas spending rows again
juaristi22 Oct 28, 2026
c4b08a5
Track the register entries that followed microcosm#1278 on #1095
juaristi22 Oct 28, 2026
785a170
Bind the taxpayers with pension income by area at HMRC's level
juaristi22 Oct 28, 2026
a195755
Re-pin the gate digests, coverage manifest and H2 fixture for gas
juaristi22 Oct 28, 2026
a58bc1a
Restate the cars fuel duty hold's ONS figure on the September edition
juaristi22 Oct 28, 2026
a2201d5
Flag the dense build's households in Rural Fuel Duty Relief areas
juaristi22 Oct 28, 2026
d95e213
Re-pin the gate digests for the RFDR export column
juaristi22 Oct 28, 2026
ec248e6
Name microcosm#1095 in the texts that still said #1278 tracks an item
juaristi22 Oct 28, 2026
dc42fa4
Merge PR A's UC take-up gate and care-hours test (microcosm#1419) int…
juaristi22 Oct 28, 2026
e72700f
Bind OBR's betting and gaming receipts in the national calibration
juaristi22 Oct 28, 2026
7870830
Merge PR A's review round (microcosm#1419) into PR C
juaristi22 Oct 29, 2026
f27442f
Regenerate the H2 spine fixture on the merged tree
juaristi22 Oct 29, 2026
7d503bc
Regenerate the local references, membership and compile-parity receipt
juaristi22 Oct 29, 2026
33e8643
Merge main (PR A, microcosm#1419) into PR C
juaristi22 Oct 29, 2026
cd8364b
Balance the Carer's Allowance take-up draws by weight within strata
juaristi22 Oct 29, 2026
accbed3
Re-pin the coverage manifest and the H2 fixture for the balanced draw
juaristi22 Oct 29, 2026
36b0d42
Pin the candidate scorer test at 27,208 active references
juaristi22 Oct 29, 2026
526d6ab
Refuse RFDR flags on malformed codes or none flagged; test the releas…
juaristi22 Oct 29, 2026
27a3e5e
Bound the ONS energy levels by Energy Trends; record Northern Ireland…
juaristi22 Oct 29, 2026
b5cefb2
Refuse a Carer's Allowance stratum that misses its share by over a re…
juaristi22 Oct 29, 2026
a967c64
Hold the pension bridge to its measured gap plus half a point
juaristi22 Oct 29, 2026
8ff6c96
Count the City of London's pension cells among the support-floor rows
juaristi22 Oct 29, 2026
43b600c
Bind SPI Table 3.7's other income by total-income band
juaristi22 Oct 29, 2026
9a3f291
Keep gas on Energy Trends and hold ONS 04.5.2 out; check the QEP method
juaristi22 Oct 29, 2026
699e3aa
Answer Anthony's re-review: count pins, RFDR per system, CA tolerance…
juaristi22 Oct 29, 2026
fdf0c79
Grow the SPI savings rows by the OBR's savings income; bind SPI Table…
juaristi22 Oct 29, 2026
6353e65
Re-pin the UK feed onto the merge of chronicle#394
juaristi22 Oct 29, 2026
b3f4ebf
Pin the national calibration test at 1,403 active references
juaristi22 Oct 29, 2026
6919f44
Pin each UK active reference count once in the tests
juaristi22 Oct 29, 2026
edad8ae
Answer the independent review and Vahid's review of #1420
juaristi22 Oct 30, 2026
ead8943
Regenerate the UK surfaces, fixture and gate digests for commit 36
juaristi22 Oct 30, 2026
216f22a
Answer the re-review of #1420 at ead8943d6
juaristi22 Oct 30, 2026
10f434d
Answer the round-3 re-review of #1420 at 216f22a5f
juaristi22 Oct 30, 2026
5aba83c
Hold the betting and gaming row out again until the solve can measure it
juaristi22 Oct 30, 2026
125d6e7
Answer the round-4 re-review of #1420 at 5aba83cd2
juaristi22 Oct 30, 2026
1626905
Answer the round-5 re-review of #1420 at 125d6e73b
juaristi22 Oct 30, 2026
059225c
Answer the round-6 re-review of #1420 at 162690525
juaristi22 Oct 30, 2026
54d9d5b
Answer the round-7 re-review of #1420 at 059225c89
juaristi22 Oct 30, 2026
0b276e5
Answer the round-8 re-review of #1420 at 54d9d5b6d
juaristi22 Oct 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/1418-carers-allowance-balanced-draw.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The Carer's Allowance take-up stage now balances its draws by weight within strata (#1418), as policyengine-uk's `stratified_uniform` does. The newly entitled carers' identity-keyed uniforms are placed within strata of route (in-household or not), the benefit unit's survey-year means-tested benefit receipt, and the survey-year weekly earnings band (one weekly rate wide above the limit, open above three). Each stratum then claims the take-up rate of its eligible weight to within one record, and the outside pool's draw is balanced the same way at its measured share. Independent draws reach the rate only in expectation: on the few source households over the limit, one draw had put the 2031 taper's cost well below its expectation. The rate, the pool and the caseload are unchanged. The receipt reports each stratum's realised share against its target, support-banded. Each stratum's placement is rotated by a start drawn per stratum (systematic sampling), keyed on its code and its lowest-draw member's identity, so a carer in a stratum of one or two records claims with probability equal to the rate rather than always or never, in any build, and a change of membership re-draws a stratum only when that member changes. The stage refuses a take-up or pool stratum whose drawn weight misses its share by more than its heaviest record, the most the rotated placement allows, and the receipt records the check (`strata_within_one_record`). The stage keys its cache on the shared balanced-draw helper too, so a change to the placement re-runs it on a resumed graph store.
18 changes: 18 additions & 0 deletions changelog.d/1418-electricity-level-ons-spending.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
The UK LCFS stage now sets household electricity at the level ONS household spending implies, not at DESNZ Energy Trends volume. The dense build's local electricity rows move with it, and the national binds ONS 04.5.1 again (#1418).

The stage still rakes kWh to the NEED shape. For a fuel it declares in `spend_level_fuels`, though, its kWh level is the kWh that ONS Consumer Trends household spending buys:
- the class for calendar 2024 (04.5.1 for electricity);
- less QEP's United Kingdom average fixed cost on every UK domestic meter DESNZ counts (Great Britain's sub-national count with Northern Ireland's row, now vendored);
- divided by QEP's United Kingdom average unit cost.

For electricity that is 84.9 TWh against Energy Trends' 91.7 TWh (−7.4%), from ONS's £28.13bn on the 30 September 2026 release. On the last national's design weights that puts the frame's electricity spend about 1% below ONS, where Energy Trends put it about 5% above. The 1% is the standing charges of the meters that aren't frame households' main homes; the next build measures it. Northern Ireland's meters stay in the standing-charge subtraction, and the level block says why: its tariffs carry the charge in the unit rate.

The national measures the ONS 04.5.1 row on the frame's own 2024 electricity column against ONS's calendar-2024 figure (`measurement_period` 2024), as it measures road fuel. Measured at 2025 through the engine, the row would put the engine's projection into the weights: consumer prices, +3.4% from 2024, against +2.5% in ONS's own series (María's ruling of 2026-10-29).

Gas keeps the Energy Trends level, and its ONS 04.5.2 row stays out of the calibration objective (María's ruling of 2026-10-29). The class counts bottled gas (COICOP 04.5.2.2), which the frame holds in `liquid_fuel_consumption`, so binding it on mains gas would count bottled gas twice: ONS's CPI weights put it at 2.2% to 3.2% of the class. #1095 tracks netting it out once Chronicle carries the split. Decision D1's electricity exclusion is deleted; its gas exclusion is replaced by one with that reason.

The local `desnz.subnational.electricity_kwh` rows bind DESNZ's split between authorities and constituencies at the stage's own level, through one factor per fuel. So they move with it: Great Britain's cells now sum to 82.7 TWh, not 89.2 TWh, and the dense build no longer binds ONS nationally against local rows about 8% higher. The gas rows keep the Energy Trends level.

The `uk_stage_lcfs_consumption_energy_rake` gate recomputes the new level from the vendored rows. Because that recomputation shares the stage's inputs, the gate also holds the ONS-spend level to between 0.85 and 1.0 of the Energy Trends volume (0.926 on this release), so a wrong ONS class or vintage, or a unit error, that moves it by more than about 8% fails. ONS's own revisions can move it as far (the June 2026 release gave 0.840), so a re-pin that crosses the bounds asks for the level to be re-adjudicated. Every QEP payment method's level falls inside those bounds, so the gate checks the stage's declared payment method (`all`) directly. The H5's `energy_price_basis` stays calendar 2024.

The level reads QEP's prices once per process, so the local compile no longer re-reads the 2.8 MB QEP resource for each of the 1,957 energy references: the independent review of #1420 measured one local compile growing from about 0.6 to about 21 minutes. The LCFS stage now keys its cache on the module that sets the level, and the spine gates on the module that holds their checks; only the transferred spine gate, which evaluates the energy rake, keys on the level too. So a code-only change re-runs what reads it on a resumed graph store, and nothing else. `tools/diagnose_uk_consumption_taxes.py` reads each row's measurement period from the run's registry, or from the contract without one, and weights every row at the calibration year. So it measures electricity and road fuel at 2024 on the solved weights, as the national binds them. A test holds each bound ONS energy row to a fuel the stage levels to ONS spending, so the gas row cannot rebind on the Energy Trends level.
1 change: 1 addition & 0 deletions changelog.d/1418-pension-income-by-area.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK local surface binds pension income by area (#1418): the new `hmrc.pension_income.amount` rows give each constituency's and local authority's taxpayers' private and State Pension income together, from HMRC SPI 2023-24 Tables 3.15 and 3.14 (count times mean). They are grown to calendar 2025 by HMRC's all-ranges total income growth in ITL Table 2.5, 1.5% below the growth the national State Pension and private pension bands show together (×1.134 against ×1.151). A fan-out sum bridge then reconciles them per grain to those bands summed, the first bridge to sum two national fan-outs. At source the bands sit 3.0% above the constituency cells and 3.2% above the local-authority cells: the area tables leave out the taxpayers HMRC places in no area, who are more often pensioners. So the reconciled cells take ×1.045 at both grains on the compiled surface, where the two authorities without SPI measures are implied from the constituencies. The bridge refuses a source factor outside 2.5% to 3.7% above one, since the gap runs one way, or an index gap past 2%: each bound is the measured gap plus or minus half a point. HMRC rounds the area counts to the thousand, so the rounded-thousand floor (`uk/local_small_cell_rule.json`) defers the 20 count cells it publishes at 5,000 or fewer (17 constituencies, 3 local authorities), where the rounding can reach 10%, reading each at its published value, before the index, and their amount cells with them, since each amount is the count times its mean (María's ruling of 2026-10-29). The deferred cells still count in their leg's control, grown by their bound neighbours' index, and in the bridge's totals at source. The frame reads each taxpayer's private and State Pension over the year, `total_pension_income_annual`. Two authorities with no SPI measures, as for employment, are signed deferrals, and the metric is appended last at both grains so no earlier metric renumbers. The `hmrc.pension_income.count` rows bind the taxpayers with either pension at HMRC's own area level, grown by ITL Table 2.5's all-ranges taxpayer growth. No national row counts that union: the national bands count State Pension and private pension recipients apart, and most taxpayers of State Pension age are both. The rows are declared a subset of ITL's taxpayers and a local family with no higher control.
1 change: 1 addition & 0 deletions changelog.d/1418-spi-interest-obr-savings-path.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The SPI Table 3.7 interest rows by band now grow from 2023-24 to the 2025 calibration period by the OBR's forecast growth of savings income (October 2026 EFO, receipts Table 4.5: about x1.36 over calendar 2025's window), the path the Budget's savings costing grows its base by, in place of ONS household interest income then GDP per head (x1.192) (#1418, María's ruling of 2026-10-29). The engine's own interest projection stays GDP per head until PolicyEngine/policyengine-uk#2638 settles it, so the calibration reweights to the OBR level at 2025.
3 changes: 3 additions & 0 deletions changelog.d/1418-spi-other-income-by-band.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
The national calibration binds SPI 2023-24 Table 3.7's other income by total-income band (`hmrc.spi.other_investment_income.amount_by_total_income_band`, taxpayers only, on `other_investment_income`), uprated to 2025 by the OBR's forecast growth of savings income (October 2026 EFO, receipts Table 4.5), as the interest rows now are (#1418). No row held that column, so the Income Tax on savings rows were met mostly through it: about £9.5bn on PR A's national against Table 3.7's £5.85bn (María's ruling of 2026-10-29).

The family sits in the zero-support floor's HMRC amounts scope, so a band no household carries, most likely £500k to £1m or £1m and over, is dropped and receipted, not refused.
1 change: 1 addition & 0 deletions changelog.d/1418-spi-self-employment-bands.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The national calibration binds SPI 2023-24 Table 3.10's self-employment bands up to £10,000 (#1418, row 15; María's ruling of 2026-10-29). The new targets are `hmrc.spi.self_employment_income.count_by_self_employment_income_band` and `amount_by_self_employment_income_band`: the individuals with self-employment income, and that income, in the eight bands from £1 to £10,000. That is the small-profit end the Budget's trading allowance measures reach. Table 3.10 counts every individual with a self-employment source, taxpayer or not, so the targets carry no taxpayer filter. Each person is banded on their own self-employment income after loss relief, and the top band is closed at £10,000. The bands above stay with Table 3.6's rows by total income, so the two tables don't pull against each other. Counts grow to 2025 with HMRC's projected growth of all taxpayers (Income Tax liabilities statistics Table 2.5). Amounts grow by that count index times the OBR's mixed income per head, so each band's mean moves as the engine moves incomes; on mixed income per head alone, each band's implied mean would have fallen 3.5%. Both families sit in the zero-support floor's HMRC scopes, so a band no household carries is dropped and receipted, not refused.
9 changes: 9 additions & 0 deletions changelog.d/1418-uk-feed-repin-53815b2.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
The UK Chronicle feed is re-pinned to `53815b2`, the merge of PolicyEngine/chronicle#394 (#1418). It adds SPI 2023-24 Table 3.10 and the OBR's October 2026 EFO beside March's, and it sits on the `0b721a4` merge of PolicyEngine/chronicle#392 (HMRC's ISA statistics). That is 1,511,063 rows, up 7,748, and the schema digest is unchanged.

A new vendored resource, `obr_savings_income_growth_facts.json`, carries the eight October 2026 EFO Table 4.5 savings income growth rates. The other vendored resources carry the same rows under the new feed identity.

On the national surface:
- the 13 SPI Table 3.7 interest rows grow by the OBR's savings income and move up 13.4%;
- Table 3.7's other income adds 13 rows, and Table 3.10 adds 16 (eight bands each of counts and amounts);
- `obr.winter_fuel_allowance` is pinned to March's expenditure table. October's table gives FY2025-26 as outturn, 0.02% above March's projection, and the selector would otherwise match both;
- no other compiled value moves. The other OBR targets window FY2024-25 and FY2025-26 into calendar 2025, and only March's edition carries FY2024-25.
1 change: 1 addition & 0 deletions changelog.d/1418-uk-rfdr-area-flag.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The dense build now sets policyengine-uk's `in_rural_fuel_duty_relief_area` on each placed household from its assigned output area, so the Rural Fuel Duty Relief prices its petrol and diesel at the relieved rate (#1418). HMRC Notice 2001 opens the relief in the Inner and Outer Hebrides, the Northern Isles, the Islands in the Clyde, the Isles of Scilly, the post town of Hawes and sixteen mainland postcode districts. It names the island groups without bounding them, so `tools/build_uk_rfdr_area_layer.py` declares Microcosm's translation into postcode districts, unit postcodes and excluded units, with sources. It then reads the ONS postcode lookup and writes `uk/rfdr_area_layer.json`, the 2021 output areas of England and Wales and 2022 output areas of Scotland more than half of whose postcodes lie in the scheme. That is 41 in England and 1,053 in Scotland, from 7,104 postcodes of the August 2026 lookup. Islands that share a sector with the mainland (Raasay, Seil, Easdale, Luing, Kerrera and Lismore) and the post town of Hawes are listed by unit postcode. The post-geography node `uk.full.geography.rfdr` (`--rfdr-localisation area`, the default with the atomic assignment; `none`, the default with the legacy assignment, is for measurement builds and refused by a release candidate) reads it. Northern Ireland has no area in the scheme. The national line, which has no areas, leaves the engine's default of false (María's ruling of 2026-10-28). The node refuses a Great Britain output-area code that is not well-formed for its system. On a full-scale build it also refuses a system (England and Wales, or Scotland) with households none of which is flagged, so a change of code format or vintage cannot publish its households unflagged; a sampled rung may place none of the 41 English areas, so it checks the codes only. The committed layer is built from the NSPL zip Chronicle registers. The tool pins the registration's package id, file and sha256, and `--check` rebuilds the layer in memory and compares it with the committed files. A test holds the provenance to the high value home stock's own record of the zip.
1 change: 1 addition & 0 deletions changelog.d/1418-uk-support-floor-signature.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK release certifier now checks a run's support-floor drops against the floor's signature alone: its tracking, approver, approval date and expiry. A floor whose reason or adjudication is re-worded can still certify a run that dropped rows under it, and a new signature still refuses (#1418).
23 changes: 23 additions & 0 deletions docs/uk-chronicle-feed-repin.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,3 +322,26 @@ national and local reference files are byte-identical, the membership moves only
and both compile-parity receipts are unchanged. Both surfaces were regenerated from the national
and local cuts, as for `200bf6d`. No reference moves, so the national register version does not
either. The high-value home stock still names `735820c`: the addition touches none of its inputs.

The `53815b2` re-pin (microcosm#1418 PR C, the merge of PolicyEngine/chronicle#394, on top of the
`0b721a4` merge of PolicyEngine/chronicle#392) brought SPI 2023-24 Table 3.10 and the OBR's October
2026 EFO beside March's, with #392's HMRC ISA statistics (1,511,063 rows; the schema digest is
unchanged). The merge commit's tree is the pull request head's (`2ba21f3`), merged onto an unmoved
base, so the artifact was built from that head's bundle and carries the merge commit's identity.

A new vendored resource, `obr_savings_income_growth_facts.json`, carries the eight October 2026 EFO
Table 4.5 savings income growth rates the SPI Table 3.7 interest and other-income rows now grow by.
The other vendored resources, and the constituency energy resource rebuilt beside them, carry the
same rows under the new feed identity. On the national surface only the intended values move: the
13 interest rows (up 13.4%), 13 other-income rows and 16 Table 3.10 rows, whose zero band and bands
from £10,000 are signed out row by row. October's edition starts the OBR lines at FY2025-26, so the
targets that window FY2024-25 and FY2025-26 into calendar 2025 still resolve to March's facts;
`obr.winter_fuel_allowance`, which reads FY2025-26 alone, matched both editions and is pinned to
March's expenditure table (`source_table`). On the local surface the statuses and the reference
file are unchanged, and the membership moves its feed label and the DESNZ gas cells, which return
to the Energy Trends level in the same PR.

The national register version moves with the new references, so the uk-candidate-eval scoring
register is re-frozen before the next national solve. The high-value home stock still names
`735820c`: its inputs are the same on `53815b2` (the feed-gated fingerprint check passes), so it
was not rebuilt.
Loading
Loading