Repository navigation
Add HMRC benefits in kind statistics, July 2026, and keep one OBR EFO number per line and year - #399
Conversation
|
Review of 9ca635f (draft, so a comment rather than a formal review). First round; no earlier comments or threads. I reviewed b555d8e. 9ca635f only adds the workbook's R2 location to the manifest, and the BIK The benefits in kind package is right: all 544 facts match the pinned workbook, and the pinned file is HMRC's current one. The #397 drop is exact for 425 of 426 cases, and the pins reconcile. One should, about a March line-year that October also publishes under a different line id, and one nit. Findings1. should: one kept March fact still has an October number for the same line and year.
2. nit: a few places still describe the pre-#397 picture. Benefits in kind package
#397: what drops and what staysI built the nine EFO packages at main (
VerdictThe BIK half is ready. On #397, item 1 needs a decision either way. Then it waits on the Microcosm re-pin order in the body and a green bundle job. What I ranFresh clone at b555d8e (BIK
Not verified
🤖 Generated with Claude Code |
Vahid's review of #399: March kept obr.efo_2026_03.receipts.betting_and_gaming_duties_accrued.fy2025, but October publishes the same line and year (concept obr.betting_and_gaming_duties, accrued, UK, FY2025) as table 4.1's 'Betting and gaming taxes', under another line id. The generators now match March and October on concept, unit, constraints, domain, geography, entity and period, not on line id, so that record set goes too. The October build keeps the line's March record set as its template, in its place, so October's facts are unchanged. The test compares facts the same way, and March keeps 73 facts. Where the build, the October packages' headers, a test docstring, the October doc and the #393 changelog fragment still described the two vintages side by side, they now say the years both publish are October's only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, Vahid. Both are fixed in 3e1e7e0. 1. Sheet 3.10's 2025-26 total: dropped. October's table 4.1 'Betting and gaming taxes' (
2. Pre-#397 wording: updated. These now say the years both vintages publish are October's only:
Tests: 🤖 Generated with Claude Code |
|
Review of 3e1e7e0, round 2 Thanks, María. Item 1 is resolved. Item 2 is resolved apart from one sentence. Status of round 11. should (kept March fact overlapping October table 4.1): resolved.
2. nit (pre-#397 wording): mostly resolved. The build docstring, No new findings. VerdictReady as far as items 1 and 2 go, apart from the one changelog sentence. It still waits on the Microcosm re-pin order in the body and a green bundle job. What I ranFresh clone at 3e1e7e0 and a fresh uv venv, both deleted afterwards. I ran one process at a time.
Not verified: the merged-bundle test and its absolute pins. CI's Chronicle bundle and surface jobs were still pending when I posted. 🤖 Generated with Claude Code |
For PolicyEngine/uk-autumn-budget-dashboard-2026#316: Budget 2026 raises company car tax by one percentage point in every band from 2030-31, and policyengine-uk 2.165.0 (PolicyEngine/policyengine-uk#2750) models the company car and car fuel benefits. hmrc-benefits-in-kind-july-2026 carries 544 facts from HMRC's Tables 1 to 4 workbook (15 July 2026): Table 1's recipients, taxable value, average taxable value, and Income Tax and Class 1A NIC liabilities, 2011-12 to 2024-25; Tables 2A, 2C and 2E's car benefit recipients by CO2 band and by fuel type and the average CO2 emissions of company cars over the same years; and Tables 3A, 3B and 4's 2024-25 recipients, taxable value and average taxable value by range of total income and by region. Markers emit no fact, never a zero; each sheet's rounding is quoted; 2024-25 is provisional and payrolled cars are under-reported from 2016-17, on every record set's notes. HMRC's averages are ported as printed, since the rounded cells do not reproduce them. Tables 3A, 3B and 4 are grossed from the SPI cases matched to the P11D and RTI records (survey_aggregate); their 'All' rows print Table 1's totals and are not ported again. Re-pin the merged bundle: +544 facts, one source table and one geography (HMRC's unknown recipient region). The merged-bundle test needs `chronicle publish-raw` for the new artifact first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#397: where two vintages publish a number for the same line and year, Chronicle keeps the later one. The March 2026 receipts, expenditure, economy and aggregates packages drop the 426 record sets whose line and year the October 2026 packages also publish (238, 141, 30 and 17). They keep the 74 October lacks: each line's 2024-25 outturn (calendar 2024 for the economy), and the 2025-26 forecasts of the lines October starts in 2026-27. build_uk_excise_packages.py and build_uk_1095_port_packages.py leave out the years the October receipts package publishes, and the hand-written parts lose the same record sets; the 74 facts kept are unchanged. The high-value council tax surcharge has no 2024-25 value, so build_obr_efo_october_2026.py keeps its March record set as the template for its October line, and the October packages stay byte-identical. The same rule covers vehicle excise duties (chronicle#395), so the excise script's outturn-only special case for them goes; March keeps VED's 2024-25 outturn. Re-pin the merged bundle (-426 facts; semantic duplicate keys 2,687 -> 2,324), and update the tests, package headers and docs that described the March years. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`chronicle publish-raw` uploaded 2026_Tables_1_to_4.ods to r2://ledger-raw/raw/uk/hmrc/hmrc-benefits-in-kind-july-2026/2026/<sha256>/ and wrote storage.r2 into the manifest. `chronicle verify-raw` finds the remote object's SHA-256 and size (42,963 bytes) equal to the manifest's, so the package's facts now carry raw_r2_uri and pass the consumer contract. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vahid's review of #399: March kept obr.efo_2026_03.receipts.betting_and_gaming_duties_accrued.fy2025, but October publishes the same line and year (concept obr.betting_and_gaming_duties, accrued, UK, FY2025) as table 4.1's 'Betting and gaming taxes', under another line id. The generators now match March and October on concept, unit, constraints, domain, geography, entity and period, not on line id, so that record set goes too. The October build keeps the line's March record set as its template, in its place, so October's facts are unchanged. The test compares facts the same way. March keeps 74 facts: 73, and chronicle#395's VED outturn. Where the build, the October packages' headers, a test docstring, the October doc and the #393 changelog fragment still described the two vintages side by side, they now say the years both publish are October's only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3e1e7e0 to
ac9d7bb
Compare
Summary
Two changes, in two commits; a third records the workbook's R2 location, and a fourth answers the first review:
hmrc-benefits-in-kind-july-2026(544 facts), for Company car tax +1 point (2030-31): needs a company car benefit in policyengine-uk and company cars in Microcosm uk-autumn-budget-dashboard-2026#316. Budget 2026 raises company car tax by one percentage point in every band from 2030-31 (Table 6.1). policyengine-uk 2.165.0 (Model the company car and car fuel benefits, with Class 1A NICs policyengine-uk#2750) models the company car and car fuel benefits and their Class 1A NICs. Microcosm draws company car CO2 and fuel at HMRC's Table 2A and 2C margins and measures its calibrated file against these tables, binding none of them. Its publisher facts have to come from Chronicle, so this package lands before Microcosm re-pins its feed.Closes #397.
1. HMRC benefits in kind statistics, July 2026
Source: Benefits in kind statistics: July 2026, published 15 July 2026 under the Open Government Licence v3.0. The Tables 1 to 4 workbook is pinned by SHA-256 in
db/data/hmrc/benefits_in_kind_july_2026/manifest.yaml.packages/hmrc/benefits_in_kind_july_2026/build_package.pywritessource_package.yamlfrom the workbook, and--checkcompares instead.What it carries (121 record sets)
company_car_co2_emissionsbounds, so bands from the two schemes never share a key.statistical_scopeidhmrc:recipient_region_unknown.Not ported:
Record-set ids follow
hmrc.benefits_in_kind_july_2026.<table>.<measure>.ty<year>, and a fact'ssource_record_idappends.<row>.<measure>. For example, 2024-25 car benefit recipients arehmrc.benefits_in_kind_july_2026.table1.recipients.ty2024.car.recipients(920,000).HMRC's semantics
period_coveragenotes. The 2024-25 liabilities also say HMRC bases them on the Survey of Personal Incomes projected forward a year (commentary section 1; background quality report section 3.5).period_coveragenotes, along with what each year could report: no RTI reporting of payrolled cars in 2016-17, voluntary reporting in 2017-18, and compulsory but incomplete reporting from 2018-19. Earlier years note the informal payrolling that kept a few employers' P11Ds out (section 8.1), and 2011-12 to 2014-15 note that electric cars are absent.Provenance
administrative.model_output.survey_aggregate, with survey instrument 'Survey of Personal Incomes matched to employers' P11D and Real Time Information company car records'.hmrc-isa-statistics-2026classes its SPI-matched tables the same way.observation, at countryK02000001except Table 4's regions.The notes cite the release's commentary and HMRC's background quality report. Both were checked against the cited sections.
2. One number per OBR EFO line and year (#397)
The rule (María, 30 October 2026): when two vintages publish a number for the same line and year, Chronicle keeps the later one. The March 2026 packages therefore drop every record set whose line and year the October 2026 package of the same kind publishes. A line and year counts as October's under any line id. That removes 427 facts: receipts 239, expenditure 141, economy 30, aggregates 17. They keep 74:
Sheet 3.10's own 2025-26 total goes: October prints it as table 4.1's 'Betting and gaming taxes', with the same concept and accrued basis under another line id (Vahid's first review).
How:
scripts/build_uk_excise_packages.pyandscripts/build_uk_1095_port_packages.pyleave out every record set whose line and year the October receipts package publishes, matched on concept, unit, constraints, domain, geography, entity and period. Re-running them reproduces the committed receipts package byte for byte. The hand-written parts of the four packages lose the same record sets.council_tax_high_value_surcharge, which has no 2024-25 value.scripts/build_obr_efo_october_2026.pykeeps their March record sets (LINES_LEFT_IN_OCTOBER_ONLY) as the templates for their October lines, each in its original place. Without them, the October lines would vanish.build_obr_efo_october_2026.py --checkpasses. Their headers now say the years both vintages publish are October's only.docs/data-sources/uk-obr-efo-october-2026.md, the chronicle#381 and UK facts for the microcosm#1095 ports: HMRC Stamp Tax Statistics and the corporate share of SDLT, DWP's Move to UC claim rates, the geography of OBR table 4.9's DWP lines, and HBAI with vintages #357 data-source docs and the source checklist's OBR rows now describe the split.Merge order
build_uk_excise_packages.py,build_obr_efo_october_2026.py, the bundle test,test_issue_381_uk_excise_facts.pyand the EFO docs. It merged first, and this PR is rebased onto it (7d5d627), with the three generators re-run and the pins re-measured against that main.Before merge
R2: done.
chronicle publish-rawuploaded the workbook tor2://ledger-raw/raw/uk/hmrc/hmrc-benefits-in-kind-july-2026/2026/a0ee5617…/2026_Tables_1_to_4.odsand wrotestorage.r2into the manifest (9ca635f).chronicle verify-rawmatches the remote SHA-256 and size (42,963 bytes), and the package's facts now pass the consumer contract.The merged-bundle test (
test_build_bundle_writes_merged_consumer_contract) runs in CI only. Its pins are updated from measured builds:hmrcsource goes to 37,216 andobrto 4,539; source tables go to 384.K0200000141,080,K0300000123,558,K040000011,180.The EFO deltas come from partial bundles of the nine EFO packages, built on main and on this branch: 4,818 → 4,391 facts and 363 → 0 duplicate keys, with no period, geography or entity key removed. The BIK counts come from the package's facts, run through
consumer_fact_rowsandbuild_bundle_coveragewith the R2 URIpublish-rawwill write. Its geography names match Chronicle's register, so it adds no name warnings.Chronicle judge reviews, per
AGENTS.md.Tests
tests/test_source_hmrc_benefits_in_kind_july_2026.py(14 tests) re-reads every value from the workbook with pandas, located by the publisher's labels. It also checks markers, notes, provenance, counts by table and year, and the generator's reproduction of the committed package.tests/test_source_obr_efo_october_2026.pygains a test that no March fact shares a line and year with October, by record-set id and by concept, unit, period, geography, entity and constraint set, and that each line keeps one year. The March-line test now also covers the two retained templates.test_issue_381_uk_excise_facts.py,test_issue_357_microcosm_1095_ports.py,test_chronicle_source_package.py(economy and aggregates),test_scale_value.pyandtest_chronicle_obr_council_tax_decomposition.py. The scale-value test swaps the dropped income tax FY2025-26 pin for NI employee FY2024-25, which still discriminates; no expenditure line besides Scottish council tax does. The council tax test checks March's 2024-25 identity and October's 2025-26 to 2031-32.chronicle validate-package hmrc-benefits-in-kind-july-2026is valid with no warnings. After the review fix, the three EFO test files it touches pass again (57 tests), and after the rebase onto UK facts for chronicle#395: Northern Ireland domestic rates, Welsh bus journeys, the Childcare Offer for Wales, VED by payer and CPI gas weights #396 the eleven test files the branch touches pass (120 tests), UK facts for chronicle#395: Northern Ireland domestic rates, Welsh bus journeys, the Childcare Offer for Wales, VED by payer and CPI gas weights #396'stest_source_uk_395_facts.pyincluded. The full suite runs in CI.🤖 Generated with Claude Code