diff --git a/changelog.d/2738.fixed.md b/changelog.d/2738.fixed.md new file mode 100644 index 0000000000..5a1d6d5767 --- /dev/null +++ b/changelog.d/2738.fixed.md @@ -0,0 +1 @@ +Projected the sixteen gambling duty inputs per person from HMRC's 2025-26 receipts and the OBR's October 2026 betting and gaming forecast (receipts Table 4.10), divided by the rates the model charges and by population growth. The inputs no longer stay at their 2024-25 cash level, their weighted totals follow each duty's base as household weights grow with population, and the 2026 and 2027 remote gambling rate rises and the 2027 higher-stake machine rate rise apply once. diff --git a/docs/engineering/gambling-duties.md b/docs/engineering/gambling-duties.md index 4ca593ca85..6aacbabec5 100644 --- a/docs/engineering/gambling-duties.md +++ b/docs/engineering/gambling-duties.md @@ -113,10 +113,11 @@ and [HMRC Notice 452](https://www.gov.uk/guidance/machine-games-duty-excise-noti ### Higher-stake machines from 1 April 2027 -Budget 2026 moves machines with a maximum charge to play of more than £1, or -a maximum prize of more than £100, from the standard rate of 20% to the higher -rate of 25% from 1 April 2027. Finance Bill 2026-27 will amend the stake and -prize thresholds that set each machine's type, and with it the rate +In Budget 2026 the government said it would move machines with a maximum +charge to play of more than £1, or a maximum prize of more than £100, from the +standard rate of 20% to the higher rate of 25% from 1 April 2027. In Finance +Bill 2026-27 the government will amend the stake and prize thresholds that set each machine's +type, and with it the rate ([OOTLAR paragraph 1.50](https://www.gov.uk/government/publications/budget-2026-overview-of-tax-legislation-and-rates-ootlar/budget-2026-overview-of-tax-legislation-and-rates-ootlar)). Today, Finance Act 2012 Schedule 24 paragraph 5(3) puts every machine outside the lower-rate class with a highest charge of at most £5 in the standard-rate @@ -146,7 +147,8 @@ exactly £1 and £100 stays at 20%, while £1 with a £500 prize, or £2 with a B3A: £2 and £500; B4: £2 and £400](https://www.gamblingcommission.gov.uk/licensees-and-businesses/guide/gaming-machine-categories)) exceed one of the thresholds. Category C (£1 and £100) and Category D stay within them. The tax class turns on the highest charge and prize a machine - actually offers, so a category split approximates it. + actually offers, so a category split approximates it. The projection + assumes 90.78% (see [the higher-stake share](#the-higher-stake-share)). - The duty floors the new input at zero and caps it at the standard-rate input. It never taxes takings that the standard-rate input leaves out. @@ -165,10 +167,10 @@ change, or move its date to delay it. It shares fiscal-year segments with the rates, so a rate reform applies to the higher-stake part at whichever rate the switch selects. -The policy costing puts the measure at £135 million in 2027-28 and £130 -million a year to 2031-32 +The Treasury puts the measure at £135 million in 2027-28 and £130 million a +year to 2031-32 ([Budget 2026 policy costings, page 67](https://assets.publishing.service.gov.uk/media/6ae1883fe4367e5c8b0467df/AB26_Policy_Costing.pdf#page=67)). -That costing grows its base with the OBR's Machine Games Duty forecast and nets +The Treasury grows its base with the OBR's Machine Games Duty forecast and nets off a behavioural response: an elasticity of -0.95, a quarter of lost stakes moving to Remote Gaming Duty, general attrition and lower Corporation Tax. The model applies the rates statically to supplied takings and reproduces none of @@ -279,6 +281,215 @@ Sources: [Finance Act 1997, sections 10 and 11](https://www.legislation.gov.uk/u [HMRC July 2026 tables](https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods) and [Gambling Commission 2024/25 industry statistics](https://www.gamblingcommission.gov.uk/statistics-and-research/publication/industry-statistics-annual-report-financial-year-april-2024-to-march-2025). +## Projection + +Each input declares its duty's index, +`gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.`, +and dataset projection grows it by the same duty's series in +`yoy_growth.yaml` (`policyengine_uk/data/uprating_indices.yaml`). A series +grows the duty's base, not its receipts: each year's receipts divided by the +rate the model charges that year. The model then applies its own rate +changes once. Grown with receipts instead, the base would carry Remote +Gaming Duty's rise to 40% and the formula would charge 40% on it again. + +Each series holds the base's growth per person: one plus the base's growth, +over one plus `ons.population` growth, minus one, to six decimals. Dataset +projection grows every household weight by `ons.population`, so each +weighted input grows with its duty's national base. Past +`PROJECTION_END_YEAR`, policyengine-core scales each input by its index and +each weight by the population index, so the weighted input keeps that path +until `create_economic_assumption_indices` stops compounding, and both hold +from then on. Grown by the base's own growth, the weighted bases would count +population growth twice and exceed Table 4.10 by 1.54% in 2031-32. +`uprate_council_tax` +divides each council tax bill's growth by the growth in household weights +for the same reason. + +| Fiscal year | Receipts | Source | +| --- | --- | --- | +| 2024-25 | Base year (index 1) | HMRC July 2026 Betting and Gaming tables, `Table_1_BG_receipts` row 45 | +| 2025-26 | Outturn | The same table, row 46 | +| 2026-27 to 2031-32 | Forecast | OBR October 2026 Economic and Fiscal Outlook, receipts Table 4.10, "Betting and gaming duties forecast breakdown" | +| 2032-33 on | None | The `average_earnings` long-run path, as for the other receipts series | + +After 2031-32 the importer divides the base's growth by one plus +population growth in every year to 2073, the last year `ons.population` +states. Each series' last value stays in force after that, as population's +does. + +`policyengine_uk/utils/import_obr_forecasts.py` writes the series from +`RECEIPTS_SERIES_SPECS`. It reads each rate from the model's parameter file +and averages it over the OBR's April-to-March year by days. The equations +below round receipts and rates for display, and each result comes from the +unrounded figures. Each duty's rate comes out as follows: + +- **Remote Gaming Duty**: 21% to 31 March 2026, 40% from 1 April 2026. + Receipts grow 108.3% into 2026-27 (£2,752.0 million against HMRC's + £1,321.0 million); the base grows (2,752.0 / 40%) / (1,321.0 / 21%) − 1 = + 9.38%. +- **General Betting Duty**: the remote part, 45.5% of the 2024-25 base, goes + from 15% to 25% on 1 April 2027, so the rate on the whole base goes from + 15% to 15% + 10% × 45.5% = 19.55%. Receipts grow 33.8% into 2027-28 + (£655.6 million to £877.4 million); the base grows (877.4 / 19.55%) / + (655.6 / 15%) − 1 = 2.70%. The shares come from Microcosm's split of + HMRC's 2024-25 receipts by the Gambling Commission's 2024-25 gross + gambling yield, held fixed in every year. That split counts all remote + horse racing as UK horse racing, which keeps 15%, so the part at 25% may + be larger, and the 2.70% then overstates the base's growth. Microcosm's + split gives spread betting no part, so it has no weight. +- **Machine Games Duty**: from 1 April 2027 the model charges the + higher-stake part of the standard-rate base at the higher rate (see + [higher-stake machines](#higher-stake-machines-from-1-april-2027)). The + OBR includes that rise and the measures announced since March in + Table 4.10 (EFO paragraph 3.2), and says the rise "increases receipts by a + further estimated £0.1 billion a year from 2027-28" (paragraph 4.45). The + series divides the line by a rate that + charges the higher-stake share of the standard-rate base, 90.78% (see + [the higher-stake share](#the-higher-stake-share)), at 25% from 2027-28. + The importer weights each band by HMRC's 2024-25 liabilities over its + rate, so the rate on the whole base rises from 19.35% to 23.68%. Receipts + grow 22.1% + into 2027-28 (£677.1 million to £826.9 million); the base falls by + (826.9 / 23.68%) / (677.1 / 19.35%) − 1 = −0.23%. The importer reads the + date from `gov.hmrc.gambling_duty.machine_games.higher_stake_machines_at_higher_rate` + and the rates from their parameter files, so the series and the formula + read the same dates and rates. + + The OBR joins the Treasury's two costings as measure 56 in policy + Table 1.1, under the head Betting: the 25% rate and the move of liability + to machine suppliers from April 2028. The Treasury nets off a behavioural + response in its rate costing: an elasticity of −0.95, with a quarter of the + stakes lost moving to Remote Gaming Duty, and a general attrition + assumption. It estimates that moving liability to suppliers reduces + non-compliance and brings payments forward + ([Budget 2026 policy costings, pages 67 and 68](https://assets.publishing.service.gov.uk/media/6ae1883fe4367e5c8b0467df/AB26_Policy_Costing.pdf#page=67)). + Dividing the line by the rate that produced it keeps all of these in the + base, as receipts over the rate does for every other duty. Taking the + measure off the line instead would drop them, and the model's static + rates restore none of them. +- **Gaming Duty**: the household formula charges one effective rate, which + does not change, so the base grows with receipts. The statutory bands do + not enter: any change in the duty's effective rate that the OBR + forecasts stays in the base. +- **Bingo Duty**: 10% until Parliament abolished the duty from 1 April 2026. + A zero rate leaves no base to divide out, so the importer holds the base + at its 2025-26 level while the rate stays at zero. Per person the series + falls as population grows, and the + weighted base holds. Current law charges nothing whatever the base; a + reform that charges Bingo Duty again charges that 2025-26 base in cash + terms. +- **Lottery Duty** (12%) and **Pool Betting Duty** (15%): one rate each, so + the base grows with receipts. + +The series rest on these approximations: + +- HMRC reports cash receipts, while the OBR forecasts accrued receipts in + Table 4.10: its totals equal the "Betting and gaming taxes" row of the + October 2026 Table 4.1, whose note reads "All taxes are on an accrued + basis." Growth into + 2026-27 compares accrued 2026-27 receipts with cash 2025-26 receipts. +- Model years run from 6 April and the OBR's from 1 April. A model year + that takes in five days of a 1 April rate change charges them at the new + rate, so FY2025 Remote Gaming Duty and FY2026 General Betting Duty exceed + the published receipts by 1.2% and 0.4%, and FY2025 Bingo Duty falls short + of them by 1.4%. With the higher-stake input at its share, FY2026 Machine + Games Duty exceeds Table 4.10 by 0.31% for the same reason: the OBR charges + no higher-stake machine at 25% in 2026-27, while the model charges 1 to + 5 April 2027 at 25% in model year 2026. +- Two lines fall where no rate changes. The OBR forecasts Lottery Duty + receipts of £830.5 million for 2026-27, which fall short of HMRC's 2025-26 + outturn of £980.1 million by 15.3%. It forecast £1,008.6 million for the + same year in March 2026's Table 3.10, lists no Lottery Duty measure in + October's policy table, and gives no reason in EFO paragraph 4.45. Pool + Betting Duty + receipts fall from £8.1 million in 2024-25 to £4.0 million in 2025-26 and + a forecast £2.8 million in 2026-27, so its index falls by 65%. + +### The higher-stake share + +`MACHINE_GAMES_HIGHER_STAKE_SHARE` in +`policyengine_uk/utils/import_obr_forecasts.py` holds the share: 90.78% of +standard-rate takings come from machines with a maximum charge above £1 +or a maximum prize above £100. It divides the Gambling Commission's +2024-25 gaming machine yield in Category B, £2,289.8 million, by that in +Categories B and C, £2,522.3 million (Industry Statistics, April 2025 to +March 2026 release, sheet 4, cells K34 and L34, revised figures). The +2007 regulations define a Category B machine as one outside Categories C +and D, and the Commission gives Category C's limits as a £1 charge and a +£100 prize +([Categories of Gaming Machine Regulations 2007, regulation 5](https://www.legislation.gov.uk/uksi/2007/2158/regulation/5); +[Gambling Commission, gaming machine categories](https://www.gamblingcommission.gov.uk/licensees-and-businesses/guide/gaming-machine-categories)). +Category D enters neither side. Its money-prize machines charge at most +20p for at most £10 in cash, which puts them in the lower-rate class +(Finance Act 2012, Schedule 24, paragraph 5(2)). Its other machines pay no +cash prize, so their games fall outside the duty (paragraph 2(2)). A machine's +category caps its charge and prize, while the tax class turns on the +highest charge and prize it actually offers, so the category split only +approximates the class. + +The statistics cover the premises the Commission licenses in Great +Britain: arcades, betting shops, bingo halls and casinos. They leave out: + +- pubs and other alcohol-licensed premises, which may hold only Category C + and D machines; +- members' and commercial clubs, which may hold Category B3A or B4 + machines as well as C and D; +- Northern Ireland, outside the statistics' Great Britain coverage. + +HMRC's 2024-25 standard-rate liabilities, £604.7 million at 20%, give a +UK base of £3,023.7 million, £501.4 million (16.6%) more than the +Commission's Category B and C yield. Differences in definition and timing +between gross gambling yield and dutiable net takings may account for +part of it. Pubs' Category C and D machines never pay the higher rate, so +leaving them out pushes the share up, though skill-with-prize machines there +fall outside the categories. Clubs' taxable Category B4 machines pay it, so +leaving them out pushes the share down. Legally operated Category B3A +lottery machines pay no duty and enter neither side. If the whole +difference stayed at 20%, the share would be £2,289.8 million over £3,023.7 million, 75.73%, and +the 2027-28 base index would rise by 3.13% from its level at 90.78%. The +Commission's provisional 2025-26 figures give 91.41%. The importer holds +the 2024-25 share in every year, as it does the band weights. + +### Datasets without the higher-stake input + +No dataset supplies +`machine_games_standard_rate_higher_stake_net_takings` yet. Until one +does, `machine_games_duty` charges the standard rate on the whole +standard-rate input, the rates before Budget 2026, on a base that grows +by receipts over the rates after it. From 2027-28 the duty therefore +falls short of Table 4.10 by 18.3%: the duty equals Table 4.10's line +times 19.35% / 23.68%. Two constructed households, weighted 700 and 1,900, +hold the 2024-25 national bases between them, and their weighted duty +comes to: + +| £ million | 2027-28 | 2028-29 | 2029-30 | 2030-31 | 2031-32 | +| --- | ---: | ---: | ---: | ---: | ---: | +| Table 4.10 | 826.9 | 857.0 | 865.2 | 875.5 | 896.3 | +| Higher-stake input at 90.78% | 826.9 | 857.0 | 865.2 | 875.5 | 896.3 | +| Higher-stake input absent | 675.5 | 700.0 | 706.8 | 715.2 | 732.2 | +| Shortfall without it | −151.4 | −156.9 | −158.4 | −160.3 | −164.1 | + +A dataset that sets the input to the share of its standard-rate input +closes the gap. Before 2026-27 the input changes nothing; in 2026-27 it +adds the five days at 25%. + +Every input in the gambling folder declares an index, including +`machine_games_standard_rate_higher_stake_net_takings`; +`policyengine_uk/tests/test_gambling_duty_base_projection.py` fails for a new +input that declares none. + +Sources: [OBR October 2026 Economic and Fiscal Outlook](https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/) +(receipts Table 4.10 and policy Table 1.1, in the +[detailed forecast tables zip file](https://obr.uk/download/october-2026-economic-and-fiscal-outlook-detailed-forecast-tables-zip-file/); +receipts workbook SHA-256 +`d115705716d77b11c190385d41cb1924f30080c9331a8a386c04d4e65a368e95`), +[HMRC July 2026 Betting and Gaming tables](https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods), +[Autumn Budget 2026 policy costings, pages 67 and 68](https://assets.publishing.service.gov.uk/media/6ae1883fe4367e5c8b0467df/AB26_Policy_Costing.pdf#page=67), +[Microcosm's 2024-25 gambling bases](https://github.com/PolicyEngine/microcosm/blob/main/docs/evidence/uk-excise-quantities-1359/outputs/national_reconciliation.json) +and the [Gambling Commission's Industry Statistics, April 2025 to March 2026](https://www.gamblingcommission.gov.uk/statistics-and-research/publication/industry-statistics-annual-report-financial-year-april-2025-to-march-2026-official-statistics) +(workbook SHA-256 +`a6ea23b5ddff2f3839ebec2a8cc0d9a0817e218f5bf10921e0172ecae768da58`). + ## Tax totals and fiscal classification The seven component duties enter `gambling_duty`, which enters household and diff --git a/policyengine_uk/data/uprating_indices.yaml b/policyengine_uk/data/uprating_indices.yaml index 1c63a4598f..bbb436d27e 100644 --- a/policyengine_uk/data/uprating_indices.yaml +++ b/policyengine_uk/data/uprating_indices.yaml @@ -170,6 +170,33 @@ gov.economic_assumptions.yoy_growth.obr.per_capita.mixed_income: - self_employment_gross_receipts - trading_loss - owned_company_income_share +# Gambling duty bases grow per person, as household weights grow by +# population, with the OBR's receipts forecast for their duty divided by the +# rates the model charges, so the model applies its own rate changes to them +# once (#2738). +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.general_betting: +- betting_exchange_commission +- financial_spread_betting_net_losses +- horse_and_dog_pool_betting_net_losses +- land_based_fixed_odds_betting_net_losses +- other_spread_betting_net_losses +- remote_other_betting_net_losses +- remote_uk_horseracing_betting_net_losses +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.pool_betting: +- pool_betting_net_receipts +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.remote_gaming: +- remote_gaming_net_losses +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.machine_games: +- machine_games_higher_rate_net_takings +- machine_games_lower_rate_net_takings +- machine_games_standard_rate_higher_stake_net_takings +- machine_games_standard_rate_net_takings +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.gaming: +- land_based_casino_net_losses +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.bingo: +- bingo_net_receipts +gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases.lottery: +- national_lottery_ticket_spending gov.economic_assumptions.yoy_growth.obr.private_pension_index: - private_pension_income gov.economic_assumptions.yoy_growth.obr.lagged_average_earnings: diff --git a/policyengine_uk/parameters/gov/economic_assumptions/README.md b/policyengine_uk/parameters/gov/economic_assumptions/README.md index 88543fbde4..954aaae5a1 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/README.md +++ b/policyengine_uk/parameters/gov/economic_assumptions/README.md @@ -35,6 +35,7 @@ GDP and population assumptions remain supported. | `stamp_duty_land_tax.non_residential` | SDLT receipts from non-residential property | EFO receipts, property transactions taxes by sector: Table 4.7 in October 2026 | | `business_rates.great_britain` | Business rates receipts, England, Scotland and Wales (cash) | EFO receipts, current receipts (cash basis): Table 4.8 in October 2026 | | `business_rates.northern_ireland` | Northern Ireland business rates receipts | EFO receipts, other taxes: Table 4.2 in October 2026 | +| `per_capita.gambling_duty_bases.` | The base of each of the seven betting and gaming duties (receipts over the model's rate), per person | HMRC Betting and Gaming tables (2024-25, 2025-26), then EFO receipts, betting and gaming duties forecast breakdown: Table 4.10 in October 2026, over `ons.population` | The four receipts series project the corporate SDLT and business rates statistics (`gov.hmrc.stamp_duty.statistics`, @@ -45,6 +46,26 @@ in an earlier EFO), the EFO's forecast years, and after them the `average_earnings` path. Years before the outturn year are kept, so each index keeps its base year. +The seven `per_capita.gambling_duty_bases` series (`general_betting`, +`pool_betting`, `remote_gaming`, `machine_games`, `gaming`, `bingo`, +`lottery`) grow the gambling duty inputs (#2738). They grow a duty's base, +not its receipts: each year's receipts divided by the rate the model charges +that fiscal year. The model then applies its own rate changes, such as +Remote Gaming Duty's rise from 21% to 40% on 1 April 2026, once. The OBR +publishes only forecasts in Table 4.10, so each series starts from HMRC's +2024-25 receipts, the model's base year, and its 2025-26 outturn, which the +importer holds in `RECEIPTS_SERIES_SPECS`. The importer holds a base flat in +any year whose rate is zero, as for Bingo Duty from 2026-27 on, and after the +forecast it grows every other base with `average_earnings`. The importer +writes each series per person, (1 + +base growth) / (1 + `ons.population` growth) - 1 to six decimals, in every +year to the last that `ons.population` states. Dataset projection grows +household weights by `ons.population`, so the weighted inputs grow with the +base. See +[gambling duties](../../../../docs/engineering/gambling-duties.md#projection) +for each duty's rates, the higher-stake share Machine Games Duty's rate +assumes, and the approximations. + `national_living_wage` projects the minimum wage rates (`gov.hmrc.minimum_wage`) after the last rate set. It is the growth of the rates set to 2027, then the OBR's estimates in Table 1.13 of the detailed @@ -458,6 +479,16 @@ the forecast levels, though not their growth. `policyengine_uk/tests/test_corporate_receipts_statistics.py` (`OBR_RECEIPTS_BN`), which pin the series to the OBR's tables. Pass `--skip-receipts-growth` only to leave them on the previous EFO. + The same run refreshes the `per_capita.gambling_duty_bases` series from + the new EFO's betting and gaming duties forecast breakdown and the + `ons.population` growth in `yoy_growth.yaml`, so rerun it after any change + to that population series. They read no previous EFO: the year before the + table's first comes from HMRC's receipts in each spec. When the table starts a year later (2027-28 in October 2027), add + HMRC's outturn for the year before from its latest Betting and Gaming + tables; the importer stops and names the year until you do. Check the + EFO's policy tables for any new measure in the betting and gaming lines. + Then update the published levels in + `policyengine_uk/tests/test_gambling_duty_base_projection.py`. 7. If the new EFO runs past `PROJECTION_END_YEAR`, move it on (see [Dataset projection horizon](#dataset-projection-horizon)). 8. Refresh the household energy price inputs (see diff --git a/policyengine_uk/parameters/gov/economic_assumptions/yoy_growth.yaml b/policyengine_uk/parameters/gov/economic_assumptions/yoy_growth.yaml index de65a776f8..651b284593 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/yoy_growth.yaml +++ b/policyengine_uk/parameters/gov/economic_assumptions/yoy_growth.yaml @@ -989,6 +989,392 @@ obr: href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ per_capita: + # Gambling duty bases per person (#2738), which project the household + # gambling inputs (data/uprating_indices.yaml). Each series grows a + # duty's base, not its receipts: each year's receipts over the rate the + # model charges, so the model applies its own rate changes once (see + # docs/engineering/gambling-duties.md). The importer writes the base's + # growth over one plus ons.population growth, to six decimals: (1 + base + # growth) / (1 + population growth) - 1. Dataset projection grows + # household weights by ons.population, so the weighted base grows with + # the duty's base. After the OBR's forecast the base grows with + # average_earnings, or holds flat while its rate is zero, and the importer + # divides that growth the same way in each year to ons.population's last. + # policyengine_uk/utils/import_obr_forecasts.py writes every year after + # the 2024 base year from HMRC's outturn, the EFO's receipts tables and + # the ons.population growth below. + gambling_duty_bases: + remote_gaming: + description: Growth per person in the Remote Gaming Duty base, the duty's receipts over the rate the model charges. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: 0.132539 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "Remote gaming duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: 0.091574 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: 0.009315 + 2028-01-01: 0.019987 + 2029-01-01: 0.020268 + 2030-01-01: 0.011003 + 2031-01-01: 0.003881 + # Beyond the OBR forecast: the average_earnings long-run path above; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: 0.026242 + 2033-01-01: 0.02874 + 2034-01-01: 0.03144 + 2035-01-01: 0.033939 + 2036-01-01: 0.036538 + 2037-01-01: 0.036641 + 2038-01-01: 0.036745 + 2040-01-01: 0.036848 + 2042-01-01: 0.036952 + 2044-01-01: 0.037056 + 2045-01-01: 0.037159 + 2047-01-01: 0.037263 + 2048-01-01: 0.037366 + 2049-01-01: 0.03747 + 2050-01-01: 0.037677 + 2051-01-01: 0.037781 + 2052-01-01: 0.037989 + 2053-01-01: 0.038092 + 2054-01-01: 0.0383 + 2055-01-01: 0.038404 + 2056-01-01: 0.038612 + 2057-01-01: 0.038715 + 2058-01-01: 0.038819 + 2059-01-01: 0.038923 + 2060-01-01: 0.039027 + 2061-01-01: 0.039131 + 2064-01-01: 0.039235 + 2071-01-01: 0.039339 + metadata: + unit: /1 + label: Remote Gaming Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + lottery: + description: Growth per person in the Lottery Duty base, the duty's receipts over the rate the model charges. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: 0.048312 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "Lottery duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: -0.154336 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: 0.021712 + 2028-01-01: 0.028957 + 2029-01-01: 0.02954 + 2030-01-01: 0.026297 + 2031-01-01: 0.026866 + # Beyond the OBR forecast: the average_earnings long-run path above; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: 0.026242 + 2033-01-01: 0.02874 + 2034-01-01: 0.03144 + 2035-01-01: 0.033939 + 2036-01-01: 0.036538 + 2037-01-01: 0.036641 + 2038-01-01: 0.036745 + 2040-01-01: 0.036848 + 2042-01-01: 0.036952 + 2044-01-01: 0.037056 + 2045-01-01: 0.037159 + 2047-01-01: 0.037263 + 2048-01-01: 0.037366 + 2049-01-01: 0.03747 + 2050-01-01: 0.037677 + 2051-01-01: 0.037781 + 2052-01-01: 0.037989 + 2053-01-01: 0.038092 + 2054-01-01: 0.0383 + 2055-01-01: 0.038404 + 2056-01-01: 0.038612 + 2057-01-01: 0.038715 + 2058-01-01: 0.038819 + 2059-01-01: 0.038923 + 2060-01-01: 0.039027 + 2061-01-01: 0.039131 + 2064-01-01: 0.039235 + 2071-01-01: 0.039339 + metadata: + unit: /1 + label: Lottery Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + general_betting: + description: Growth per person in the General Betting Duty base, the duty's receipts over the rates the model charges, each weighted by its part's 2024-25 share of the base. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: -0.044277 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "General betting duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: -0.044128 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: 0.025705 + 2028-01-01: 0.043246 + 2029-01-01: 0.032385 + 2030-01-01: 0.026778 + 2031-01-01: 0.025912 + # Beyond the OBR forecast: the average_earnings long-run path above; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: 0.026242 + 2033-01-01: 0.02874 + 2034-01-01: 0.03144 + 2035-01-01: 0.033939 + 2036-01-01: 0.036538 + 2037-01-01: 0.036641 + 2038-01-01: 0.036745 + 2040-01-01: 0.036848 + 2042-01-01: 0.036952 + 2044-01-01: 0.037056 + 2045-01-01: 0.037159 + 2047-01-01: 0.037263 + 2048-01-01: 0.037366 + 2049-01-01: 0.03747 + 2050-01-01: 0.037677 + 2051-01-01: 0.037781 + 2052-01-01: 0.037989 + 2053-01-01: 0.038092 + 2054-01-01: 0.0383 + 2055-01-01: 0.038404 + 2056-01-01: 0.038612 + 2057-01-01: 0.038715 + 2058-01-01: 0.038819 + 2059-01-01: 0.038923 + 2060-01-01: 0.039027 + 2061-01-01: 0.039131 + 2064-01-01: 0.039235 + 2071-01-01: 0.039339 + metadata: + unit: /1 + label: General Betting Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + - title: Microcosm, FY2024-25 General Betting Duty bases split by the Gambling Commission's gross gambling yield (docs/evidence/uk-excise-quantities-1359/outputs/national_reconciliation.json) + href: https://github.com/PolicyEngine/microcosm/blob/main/docs/evidence/uk-excise-quantities-1359/outputs/national_reconciliation.json + - title: Gambling Commission, industry statistics, April 2024 to March 2025 + href: https://www.gamblingcommission.gov.uk/statistics-and-research/publication/industry-statistics-annual-report-financial-year-april-2024-to-march-2025 + machine_games: + description: Growth per person in the Machine Games Duty base, the duty's receipts over the rates the model charges, each weighted by its band's 2024-25 share of the base, with the higher-stake share of the standard-rate band at the higher rate from 1 April 2027. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: 0.048678 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "Machine games duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: 0.054148 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: -0.003614 + 2028-01-01: 0.034551 + 2029-01-01: 0.007554 + 2030-01-01: 0.009337 + 2031-01-01: 0.021314 + # Beyond the OBR forecast: the average_earnings long-run path above; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: 0.026242 + 2033-01-01: 0.02874 + 2034-01-01: 0.03144 + 2035-01-01: 0.033939 + 2036-01-01: 0.036538 + 2037-01-01: 0.036641 + 2038-01-01: 0.036745 + 2040-01-01: 0.036848 + 2042-01-01: 0.036952 + 2044-01-01: 0.037056 + 2045-01-01: 0.037159 + 2047-01-01: 0.037263 + 2048-01-01: 0.037366 + 2049-01-01: 0.03747 + 2050-01-01: 0.037677 + 2051-01-01: 0.037781 + 2052-01-01: 0.037989 + 2053-01-01: 0.038092 + 2054-01-01: 0.0383 + 2055-01-01: 0.038404 + 2056-01-01: 0.038612 + 2057-01-01: 0.038715 + 2058-01-01: 0.038819 + 2059-01-01: 0.038923 + 2060-01-01: 0.039027 + 2061-01-01: 0.039131 + 2064-01-01: 0.039235 + 2071-01-01: 0.039339 + metadata: + unit: /1 + label: Machine Games Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + - title: Gambling Commission, Industry Statistics, April 2025 to March 2026 (sheet 4, 2024-25 gaming machine yield by category) + href: https://www.gamblingcommission.gov.uk/statistics-and-research/publication/industry-statistics-annual-report-financial-year-april-2025-to-march-2026-official-statistics + gaming: + description: Growth per person in the Gaming Duty base, the duty's receipts over the casino effective rate the model charges. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: 0.047963 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "Gaming duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: -0.000114 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: 0.059753 + 2028-01-01: 0.000213 + 2029-01-01: -0.000768 + 2030-01-01: 0.00304 + 2031-01-01: -0.006826 + # Beyond the OBR forecast: the average_earnings long-run path above; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: 0.026242 + 2033-01-01: 0.02874 + 2034-01-01: 0.03144 + 2035-01-01: 0.033939 + 2036-01-01: 0.036538 + 2037-01-01: 0.036641 + 2038-01-01: 0.036745 + 2040-01-01: 0.036848 + 2042-01-01: 0.036952 + 2044-01-01: 0.037056 + 2045-01-01: 0.037159 + 2047-01-01: 0.037263 + 2048-01-01: 0.037366 + 2049-01-01: 0.03747 + 2050-01-01: 0.037677 + 2051-01-01: 0.037781 + 2052-01-01: 0.037989 + 2053-01-01: 0.038092 + 2054-01-01: 0.0383 + 2055-01-01: 0.038404 + 2056-01-01: 0.038612 + 2057-01-01: 0.038715 + 2058-01-01: 0.038819 + 2059-01-01: 0.038923 + 2060-01-01: 0.039027 + 2061-01-01: 0.039131 + 2064-01-01: 0.039235 + 2071-01-01: 0.039339 + metadata: + unit: /1 + label: Gaming Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + bingo: + description: Growth per person in the Bingo Duty base, the duty's receipts over the rate the model charges; the base holds flat once the rate is zero. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: 0.029891 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "Bingo duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: -0.001996 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: -0.001298 + 2028-01-01: -0.001697 + 2029-01-01: -0.002096 + 2030-01-01: -0.002494 + 2031-01-01: -0.002394 + # Beyond the OBR forecast: held flat, as the rate is zero; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: -0.002195 + 2033-01-01: -0.002096 + 2034-01-01: -0.001896 + 2035-01-01: -0.001797 + 2036-01-01: -0.001697 + 2037-01-01: -0.001597 + 2038-01-01: -0.001498 + 2040-01-01: -0.001398 + 2042-01-01: -0.001298 + 2044-01-01: -0.001199 + 2045-01-01: -0.001099 + 2047-01-01: -0.000999 + 2048-01-01: -0.000899 + 2049-01-01: -0.000799 + 2050-01-01: -0.0006 + 2051-01-01: -0.0005 + 2052-01-01: -0.0003 + 2053-01-01: -0.0002 + 2054-01-01: 0.0000 + 2055-01-01: 0.0001 + 2056-01-01: 0.0003 + 2057-01-01: 0.0004 + 2058-01-01: 0.0005 + 2059-01-01: 0.0006 + 2060-01-01: 0.0007 + 2061-01-01: 0.000801 + 2064-01-01: 0.000901 + 2071-01-01: 0.001001 + metadata: + unit: /1 + label: Bingo Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + pool_betting: + description: Growth per person in the Pool Betting Duty base, the duty's receipts over the rate the model charges. + values: + # Base year (2024-25): the gambling duty inputs start in model year 2024 + 2024-01-01: 0.0000 + # Outturn (2025-26 over 2024-25) from HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2025-01-01: -0.512992 + # Forecast (2026-27 over 2025-26) from OBR EFO October 2026, receipts Table 4.10 "Pool betting duty", over HMRC's receipts (July 2026 Betting and Gaming tables, Table_1_BG_receipts); per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2026-01-01: -0.288858 + # Forecast (2027-2031) from OBR EFO October 2026, receipts Table 4.10; per head: base growth (receipts over the rate the model charges) over one plus ons.population growth + 2027-01-01: -0.053757 + 2028-01-01: 0.064652 + 2029-01-01: -0.009525 + 2030-01-01: 0.053325 + 2031-01-01: 0.040869 + # Beyond the OBR forecast: the average_earnings long-run path above; per head: (1 + that growth) / (1 + ons.population growth) - 1 + 2032-01-01: 0.026242 + 2033-01-01: 0.02874 + 2034-01-01: 0.03144 + 2035-01-01: 0.033939 + 2036-01-01: 0.036538 + 2037-01-01: 0.036641 + 2038-01-01: 0.036745 + 2040-01-01: 0.036848 + 2042-01-01: 0.036952 + 2044-01-01: 0.037056 + 2045-01-01: 0.037159 + 2047-01-01: 0.037263 + 2048-01-01: 0.037366 + 2049-01-01: 0.03747 + 2050-01-01: 0.037677 + 2051-01-01: 0.037781 + 2052-01-01: 0.037989 + 2053-01-01: 0.038092 + 2054-01-01: 0.0383 + 2055-01-01: 0.038404 + 2056-01-01: 0.038612 + 2057-01-01: 0.038715 + 2058-01-01: 0.038819 + 2059-01-01: 0.038923 + 2060-01-01: 0.039027 + 2061-01-01: 0.039131 + 2064-01-01: 0.039235 + 2071-01-01: 0.039339 + metadata: + unit: /1 + label: Pool Betting Duty base growth per person + reference: + - title: OBR EFO October 2026 (detailed forecast tables, receipts, Table 4.10) + href: https://obr.uk/efo/economic-and-fiscal-outlook-october-2026/ + - title: HMRC UK Betting and Gaming Statistics, July 2026 tables (Table_1_BG_receipts, 2024-25 and 2025-26 receipts) + href: https://assets.publishing.service.gov.uk/media/6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods + gdp: description: Per capita nominal GDP year-on-year growth. values: diff --git a/policyengine_uk/tests/fixtures/obr_receipts/README.md b/policyengine_uk/tests/fixtures/obr_receipts/README.md index 825e379a3d..cf59073008 100644 --- a/policyengine_uk/tests/fixtures/obr_receipts/README.md +++ b/policyengine_uk/tests/fixtures/obr_receipts/README.md @@ -2,10 +2,11 @@ These parser fixtures retain the original worksheet XML for Contents, Other taxes (3.2, or 4.2 in October 2026), Property transactions taxes: receipts by -sector (3.7, or 4.7), and Current receipts (on a cash basis: 3.9 in March -2025, 3.8 in March 2026, 4.8 in October 2026), plus the original shared -strings. Only the workbook's sheet list and relationships were reduced to those -entries. The published unit cells, fiscal-year headers, markers, row labels and +sector (3.7, or 4.7), Current receipts (on a cash basis: 3.9 in March 2025, +3.8 in March 2026, 4.8 in October 2026) and, from March 2026, Betting and +gaming duties forecast breakdown (3.10, or 4.10), plus the original shared +strings. The March 2025 tables have no betting and gaming breakdown. Only +the workbook's sheet list and relationships were reduced to those entries. The published unit cells, fiscal-year headers, markers, row labels and numeric cells are unchanged. These are minimal archives for the importer's XLSX reader, rather than complete Excel documents with styles and charts. The October 2026 workbook titles sheet 4.8 "43.8 Current receipts (on a cash diff --git a/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_march_2026_receipts.xlsx b/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_march_2026_receipts.xlsx index 6b8b16f9bf..4ccc9c019b 100644 Binary files a/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_march_2026_receipts.xlsx and b/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_march_2026_receipts.xlsx differ diff --git a/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_october_2026_receipts.xlsx b/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_october_2026_receipts.xlsx index 5a4c58ec67..fed8a21a1b 100644 Binary files a/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_october_2026_receipts.xlsx and b/policyengine_uk/tests/fixtures/obr_receipts/obr_efo_october_2026_receipts.xlsx differ diff --git a/policyengine_uk/tests/test_gambling_duty_base_projection.py b/policyengine_uk/tests/test_gambling_duty_base_projection.py new file mode 100644 index 0000000000..c65d3150b5 --- /dev/null +++ b/policyengine_uk/tests/test_gambling_duty_base_projection.py @@ -0,0 +1,745 @@ +"""The gambling duty inputs grow with their duty's base, not its receipts (#2738). + +Each input declares its duty's index under +``gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases``, and +dataset projection grows it by the same ``yoy_growth`` series. A series is the +growth in HMRC's 2024-25 and 2025-26 receipts, then the OBR's October 2026 +forecast (receipts Table 4.10), each year over the rate the model charges, +and then over one plus ``ons.population`` growth. Household weights grow by +``ons.population``, so the weighted inputs grow with the duty's base. For +Machine Games Duty that rate charges the higher-stake share of the +standard-rate base at the higher rate from 1 April 2027. Weighted bases that +give HMRC's 2024-25 receipts, with the higher-stake input at that share, +therefore give the OBR's receipts in later years, and a rate reform moves +receipts through the rate alone. +""" + +from datetime import date +from pathlib import Path + +import numpy as np +import pandas as pd +import pytest +import yaml +from hypothesis import given, settings +from hypothesis import strategies as st + +from policyengine_uk import CountryTaxBenefitSystem, Simulation +from policyengine_uk.data import UKSingleYearDataset +from policyengine_uk.data.economic_assumptions import extend_single_year_dataset +from policyengine_uk.model_api import Scenario +from policyengine_uk.utils import import_obr_forecasts +from policyengine_uk.utils.import_obr_forecasts import ( + MACHINE_GAMES_HIGHER_STAKE_SHARE, + RECEIPTS_SERIES_SPECS, + line_rate, +) + +pytestmark = pytest.mark.usefixtures("cloned_uk_tax_benefit_system") + +GROWTH = "gov.economic_assumptions.yoy_growth.obr.per_capita.gambling_duty_bases" +INDEX = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases" +POPULATION = "gov.economic_assumptions.yoy_growth.ons.population" +POPULATION_INDEX = "gov.economic_assumptions.indices.ons.population" +AVERAGE_EARNINGS = "gov.economic_assumptions.yoy_growth.obr.average_earnings" +INPUTS = { + "general_betting": ( + "betting_exchange_commission", + "financial_spread_betting_net_losses", + "horse_and_dog_pool_betting_net_losses", + "land_based_fixed_odds_betting_net_losses", + "other_spread_betting_net_losses", + "remote_other_betting_net_losses", + "remote_uk_horseracing_betting_net_losses", + ), + "pool_betting": ("pool_betting_net_receipts",), + "remote_gaming": ("remote_gaming_net_losses",), + "machine_games": ( + "machine_games_higher_rate_net_takings", + "machine_games_lower_rate_net_takings", + "machine_games_standard_rate_higher_stake_net_takings", + "machine_games_standard_rate_net_takings", + ), + "gaming": ("land_based_casino_net_losses",), + "bingo": ("bingo_net_receipts",), + "lottery": ("national_lottery_ticket_spending",), +} +DUTIES = { + "general_betting": "general_betting_duty", + "pool_betting": "pool_betting_duty", + "remote_gaming": "remote_gaming_duty", + "machine_games": "machine_games_duty", + "gaming": "gaming_duty", + "bingo": "bingo_duty", + "lottery": "lottery_duty", +} +# OBR, Economic and fiscal outlook, October 2026, detailed forecast tables: +# receipts, Table 4.10 (£ billion), 2026-27 to 2031-32, as published. +TABLE_4_10 = { + "remote_gaming": (2.7520296250837784, 2.7812750734699425, 2.8416867878041874, 2.9053716896455919, 2.9446821891345443, 2.963204761653297), + "lottery": (0.83052038459597621, 0.84965566898040046, 0.87574553838772884, 0.90350845044750805, 0.92958587301870155, 0.95685079671453654), + "general_betting": (0.65560522982822456, 0.87744453968275893, 0.91694664204685206, 0.94862995017184315, 0.97646741817020943, 1.0041740187990271), + "machine_games": (0.67707068705786499, 0.82692873727074745, 0.85695449075649544, 0.86524127517297356, 0.87550370868921701, 0.89631020781238802), + "gaming": (0.17422987982556123, 0.18488058456996015, 0.18523427331264838, 0.18548069649974164, 0.18650974968911124, 0.18568125218312026), + "bingo": (1.8867652000000002e-4, 0.0, 0.0, 0.0, 0.0, 0.0), + "pool_betting": (2.8288821260900035e-3, 2.6802893699237468e-3, 2.85842704577818e-3, 2.8371447589557743e-3, 2.9959073747097458e-3, 3.1258296515431082e-3), +} # fmt: skip +# HMRC, UK Betting and Gaming Statistics, July 2026 tables, +# Table_1_BG_receipts rows 45 (2024-25) and 46 (2025-26), £ thousand. +HMRC = { + "general_betting": (713_858, 684_502), + "pool_betting": (8_125, 3_970), + "remote_gaming": (1_162_539, 1_320_966), + "machine_games": (609_245, 641_010), + "gaming": (165_397, 173_902), + "bingo": (25_233, 26_073), + "lottery": (931_887, 980_132), +} +# The 2024-25 bases of the duties with more than one rate, in £ (Microcosm's +# national levels): General Betting Duty's split by Gambling Commission +# yield, and Machine Games Duty's bands from HMRC's liabilities over their +# rates. The remote part pays 25% from 1 April 2027; the rest 15%. From the +# same date the higher-stake part of the standard-rate band pays 25%. +GENERAL_BETTING_PARTS = { + "remote_uk_horseracing_betting_net_losses": 888_079_509.7245454, + "land_based_fixed_odds_betting_net_losses": 1_527_826_124.9133615, + "horse_and_dog_pool_betting_net_losses": 55_219_132.22492455, + "remote_other_betting_net_losses": 2_163_978_657.2782593, + "betting_exchange_commission": 123_949_909.1922428, +} +MACHINE_GAMES_BANDS = { + "machine_games_lower_rate_net_takings": (138_420_000, 0.05), + "machine_games_standard_rate_net_takings": (3_023_740_000, 0.20), + "machine_games_higher_rate_net_takings": (2_384_000, 0.25), +} +SINGLE_RATES = { + "remote_gaming_net_losses": 0.21, + "national_lottery_ticket_spending": 0.12, + "pool_betting_net_receipts": 0.15, + "land_based_casino_net_losses": 0.235487, + "bingo_net_receipts": 0.10, +} +SINGLE_RATE_DUTY = { + "remote_gaming_net_losses": "remote_gaming", + "national_lottery_ticket_spending": "lottery", + "pool_betting_net_receipts": "pool_betting", + "land_based_casino_net_losses": "gaming", + "bingo_net_receipts": "bingo", +} +REMOTE_SHARE = GENERAL_BETTING_PARTS["remote_other_betting_net_losses"] / sum( + GENERAL_BETTING_PARTS.values() +) +STANDARD = "machine_games_standard_rate_net_takings" +HIGHER_STAKE = "machine_games_standard_rate_higher_stake_net_takings" +# The Gambling Commission's 2024-25 gaming machine yield, £ million, in +# Category B and in Category C (Industry Statistics, April 2025 to March 2026 +# release, sheet 4, K34 and L34). +HIGHER_STAKE_SHARE = 2_289.84969448829 / (2_289.84969448829 + 232.47489298159996) + + +def machine_games_rate(higher_stake_rate, share=HIGHER_STAKE_SHARE): + """Machine Games Duty's rate on its 2024-25 bands, with ``share`` of the + standard-rate band at ``higher_stake_rate``.""" + total = sum(base for base, _ in MACHINE_GAMES_BANDS.values()) + charged = sum(base * rate for base, rate in MACHINE_GAMES_BANDS.values()) + standard = MACHINE_GAMES_BANDS[STANDARD][0] + return (charged + standard * share * (higher_stake_rate - 0.20)) / total + + +MACHINE_GAMES_BEFORE = machine_games_rate(0.20) +MACHINE_GAMES_AFTER = machine_games_rate(0.25) +YEARS = range(2024, 2033) + + +@pytest.fixture(scope="module") +def parameters(): + return CountryTaxBenefitSystem().parameters + + +def index(parameters, duty, year): + return parameters.get_child(f"{INDEX}.{duty}")(f"{year}-01-01") + + +def population(parameters, year): + """One plus ``ons.population`` growth into ``year``.""" + return 1 + parameters.get_child(POPULATION)(f"{year}-01-01") + + +def population_index(parameters, year): + """The ``ons.population`` index in ``year`` over its 2024 level (the + index itself starts at 1 in 2021).""" + index = parameters.get_child(POPULATION_INDEX) + return index(f"{year}-01-01") / index("2024-01-01") + + +def weighted_index(parameters, duty, year): + """The duty's index times the population index: the growth of a + weighted input since 2024.""" + return index(parameters, duty, year) * population_index(parameters, year) + + +def obr_rate(duty, year): + """The rate on a duty's 2024-25 base over the OBR's year from 1 April + ``year``. Every gambling rate change starts on 1 April.""" + if duty == "remote_gaming": + return 0.21 if year < 2026 else 0.40 + if duty == "bingo": + return 0.10 if year < 2026 else 0.0 + if duty == "general_betting": + return 0.15 if year < 2027 else 0.15 + 0.10 * REMOTE_SHARE + if duty == "machine_games": + return MACHINE_GAMES_BEFORE if year < 2027 else MACHINE_GAMES_AFTER + return {"lottery": 0.12, "pool_betting": 0.15, "gaming": 0.235487}[duty] + + +def aggregate_index(parameters, duty, year): + """The duty's national base in ``year`` over its 2024-25 base: HMRC's + receipts to 2025-26 and Table 4.10's to 2031-32, each over the rate of + the OBR's year, then average earnings growth after the forecast. A base + whose rate is zero holds flat.""" + if year == 2024: + return 1.0 + if obr_rate(duty, year) == 0: + return aggregate_index(parameters, duty, year - 1) + if year > 2031: + earnings = parameters.get_child(AVERAGE_EARNINGS)(f"{year}-01-01") + return aggregate_index(parameters, duty, year - 1) * (1 + earnings) + + def base(year): + if year < 2026: + receipts = HMRC[duty][year - 2024] * 1e3 + else: + receipts = TABLE_4_10[duty][year - 2026] * 1e9 + return receipts / obr_rate(duty, year) + + return base(year) / base(2024) + + +def test_every_gambling_input_declares_its_duty_index(): + """Every input in the gambling folder declares its duty's index, and + dataset projection grows it by the same duty's series.""" + from policyengine_uk.system import system + + folder = Path(__file__).parents[1] / "variables/input/consumption/gambling" + inputs = {path.stem for path in folder.glob("*.py")} + declared = {name: duty for duty, names in INPUTS.items() for name in names} + assert set(declared) == inputs + for name, duty in declared.items(): + assert system.variables[name].uprating == f"{INDEX}.{duty}", name + mapping = yaml.safe_load( + (Path(__file__).parents[1] / "data/uprating_indices.yaml").read_text() + ) + gambling = {key: value for key, value in mapping.items() if "gambling" in key} + assert gambling == { + f"{GROWTH}.{duty}": list(names) for duty, names in INPUTS.items() + } + + +def test_indices_are_positive_from_the_2024_base_year(parameters): + for duty in INPUTS: + assert index(parameters, duty, 2024) == 1 + for year in range(2024, 2040): + assert index(parameters, duty, year) > 0, (duty, year) + + +def test_remote_gaming_index_takes_out_the_april_2026_rate_rise(parameters): + """Receipts more than double into 2026-27 as the rate goes from 21% to + 40%; the base grows 9.4%, and per head 9.4% over population growth.""" + receipts_growth = TABLE_4_10["remote_gaming"][0] / (HMRC["remote_gaming"][1] / 1e6) + assert receipts_growth - 1 == pytest.approx(1.0834, abs=1e-4) + base_growth = receipts_growth * 0.21 / 0.40 + assert round(base_growth - 1, 6) == 0.093757 + growth = parameters.get_child(f"{GROWTH}.remote_gaming") + people = population(parameters, 2026) + assert growth("2026-01-01") == round(base_growth / people - 1, 6) + assert weighted_index(parameters, "remote_gaming", 2026) / weighted_index( + parameters, "remote_gaming", 2025 + ) == pytest.approx(base_growth, rel=2e-5) + + +def test_general_betting_index_takes_out_the_april_2027_remote_rise(parameters): + """The remote part (45.5% of the 2024-25 base) goes from 15% to 25%, so + the duty's rate on the whole base goes from 15% to 19.5%.""" + rate_2027 = 0.15 + 0.10 * REMOTE_SHARE + assert rate_2027 == pytest.approx(0.19547, abs=1e-5) + receipts = TABLE_4_10["general_betting"] + base_growth = (receipts[1] / rate_2027) / (receipts[0] / 0.15) + assert round(base_growth - 1, 6) == 0.027038 + growth = parameters.get_child(f"{GROWTH}.general_betting") + people = population(parameters, 2027) + assert growth("2027-01-01") == round(base_growth / people - 1, 6) + # Receipts grow 33.8%. + assert receipts[1] / receipts[0] - 1 == pytest.approx(0.3384, abs=1e-4) + + +def test_the_higher_stake_share_is_category_b_over_b_and_c(): + assert MACHINE_GAMES_HIGHER_STAKE_SHARE == HIGHER_STAKE_SHARE + assert HIGHER_STAKE_SHARE == pytest.approx(0.90783, abs=1e-5) + + +def test_machine_games_index_takes_out_the_april_2027_higher_stake_rise(parameters): + """Table 4.10's Machine Games Duty line includes the Budget's higher rate + on higher-stake machines (OBR paragraph 4.45): receipts grow 22.1% into + 2027-28. With 90.8% of the standard-rate base at 25%, the rate on the + whole base goes from 19.35% to 23.68%, so the base falls 0.23%.""" + receipts = TABLE_4_10["machine_games"] + assert receipts[1] / receipts[0] - 1 == pytest.approx(0.2213, abs=1e-4) + assert MACHINE_GAMES_BEFORE == pytest.approx(0.193477, abs=1e-6) + assert MACHINE_GAMES_AFTER == pytest.approx(0.236849, abs=1e-6) + base_growth = (receipts[1] / MACHINE_GAMES_AFTER) / ( + receipts[0] / MACHINE_GAMES_BEFORE + ) + assert round(base_growth - 1, 6) == -0.002319 + growth = parameters.get_child(f"{GROWTH}.machine_games") + people = population(parameters, 2027) + assert growth("2027-01-01") == round(base_growth / people - 1, 6) + + +def test_bingo_base_holds_flat_once_its_rate_is_zero(parameters): + """Bingo Duty ends on 1 April 2026; its base stays at its 2025-26 level. + Per head it falls as population grows: in every year to the end of + ons.population, one plus its growth, times one plus population growth, is + one, so the weighted index holds.""" + at_2025 = weighted_index(parameters, "bingo", 2025) + assert at_2025 == pytest.approx(HMRC["bingo"][1] / HMRC["bingo"][0], rel=2e-5) + growth = parameters.get_child(f"{GROWTH}.bingo") + for year in range(2026, 2075): + people = population(parameters, year) + assert (1 + growth(f"{year}-01-01")) * people == pytest.approx( + 1, abs=5e-7 * people + 1e-12 + ), year + for year in range(2026, 2040): + assert weighted_index(parameters, "bingo", year) == pytest.approx( + at_2025, rel=2e-4 + ), year + + +def households_frame(year, columns, weights): + """Households of one adult each, with the given weights and household + columns.""" + ids = list(range(1, len(weights) + 1)) + count = len(ids) + return UKSingleYearDataset( + person=pd.DataFrame( + { + "person_id": ids, + "person_benunit_id": ids, + "person_household_id": ids, + "age": [40] * count, + } + ), + benunit=pd.DataFrame({"benunit_id": ids}), + household=pd.DataFrame( + { + "household_id": ids, + "region": ["LONDON"] * count, + "tenure_type": ["OWNED_OUTRIGHT"] * count, + "council_tax": [0.0] * count, + "rent": [0.0] * count, + "household_weight": list(weights), + **columns, + } + ), + fiscal_year=year, + ) + + +def household_frame(year, inputs): + """One household of one adult, weight 1, with the given household + inputs.""" + return households_frame( + year, {name: [value] for name, value in inputs.items()}, (1.0,) + ) + + +# Two households hold each total between them, 30% at weight 700 and 70% at +# weight 1,900, as survey households hold the national bases. +WEIGHTS = (700.0, 1_900.0) +SHARES = (0.3, 0.7) + + +def weighted_frame(year, totals): + """Two households whose weighted inputs sum to ``totals``: each holds + its share of a total over its weight.""" + return households_frame( + year, + { + name: [total * share / weight for share, weight in zip(SHARES, WEIGHTS)] + for name, total in totals.items() + }, + WEIGHTS, + ) + + +def weighted(simulation, variable, year): + """``variable``'s total over the households in ``year``, each weighted by + its household weight.""" + values = np.asarray(simulation.calculate(variable, year), dtype=float) + weights = np.asarray(simulation.calculate("household_weight", year), dtype=float) + return float((values * weights).sum()) + + +def weighted_simulation(parameters, totals): + """A simulation of ``weighted_frame``'s households, projected as a survey + dataset is: by extend_single_year_dataset to its last year, and by Core's + uprating after it.""" + projected = extend_single_year_dataset(weighted_frame(2024, totals), parameters) + return Simulation(dataset=projected) + + +def situation(inputs, year=2024): + return { + "people": {"person": {"age": {year: 40}}}, + "benunits": {"benunit": {"members": ["person"]}}, + "households": { + "household": { + "members": ["person"], + **{name: {year: value} for name, value in inputs.items()}, + } + }, + } + + +ALL_INPUTS = { + name: 1_000.0 * (position + 1) + for position, name in enumerate(name for names in INPUTS.values() for name in names) +} + + +def test_dataset_projection_matches_core_uprating(parameters): + """Two routes grow a 2024 input: dataset projection compounds the + yoy_growth series a year at a time; Core scales a situation input by the + ratio of the index (rounded to five decimals). They agree.""" + projected = extend_single_year_dataset( + household_frame(2024, ALL_INPUTS), parameters, end_year=2032 + ) + simulation = Simulation(situation=situation(ALL_INPUTS)) + for year in YEARS: + for name, value in ALL_INPUTS.items(): + stored = projected[year].household[name].iloc[0] + uprated = simulation.calculate(name, year)[0] + assert stored == pytest.approx(uprated, rel=1e-4), (name, year) + # Remote gaming grows by its base's growth over population growth. + stored = projected[2026].household["remote_gaming_net_losses"].iloc[0] + people = population(parameters, 2025) * population(parameters, 2026) + base_growth = aggregate_index(parameters, "remote_gaming", 2026) + assert stored == pytest.approx( + ALL_INPUTS["remote_gaming_net_losses"] * base_growth / people, rel=3e-6 + ) + + +def test_weighted_gambling_bases_follow_the_aggregate(parameters): + """Household weights grow by ons.population, so each household's input + must grow per head for the weighted input to grow as its duty's base + does. Projected by extend_single_year_dataset to its last year and by + Core's uprating after it, each of the sixteen weighted inputs grows in + every year to 2039 as its duty's national base: receipts over the rate + to 2031-32, then average earnings, or flat while the rate is zero.""" + names = [name for names in INPUTS.values() for name in names] + totals = {name: 1e6 * (position + 1) for position, name in enumerate(names)} + simulation = weighted_simulation(parameters, totals) + for duty, duty_inputs in INPUTS.items(): + for year in range(2024, 2040): + expected = aggregate_index(parameters, duty, year) + for name in duty_inputs: + growth = weighted(simulation, name, year) / totals[name] + assert growth == pytest.approx(expected, rel=1e-4), (name, year) + + +def test_higher_stake_machines_pay_the_april_2027_rise_once(): + """The Machine Games Duty series divides Table 4.10's line by the rate + with the higher-stake rise in it, so the projected base holds no rise and + the model adds it once (#2731): from model year 2027, 25% on the + higher-stake part of the standard-rate base and 20% on the rest, both + grown by the one index.""" + standard, higher_stake = STANDARD, HIGHER_STAKE + simulation = Simulation( + situation=situation({standard: 1_000.0, higher_stake: 400.0}) + ) + without = Simulation(situation=situation({standard: 1_000.0})) + for year in range(2027, 2032): + base = simulation.calculate(standard, year)[0] + part = simulation.calculate(higher_stake, year)[0] + assert base > 1_000 + # Inputs and duties are float32, and the rise is a difference of two. + assert part == pytest.approx(0.4 * base, rel=1e-6) + duty = simulation.calculate("machine_games_duty", year)[0] + assert duty == pytest.approx(0.20 * (base - part) + 0.25 * part, rel=1e-6) + rise = duty - without.calculate("machine_games_duty", year)[0] + assert rise == pytest.approx(0.05 * part, rel=1e-5) + + +def test_a_rise_that_behaviour_offsets_leaves_the_duty_where_it_was(): + """£100 of higher-stake takings pay £20 at 20%. If the rise to 25% + cuts them to £80, they pay the same £20, receipts hold flat, and the + base index is 0.8: the model's 25% on the projected £80 gives the £20.""" + spec = import_obr_forecasts.machine_games_spec(1.0) + rates = tuple(part for part in spec.rates if part.parameter.endswith("standard")) + standard_only = import_obr_forecasts.ReceiptsSeriesSpec( + path=spec.path, title=spec.title, row=spec.row, rates=rates + ) + before, after = line_rate(standard_only, 2026), line_rate(standard_only, 2027) + assert (before, after) == (0.20, 0.25) + index = (20 / after) / (20 / before) + assert index == pytest.approx(0.8, abs=1e-12) + takings = 100 * index + simulation = Simulation( + situation=situation({STANDARD: takings, HIGHER_STAKE: takings}, year=2027) + ) + assert simulation.calculate("machine_games_duty", 2027)[0] == pytest.approx( + 20, rel=1e-6 + ) + earlier = Simulation(situation=situation({STANDARD: 100, HIGHER_STAKE: 100})) + assert earlier.calculate("machine_games_duty", 2024)[0] == pytest.approx( + 20, rel=1e-6 + ) + + +def national_bases(higher_stake_share=HIGHER_STAKE_SHARE): + """2024-25 bases that give HMRC's 2024-25 receipts at the model's rates, + with the higher-stake input at ``higher_stake_share`` of the + standard-rate input (none if None).""" + bases = { + name: HMRC[SINGLE_RATE_DUTY[name]][0] * 1e3 / rate + for name, rate in SINGLE_RATES.items() + } + general_betting = HMRC["general_betting"][0] * 1e3 / 0.15 + total = sum(GENERAL_BETTING_PARTS.values()) + bases |= { + name: general_betting * part / total + for name, part in GENERAL_BETTING_PARTS.items() + } + machine_games = ( + HMRC["machine_games"][0] + * 1e3 + / sum(base * rate for base, rate in MACHINE_GAMES_BANDS.values()) + ) + bases |= { + name: base * machine_games for name, (base, _) in MACHINE_GAMES_BANDS.items() + } + if higher_stake_share is not None: + bases[HIGHER_STAKE] = bases[STANDARD] * higher_stake_share + return bases + + +def model_year_share(change: date, year: int) -> float: + """The share of model year ``year`` (6 April to 5 April) from ``change``.""" + start, end = date(year, 4, 6), date(year + 1, 4, 6) + after = max(start, min(change, end)) + return (end - after).days / (end - start).days + + +def expected_receipts(duty, year): + """The duty in model year ``year`` on bases that give HMRC's 2024-25 + receipts: HMRC's receipts to 2025-26, then Table 4.10. Model years run + 6 April to 5 April, so those that take in five days of a 1 April rate + change charge them at the new rate. Bingo Duty is zero from 2026-27, + whatever late receipts Table 4.10 records.""" + if duty == "bingo" and year >= 2026: + return 0.0 + if year < 2026: + receipts = HMRC[duty][year - 2024] * 1e3 + else: + receipts = TABLE_4_10[duty][min(year, 2031) - 2026] * 1e9 + if duty == "machine_games" and year == 2026: + share = model_year_share(date(2027, 4, 1), 2026) + receipts *= ( + MACHINE_GAMES_BEFORE * (1 - share) + MACHINE_GAMES_AFTER * share + ) / MACHINE_GAMES_BEFORE + if duty == "remote_gaming" and year == 2025: + share = model_year_share(date(2026, 4, 1), 2025) + receipts *= (0.21 * (1 - share) + 0.40 * share) / 0.21 + if duty == "bingo" and year == 2025: + receipts *= 1 - model_year_share(date(2026, 4, 1), 2025) + if duty == "general_betting" and year == 2026: + share = model_year_share(date(2027, 4, 1), 2026) + receipts *= (0.15 + 0.10 * REMOTE_SHARE * share) / 0.15 + return receipts + + +@pytest.fixture(scope="module") +def national_simulation(parameters): + return weighted_simulation(parameters, national_bases()) + + +@pytest.fixture(scope="module") +def national_simulation_without_higher_stake(parameters): + return weighted_simulation(parameters, national_bases(None)) + + +@pytest.mark.parametrize("duty", sorted(DUTIES)) +def test_bases_that_give_hmrc_receipts_give_the_obr_forecast(national_simulation, duty): + """Receipts are each base times its rate. The weighted bases grow by the + OBR's receipts over those rates: household inputs per head, and household + weights by population. So weighted bases that give HMRC's 2024-25 + receipts give HMRC's 2025-26 receipts and then Table 4.10 in every year + to 2031-32, with the higher-stake input at its share.""" + for year in range(2024, 2032): + model = weighted(national_simulation, DUTIES[duty], year) + assert model == pytest.approx(expected_receipts(duty, year), rel=2e-4, abs=1), ( + duty, + year, + ) + + +def test_without_the_higher_stake_input_the_duty_falls_short_of_table_4_10( + national_simulation_without_higher_stake, +): + """A dataset without the higher-stake input charges the standard rate on + all of the standard-rate base, the rates before Budget 2026, while the + base grows by receipts over the rates after it. From 2027-28 the duty is + therefore Table 4.10's line times 19.35% / 23.68%, 18.3% short.""" + simulation = national_simulation_without_higher_stake + assert 1 - MACHINE_GAMES_BEFORE / MACHINE_GAMES_AFTER == pytest.approx( + 0.1831, abs=1e-4 + ) + for year in range(2024, 2032): + model = weighted(simulation, "machine_games_duty", year) + expected = expected_receipts("machine_games", year) + if year == 2026: + expected = TABLE_4_10["machine_games"][0] * 1e9 + if year >= 2027: + expected = expected * MACHINE_GAMES_BEFORE / MACHINE_GAMES_AFTER + assert model == pytest.approx(expected, rel=2e-4, abs=1), year + + +# Each example builds a reform, which reprocesses the parameter tree. +@settings(max_examples=4, deadline=None, derandomize=True) +@given( + bases=st.lists( + st.floats(min_value=0, max_value=1e7, allow_nan=False), + min_size=4, + max_size=4, + ), + factor=st.floats(min_value=0.1, max_value=3, allow_nan=False), + switch=st.booleans(), + year=st.integers(min_value=2027, max_value=2031), +) +def test_a_rate_reform_moves_receipts_through_the_rate_alone( + bases, factor, switch, year +): + """Scaling the Remote Gaming Duty rate, the remote betting rate and the + Machine Games Duty standard and higher rates for a year, and turning the + higher-stake switch on or off, leaves every projected base as it was and + moves each duty through its rates alone.""" + inputs = { + "remote_gaming_net_losses": bases[0], + "remote_other_betting_net_losses": bases[1], + STANDARD: bases[2], + HIGHER_STAKE: min(bases[3], bases[2]), + } + machine_games = "gov.hmrc.gambling_duty.machine_games" + baseline = Simulation(situation=situation(inputs)) + reform = Simulation( + situation=situation(inputs), + scenario=Scenario( + parameter_changes={ + "gov.hmrc.gambling_duty.remote_gaming.rate": {str(year): 0.40 * factor}, + "gov.hmrc.gambling_duty.general_betting.rates.remote": { + str(year): 0.25 * factor + }, + f"{machine_games}.rates.standard": {str(year): 0.20 * factor}, + f"{machine_games}.rates.higher": {str(year): 0.25 * factor}, + f"{machine_games}.higher_stake_machines_at_higher_rate": { + str(year): switch + }, + } + ), + ) + for name in inputs: + np.testing.assert_array_equal( + reform.calculate(name, year), baseline.calculate(name, year) + ) + for name, duty in ( + ("remote_gaming_net_losses", "remote_gaming_duty"), + ("remote_other_betting_net_losses", "general_betting_duty"), + ): + base = baseline.calculate(name, year)[0] + before = baseline.calculate(duty, year)[0] + after = reform.calculate(duty, year)[0] + assert after == pytest.approx(before * factor, rel=1e-5, abs=0.05) + rate = 0.40 if name == "remote_gaming_net_losses" else 0.25 + assert before == pytest.approx(base * rate, rel=1e-5, abs=0.05) + standard = baseline.calculate(STANDARD, year)[0] + part = baseline.calculate(HIGHER_STAKE, year)[0] + before = baseline.calculate("machine_games_duty", year)[0] + after = reform.calculate("machine_games_duty", year)[0] + assert before == pytest.approx( + 0.20 * (standard - part) + 0.25 * part, rel=1e-5, abs=0.05 + ) + part_rate = 0.25 if switch else 0.20 + assert after == pytest.approx( + factor * (0.20 * (standard - part) + part_rate * part), rel=1e-5, abs=0.05 + ) + + +# A distinct rate for every rate parameter the duty formulas read, so a part +# of an importer series wired to the wrong parameter takes the wrong rate. +WIRING_RATES = { + "gov.hmrc.gambling_duty.remote_gaming.rate": 0.31, + "gov.hmrc.gambling_duty.lottery.rate": 0.17, + "gov.hmrc.gambling_duty.general_betting.rates.fixed_odds": 0.11, + "gov.hmrc.gambling_duty.general_betting.rates.remote": 0.29, + "gov.hmrc.gambling_duty.general_betting.rates.exchange_commission": 0.43, + "gov.hmrc.gambling_duty.general_betting.rates.financial_spread": 0.07, + "gov.hmrc.gambling_duty.general_betting.rates.other_spread": 0.13, + "gov.hmrc.gambling_duty.machine_games.rates.lower": 0.03, + "gov.hmrc.gambling_duty.machine_games.rates.standard": 0.19, + "gov.hmrc.gambling_duty.machine_games.rates.higher": 0.37, + "gov.hmrc.gambling_duty.bingo.rate": 0.23, + "gov.hmrc.gambling_duty.pool_betting.rate": 0.41, + "household.consumption.gambling.casino_effective_rate": 0.47, +} +SWITCH = "gov.hmrc.gambling_duty.machine_games.higher_stake_machines_at_higher_rate" + + +def test_the_wiring_rates_cover_every_gambling_rate_parameter(parameters): + """Every rate parameter under the gambling duties has a wiring rate, + except Gaming Duty's statutory bands, an operator catalogue the household + formula does not read.""" + leaves = { + parameter.name + for parameter in parameters.gov.hmrc.gambling_duty.get_descendants() + if not hasattr(parameter, "children") and not hasattr(parameter, "brackets") + } + leaves = {name for name in leaves if ".gaming.normal_six_month_bands" not in name} + assert leaves == set(WIRING_RATES) - { + "household.consumption.gambling.casino_effective_rate" + } | {SWITCH} + + +@pytest.mark.parametrize("switch", [True, False]) +def test_each_series_charges_the_rates_the_model_charges(switch): + """Differential: with every rate parameter set to its own value in + 2029-30, a household holding the national bases pays, over each duty's + base, the rate the importer divides that duty's receipts by. The model's + formulas decide which rate each input pays, so a part of a series wired + to another parameter, or weighted as another part, fails.""" + year = 2029 + changes = {path: {str(year): rate} for path, rate in WIRING_RATES.items()} + changes[SWITCH] = {str(year): switch} + simulation = Simulation( + dataset=household_frame(2024, national_bases()), + scenario=Scenario(parameter_changes=changes), + ) + values = {**WIRING_RATES, SWITCH: float(switch)} + with pytest.MonkeyPatch.context() as patch: + patch.setattr( + import_obr_forecasts, + "parameter_values", + lambda path: ((date(2000, 1, 1), values[path]),), + ) + rates = { + spec.path[-1]: line_rate(spec, year) + for spec in RECEIPTS_SERIES_SPECS + if "gambling_duty_bases" in spec.path + } + assert rates.keys() == DUTIES.keys() + for duty, names in INPUTS.items(): + base = sum( + simulation.calculate(name, year)[0] + for name in names + if name != HIGHER_STAKE + ) + charged = simulation.calculate(DUTIES[duty], year)[0] + assert charged / base == pytest.approx(rates[duty], rel=1e-5), duty diff --git a/policyengine_uk/tests/test_import_obr_forecasts.py b/policyengine_uk/tests/test_import_obr_forecasts.py index 5e9d9ecb18..3c32efc24f 100644 --- a/policyengine_uk/tests/test_import_obr_forecasts.py +++ b/policyengine_uk/tests/test_import_obr_forecasts.py @@ -1,13 +1,15 @@ import os +from dataclasses import replace from datetime import date from pathlib import Path from io import BytesIO +from textwrap import indent from zipfile import ZipFile import xml.etree.ElementTree as ET import pytest import yaml -from hypothesis import given, settings +from hypothesis import event, given, settings from hypothesis import strategies as st from policyengine_uk.utils import import_obr_forecasts @@ -18,6 +20,9 @@ STATUTORY_GAP_SPECS, _sheet_paths, ForecastGap, + MACHINE_GAMES_HIGHER_STAKE_SHARE, + ReceiptsSeriesSpec, + machine_games_spec, average_earnings_path, compute_receipts_growth, find_receipts_line, @@ -37,6 +42,8 @@ extract_september_cpi_from_receipts, infer_forecast_start_year, infer_release, + per_head_growth, + population_growth, update_forecast_gap_yaml, update_yoy_growth_yaml, ) @@ -472,6 +479,28 @@ def receipts_table( "great_britain": (29.0, 30.1, 31.0), "northern_ireland": (0.39, 0.395, 0.4), } +# Synthetic betting and gaming duty receipts, £bn, from the year after each +# EFO's first (outturn) year, as the OBR's forecast breakdown starts. Remote +# Gaming Duty doubles in 2026-27 (2.8 = 1.4 x 40% / 21% x 1.05), when its +# rate rises from 21% to 40%, so its base grows 5%. +GAMBLING_LEVELS = { + "remote_gaming": (1.4, 2.8, 2.94), + "lottery": (1.0, 1.05, 1.1), + "general_betting": (0.7, 0.72, 0.9), + "machine_games": (0.65, 0.66, 0.8), + "gaming": (0.17, 0.18, 0.19), + "bingo": (0.03, 0.0001, 0.0), + "pool_betting": (0.004, 0.0042, 0.0044), +} +GAMBLING_LABELS = { + "remote_gaming": "Remote gaming duty", + "lottery": "Lottery duty", + "general_betting": "General betting duty", + "machine_games": "Machine games duty", + "gaming": "Gaming duty", + "bingo": "Bingo duty", + "pool_betting": "Pool betting duty", +} def make_receipts_xlsx( @@ -483,10 +512,12 @@ def make_receipts_xlsx( outturn_years: int = 1, sdlt_rows: list[tuple[str, tuple]] | None = None, chapter: int = 3, + gambling_levels: dict[str, tuple] | None = None, ) -> bytes: """Receipts tables numbered in ``chapter``: 3 to March 2026, 4 from October 2026.""" levels = levels or RECEIPTS_LEVELS + gambling_levels = gambling_levels or GAMBLING_LEVELS sheet_318 = make_sheet({2: [make_inline_cell("B2", "Other table")]}) sheet_319 = make_sheet( { @@ -571,6 +602,24 @@ def make_receipts_xlsx( first_column=4, outturn_years=outturn_years, ) + # The betting and gaming breakdown publishes forecasts only, from the + # year after the outturn year, under a total and an "of which" row. + sheets[f"{chapter}.10"] = receipts_table( + f"{chapter}.10 Betting and gaming duties forecast breakdown", + first_year + 1, + [ + ( + "Betting and gaming duties", + tuple( + sum(float(level) for level in year) + for year in zip(*gambling_levels.values()) + ), + ), + ("of which:", ()), + *((GAMBLING_LABELS[key], gambling_levels[key]) for key in GAMBLING_LABELS), + ], + outturn_years=0, + ) sheets[f"{chapter}.18"] = sheet_318 sheets[f"{chapter}.19"] = sheet_319 return make_workbook(sheets) @@ -753,16 +802,41 @@ def test_small_gaps_are_written_in_fixed_point(value, text): + RECEIPTS_SERIES_FIXTURE.format(key="northern_ireland") + "\n" ) +# The gambling duty bases, which the importer writes per head of +# ons.population under per_capita. +PER_CAPITA_FIXTURE = ( + " per_capita:\n" + " gambling_duty_bases:\n" + + "".join( + indent(RECEIPTS_SERIES_FIXTURE.format(key=key), " ") + for key in ( + "remote_gaming", + "lottery", + "general_betting", + "machine_games", + "gaming", + "bingo", + "pool_betting", + ) + ) + + "\n" +) +# Population growth changes in 2026 and 2030, so each year's per-head growth +# must take that year's population growth out. +POPULATION_FIXTURE = """ons: + population: + values: + 2024-01-01: 0.0100 + 2026-01-01: 0.0200 + 2030-01-01: -0.0050 +""" YOY_GROWTH_FIXTURE = ( "obr:\n" + "".join(FLAT_SERIES_FIXTURE.format(key=key) for key in SERIES_KEYS[:2]) + RECEIPTS_FIXTURE + + PER_CAPITA_FIXTURE + "".join(FLAT_SERIES_FIXTURE.format(key=key) for key in SERIES_KEYS[2:]) - + """ons: - population: - values: - 2024-01-01: 0.0100 -""" + + POPULATION_FIXTURE ) GAP_FIXTURE = """description: Gap. values: @@ -951,9 +1025,19 @@ def failing_replace(source, target): assert list(tmp_path.rglob("*.staged")) == [] -# Receipts growth series: stamp_duty_land_tax and business_rates under obr. +# Receipts growth series: stamp_duty_land_tax and business_rates under obr, +# which link their first year to the previous EFO's outturn, and the +# gambling_duty_bases under per_capita, which grow a duty's base per head +# from HMRC's outturn. -RECEIPTS_PATHS = {spec.path[-1]: spec.path for spec in RECEIPTS_SERIES_SPECS} +RECEIPTS_SPECS = tuple(spec for spec in RECEIPTS_SERIES_SPECS if spec.outturn is None) +RECEIPTS_PATHS = {spec.path[-1]: spec.path for spec in RECEIPTS_SPECS} +GAMBLING_PATHS = { + spec.path[-1]: spec.path + for spec in RECEIPTS_SERIES_SPECS + if spec.outturn is not None +} +ALL_RECEIPTS_PATHS = {**RECEIPTS_PATHS, **GAMBLING_PATHS} def _series(tree, path): @@ -994,7 +1078,7 @@ def test_receipts_lines_are_found_by_title_and_label_not_table_number(): find_receipts_line(current, spec), find_receipts_line(previous, spec), ) - for spec in RECEIPTS_SERIES_SPECS + for spec in RECEIPTS_SPECS } great_britain, great_britain_before = tables[RECEIPTS_PATHS["great_britain"]] @@ -1019,7 +1103,11 @@ def test_receipts_growth_is_the_ratio_of_published_levels(): ) # The first year is this EFO's outturn over the previous EFO's outturn, # e.g. residential 10.0 / 8.0; later years are this EFO's own ratios. - assert {path[-1]: series.values for path, series in growth.items()} == { + assert { + path[-1]: series.values + for path, series in growth.items() + if path in RECEIPTS_PATHS.values() + } == { "residential": {2024: 0.25, 2025: 0.1, 2026: 0.1}, "non_residential": {2024: 0.25, 2025: 0.25, 2026: -0.1}, "great_britain": {2024: 0.034483, 2025: 0.05, 2026: 0.05}, @@ -1101,6 +1189,54 @@ def test_a_last_level_at_or_below_zero_is_refused(last): compute_receipts_growth(levels, make_previous_receipts_xlsx()) +def test_a_tail_that_rounds_the_index_to_zero_is_refused(): + # The forecast keeps the index above zero; the average_earnings tail + # takes it there, after the forecast's own check has run. + levels = make_receipts_xlsx( + sdlt_rows=[ + ("SDLT from residential property", (1.0, 1.0, 0.00005)), + ("SDLT from non-residential property", (4.0, 5.0, 4.5)), + ] + ) + falling = FLAT_SERIES_FIXTURE.format(key="average_earnings").replace( + "0.0200", "-0.5100" + ) + content = YOY_GROWTH_FIXTURE.replace( + FLAT_SERIES_FIXTURE.format(key="average_earnings"), falling + ) + assert content != YOY_GROWTH_FIXTURE + render(current=levels) + with pytest.raises(ValueError, match="stamp_duty_land_tax.residential: the index"): + render(content, current=levels) + + +def test_the_index_check_reaches_the_end_of_the_growth_file(): + # A fall from 2035 takes the index to zero at five decimals in 2044. + with pytest.raises(ValueError, match="in 2044"): + import_obr_forecasts.check_index_stays_positive( + ("example",), + { + date(2024, 1, 1): 0.0, + date(2035, 1, 1): -0.75, + date(2074, 1, 1): 0.0, + }, + ) + + +def test_the_index_check_reads_the_value_in_force_each_january(): + # A value dated mid-year applies from the next January, so it must not + # replace that year's January value. + with pytest.raises(ValueError, match="in 2023"): + import_obr_forecasts.check_index_stays_positive( + ("example",), + { + date(2022, 1, 1): 0.0, + date(2023, 1, 1): -1.0, + date(2023, 7, 1): 0.0, + }, + ) + + def test_receipts_labels_lose_footnote_markers_only(): line = find_receipts_line( read_workbook_sheets( @@ -1232,7 +1368,7 @@ def test_render_changes_only_the_receipts_values_and_references(): rendered = render() before, after = yaml.safe_load(YOY_GROWTH_FIXTURE), yaml.safe_load(rendered) for tree in (before, after): - for path in RECEIPTS_PATHS.values(): + for path in ALL_RECEIPTS_PATHS.values(): del _series(tree, path)["values"] del _series(tree, path)["metadata"]["reference"] assert after == before @@ -1371,7 +1507,7 @@ def test_read_calendar_values_includes_nested_series(tmp_path): 2031: 0.02, } assert set(values) == set(SERIES_KEYS) | { - ".".join(path) for path in RECEIPTS_PATHS.values() + ".".join(path) for path in ALL_RECEIPTS_PATHS.values() } @@ -1422,6 +1558,17 @@ def test_rendered_growth_compounds_the_published_levels(years, data): "great_britain": ("33.929814327646056", "35.43608032863263", "37.21735218338801", "38.66510919685915", "41.462982206056644", "42.42035811098974", "43.447399285510656"), "northern_ireland": ("0.3955394050104385", "0.40595301023058955", "0.414556055466312", "0.4226801104356366", "0.43095512116333085", "0.4394491849889923", "0.44783993918412923"), } # fmt: skip +# The October 2026 EFO's betting and gaming duties (£bn, Table 4.10) from +# 2026-27, which publishes forecasts only. +OCTOBER_2026_GAMBLING = { + "remote_gaming": ("2.7520296250837784", "2.7812750734699425", "2.8416867878041874", "2.9053716896455919", "2.9446821891345443", "2.963204761653297"), + "lottery": ("0.83052038459597621", "0.84965566898040046", "0.87574553838772884", "0.90350845044750805", "0.92958587301870155", "0.95685079671453654"), + "general_betting": ("0.65560522982822456", "0.87744453968275893", "0.91694664204685206", "0.94862995017184315", "0.97646741817020943", "1.0041740187990271"), + "machine_games": ("0.67707068705786499", "0.82692873727074745", "0.85695449075649544", "0.86524127517297356", "0.87550370868921701", "0.89631020781238802"), + "gaming": ("0.17422987982556123", "0.18488058456996015", "0.18523427331264838", "0.18548069649974164", "0.18650974968911124", "0.18568125218312026"), + "bingo": ("1.8867652000000002E-4", "0", "0", "0", "0", "0"), + "pool_betting": ("2.8288821260900035E-3", "2.6802893699237468E-3", "2.85842704577818E-3", "2.8371447589557743E-3", "2.9959073747097458E-3", "3.1258296515431082E-3"), +} # fmt: skip MARCH_2026_RECEIPTS = { "residential": ("10.379999999999999", "11.731471524432562", "13.428906667032589", "14.984943814530874", "16.673948133476184", "17.964513468126285", "19.057301113969185"), "non_residential": ("3.5049999999999999", "3.6750451743530199", "4.72127759316189", "5.3288288104799904", "5.9009139798817092", "6.5330695355310402", "7.1046581772762902"), @@ -1432,9 +1579,9 @@ def test_rendered_growth_compounds_the_published_levels(years, data): def test_october_2026_tables_reproduce_the_committed_series(): """yoy_growth.yaml holds exactly what the importer writes from the - October 2026 receipts tables (Tables 4.7, 4.8 and 4.2) and the March 2026 - ones (3.7, 3.8 and 3.2): rerunning it on the committed file changes - nothing.""" + October 2026 receipts tables (Tables 4.7, 4.8, 4.2 and 4.10) and the + March 2026 ones (3.7, 3.8 and 3.2): rerunning it on the committed file + changes nothing.""" committed = get_yoy_growth_path().read_text() rendered = render( committed, @@ -1444,6 +1591,7 @@ def test_october_2026_tables_reproduce_the_committed_series(): levels=OCTOBER_2026_RECEIPTS, cash_table="4.8", chapter=4, + gambling_levels=OCTOBER_2026_GAMBLING, ), previous=make_previous_receipts_xlsx( release="March 2026", @@ -1709,7 +1857,7 @@ def test_main_requires_the_previous_receipts_tables(tmp_path): ) content = yoy.read_text() assert "2026-01-01: 0.0248" in content - for path in RECEIPTS_PATHS.values(): + for path in ALL_RECEIPTS_PATHS.values(): assert _values(content, path) == _values(YOY_GROWTH_FIXTURE, path) @@ -1927,3 +2075,680 @@ def test_main_writes_receipts_growth_to_the_output_path(tmp_path): ) assert yoy.read_text() == YOY_GROWTH_FIXTURE assert _values(output.read_text(), RECEIPTS_PATHS["residential"])[2024] == 0.25 + + +# Gambling duty bases (#2738): each line's receipts over the rate the model +# charges, after HMRC's outturn. + +GAMBLING_SPECS = tuple(spec for spec in RECEIPTS_SERIES_SPECS if spec.outturn) +# HMRC's 2024-25 and 2025-26 receipts, £bn (July 2026 Betting and Gaming +# tables, Table_1_BG_receipts rows 45 and 46). +HMRC_RECEIPTS = { + "remote_gaming": (1.162539, 1.320966), + "lottery": (0.931887, 0.980132), + "general_betting": (0.713858, 0.684502), + "machine_games": (0.609245, 0.641010), + "gaming": (0.165397, 0.173902), + "bingo": (0.025233, 0.026073), + "pool_betting": (0.008125, 0.003970), +} +# The remote part of the General Betting Duty base, which pays 25% from +# 1 April 2027; the rest pays 15% throughout. +REMOTE_BETTING_SHARE = 2_163_978_657 / (2_471_124_767 + 2_163_978_657 + 123_949_909) +# Machine Games Duty's 2024-25 bands (£): HMRC's liabilities (Table_2_MGD +# row 20, £ thousand) over their rates, 5%, 20% and 25%. +MACHINE_GAMES_BANDS = { + "lower": (6_921e3 / 0.05, 0.05), + "standard": (604_748e3 / 0.20, 0.20), + "higher": (596e3 / 0.25, 0.25), +} +# The Gambling Commission's 2024-25 gaming machine yield, £ million, in +# Category B and in Category C (Industry Statistics, April 2025 to March 2026 +# release, sheet 4, K34 and L34). +HIGHER_STAKE_SHARE = 2_289.84969448829 / (2_289.84969448829 + 232.47489298159996) + + +def machine_games_rate(year, share=HIGHER_STAKE_SHARE): + """Machine Games Duty's rate on its 2024-25 bands in the OBR's fiscal + year ``year``: Budget 2026 moves ``share`` of the standard-rate band from + 20% to 25% on 1 April 2027, the first day of 2027-28.""" + total = sum(base for base, _ in MACHINE_GAMES_BANDS.values()) + charged = sum(base * rate for base, rate in MACHINE_GAMES_BANDS.values()) + if year >= 2027: + charged += MACHINE_GAMES_BANDS["standard"][0] * share * 0.05 + return charged / total + + +def gambling_growth(**overrides): + growth = compute_receipts_growth( + make_receipts_xlsx(**overrides), b"not read", specs=GAMBLING_SPECS + ) + return {path[-1]: series.values for path, series in growth.items()} + + +def ratio(new, old): + return round(new / old - 1, 6) + + +def test_gambling_bases_take_out_the_rates_the_model_charges(): + """Synthetic March 2026 tables: growth from HMRC's 2024-25 receipts into + the table's first year (2025-26), then the table's own years. Remote + Gaming Duty's rise from 21% to 40% (1 April 2026) and the remote betting + rise from 15% to 25% (1 April 2027) come out of the base, and so does the + higher-stake machines' rise from 20% to 25% (1 April 2027); Bingo Duty's + abolition (1 April 2026) holds its base flat.""" + growth = gambling_growth() + hmrc = {key: values[0] for key, values in HMRC_RECEIPTS.items()} + remote = 0.15 * (1 - REMOTE_BETTING_SHARE) + 0.25 * REMOTE_BETTING_SHARE + expected = { + "remote_gaming": { + 2025: ratio(1.4, hmrc["remote_gaming"]), + # Receipts double (1.4 to 2.8), the base grows 5%. + 2026: ratio(2.8 / 0.40, 1.4 / 0.21), + 2027: ratio(2.94, 2.8), + }, + "general_betting": { + 2025: ratio(0.7, hmrc["general_betting"]), + 2026: ratio(0.72, 0.7), + 2027: ratio(0.9 / remote, 0.72 / 0.15), + }, + "bingo": {2025: ratio(0.03, hmrc["bingo"]), 2026: 0.0, 2027: 0.0}, + # Higher-stake machines pay 25% from 2027-28. + "machine_games": { + 2025: ratio(0.65, hmrc["machine_games"]), + 2026: ratio(0.66, 0.65), + 2027: ratio( + 0.8 / machine_games_rate(2027), 0.66 / machine_games_rate(2026) + ), + }, + } + for key in ("lottery", "gaming", "pool_betting"): + levels = (hmrc[key], *GAMBLING_LEVELS[key]) + expected[key] = {2025 + i: ratio(levels[i + 1], levels[i]) for i in range(3)} + assert growth.keys() == expected.keys() + for key, values in expected.items(): + assert growth[key] == pytest.approx(values, abs=1.5e-6), key + # Without the rate taken out, Remote Gaming Duty's 2026 growth would be + # the receipts' 100%. + assert growth["remote_gaming"][2026] == pytest.approx(0.05, abs=1e-6) + + +def test_real_october_2026_gambling_series(): + """The committed series, from Table 4.10 of the real October 2026 tables + and HMRC's outturn, by hand.""" + growth = compute_receipts_growth( + REAL_OCTOBER_2026_RECEIPTS, b"not read", specs=GAMBLING_SPECS + ) + values = {path[-1]: series.values for path, series in growth.items()} + # Remote Gaming Duty: 2026-27 at 40% over HMRC's 2025-26 at 21%. + assert values["remote_gaming"][2026] == round( + (2.7520296250837784 / 0.40) / (1.320966 / 0.21) - 1, 6 + ) + assert values["remote_gaming"][2026] == 0.093757 + # General Betting Duty: 2027-28 with its remote part at 25%. + remote = 0.15 + 0.10 * REMOTE_BETTING_SHARE + assert values["general_betting"][2027] == round( + (0.87744453968275893 / remote) / (0.65560522982822456 / 0.15) - 1, 6 + ) + assert values["general_betting"][2027] == 0.027038 + # Machine Games Duty: 2027-28 with the higher-stake share at 25%. + assert values["machine_games"][2027] == round( + (0.82692873727074745 / machine_games_rate(2027)) + / (0.67707068705786499 / machine_games_rate(2026)) + - 1, + 6, + ) + assert values["machine_games"][2027] == -0.002319 + assert values["bingo"] == dict.fromkeys(range(2026, 2032), 0.0) | {2025: 0.03329} + for series in values.values(): + assert sorted(series) == list(range(2025, 2032)) + + +def test_committed_gambling_series_are_the_real_bases_per_head(): + """yoy_growth.yaml holds each real October 2026 base's growth over one + plus the committed ons.population growth: Remote Gaming Duty's 2026-27 + base growth of 9.38% is 9.16% per head as population grows 0.2%. One plus + each value, times one plus population growth, is one plus the base's + growth, to the rounding of the six decimals both carry.""" + committed = get_yoy_growth_path().read_text() + population = population_growth(committed) + assert population[2026] == 0.002 + growth = compute_receipts_growth( + REAL_OCTOBER_2026_RECEIPTS, b"not read", specs=GAMBLING_SPECS + ) + per_head = { + path[-1]: per_head_growth(series, population).values + for path, series in growth.items() + } + base_growth = (2.7520296250837784 / 0.40) / (1.320966 / 0.21) + assert per_head["remote_gaming"][2026] == round( + base_growth / (1 + population[2026]) - 1, 6 + ) + for key, values in per_head.items(): + stored = _values(committed, GAMBLING_PATHS[key]) + assert stored[2024] == 0.0 + assert {year: stored[year] for year in values} == values, key + aggregate = growth[GAMBLING_PATHS[key]].values + for year, value in values.items(): + assert (1 + value) * (1 + population[year]) == pytest.approx( + 1 + aggregate[year], abs=1.1e-6 + ), (key, year) + + +def test_gambling_lines_read_no_previous_efo(): + # March 2025's tables have no betting and gaming breakdown; the gambling + # lines take 2024-25 and 2025-26 from HMRC instead. + real = compute_receipts_growth(REAL_RECEIPTS[2026], REAL_RECEIPTS[2025]) + alone = compute_receipts_growth(REAL_RECEIPTS[2026], b"", specs=GAMBLING_SPECS) + assert {path: real[path].values for path in alone} == { + path: series.values for path, series in alone.items() + } + assert all(series.previous_line is None for series in alone.values()) + + +def test_gambling_outturn_must_reach_the_year_before_the_table(): + # A table from 2027-28 needs 2026-27 receipts, which HMRC has not + # published in the July 2026 tables. + with pytest.raises(ValueError, match="have no 2026-27, the year before Table"): + compute_receipts_growth( + make_receipts_xlsx(first_year=2026), b"", specs=GAMBLING_SPECS + ) + + +def test_bingo_holds_its_base_flat_after_the_forecast(): + """Bingo Duty's rate is zero from 1 April 2026, so its base holds flat. + Per head it falls with population in every year to the last that + ons.population states, so the weighted base stays where it was.""" + rendered = render() + start = rendered.index(" bingo:") + block = rendered[start : rendered.index(" pool_betting:")] + assert ( + "# Beyond the OBR forecast: held flat, as the rate is zero; per head: " + "(1 + that growth) / (1 + ons.population growth) - 1" + ) in block + bingo = _values(rendered, GAMBLING_PATHS["bingo"]) + population = population_growth(rendered) + for year in range(2026, 2040): + flat = round(1 / (1 + _in_effect(population, year)) - 1, 6) + assert _in_effect(bingo, year) == flat, year + # Population last changes in 2030. + assert max(bingo) == 2030 + # Remote Gaming Duty follows average earnings, as the receipts lines do, + # over population growth: 2% to 2029, then -0.5%. + remote_gaming = _values(rendered, GAMBLING_PATHS["remote_gaming"]) + assert {year: remote_gaming[year] for year in (2028, 2030, 2031)} == { + 2028: round(1 / 1.02 - 1, 6), + 2030: round(1 / 0.995 - 1, 6), + 2031: round(1.02 / 0.995 - 1, 6), + } + + +def test_gambling_comments_and_references_name_hmrc_and_the_rates(): + rendered = render() + block = rendered[ + rendered.index(" remote_gaming:") : rendered.index(" lottery:") + ] + # March 2026's table starts in 2025-26, so no year comes from HMRC alone. + assert "# Outturn (2025" not in block + assert ( + "# Forecast (2025-26 over 2024-25) from OBR EFO March 2026, receipts " + 'Table 3.10 "Remote gaming duty", over HMRC\'s receipts (July 2026 ' + "Betting and Gaming tables, Table_1_BG_receipts); per head: base growth " + "(receipts over the rate the model charges) over one plus ons.population " + "growth" + ) in block + references = _series(yaml.safe_load(rendered), GAMBLING_PATHS["remote_gaming"])[ + "metadata" + ]["reference"] + assert [reference["title"] for reference in references] == [ + "OBR EFO March 2026 (detailed forecast tables, receipts, Table 3.10)", + "HMRC UK Betting and Gaming Statistics, July 2026 tables " + "(Table_1_BG_receipts, 2024-25 and 2025-26 receipts)", + ] + + +def test_gambling_series_are_written_per_head(): + """The importer writes each gambling base's growth over one plus that + year's ons.population growth. In the synthetic March 2026 tables Remote + Gaming Duty's receipts double into 2026-27 as its rate rises from 21% to + 40%, so its base grows 5%, and per head 5% over population growth of 2%. + Every other year and duty divides the same way.""" + rendered = render() + remote_gaming = _values(rendered, GAMBLING_PATHS["remote_gaming"]) + assert remote_gaming[2026] == round((2.8 / 0.40) / (1.4 / 0.21) / 1.02 - 1, 6) + population = population_growth(YOY_GROWTH_FIXTURE) + for key, aggregate in gambling_growth().items(): + values = _values(rendered, GAMBLING_PATHS[key]) + assert sorted(aggregate) == [2025, 2026, 2027] + for year, growth in aggregate.items(): + per_head = (1 + growth) / (1 + _in_effect(population, year)) - 1 + # Both carry six decimals. + assert values[year] == pytest.approx(per_head, abs=1.1e-6), (key, year) + + +def test_the_index_check_reads_the_per_head_series(): + """The model compounds the per-head series, so the index check reads it. + Bingo Duty's 2025-26 receipts here leave its index at 0.00002, and its + base holds flat from 2026-27, so the aggregate index stays there. If + population triples in 2028 and 2029, the per-head index falls to 0.00001 + and then to zero at five decimals, and the importer refuses it.""" + bingo = HMRC_RECEIPTS["bingo"][0] * 0.00002 + workbook = make_receipts_xlsx( + gambling_levels=GAMBLING_LEVELS | {"bingo": (bingo, 0.0001, 0.0)} + ) + aggregate = compute_receipts_growth(workbook, b"", specs=GAMBLING_SPECS) + assert aggregate[GAMBLING_PATHS["bingo"]].values == { + 2025: -0.99998, + 2026: 0.0, + 2027: 0.0, + } + render(current=workbook) + tripling = YOY_GROWTH_FIXTURE.replace( + " 2030-01-01: -0.0050\n", + " 2028-01-01: 2.0000\n 2030-01-01: -0.0050\n", + ) + assert population_growth(tripling)[2028] == 2.0 + with pytest.raises( + ValueError, + match="per_capita.gambling_duty_bases.bingo: the index reaches 0.0 in 2029", + ): + render(tripling, current=workbook) + + +def test_the_machine_games_parameters_are_the_oracles(): + """The hand-written Machine Games Duty rates above are the model's: 5%, + 20% and 25%, with higher-stake machines at the higher rate from 1 April + 2027.""" + folder = Path(__file__).parents[1] / "parameters/gov/hmrc/gambling_duty" + + def values(name): + data = yaml.safe_load((folder / f"machine_games/{name}.yaml").read_text()) + return { + str(day): value["value"] if isinstance(value, dict) else value + for day, value in data["values"].items() + } + + assert values("rates/lower") == {"2013-02-01": 0.05} + assert values("rates/standard") == {"2013-02-01": 0.2} + assert values("rates/higher") == {"2015-03-01": 0.25} + assert values("higher_stake_machines_at_higher_rate") == { + "2013-02-01": False, + "2027-04-01": True, + } + + +def test_the_higher_stake_share_is_category_b_over_b_and_c(): + assert MACHINE_GAMES_HIGHER_STAKE_SHARE == HIGHER_STAKE_SHARE + assert HIGHER_STAKE_SHARE == pytest.approx(0.907833, abs=1e-6) + spec = machine_games_spec(HIGHER_STAKE_SHARE) + assert spec in GAMBLING_SPECS + standard = MACHINE_GAMES_BANDS["standard"][0] + assert [part.weight for part in spec.rates] == pytest.approx( + [ + MACHINE_GAMES_BANDS["lower"][0], + standard * (1 - HIGHER_STAKE_SHARE), + standard * HIGHER_STAKE_SHARE, + MACHINE_GAMES_BANDS["higher"][0], + ] + ) + + +def machine_games_levels(levels): + """A synthetic March 2026 workbook whose Machine Games Duty line is + ``levels`` from 2025-26 (the other lines are 1).""" + return make_receipts_xlsx( + gambling_levels={key: (1.0,) * len(levels) for key in GAMBLING_LABELS} + | {"machine_games": tuple(levels)} + ) + + +def machine_games_growth(spec, levels): + growth = compute_receipts_growth(machine_games_levels(levels), b"", specs=(spec,)) + return growth[spec.path].values + + +def test_a_rise_that_behaviour_offsets_gives_a_base_index_of_0_8(): + """£100 of higher-stake takings pay £20 at 20%. If the rise to 25% cuts + them to £80, they pay the same £20 and receipts hold flat. The base index + falls to 0.8, so the model's 25% on the projected £80 gives back £20.""" + spec = machine_games_spec(1.0) + standard_only = replace( + spec, + rates=tuple(part for part in spec.rates if part.parameter.endswith("standard")), + outturn=replace(spec.outturn, receipts_thousands=((2024, 20e3),)), + ) + values = machine_games_growth(standard_only, (0.02, 0.02, 0.02)) + assert values == {2025: 0.0, 2026: 0.0, 2027: -0.2} + index = (1 + values[2025]) * (1 + values[2026]) * (1 + values[2027]) + assert index == pytest.approx(0.8, abs=1e-12) + assert 100 * index * 0.25 == pytest.approx(20, abs=1e-9) + + +SHARES = st.floats(min_value=0, max_value=1, allow_nan=False) + + +@settings(max_examples=60, deadline=None, derandomize=True) +@given(share=SHARES, levels=st.lists(LEVELS, min_size=5, max_size=5)) +def test_machine_games_bases_reconcile_to_receipts_at_any_share(share, levels): + """For any higher-stake share and any receipts path from 2025-26 to + 2029-30: each year's growth is the receipts' growth over the growth in + the rate the model charges with that share at 25% from 2027-28. So a base + that gives 2024-25's receipts gives every later year's at that rate, to + within the rounding of the six-decimal growth.""" + values = machine_games_growth(machine_games_spec(share), levels) + receipts = {2024: HMRC_RECEIPTS["machine_games"][0]} + receipts |= dict(zip(range(2025, 2030), levels)) + rates = {year: machine_games_rate(year, share) for year in receipts} + assert sorted(values) == list(range(2025, 2030)) + base = receipts[2024] / rates[2024] + tolerance = 1.0 + for year in range(2025, 2030): + exact = (receipts[year] / rates[year]) / (receipts[year - 1] / rates[year - 1]) + assert values[year] == pytest.approx(exact - 1, abs=5.1e-7) + base *= 1 + values[year] + tolerance *= 1 + 5.1e-7 / exact + assert base * rates[year] == pytest.approx( + receipts[year], rel=tolerance - 1 + 1e-12 + ) + + +@settings(max_examples=40, deadline=None, derandomize=True) +@given(levels=st.lists(LEVELS, min_size=5, max_size=5)) +def test_a_zero_share_grows_the_machine_games_base_with_receipts(levels): + """With no higher-stake share the rate does not change, so the base grows + with receipts: the plain rule of receipts over the rate, the series of + the three bands alone.""" + zero = machine_games_spec(0.0) + bands = replace(zero, rates=tuple(part for part in zero.rates if not part.switch)) + assert len(bands.rates) == 3 + values = machine_games_growth(zero, levels) + assert values == machine_games_growth(bands, levels) + receipts = {2024: HMRC_RECEIPTS["machine_games"][0]} + receipts |= dict(zip(range(2025, 2030), levels)) + assert values == pytest.approx( + {year: receipts[year] / receipts[year - 1] - 1 for year in range(2025, 2030)}, + abs=5.1e-7, + ) + + +GAMBLING_RATE_PARAMETERS = sorted( + {path for spec in GAMBLING_SPECS for part in spec.rates for path in part.paths} +) +SWITCHES = {part.switch for spec in GAMBLING_SPECS for part in spec.rates} - {None} +RATE = st.floats(min_value=0.01, max_value=0.6, allow_nan=False) + + +@st.composite +def rate_schedules(draw): + """Dated values for every gambling rate parameter: a first value, then + perhaps a change on any day from 2024 to 2028, to zero or not. A switch + takes true or false, read as 1 or 0.""" + schedules = {} + for parameter in GAMBLING_RATE_PARAMETERS: + value = st.booleans().map(float) if parameter in SWITCHES else RATE + changed = st.booleans().map(float) if parameter in SWITCHES else RATE + entries = [(date(2015, 1, 1), draw(value))] + if draw(st.booleans()): + day = draw(st.dates(date(2024, 1, 1), date(2028, 12, 31))) + entries.append((day, draw(st.one_of(st.just(0.0), changed)))) + schedules[parameter] = tuple(entries) + return schedules + + +def oracle_rate(spec, schedules, year): + """The weighted mean of each part's daily rate over 1 April to 31 March. + A switched part pays its switched rate on the days its switch is on.""" + days = [ + date.fromordinal(day) + for day in range(date(year, 4, 1).toordinal(), date(year + 1, 4, 1).toordinal()) + ] + + def on(parameter, day): + entries = schedules[parameter] + return max((e for e in entries if e[0] <= day), default=entries[0])[1] + + total = sum(component.weight for component in spec.rates) + mean = 0.0 + for component in spec.rates: + daily = [ + on(component.switched, day) + if component.switch and on(component.switch, day) + else on(component.parameter, day) + for day in days + ] + mean += component.weight * sum(daily) / len(daily) + return mean / total + + +def oracle_growth(spec, receipts, schedules): + """Each year's base growth to six decimals, and whether the index the + model compounds from it, rounded to five decimals, reaches zero.""" + rates = {year: oracle_rate(spec, schedules, year) for year in receipts} + growth, index, zero = {}, 1.0, False + for year in sorted(receipts)[1:]: + if rates[year] < 1e-12 or rates[year - 1] < 1e-12: + growth[year] = 0.0 + else: + exact = (receipts[year] / rates[year]) / ( + receipts[year - 1] / rates[year - 1] + ) + growth[year] = round(exact - 1, 6) + index = round(index * (1 + growth[year]), 5) + zero = zero or index <= 0 + return growth, rates, zero + + +def gambling_receipts(key, levels): + return { + 2024: HMRC_RECEIPTS[key][0], + **dict(zip(range(2025, 2025 + len(levels)), levels)), + } + + +@settings(max_examples=40, deadline=None, derandomize=True) +@given( + levels=st.fixed_dictionaries( + {key: st.tuples(LEVELS, LEVELS, LEVELS) for key in GAMBLING_LABELS} + ), + schedules=rate_schedules(), + factor=st.floats(min_value=0.1, max_value=10, allow_nan=False), +) +def test_base_growth_is_receipts_growth_over_rate_growth(levels, schedules, factor): + """For any positive receipts and dated rates: one plus each year's growth + times the rate's growth is the receipts' growth (to the six decimals + written); a year whose rate, or the year before's, is zero holds the base + flat; and one factor on every rate, a rate reform the model applies + itself, changes no growth. A series whose index would round to zero is + refused instead.""" + module = import_obr_forecasts + workbook = make_receipts_xlsx(gambling_levels=levels) + scaled = { + parameter: tuple( + (day, value if parameter in SWITCHES else value * factor) + for day, value in entries + ) + for parameter, entries in schedules.items() + } + oracles = { + spec.path: oracle_growth( + spec, gambling_receipts(spec.path[-1], levels[spec.path[-1]]), schedules + ) + for spec in GAMBLING_SPECS + } + with pytest.MonkeyPatch.context() as patch: + if any(zero for _, _, zero in oracles.values()): + event("refused") + patch.setattr(module, "parameter_values", lambda p: schedules[p]) + with pytest.raises(ValueError, match="too small to grow from"): + compute_receipts_growth(workbook, b"", specs=GAMBLING_SPECS) + return + event("written") + patch.setattr(module, "parameter_values", lambda p: schedules[p]) + growth = compute_receipts_growth(workbook, b"", specs=GAMBLING_SPECS) + patch.setattr(module, "parameter_values", lambda p: scaled[p]) + growth_scaled = compute_receipts_growth(workbook, b"", specs=GAMBLING_SPECS) + for spec in GAMBLING_SPECS: + values = growth[spec.path].values + receipts = gambling_receipts(spec.path[-1], levels[spec.path[-1]]) + expected, rates, _ = oracles[spec.path] + assert sorted(values) == [2025, 2026, 2027] + for year, value in values.items(): + if rates[year] < 1e-12 or rates[year - 1] < 1e-12: + assert value == 0.0 + else: + exact = (receipts[year] / rates[year]) / ( + receipts[year - 1] / rates[year - 1] + ) + assert value == pytest.approx(exact - 1, abs=1e-6, rel=1e-9) + assert 1 + value > 0 + assert growth_scaled[spec.path].values == pytest.approx(values, abs=1.5e-6) + + +# Per head (#2738): dataset projection grows household weights by +# ons.population, so each gambling series is its base's growth over one plus +# population growth, and compounding it with population growth gives back +# the aggregate base. + +GROWTH_FACTOR = st.floats(min_value=0.5, max_value=2, allow_nan=False) +POPULATION_GROWTH = st.floats(min_value=-0.02, max_value=0.03, allow_nan=False).map( + lambda value: round(value, 4) +) +REAL_RATES = { + parameter: import_obr_forecasts.parameter_values(parameter) + for parameter in GAMBLING_RATE_PARAMETERS +} +# The model's gambling rates last change in 2027-28, so each line's rate holds +# from 2028-29. +assert max(day.year for entries in REAL_RATES.values() for day, _ in entries) < 2028 +LINE_RATES = { + spec.path: {year: oracle_rate(spec, REAL_RATES, year) for year in range(2024, 2029)} + for spec in GAMBLING_SPECS +} + + +@settings(max_examples=40, deadline=None, derandomize=True) +@given(last=st.integers(2025, 2045), data=st.data()) +def test_per_head_series_compound_to_the_aggregate_index(last, data): + """For any gambling receipts and any ons.population series ending in any + year: in every year to 2060, one plus a rendered series' growth, times one + plus population growth, is one plus the base's growth, to the rounding of + the six decimals written. The base grows by receipts over the rate the + model charges to the end of the forecast, then with average_earnings, or + holds flat while its rate is zero. So the per-head index, compounded with + the population index, gives the aggregate index, and a second run changes + nothing.""" + levels = {} + for key in GAMBLING_LABELS: + level = HMRC_RECEIPTS[key][0] + series = [] + for _ in range(3): + level *= data.draw(GROWTH_FACTOR) + series.append(level) + levels[key] = tuple(series) + population = { + year: data.draw(POPULATION_GROWTH, label=f"population {year}") + for year in range(2024, last + 1) + } + content = YOY_GROWTH_FIXTURE.replace( + POPULATION_FIXTURE, + "ons:\n population:\n values:\n" + + "".join( + f" {year}-01-01: {format_growth(value)}\n" + for year, value in population.items() + ), + ) + current = make_receipts_xlsx(gambling_levels=levels) + rendered = render(content, current=current) + earnings = _values(rendered, ("average_earnings",)) + for spec in GAMBLING_SPECS: + values = _values(rendered, spec.path) + # Years before the first growth year keep the fixture's values. + assert (values[2023], values[2024]) == (0.05, 0.01) + receipts = gambling_receipts(spec.path[-1], levels[spec.path[-1]]) + rates = { + year: LINE_RATES[spec.path][min(year, 2028)] for year in range(2024, 2061) + } + per_head_index = population_index = aggregate_index = tolerance = 1.0 + for year in range(2025, 2061): + if rates[year] < 1e-12 or rates[year - 1] < 1e-12: + exact = 1.0 + elif year in receipts: + exact = (receipts[year] / rates[year]) / ( + receipts[year - 1] / rates[year - 1] + ) + else: + exact = 1 + _in_effect(earnings, year) + growth = _in_effect(values, year) + people = 1 + _in_effect(population, year) + assert growth > -1 + # Six-decimal rounding moves the product by at most 5e-7 times + # one plus the population growth. + error = 5e-7 * people + 1e-12 + assert abs((1 + growth) * people - exact) <= error, (spec.path, year) + per_head_index *= 1 + growth + population_index *= people + aggregate_index *= exact + tolerance *= 1 + error / exact + assert per_head_index * population_index == pytest.approx( + aggregate_index, rel=tolerance - 1 + 1e-12 + ), (spec.path, year) + assert render(rendered, current=current) == rendered + + +def test_a_fall_that_rounds_the_index_to_zero_is_refused(): + """Receipts that fall from £1,000bn to £0.01bn as the rate rises from 1% + to 60% shrink the base by a factor of six million. To six decimals that + growth is -100%, which would hold the index at zero in every later year, + so the importer refuses it.""" + spec = next(spec for spec in GAMBLING_SPECS if spec.path[-1] == "remote_gaming") + workbook = make_receipts_xlsx( + gambling_levels=GAMBLING_LEVELS | {"remote_gaming": (1000.0, 0.01, 0.01)} + ) + schedule = ((date(2015, 1, 1), 0.01), (date(2026, 4, 1), 0.6)) + with pytest.MonkeyPatch.context() as patch: + patch.setattr(import_obr_forecasts, "parameter_values", lambda p: schedule) + with pytest.raises( + ValueError, + match="2026-27 growth of -1.0 takes the index to 0.0 at five decimals", + ): + compute_receipts_growth(workbook, b"", specs=(spec,)) + + +POSITIVE = st.one_of(LEVELS, st.floats(min_value=1e-9, max_value=1e-2, allow_nan=False)) + + +@settings(max_examples=60, deadline=None, derandomize=True) +@given( + levels=st.fixed_dictionaries( + {key: st.tuples(POSITIVE, POSITIVE, POSITIVE) for key in GAMBLING_LABELS} + ), + schedules=rate_schedules(), +) +def test_a_written_series_keeps_every_index_positive(levels, schedules): + """For any positive receipts, however small, and any dated rates: the + importer either refuses, when the written growth would round the model's + index to zero, or writes series whose index the model compounds to five + decimals stays above zero in every year.""" + workbook = make_receipts_xlsx(gambling_levels=levels) + with pytest.MonkeyPatch.context() as patch: + patch.setattr(import_obr_forecasts, "parameter_values", lambda p: schedules[p]) + try: + growth = compute_receipts_growth(workbook, b"", specs=GAMBLING_SPECS) + except ValueError as error: + event("refused") + assert "too small to grow from" in str(error) + assert any( + oracle_growth( + spec, + gambling_receipts(spec.path[-1], levels[spec.path[-1]]), + schedules, + )[2] + for spec in GAMBLING_SPECS + ) + return + event("written") + for series in growth.values(): + index = 1.0 + for year in sorted(series.values): + index = round(index * (1 + series.values[year]), 5) + assert index > 0 diff --git a/policyengine_uk/tests/test_population_vintage.py b/policyengine_uk/tests/test_population_vintage.py index 62307a4679..13dbbb5ece 100644 --- a/policyengine_uk/tests/test_population_vintage.py +++ b/policyengine_uk/tests/test_population_vintage.py @@ -14,8 +14,10 @@ (28 May 2026), which from 2031-32 is the principal projection of the ONS 2024-based national population projections. -per_capita.*: (1 + aggregate growth) / (1 + ons.population) - 1, rounded to 4 -decimals, with calendar-year aggregates from the October 2026 EFO tables. +per_capita.gdp, mixed_income and non_labour_income: (1 + aggregate growth) / +(1 + ons.population) - 1, rounded to 4 decimals, with calendar-year +aggregates from the October 2026 EFO tables. The importer writes the +per_capita gambling series to six decimals (test_import_obr_forecasts.py). """ import pytest diff --git a/policyengine_uk/utils/import_obr_forecasts.py b/policyengine_uk/utils/import_obr_forecasts.py index fe271e864f..e9423bdd02 100644 --- a/policyengine_uk/utils/import_obr_forecasts.py +++ b/policyengine_uk/utils/import_obr_forecasts.py @@ -5,9 +5,10 @@ import os import re import shutil -from dataclasses import dataclass +from dataclasses import dataclass, replace from datetime import date from decimal import Decimal +from functools import lru_cache from io import BytesIO from pathlib import Path from urllib.parse import unquote @@ -589,6 +590,59 @@ def update_forecast_gap_yaml( TABLE_NUMBER_RE = re.compile(r"\d+\.\d+") FOOTNOTE_RE = re.compile(r"(?<=[a-z])\d+$", re.IGNORECASE) VALUE_LINE_RE = re.compile(r"^\s*(\d{4})-\d{2}-\d{2}:") +# The last year any growth series reaches: yoy_growth.yaml's last entries are +# for 2073, and lag_source_years runs a lagged series a year past its source. +# create_economic_assumption_indices stops at or before 2074, so a series that +# stays positive to here stays positive in the model. +LAST_INDEX_YEAR = 2074 + + +@dataclass(frozen=True) +class RateComponent: + """A part of a duty's base and the rate parameter the model charges on it. + + ``parameter`` is the rate's path in the model's parameter tree, such as + ``gov.hmrc.gambling_duty.remote_gaming.rate``. ``weight`` is the part's + size, in any unit: divided by the total of a line's weights, it is the + part's share of the base, held at that share in every year. + + A part with a ``switch``, the path of a dated true-or-false parameter, + pays the rate at ``switched`` on the days the switch is true, as the + model's formula does for a part that moves between rates. + """ + + parameter: str + weight: float = 1.0 + switch: str | None = None + switched: str | None = None + + def __post_init__(self): + if (self.switch is None) != (self.switched is None): + raise ValueError( + f"{self.parameter}: a switched part needs both a switch and " + "the rate it switches to" + ) + + @property + def paths(self) -> tuple[str, ...]: + """The parameters the part's rate reads.""" + return tuple( + path + for path in (self.parameter, self.switch, self.switched) + if path is not None + ) + + +@dataclass(frozen=True) +class ReceiptsOutturn: + """A line's outturn receipts from outside the EFOs, in £ thousand (as + published) by the calendar year in which each fiscal year starts. + ``source`` names them in comments and ``title`` in references.""" + + source: str + title: str + href: str + receipts_thousands: tuple[tuple[int, float], ...] @dataclass(frozen=True) @@ -600,11 +654,121 @@ class ReceiptsSeriesSpec: receipts on a cash basis are Table 3.8 in March 2026 and 3.9 in March 2025. Titles and row labels are compared lowercased, without footnote markers. + + A line with ``rates`` grows a duty's base rather than its receipts, so a + rate change the model applies itself moves receipts only once. Each + year's base is its receipts over the rate the model charges in that + fiscal year (see ``line_rate``). A year whose rate is zero has no base, + so the series holds the base flat into and out of it. A line with + ``outturn`` takes the years before the EFO's first from it, not from the + previous EFO, which it does not read. ``references`` are cited after the + sources of the levels. + + A series under ``per_capita`` grows its line per head (``per_head``). """ path: tuple[str, ...] title: str row: str + rates: tuple[RateComponent, ...] = () + outturn: ReceiptsOutturn | None = None + references: tuple[tuple[str, str], ...] = () + + @property + def per_head(self) -> bool: + """Whether the series sits under ``per_capita``. Dataset projection + grows household weights by ``ons.population``, so + ``render_receipts_growth_yaml`` writes such a series' growth over one + plus that growth (``per_head_growth``, ``per_head_tail``), and the + weighted total grows with the line in every year.""" + return self.path[0] == "per_capita" + + +GAMBLING_RATES = "gov.hmrc.gambling_duty" +MACHINE_GAMES = f"{GAMBLING_RATES}.machine_games" + +# The higher-stake share of Machine Games Duty's standard-rate base: the part +# from machines with a maximum charge above £1 or a maximum prize above £100, +# which pays the higher rate from 1 April 2027 (#2731). It is the Gambling +# Commission's 2024-25 gaming machine yield in Category B over that in +# Categories B and C, £ million (Industry Statistics, April 2025 to March +# 2026 release, sheet 4, cells K34 and L34, revised). Category B holds the +# machines outside Categories C and D, and Category C's limits are a £1 +# charge and a £100 prize. Category D machines either offer no cash prize, +# so their games are not dutiable, or charge at most 20p for at most £10 in +# cash, the lower-rate class (Finance Act 2012, Schedule 24, paragraphs 2(2) +# and 5(2)). The statistics cover only the premises the Commission licenses +# in Great Britain, not pubs, clubs or Northern Ireland: see +# docs/engineering/gambling-duties.md for how that bears on the share. +MACHINE_GAMES_HIGHER_STAKE_SHARE = 2_289.84969448829 / ( + 2_289.84969448829 + 232.47489298159996 +) + + +def gambling_spec( + key: str, + row: str, + rates: tuple[RateComponent, ...], + receipts_thousands: tuple[tuple[int, float], ...], + **extra, +) -> ReceiptsSeriesSpec: + """A betting and gaming duty's base per head (#2738): its line in the + receipts tables' forecast breakdown (Table 4.10 in October 2026, 3.10 in + March 2026), which publishes forecasts only, after HMRC's outturn. The + model's gambling inputs start in 2024-25, the first year here.""" + return ReceiptsSeriesSpec( + path=("per_capita", "gambling_duty_bases", key), + title="betting and gaming duties forecast breakdown", + row=row, + rates=rates, + outturn=ReceiptsOutturn( + source="HMRC's receipts (July 2026 Betting and Gaming tables, " + "Table_1_BG_receipts)", + title="HMRC UK Betting and Gaming Statistics, July 2026 tables " + "(Table_1_BG_receipts, 2024-25 and 2025-26 receipts)", + href="https://assets.publishing.service.gov.uk/media/" + "6ab54fd9fe72ed1e2b02f06b/2026_Jul_Bet_and_Gam_Tab.ods", + receipts_thousands=receipts_thousands, + ), + **extra, + ) + + +def machine_games_spec(higher_stake_share: float) -> ReceiptsSeriesSpec: + """Machine Games Duty's base. Each band's weight is its 2024-25 base (£): + HMRC's liabilities (Table_2_MGD row 20) over its rate. Table 4.10's line + includes Autumn Budget 2026's higher rate on higher-stake machines + (paragraph 4.45), so ``higher_stake_share`` of the standard-rate band + pays the higher rate on the days the model's switch is on, as + ``machine_games_duty`` charges a higher-stake input at that share.""" + standard = 3_023_740_000 + return gambling_spec( + "machine_games", + "machine games duty", + ( + RateComponent(f"{MACHINE_GAMES}.rates.lower", 138_420_000), + RateComponent( + f"{MACHINE_GAMES}.rates.standard", standard * (1 - higher_stake_share) + ), + RateComponent( + f"{MACHINE_GAMES}.rates.standard", + standard * higher_stake_share, + switch=f"{MACHINE_GAMES}.higher_stake_machines_at_higher_rate", + switched=f"{MACHINE_GAMES}.rates.higher", + ), + RateComponent(f"{MACHINE_GAMES}.rates.higher", 2_384_000), + ), + ((2024, 609_245), (2025, 641_010)), + references=( + ( + "Gambling Commission, Industry Statistics, April 2025 to March " + "2026 (sheet 4, 2024-25 gaming machine yield by category)", + "https://www.gamblingcommission.gov.uk/statistics-and-research/" + "publication/industry-statistics-annual-report-financial-year-april-" + "2025-to-march-2026-official-statistics", + ), + ), + ) RECEIPTS_SERIES_SPECS = ( @@ -628,9 +792,163 @@ class ReceiptsSeriesSpec: title="other taxes", row="northern ireland business rates", ), + # The seven betting and gaming duties' bases. HMRC's receipts are row 45 + # (2024-25) and row 46 (2025-26) of Table_1_BG_receipts, £ thousand. + gambling_spec( + "remote_gaming", + "remote gaming duty", + (RateComponent(f"{GAMBLING_RATES}.remote_gaming.rate"),), + ((2024, 1_162_539), (2025, 1_320_966)), + ), + gambling_spec( + "lottery", + "lottery duty", + (RateComponent(f"{GAMBLING_RATES}.lottery.rate"),), + ((2024, 931_887), (2025, 980_132)), + ), + # Each part's 2024-25 base (£): HMRC's General Betting Duty receipts over + # 15%, split by the Gambling Commission's 2024-25 gross gambling yield as + # Microcosm splits the model's inputs. Fixed odds take remote UK horse + # racing, land-based fixed odds and racing pools. Microcosm's split gives + # spread betting no part, so it has no weight. + gambling_spec( + "general_betting", + "general betting duty", + ( + RateComponent( + f"{GAMBLING_RATES}.general_betting.rates.fixed_odds", 2_471_124_767 + ), + RateComponent( + f"{GAMBLING_RATES}.general_betting.rates.remote", 2_163_978_657 + ), + RateComponent( + f"{GAMBLING_RATES}.general_betting.rates.exchange_commission", + 123_949_909, + ), + ), + ((2024, 713_858), (2025, 684_502)), + references=( + ( + "Microcosm, FY2024-25 General Betting Duty bases split by the " + "Gambling Commission's gross gambling yield (docs/evidence/" + "uk-excise-quantities-1359/outputs/national_reconciliation.json)", + "https://github.com/PolicyEngine/microcosm/blob/main/docs/evidence/" + "uk-excise-quantities-1359/outputs/national_reconciliation.json", + ), + ( + "Gambling Commission, industry statistics, April 2024 to March 2025", + "https://www.gamblingcommission.gov.uk/statistics-and-research/" + "publication/industry-statistics-annual-report-financial-year-april-" + "2024-to-march-2025", + ), + ), + ), + machine_games_spec(MACHINE_GAMES_HIGHER_STAKE_SHARE), + # The household Gaming Duty formula charges one effective rate, not the + # statutory bands. + gambling_spec( + "gaming", + "gaming duty", + (RateComponent("household.consumption.gambling.casino_effective_rate"),), + ((2024, 165_397), (2025, 173_902)), + ), + gambling_spec( + "bingo", + "bingo duty", + (RateComponent(f"{GAMBLING_RATES}.bingo.rate"),), + ((2024, 25_233), (2025, 26_073)), + ), + gambling_spec( + "pool_betting", + "pool betting duty", + (RateComponent(f"{GAMBLING_RATES}.pool_betting.rate"),), + ((2024, 8_125), (2025, 3_970)), + ), ) +@lru_cache(maxsize=None) +def parameter_values(parameter: str) -> tuple[tuple[date, float], ...]: + """A parameter's dated values, earliest first, from its file in the + model's parameter tree. True and false read as 1 and 0.""" + path = get_repo_root() / "policyengine_uk/parameters" / Path(*parameter.split(".")) + values = yaml.safe_load(path.with_suffix(".yaml").read_text())["values"] + entries = [] + for instant, value in values.items(): + if isinstance(value, dict): + value = value["value"] + entries.append((date.fromisoformat(str(instant)), float(value))) + if not entries: + raise ValueError(f"{parameter} has no values") + return tuple(sorted(entries)) + + +def value_on(parameter: str, day: date) -> float: + """A parameter's value on ``day``. Before its first value it takes that + value, as the model does: it backdates each parameter's first value.""" + entries = parameter_values(parameter) + value = entries[0][1] + for instant, entry in entries: + if instant <= day: + value = entry + return value + + +def component_rate(component: RateComponent, day: date) -> float: + """The rate a part of a base pays on ``day``.""" + if component.switch is not None and value_on(component.switch, day): + return value_on(component.switched, day) + return value_on(component.parameter, day) + + +def fiscal_year_rate(component: RateComponent, year: int) -> float: + """A part's rate over the OBR's fiscal year ``year``, 1 April to 31 + March, averaged by days.""" + start, end = date(year, 4, 1), date(year + 1, 4, 1) + changes = { + instant + for path in component.paths + for instant, _ in parameter_values(path) + if start < instant < end + } + bounds = [start, *sorted(changes), end] + segments = [ + (component_rate(component, lower), (upper - lower).days) + for lower, upper in zip(bounds, bounds[1:]) + ] + if len({value for value, _ in segments}) == 1: + return segments[0][0] + return sum(value * days for value, days in segments) / (end - start).days + + +def line_rate(spec: ReceiptsSeriesSpec, year: int) -> float: + """The rate the model charges on a line's base in fiscal year ``year``: + its parts' fiscal-year rates, averaged by their weights.""" + total = sum(component.weight for component in spec.rates) + return ( + sum( + component.weight * fiscal_year_rate(component, year) + for component in spec.rates + ) + / total + ) + + +def growth_ratio( + spec: ReceiptsSeriesSpec, levels: dict[int, float], year: int +) -> float: + """One plus a line's growth into ``year``, before rounding: the ratio of + its levels, or for a line with rates, of its bases, each year's receipts + over the rate the model charges (``line_rate``). A year whose rate, or + the year before's, is zero has no base, so the base holds flat.""" + if not spec.rates: + return levels[year] / levels[year - 1] + rate, previous_rate = line_rate(spec, year), line_rate(spec, year - 1) + if rate == 0 or previous_rate == 0: + return 1.0 + return (levels[year] / rate) / (levels[year - 1] / previous_rate) + + @dataclass(frozen=True) class ReceiptsLine: """A receipts line as one EFO publishes it: £bn by the calendar year in @@ -648,12 +966,15 @@ class ReceiptsGrowth: The first year is outturn growth: this EFO's first (outturn) year over the year before in the previous EFO, where it is outturn. Each later - year is the ratio of this EFO's consecutive levels. + year is the ratio of this EFO's consecutive levels. A line with its own + outturn starts at that outturn's second year and has no previous line; + one with rates grows its base instead (see ``ReceiptsSeriesSpec``). """ values: dict[int, float] line: ReceiptsLine - previous_line: ReceiptsLine + previous_line: ReceiptsLine | None + spec: ReceiptsSeriesSpec | None = None def fiscal_year_label(year: int) -> str: @@ -860,51 +1181,149 @@ def find_receipts_line( def compute_receipts_growth( - xlsx_bytes: bytes, previous_xlsx_bytes: bytes + xlsx_bytes: bytes, + previous_xlsx_bytes: bytes, + specs: tuple[ReceiptsSeriesSpec, ...] = RECEIPTS_SERIES_SPECS, ) -> dict[tuple[str, ...], ReceiptsGrowth]: """Growth in each receipts line, from an EFO's receipts tables and an earlier EFO's, whose outturn is the base for the first year's growth. The first year must be outturn in this EFO, and the year before it - outturn in the earlier one (March 2025 for March 2026). + outturn in the earlier one (March 2025 for March 2026). A line with its + own outturn needs that outturn to reach the year before this EFO's + first instead. """ sheets = read_workbook_sheets(xlsx_bytes) - previous_sheets = read_workbook_sheets(previous_xlsx_bytes) + previous_sheets = None result = {} - for spec in RECEIPTS_SERIES_SPECS: + for spec in specs: name = ".".join(spec.path) line = find_receipts_line(sheets, spec) - previous = find_receipts_line(previous_sheets, spec) first_year = min(line.levels) base_year = first_year - 1 - if first_year not in line.outturn_years: - raise ValueError( - f"{name}: {fiscal_year_label(first_year)}, the first year in " - f"Table {line.table}, is not outturn" - ) - if base_year not in previous.outturn_years: - raise ValueError( - f"{name}: the previous receipts tables have no " - f"{fiscal_year_label(base_year)} outturn; pass the EFO whose " - f"outturn year is {fiscal_year_label(base_year)}" - ) - levels = {base_year: previous.levels[base_year], **line.levels} + if spec.outturn is not None: + previous = None + outturn = { + year: value / 1e6 + for year, value in spec.outturn.receipts_thousands + if year < first_year + } + if base_year not in outturn: + raise ValueError( + f"{name}: {spec.outturn.source} have no " + f"{fiscal_year_label(base_year)}, the year before Table " + f"{line.table}'s first; add them" + ) + levels = {**outturn, **line.levels} + if sorted(levels) != list(range(min(levels), max(levels) + 1)): + raise ValueError(f"{name}: the outturn skips a year: {sorted(levels)}") + else: + if previous_sheets is None: + previous_sheets = read_workbook_sheets(previous_xlsx_bytes) + previous = find_receipts_line(previous_sheets, spec) + if first_year not in line.outturn_years: + raise ValueError( + f"{name}: {fiscal_year_label(first_year)}, the first year in " + f"Table {line.table}, is not outturn" + ) + if base_year not in previous.outturn_years: + raise ValueError( + f"{name}: the previous receipts tables have no " + f"{fiscal_year_label(base_year)} outturn; pass the EFO whose " + f"outturn year is {fiscal_year_label(base_year)}" + ) + levels = {base_year: previous.levels[base_year], **line.levels} + rates = {year: line_rate(spec, year) for year in levels} if spec.rates else {} # Growth to or from a level at or below zero means nothing, and a - # zero last level would zero the series' index from then on. + # zero last level would zero the series' index from then on. A year + # whose rate is zero has no base to grow, whatever its receipts. for year, level in sorted(levels.items()): - if level <= 0: + if level <= 0 and rates.get(year) != 0: raise ValueError( f"{name}: {fiscal_year_label(year)} receipts of {level} are " "not positive, so there is no growth to or from them" ) values = { - year: round(levels[year] / levels[year - 1] - 1, 6) + 0.0 - for year in sorted(line.levels) + year: round(growth_ratio(spec, levels, year) - 1, 6) + 0.0 + for year in sorted(levels)[1:] } - result[spec.path] = ReceiptsGrowth(values, line, previous) + check_index(name, values) + result[spec.path] = ReceiptsGrowth(values, line, previous, spec) return result +def check_index(name: str, values: dict[int, float]) -> None: + """Refuse growth that zeroes the index the model builds from it. + + The model compounds the growth into an index rounded to five decimals + (create_economic_assumption_indices), from 1 in a line's first year when + the line has its own outturn. A positive base that rounds the index to + zero would zero it in every later year. + """ + index = 1.0 + for year, value in values.items(): + index = round(index * (1 + value), 5) + if index <= 0: + raise ValueError( + f"{name}: {fiscal_year_label(year)} growth of {value} takes " + f"the index to {index} at five decimals, so the base is " + "too small to grow from" + ) + + +def population_growth(content: str) -> dict[int, float]: + """``ons.population`` growth by year, as a yoy_growth.yaml holds it.""" + values = yaml.safe_load(content)["ons"]["population"]["values"] + return dict( + sorted( + (int(str(instant)[:4]), float(value)) for instant, value in values.items() + ) + ) + + +def growth_in_effect(series: dict[int, float], year: int) -> float: + """A growth series' value in ``year`` as the model reads it: its latest + entry at or before the year, or its first entry before that, since the + model backdates each parameter's first value.""" + earlier = [entry for entry in series if entry <= year] + return series[max(earlier) if earlier else min(series)] + + +def per_head_growth( + growth: ReceiptsGrowth, population: dict[int, float] +) -> ReceiptsGrowth: + """A series' growth per head, to six decimals: each year's growth before + rounding (``growth_ratio``), over one plus that year's ``ons.population`` + growth, minus one. + + Dataset projection grows each household's input by this and its weight + by the population growth, so the weighted input grows by their product, + the line's growth: a duty's base, for a line with rates. + """ + spec = growth.spec + first = min(growth.line.levels) + if growth.previous_line is None: + earlier = { + year: value / 1e6 + for year, value in spec.outturn.receipts_thousands + if year < first + } + else: + earlier = {first - 1: growth.previous_line.levels[first - 1]} + levels = {**earlier, **growth.line.levels} + values = { + year: round( + growth_ratio(spec, levels, year) / (1 + growth_in_effect(population, year)) + - 1, + 6, + ) + + 0.0 + for year in growth.values + } + check_index(".".join(growth.spec.path), values) + return replace(growth, values=values) + + def format_growth(value: float) -> str: """Fixed-point text with at least four decimals that reads back as the same float: 0.038 is written 0.0380, 1e-05 as 0.00001.""" @@ -974,6 +1393,7 @@ def render_receipts_values( tail: dict[int, float], release: tuple[str, int], previous_release: tuple[str, int], + tail_comment: str = "the average_earnings long-run path above", ) -> list[str]: """A series' values entries: earlier years as they were, then this EFO's growth, then the average_earnings path.""" @@ -994,17 +1414,44 @@ def render_receipts_values( month, year = release previous_month, previous_year = previous_release published, previous = growth.line, growth.previous_line - table = f"Table {published.table}" - if previous.table != published.table: - table += f" ({previous_month} {previous_year}: Table {previous.table})" + efo_first = min(published.levels) pad = " " * indent - lines = kept + [ - f"{pad}# Outturn ({fiscal_year_label(first_year)} over " - f"{fiscal_year_label(first_year - 1)}) from OBR EFO {month} {year} over " - f'{previous_month} {previous_year}, receipts {table} "{published.label}"', - f"{pad}{first_year}-01-01: {format_growth(growth.values[first_year])}", - ] - later = sorted(y for y in growth.values if y > first_year) + first_kind = "Outturn" if efo_first in published.outturn_years else "Forecast" + basis = receipts_basis(growth.spec) + lines = list(kept) + if previous is None: + # The years before the EFO's first, and the base of its first, are + # the line's own outturn. + source = growth.spec.outturn.source + years = sorted(y for y in growth.values if y < efo_first) + if years: + span = ( + f"{fiscal_year_label(years[0])} over {fiscal_year_label(years[0] - 1)}" + if len(years) == 1 + else f"{years[0]}-{years[-1]}" + ) + lines.append(f"{pad}# Outturn ({span}) from {source}{basis}") + lines += [ + f"{pad}{y}-01-01: {format_growth(growth.values[y])}" for y in years + ] + lines.append( + f"{pad}# {first_kind} ({fiscal_year_label(efo_first)} over " + f"{fiscal_year_label(efo_first - 1)}) from OBR EFO {month} {year}, " + f'receipts Table {published.table} "{published.label}", over ' + f"{source}{basis}" + ) + else: + table = f"Table {published.table}" + if previous.table != published.table: + table += f" ({previous_month} {previous_year}: Table {previous.table})" + lines.append( + f"{pad}# {first_kind} ({fiscal_year_label(efo_first)} over " + f"{fiscal_year_label(efo_first - 1)}) from OBR EFO {month} {year} over " + f'{previous_month} {previous_year}, receipts {table} "{published.label}"' + f"{basis}" + ) + lines.append(f"{pad}{efo_first}-01-01: {format_growth(growth.values[efo_first])}") + later = sorted(y for y in growth.values if y > efo_first) for kind in ("Outturn", "Forecast"): years = [ y for y in later if (y in published.outturn_years) == (kind == "Outturn") @@ -1014,16 +1461,99 @@ def render_receipts_values( span = f"{years[0]}" if len(years) == 1 else f"{years[0]}-{years[-1]}" lines.append( f"{pad}# {kind} ({span}) from OBR EFO {month} {year}, receipts " - f"Table {published.table}" + f"Table {published.table}{basis}" ) lines += [f"{pad}{y}-01-01: {format_growth(growth.values[y])}" for y in years] - lines.append( - f"{pad}# Beyond the OBR forecast: the average_earnings long-run path above" - ) + lines.append(f"{pad}# Beyond the OBR forecast: {tail_comment}") lines += [f"{pad}{y}-01-01: {format_growth(v)}" for y, v in sorted(tail.items())] return lines +def receipts_basis(spec: ReceiptsSeriesSpec | None) -> str: + """How a line's growth is measured, for its comments: nothing for + receipts, the rates taken out for a base, and population for a base per + head.""" + if spec is None or not spec.rates: + return "" + if spec.per_head: + return ( + "; per head: base growth (receipts over the rate the model charges) " + "over one plus ons.population growth" + ) + return "; base growth: receipts over the rate the model charges" + + +def per_head_tail( + tail: dict[int, float], comment: str, population: dict[int, float] +) -> tuple[dict[int, float], str]: + """A series' growth after the OBR's forecast per head, and its comment. + + In each year from the tail's first to the last that it or + ``ons.population`` states, (1 + the tail's growth) / (1 + population + growth) - 1, to six decimals, written where the value changes. The model + holds each series' last value, so the weighted total grows by the tail in + every later year too. + """ + first = min(tail) + yearly = { + year: round( + (1 + growth_in_effect(tail, year)) + / (1 + growth_in_effect(population, year)) + - 1, + 6, + ) + + 0.0 + for year in range(first, max(max(tail), max(population), first) + 1) + } + return { + year: value + for year, value in yearly.items() + if year == first or value != yearly[year - 1] + }, f"{comment}; per head: (1 + that growth) / (1 + ons.population growth) - 1" + + +def receipts_tail(content: str, growth: ReceiptsGrowth) -> tuple[dict[int, float], str]: + """A series' growth after the OBR's forecast, and its comment: the + average_earnings path, held at zero in years a base's rate is zero in or + comes from. A base under ``per_capita`` divides that growth by one plus + ``ons.population`` growth (``per_head_tail``).""" + last = max(growth.values) + path = average_earnings_path(content, last) + spec = growth.spec + if spec is None or not spec.rates: + return path, "the average_earnings long-run path above" + # Rates hold their last value after their last change, as the path does. + changes = [ + instant.year + for component in spec.rates + for parameter in component.paths + for instant, _ in parameter_values(parameter) + ] + end = max(max(path), max(changes) + 1, last + 1) + yearly = {} + for year in range(last + 1, end + 1): + if line_rate(spec, year) == 0 or line_rate(spec, year - 1) == 0: + yearly[year] = 0.0 + else: + yearly[year] = path[max(y for y in path if y <= year)] + tail = { + year: value + for year, value in yearly.items() + if year == last + 1 or value != yearly[year - 1] + } + if all(value == 0 for value in yearly.values()): + comment = "held flat, as the rate is zero" + elif any(value == 0 for value in yearly.values()): + comment = ( + "the average_earnings long-run path above, held flat where the rate is zero" + ) + else: + comment = "the average_earnings long-run path above" + if spec.per_head: + return per_head_tail(tail, comment, population_growth(content)) + return tail, comment + + def receipts_references( growth: ReceiptsGrowth, release: tuple[str, int], @@ -1031,22 +1561,48 @@ def receipts_references( ) -> list[dict[str, str]]: month, year = release previous_month, previous_year = previous_release + current = { + "title": f"OBR EFO {month} {year} (detailed forecast tables, receipts, " + f"Table {growth.line.table})", + "href": build_efo_href(month, year), + } + spec = growth.spec + extra = [ + {"title": title, "href": href} + for title, href in (spec.references if spec is not None else ()) + ] + if growth.previous_line is None: + outturn = {"title": spec.outturn.title, "href": spec.outturn.href} + return [current, outturn, *extra] base_year = min(growth.values) - 1 return [ - { - "title": f"OBR EFO {month} {year} (detailed forecast tables, receipts, " - f"Table {growth.line.table})", - "href": build_efo_href(month, year), - }, + current, { "title": f"OBR EFO {previous_month} {previous_year} (detailed forecast " f"tables, receipts, Table {growth.previous_line.table}; " f"{fiscal_year_label(base_year)} outturn)", "href": build_efo_href(previous_month, previous_year), }, + *extra, ] +def check_index_stays_positive(path: tuple[str, ...], values: dict) -> None: + """Compound a rendered series as create_economic_assumption_indices does, + from 1 in its first listed year, rounding to five decimals. + An index that rounds to zero would zero the base in every later year.""" + growth = {str(instant): value for instant, value in values.items()} + index = 1.0 + for year in range(int(min(growth)[:4]) + 1, LAST_INDEX_YEAR + 1): + in_force = max(instant for instant in growth if instant <= f"{year}-01-01") + index = round(index * (1 + growth[in_force]), 5) + if index <= 0: + raise ValueError( + f"{'.'.join(path)}: the index reaches {index} in {year} at five " + "decimals, so the base is too small to grow from" + ) + + def render_receipts_growth_yaml( content: str, growth: dict[tuple[str, ...], ReceiptsGrowth], @@ -1058,14 +1614,20 @@ def render_receipts_growth_yaml( Years before this EFO's first year are kept, so a series' index keeps its base year. Years after the OBR's receipts forecast follow - average_earnings as ``content`` holds it. The result is checked to - differ from ``content`` only in those values and references. + average_earnings as ``content`` holds it. A series under ``per_capita`` + is written per head of ``ons.population`` as ``content`` holds it, in + every year to the last that population or the tail states. The result is + checked to differ from ``content`` only in those values and references. """ before = yaml.safe_load(content) expected = copy.deepcopy(before) lines = content.split("\n") + population = None for path, series in growth.items(): - tail = average_earnings_path(content, max(series.values)) + if series.spec is not None and series.spec.per_head: + population = population or population_growth(content) + series = per_head_growth(series, population) + tail, tail_comment = receipts_tail(content, series) references = receipts_references(series, release, previous_release) start, end = _series_block(lines, ("obr", *path)) @@ -1077,7 +1639,13 @@ def render_receipts_growth_yaml( indent + 2, ) new_values = render_receipts_values( - old_values, entry_indent, series, tail, release, previous_release + old_values, + entry_indent, + series, + tail, + release, + previous_release, + tail_comment, ) lines = [*lines[: values_start + 1], *new_values, *lines[values_end:]] @@ -1112,6 +1680,7 @@ def render_receipts_growth_yaml( for year, value in {**series.values, **tail}.items() }, } + check_index_stays_positive(path, node["values"]) node["metadata"]["reference"] = references rendered = "\n".join(lines) @@ -1320,14 +1889,26 @@ def print_summary( print(f"- {spec.key}: {', '.join(window)}") -def print_receipts_summary(growth: dict[tuple[str, ...], ReceiptsGrowth]) -> None: +def print_receipts_summary( + growth: dict[tuple[str, ...], ReceiptsGrowth], + population: dict[int, float] | None = None, +) -> None: + """Print each series' growth: a series under ``per_capita`` per head when + ``population`` is given, as it is written.""" print(f"Updating {len(growth)} OBR receipts growth series") for path, series in growth.items(): + basis = "" + if series.spec is not None and series.spec.per_head: + if population is None: + basis = ", aggregate" + else: + series = per_head_growth(series, population) + basis = ", per head" window = ", ".join( f"{year}: {format_growth(value)}" for year, value in sorted(series.values.items()) ) - print(f"- {'.'.join(path)} (Table {series.line.table}): {window}") + print(f"- {'.'.join(path)} (Table {series.line.table}{basis}): {window}") def build_arg_parser() -> argparse.ArgumentParser: @@ -1392,7 +1973,9 @@ def build_arg_parser() -> argparse.ArgumentParser: action="store_true", help=( "Do not update the receipts growth series (stamp_duty_land_tax, " - "business_rates) that project corporate SDLT and business rates" + "business_rates) that project corporate SDLT and business rates, " + "or the gambling_duty_bases series that project the gambling " + "duty inputs" ), ) parser.add_argument( @@ -1581,7 +2164,9 @@ def main(argv: list[str] | None = None) -> int: f"an EFO before {month} {year}" ) receipts_growth = compute_receipts_growth(receipts_bytes, previous_bytes) - print_receipts_summary(receipts_growth) + print_receipts_summary( + receipts_growth, population_growth(source_yaml_path.read_text()) + ) calendar_values = ( read_calendar_values(source_yaml_path) if args.gaps_only else series_values ) diff --git a/policyengine_uk/variables/input/consumption/gambling/betting_exchange_commission.py b/policyengine_uk/variables/input/consumption/gambling/betting_exchange_commission.py index 453f5c85b9..3935e900d4 100644 --- a/policyengine_uk/variables/input/consumption/gambling/betting_exchange_commission.py +++ b/policyengine_uk/variables/input/consumption/gambling/betting_exchange_commission.py @@ -7,5 +7,6 @@ class betting_exchange_commission(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Chargeable betting exchange commission attributed to the household" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/141" diff --git a/policyengine_uk/variables/input/consumption/gambling/bingo_net_receipts.py b/policyengine_uk/variables/input/consumption/gambling/bingo_net_receipts.py index ad67462e6d..9bd6ed1f4a 100644 --- a/policyengine_uk/variables/input/consumption/gambling/bingo_net_receipts.py +++ b/policyengine_uk/variables/input/consumption/gambling/bingo_net_receipts.py @@ -7,5 +7,8 @@ class bingo_net_receipts(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = ( + "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.bingo" + ) label = "Attributed dutiable bingo promotion profits" reference = "https://www.legislation.gov.uk/ukpga/1981/63/section/17/2024-04-06" diff --git a/policyengine_uk/variables/input/consumption/gambling/financial_spread_betting_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/financial_spread_betting_net_losses.py index 8aefb4c920..6ef38f34ab 100644 --- a/policyengine_uk/variables/input/consumption/gambling/financial_spread_betting_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/financial_spread_betting_net_losses.py @@ -7,5 +7,6 @@ class financial_spread_betting_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Attributed dutiable financial spread betting losses" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/128" diff --git a/policyengine_uk/variables/input/consumption/gambling/horse_and_dog_pool_betting_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/horse_and_dog_pool_betting_net_losses.py index f628b80280..ff2b49b094 100644 --- a/policyengine_uk/variables/input/consumption/gambling/horse_and_dog_pool_betting_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/horse_and_dog_pool_betting_net_losses.py @@ -7,5 +7,6 @@ class horse_and_dog_pool_betting_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Attributed dutiable horse and dog racing pool losses" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/134" diff --git a/policyengine_uk/variables/input/consumption/gambling/land_based_casino_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/land_based_casino_net_losses.py index bb19e35e74..5a493d52df 100644 --- a/policyengine_uk/variables/input/consumption/gambling/land_based_casino_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/land_based_casino_net_losses.py @@ -7,5 +7,8 @@ class land_based_casino_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = ( + "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.gaming" + ) label = "Attributed dutiable land-based casino gaming yield" reference = "https://www.legislation.gov.uk/ukpga/1997/16/section/11" diff --git a/policyengine_uk/variables/input/consumption/gambling/land_based_fixed_odds_betting_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/land_based_fixed_odds_betting_net_losses.py index 0156162579..975db86a41 100644 --- a/policyengine_uk/variables/input/consumption/gambling/land_based_fixed_odds_betting_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/land_based_fixed_odds_betting_net_losses.py @@ -7,5 +7,6 @@ class land_based_fixed_odds_betting_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Attributed dutiable land-based fixed-odds losses" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/126" diff --git a/policyengine_uk/variables/input/consumption/gambling/machine_games_higher_rate_net_takings.py b/policyengine_uk/variables/input/consumption/gambling/machine_games_higher_rate_net_takings.py index 5a80acaa9c..2602ce0e38 100644 --- a/policyengine_uk/variables/input/consumption/gambling/machine_games_higher_rate_net_takings.py +++ b/policyengine_uk/variables/input/consumption/gambling/machine_games_higher_rate_net_takings.py @@ -7,5 +7,6 @@ class machine_games_higher_rate_net_takings(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.machine_games" label = "Attributed dutiable higher-rate machine games net takings" reference = "https://www.gov.uk/guidance/machine-games-duty-excise-notice-452" diff --git a/policyengine_uk/variables/input/consumption/gambling/machine_games_lower_rate_net_takings.py b/policyengine_uk/variables/input/consumption/gambling/machine_games_lower_rate_net_takings.py index f746d877d6..2b6aa332b4 100644 --- a/policyengine_uk/variables/input/consumption/gambling/machine_games_lower_rate_net_takings.py +++ b/policyengine_uk/variables/input/consumption/gambling/machine_games_lower_rate_net_takings.py @@ -7,5 +7,6 @@ class machine_games_lower_rate_net_takings(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.machine_games" label = "Attributed dutiable lower-rate machine games net takings" reference = "https://www.gov.uk/guidance/machine-games-duty-excise-notice-452" diff --git a/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_higher_stake_net_takings.py b/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_higher_stake_net_takings.py index c4121a45b6..b5676691d6 100644 --- a/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_higher_stake_net_takings.py +++ b/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_higher_stake_net_takings.py @@ -7,6 +7,7 @@ class machine_games_standard_rate_higher_stake_net_takings(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.machine_games" label = "Attributed higher-stake part of standard-rate machine games net takings" documentation = ( "The part of machine_games_standard_rate_net_takings that comes from " diff --git a/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_net_takings.py b/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_net_takings.py index c4acb3d525..21798450d9 100644 --- a/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_net_takings.py +++ b/policyengine_uk/variables/input/consumption/gambling/machine_games_standard_rate_net_takings.py @@ -7,5 +7,6 @@ class machine_games_standard_rate_net_takings(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.machine_games" label = "Attributed dutiable standard-rate machine games net takings" reference = "https://www.gov.uk/guidance/machine-games-duty-excise-notice-452" diff --git a/policyengine_uk/variables/input/consumption/gambling/national_lottery_ticket_spending.py b/policyengine_uk/variables/input/consumption/gambling/national_lottery_ticket_spending.py index e493e2bbbf..6c8fc049b8 100644 --- a/policyengine_uk/variables/input/consumption/gambling/national_lottery_ticket_spending.py +++ b/policyengine_uk/variables/input/consumption/gambling/national_lottery_ticket_spending.py @@ -7,5 +7,8 @@ class national_lottery_ticket_spending(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = ( + "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.lottery" + ) label = "Chargeable National Lottery ticket consideration" reference = "https://www.legislation.gov.uk/ukpga/1993/34/section/25" diff --git a/policyengine_uk/variables/input/consumption/gambling/other_spread_betting_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/other_spread_betting_net_losses.py index 075809c3b2..c23e31b5f1 100644 --- a/policyengine_uk/variables/input/consumption/gambling/other_spread_betting_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/other_spread_betting_net_losses.py @@ -7,5 +7,6 @@ class other_spread_betting_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Attributed dutiable other spread betting losses" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/130" diff --git a/policyengine_uk/variables/input/consumption/gambling/pool_betting_net_receipts.py b/policyengine_uk/variables/input/consumption/gambling/pool_betting_net_receipts.py index d3f88029c1..cb8293872a 100644 --- a/policyengine_uk/variables/input/consumption/gambling/pool_betting_net_receipts.py +++ b/policyengine_uk/variables/input/consumption/gambling/pool_betting_net_receipts.py @@ -7,5 +7,6 @@ class pool_betting_net_receipts(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.pool_betting" label = "Attributed dutiable non-racing pool betting net receipts" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/143" diff --git a/policyengine_uk/variables/input/consumption/gambling/remote_gaming_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/remote_gaming_net_losses.py index ecaae21921..b170dfd7a7 100644 --- a/policyengine_uk/variables/input/consumption/gambling/remote_gaming_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/remote_gaming_net_losses.py @@ -7,5 +7,6 @@ class remote_gaming_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.remote_gaming" label = "Attributed chargeable non-exempt remote gaming losses" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/161" diff --git a/policyengine_uk/variables/input/consumption/gambling/remote_other_betting_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/remote_other_betting_net_losses.py index c2ee45223c..251d9c507f 100644 --- a/policyengine_uk/variables/input/consumption/gambling/remote_other_betting_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/remote_other_betting_net_losses.py @@ -7,5 +7,6 @@ class remote_other_betting_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Attributed dutiable other remote fixed-odds losses" reference = "https://www.legislation.gov.uk/ukpga/2026/11/section/87" diff --git a/policyengine_uk/variables/input/consumption/gambling/remote_uk_horseracing_betting_net_losses.py b/policyengine_uk/variables/input/consumption/gambling/remote_uk_horseracing_betting_net_losses.py index 367b3e80e3..d508614d96 100644 --- a/policyengine_uk/variables/input/consumption/gambling/remote_uk_horseracing_betting_net_losses.py +++ b/policyengine_uk/variables/input/consumption/gambling/remote_uk_horseracing_betting_net_losses.py @@ -7,5 +7,6 @@ class remote_uk_horseracing_betting_net_losses(Variable): definition_period = YEAR default_value = 0 unit = GBP + uprating = "gov.economic_assumptions.indices.obr.per_capita.gambling_duty_bases.general_betting" label = "Attributed dutiable remote UK horseracing fixed-odds losses" reference = "https://www.legislation.gov.uk/ukpga/2014/26/section/127"