From 98ab77093b0f48e4cb50677c2c9403a5533631f9 Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Thu, 8 Oct 2026 11:45:32 +0100 Subject: [PATCH 1/3] Split VAT grossing into household share and survey coverage (#1996) Replace the undated gov.simulation.microdata_vat_coverage (0.38, added in January 2023 with no derivation) with two documented factors: gov.simulation.vat.household_share_of_receipts (0.7, HMRC's share of VAT liabilities on household consumption) and gov.simulation.vat.survey_consumption_coverage (0.67, derived on the certified Microcosm UK 2024-25 release against OBR's 2024-25 VAT outturn and cross-checked with HMRC's ready reckoner). Document the derivation and rescale the 2pp VAT reform fixture. Co-Authored-By: Claude Opus 5.5 --- changelog.d/1996.changed.md | 1 + docs/engineering/vat-coverage.md | 80 +++++++++++++++++++ myst.yml | 1 + .../simulation/microdata_vat_coverage.yaml | 9 --- .../vat/household_share_of_receipts.yaml | 12 +++ .../vat/survey_consumption_coverage.yaml | 24 ++++++ .../tests/microsimulation/reforms_config.yaml | 2 +- .../baseline/gov/hmrc/vat/vat_grossing.yaml | 32 ++++++++ .../variables/gov/hmrc/baseline_vat.py | 8 +- policyengine_uk/variables/gov/hmrc/vat.py | 8 +- 10 files changed, 165 insertions(+), 12 deletions(-) create mode 100644 changelog.d/1996.changed.md create mode 100644 docs/engineering/vat-coverage.md delete mode 100644 policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml create mode 100644 policyengine_uk/parameters/gov/simulation/vat/household_share_of_receipts.yaml create mode 100644 policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml create mode 100644 policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml diff --git a/changelog.d/1996.changed.md b/changelog.d/1996.changed.md new file mode 100644 index 0000000000..cfcd90d952 --- /dev/null +++ b/changelog.d/1996.changed.md @@ -0,0 +1 @@ +Replaced the undated `gov.simulation.microdata_vat_coverage` (0.38) with two documented factors: `gov.simulation.vat.household_share_of_receipts` (0.7, HMRC's share of VAT liabilities on household consumption) and `gov.simulation.vat.survey_consumption_coverage` (0.67, derived on Microcosm UK 2024-25 against OBR VAT receipts). Total VAT on that dataset now matches 2024-25 receipts instead of overshooting by 24%. diff --git a/docs/engineering/vat-coverage.md b/docs/engineering/vat-coverage.md new file mode 100644 index 0000000000..691e285934 --- /dev/null +++ b/docs/engineering/vat-coverage.md @@ -0,0 +1,80 @@ +# VAT grossing to receipts + +`vat` and `baseline_vat` compute the statutory liability on each household's +recorded consumption (the standard rate on `full_rate_vat_consumption` plus +the reduced rate on `reduced_rate_vat_consumption`) and divide it by two +factors, so that total VAT matches receipts: + +- `gov.simulation.vat.household_share_of_receipts` (0.7): the share of VAT + liabilities that falls on household final consumption. HMRC's VAT gap + estimates put household consumption at around 70% of the VAT total + theoretical liability; the rest falls on government, charities, VAT-exempt + businesses and new housing. This is a published quantity and does not depend + on the dataset. +- `gov.simulation.vat.survey_consumption_coverage` (0.67): the share of + household VAT liability that the dataset's consumption captures. This + depends on the dataset's consumption imputation and must be re-derived when + the default dataset changes. + +The incidence assumption is that every pound of receipts, including VAT that +government, charities and exempt businesses pay on their inputs, is passed on +to households in proportion to their own VAT-liable spending. + +These two factors replace `gov.simulation.microdata_vat_coverage`, a single +undated 0.38 added in January 2023 against an older LCFS-based consumption +imputation, with no derivation recorded. Its product with today's datasets put +2024-25 VAT 24% above OBR receipts on Microcosm UK 2024-25 and 71% above on +Enhanced FRS 2024-25. + +## Derivation (October 2026) + +The product of the two factors is the dataset's pre-scaling VAT divided by +receipts for the same year. The survey coverage is that product divided by the +household share. + +| Dataset | Year | Pre-scaling VAT | OBR VAT receipts | Product | Survey coverage | +|---|---|---|---|---|---| +| Microcosm UK 2024-25 (populace-uk-private `f9d1922c`) | 2024-25 | £80.5bn | £171.0bn (outturn) | 0.471 | 0.672 | +| Microcosm UK 2024-25 | 2025-26 | £83.8bn | £180.2bn | 0.465 | 0.665 | +| Microcosm UK 2024-25 | 2026-27 | £86.1bn | £187.7bn | 0.459 | 0.655 | +| Enhanced FRS 2024-25 | 2024-25 | £111.0bn | £171.0bn (outturn) | 0.649 | 0.927 | + +Receipts are the cash-basis VAT line of OBR March 2026 EFO detailed forecast +table 3.8. Model year 2024 is the 2024-25 fiscal year. + +HMRC's ready reckoner gives an independent check that does not use the level +of receipts: one percentage point on the standard rate raises £8.8bn in +2026-27 (Direct effects of illustrative tax changes, June 2025). On Microcosm +UK 2024-25, one point on the standard rate raises £4.24bn before scaling, so +matching HMRC needs a product of 0.48, within 3% of the level-based 0.469. + +The value is set against the certified Microcosm UK 2024-25 national release. +On Enhanced FRS 2024-25 it puts 2024-25 VAT about 38% above receipts (it was +71% above with 0.38). The implied product moves by less than 3% between +2024-25 and 2026-27, so the factors are not dated by year. + +## Reproducing + +```python +from policyengine_uk import Microsimulation + +sim = Microsimulation(dataset="microcosm_uk_2024_25.h5") +year = 2024 +p = sim.tax_benefit_system.parameters(f"{year}-01-01").gov.hmrc.vat +raw = ( + sim.calculate("full_rate_vat_consumption", year).sum() * p.standard_rate + + sim.calculate("reduced_rate_vat_consumption", year).sum() * p.reduced_rate +) +obr_receipts = 170.994e9 # OBR March 2026 EFO, table 3.8, 2024-25 +product = raw / obr_receipts +print(product, product / 0.7) +``` + +## Interaction with an explicit energy VAT base + +A domestic fuel and power VAT base built from `electricity_consumption` and +`gas_consumption` (PolicyEngine/policyengine-uk#2166) changes what the survey +coverage applies to. If those energy inputs are already calibrated to an +aggregate, they should not be divided by the survey coverage again, and the +reconciled energy part of `reduced_rate_vat_consumption` should come out of the +base used to derive it. diff --git a/myst.yml b/myst.yml index b5b3053685..54e94000bb 100644 --- a/myst.yml +++ b/myst.yml @@ -33,6 +33,7 @@ project: children: - file: docs/engineering/excise-duties.md - file: docs/engineering/bus-fares.md + - file: docs/engineering/vat-coverage.md - pattern: docs/book/policy/*.md - title: Programmes children: diff --git a/policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml b/policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml deleted file mode 100644 index f82936e097..0000000000 --- a/policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml +++ /dev/null @@ -1,9 +0,0 @@ -description: LCFS (from which ETB data is derived) microdata is known to under-report household consumption. We scale up the VAT liability to hit HMRC-reported VAT receipts (following the approach of IFS' TAXBEN, though they might have better coverage anyway from access to the non-EUL dataset). For HMRC statistics see https://www.gov.uk/government/statistics/value-added-tax-vat-annual-statistics. -metadata: - unit: /1 - period: year - reference: - - title: Value Added Tax (VAT) annual statistics - href: https://www.gov.uk/government/statistics/value-added-tax-vat-annual-statistics -values: - 2010-01-01: 0.38 \ No newline at end of file diff --git a/policyengine_uk/parameters/gov/simulation/vat/household_share_of_receipts.yaml b/policyengine_uk/parameters/gov/simulation/vat/household_share_of_receipts.yaml new file mode 100644 index 0000000000..bdfc270c35 --- /dev/null +++ b/policyengine_uk/parameters/gov/simulation/vat/household_share_of_receipts.yaml @@ -0,0 +1,12 @@ +description: Share of VAT liabilities that falls on household final consumption. The rest falls on government, charities, VAT-exempt businesses and new housing. Household VAT is grossed up by this share so that total VAT matches receipts. +metadata: + unit: /1 + period: year + economy: false + reference: + - title: HMRC, Second estimate of the VAT gap (tax year 2024 to 2025), 3 March 2026 (household consumption is around 70% of the VAT total theoretical liability) + href: https://www.gov.uk/government/statistics/vat-gap-estimates/second-estimate-of-the-vat-gap-tax-year-2023-to-2024 + - title: HMRC, Measuring tax gaps methodological annex, VAT (components of the VAT total theoretical liability) + href: https://www.gov.uk/government/statistics/measuring-tax-gaps-methodological-annex/2-tax-gaps-value-added-tax +values: + 2010-01-01: 0.7 diff --git a/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml b/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml new file mode 100644 index 0000000000..afdce1f8e6 --- /dev/null +++ b/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml @@ -0,0 +1,24 @@ +description: Share of household VAT liability captured by consumption in the microdata. Household VAT computed from survey consumption is divided by this share and by the household share of VAT liabilities to match receipts. +metadata: + unit: /1 + period: year + economy: false + derivation: >- + Pre-scaling VAT (standard rate times full_rate_vat_consumption plus reduced + rate times reduced_rate_vat_consumption) on the certified Microcosm UK + 2024-25 national release (policyengine/populace-uk-private, revision + f9d1922c) is £80.5bn in 2024. OBR VAT receipts for 2024-25 (outturn, March + 2026 EFO table 3.8) are £171.0bn. 80.5 / 171.0 = 0.471 for the product of + the two factors; divided by the household share of 0.7 gives 0.67. The + HMRC ready reckoner gives a consistent level: £8.8bn for one percentage + point on the standard rate in 2026-27 implies a product of 0.48 on the same + data. On Enhanced FRS 2024-25 the same calculation gives 0.65, so this + value overstates VAT on that dataset by about 38%. Re-derive when the + default dataset changes, with docs/engineering/vat-coverage.md. + reference: + - title: OBR, Economic and fiscal outlook March 2026, detailed forecast tables, receipts, table 3.8 + href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ + - title: HMRC, Direct effects of illustrative tax changes bulletin (June 2025) + href: https://www.gov.uk/government/statistics/direct-effects-of-illustrative-tax-changes/direct-effects-of-illustrative-tax-changes-bulletin-january-2025 +values: + 2010-01-01: 0.67 diff --git a/policyengine_uk/tests/microsimulation/reforms_config.yaml b/policyengine_uk/tests/microsimulation/reforms_config.yaml index 57bcc7d2ef..b13ec23de1 100644 --- a/policyengine_uk/tests/microsimulation/reforms_config.yaml +++ b/policyengine_uk/tests/microsimulation/reforms_config.yaml @@ -29,7 +29,7 @@ reforms: parameters: gov.hmrc.national_insurance.class_1.rates.employee.main: 0.1 - name: Raise VAT standard rate by 2pp - expected_impact: 21.3 + expected_impact: 17.3 tolerance: 1.5 parameters: gov.hmrc.vat.standard_rate: 0.22 diff --git a/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml b/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml new file mode 100644 index 0000000000..92c791c24d --- /dev/null +++ b/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml @@ -0,0 +1,32 @@ +- name: VAT is grossed up by survey coverage times the household share of liabilities + period: 2025 + absolute_error_margin: 0.01 + input: + full_rate_vat_consumption: 10_000 + reduced_rate_vat_consumption: 1_000 + output: + # (10,000 x 20% + 1,000 x 5%) / (0.67 x 0.7) = 2,050 / 0.469 + vat: 4_371.00 + baseline_vat: 4_371.00 + +- name: With both factors at one, VAT is the statutory liability on recorded consumption + period: 2025 + absolute_error_margin: 0.01 + input: + gov.simulation.vat.survey_consumption_coverage: 1 + gov.simulation.vat.household_share_of_receipts: 1 + full_rate_vat_consumption: 10_000 + reduced_rate_vat_consumption: 1_000 + output: + vat: 2_050 + +- name: The household share alone grosses survey-complete VAT up to receipts + period: 2025 + absolute_error_margin: 0.01 + input: + gov.simulation.vat.survey_consumption_coverage: 1 + full_rate_vat_consumption: 7_000 + reduced_rate_vat_consumption: 0 + output: + # 7,000 x 20% / 0.7 + vat: 2_000 diff --git a/policyengine_uk/variables/gov/hmrc/baseline_vat.py b/policyengine_uk/variables/gov/hmrc/baseline_vat.py index 4343701b56..2ada621794 100644 --- a/policyengine_uk/variables/gov/hmrc/baseline_vat.py +++ b/policyengine_uk/variables/gov/hmrc/baseline_vat.py @@ -16,4 +16,10 @@ def formula(household, period, parameters): full_rate_consumption * p.hmrc.vat.standard_rate + reduced_rate_consumption * p.hmrc.vat.reduced_rate ) - return raw_vat / p.simulation.microdata_vat_coverage + # Survey consumption under-records spending, and households bear only + # part of VAT liabilities, so household VAT is grossed up to receipts. + coverage = ( + p.simulation.vat.survey_consumption_coverage + * p.simulation.vat.household_share_of_receipts + ) + return raw_vat / coverage diff --git a/policyengine_uk/variables/gov/hmrc/vat.py b/policyengine_uk/variables/gov/hmrc/vat.py index e8ec1a5897..08410eef36 100644 --- a/policyengine_uk/variables/gov/hmrc/vat.py +++ b/policyengine_uk/variables/gov/hmrc/vat.py @@ -16,4 +16,10 @@ def formula(household, period, parameters): full_rate_consumption * p.hmrc.vat.standard_rate + reduced_rate_consumption * p.hmrc.vat.reduced_rate ) - return raw_vat / p.simulation.microdata_vat_coverage + # Survey consumption under-records spending, and households bear only + # part of VAT liabilities, so household VAT is grossed up to receipts. + coverage = ( + p.simulation.vat.survey_consumption_coverage + * p.simulation.vat.household_share_of_receipts + ) + return raw_vat / coverage From 691267f5cce6dc755fc51649ec6b5c8421ecc35d Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Thu, 8 Oct 2026 13:12:31 +0100 Subject: [PATCH 2/3] Keep microdata_vat_coverage as a deprecated combined factor Saved reforms that set gov.simulation.microdata_vat_coverage keep working: the parameter is restored with the product of the two new factors (0.469), and vat and baseline_vat use it in place of the two factors whenever it is moved away from that declared value (policyengine_uk/utils/vat.py). The check compares against the declared default rather than the baseline tree, so it also holds for Scenario parameter changes. Co-Authored-By: Claude Opus 5.5 --- changelog.d/1996.changed.md | 2 +- docs/engineering/vat-coverage.md | 7 +++ .../simulation/microdata_vat_coverage.yaml | 11 ++++ .../baseline/gov/hmrc/vat/vat_grossing.yaml | 11 ++++ .../tests/test_vat_grossing_legacy.py | 56 +++++++++++++++++++ policyengine_uk/utils/vat.py | 34 +++++++++++ .../variables/gov/hmrc/baseline_vat.py | 6 +- policyengine_uk/variables/gov/hmrc/vat.py | 6 +- 8 files changed, 124 insertions(+), 9 deletions(-) create mode 100644 policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml create mode 100644 policyengine_uk/tests/test_vat_grossing_legacy.py create mode 100644 policyengine_uk/utils/vat.py diff --git a/changelog.d/1996.changed.md b/changelog.d/1996.changed.md index cfcd90d952..de1a3bc2e5 100644 --- a/changelog.d/1996.changed.md +++ b/changelog.d/1996.changed.md @@ -1 +1 @@ -Replaced the undated `gov.simulation.microdata_vat_coverage` (0.38) with two documented factors: `gov.simulation.vat.household_share_of_receipts` (0.7, HMRC's share of VAT liabilities on household consumption) and `gov.simulation.vat.survey_consumption_coverage` (0.67, derived on Microcosm UK 2024-25 against OBR VAT receipts). Total VAT on that dataset now matches 2024-25 receipts instead of overshooting by 24%. +Split the undated VAT grossing factor `gov.simulation.microdata_vat_coverage` (0.38) into two documented factors: `gov.simulation.vat.household_share_of_receipts` (0.7, HMRC's share of VAT liabilities on household consumption) and `gov.simulation.vat.survey_consumption_coverage` (0.67, derived on Microcosm UK 2024-25 against OBR VAT receipts). Total VAT on that dataset now matches 2024-25 receipts instead of overshooting by 24%. `microdata_vat_coverage` is kept as a deprecated parameter equal to their product (0.469); a reform that sets it to another value still takes effect. diff --git a/docs/engineering/vat-coverage.md b/docs/engineering/vat-coverage.md index 691e285934..0cc8fa7f41 100644 --- a/docs/engineering/vat-coverage.md +++ b/docs/engineering/vat-coverage.md @@ -26,6 +26,13 @@ imputation, with no derivation recorded. Its product with today's datasets put 2024-25 VAT 24% above OBR receipts on Microcosm UK 2024-25 and 71% above on Enhanced FRS 2024-25. +`gov.simulation.microdata_vat_coverage` is kept as a deprecated parameter so +that existing reforms to it still work. Its value is now the product of the two +factors (0.469). If a reform sets it to any other value, `vat` and +`baseline_vat` use that value in place of the two factors +(`policyengine_uk/utils/vat.py`); otherwise they use the two factors. New +reforms should change the two factors instead. + ## Derivation (October 2026) The product of the two factors is the dataset's pre-scaling VAT divided by diff --git a/policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml b/policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml new file mode 100644 index 0000000000..54e8d87667 --- /dev/null +++ b/policyengine_uk/parameters/gov/simulation/microdata_vat_coverage.yaml @@ -0,0 +1,11 @@ +description: Deprecated. Combined VAT grossing factor, kept so that existing reforms to this path still work. Its value is gov.simulation.vat.survey_consumption_coverage times gov.simulation.vat.household_share_of_receipts (0.67 x 0.7). If a reform sets it to a different value, vat uses it in place of the two factors; otherwise vat uses the two factors. New reforms should change the two factors instead. +metadata: + unit: /1 + period: year + economy: false + deprecated: true + reference: + - title: docs/engineering/vat-coverage.md (derivation of the two factors this combines) + href: https://github.com/PolicyEngine/policyengine-uk/blob/main/docs/engineering/vat-coverage.md +values: + 2010-01-01: 0.469 diff --git a/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml b/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml index 92c791c24d..934f838e84 100644 --- a/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml +++ b/policyengine_uk/tests/policy/baseline/gov/hmrc/vat/vat_grossing.yaml @@ -30,3 +30,14 @@ output: # 7,000 x 20% / 0.7 vat: 2_000 + +- name: A reform to the deprecated combined factor still takes effect + period: 2025 + absolute_error_margin: 0.01 + input: + gov.simulation.microdata_vat_coverage: 0.5 + full_rate_vat_consumption: 10_000 + reduced_rate_vat_consumption: 1_000 + output: + # 2,050 / 0.5 + vat: 4_100 diff --git a/policyengine_uk/tests/test_vat_grossing_legacy.py b/policyengine_uk/tests/test_vat_grossing_legacy.py new file mode 100644 index 0000000000..32c98a28c7 --- /dev/null +++ b/policyengine_uk/tests/test_vat_grossing_legacy.py @@ -0,0 +1,56 @@ +import pytest +from policyengine_core.reforms import Reform + +from policyengine_uk import Simulation +from policyengine_uk.model_api import Scenario +from policyengine_uk.system import system +from policyengine_uk.utils.vat import LEGACY_COVERAGE_DEFAULT + +LEGACY = "gov.simulation.microdata_vat_coverage" +SITUATION = { + "people": {"adult": {"age": {2025: 40}}}, + "benunits": {"benunit": {"members": ["adult"]}}, + "households": { + "household": { + "members": ["adult"], + "full_rate_vat_consumption": {2025: 10_000}, + "reduced_rate_vat_consumption": {2025: 1_000}, + } + }, +} +RAW_VAT = 10_000 * 0.2 + 1_000 * 0.05 + + +def test_legacy_default_is_the_product_of_the_two_factors(): + p = system.parameters.gov.simulation + for year in range(2015, 2031): + product = p.vat.survey_consumption_coverage( + year + ) * p.vat.household_share_of_receipts(year) + assert p.microdata_vat_coverage(year) == pytest.approx(product) + assert LEGACY_COVERAGE_DEFAULT == pytest.approx(0.469) + + +def test_reform_dict_to_legacy_path_changes_vat(): + reform = Reform.from_dict({LEGACY: {"2010-01-01.2100-12-31": 0.5}}, "uk") + sim = Simulation(situation=SITUATION, reform=reform) + assert sim.calculate("vat", 2025)[0] == pytest.approx(RAW_VAT / 0.5) + + +def test_scenario_to_legacy_path_changes_vat(): + scenario = Scenario(parameter_changes={LEGACY: {"year:2025:1": 0.5}}) + sim = Simulation(situation=SITUATION, scenario=scenario) + assert sim.calculate("vat", 2025)[0] == pytest.approx(RAW_VAT / 0.5) + + +def test_untouched_legacy_path_uses_the_two_factors(): + reform = Reform.from_dict( + { + "gov.simulation.vat.survey_consumption_coverage": { + "2010-01-01.2100-12-31": 1 + } + }, + "uk", + ) + sim = Simulation(situation=SITUATION, reform=reform) + assert sim.calculate("vat", 2025)[0] == pytest.approx(RAW_VAT / 0.7) diff --git a/policyengine_uk/utils/vat.py b/policyengine_uk/utils/vat.py new file mode 100644 index 0000000000..fdf6a95329 --- /dev/null +++ b/policyengine_uk/utils/vat.py @@ -0,0 +1,34 @@ +from pathlib import Path + +import numpy as np +import yaml + +_LEGACY_PATH = ( + Path(__file__).parents[1] + / "parameters" + / "gov" + / "simulation" + / "microdata_vat_coverage.yaml" +) +# The declared value of the deprecated combined factor. A reform that moves the +# parameter away from it is honoured; otherwise the two factors are used. +LEGACY_COVERAGE_DEFAULT = float( + list(yaml.safe_load(_LEGACY_PATH.read_text())["values"].values())[-1] +) + + +def vat_grossing_factor(simulation_parameters): + """Factor that household VAT on recorded consumption is divided by. + + The product of survey consumption coverage and the household share of VAT + liabilities, unless a reform has set the deprecated combined parameter + `microdata_vat_coverage` to another value, in which case that value is used + so that existing reforms keep working. + """ + legacy = simulation_parameters.microdata_vat_coverage + if not np.isclose(legacy, LEGACY_COVERAGE_DEFAULT): + return legacy + return ( + simulation_parameters.vat.survey_consumption_coverage + * simulation_parameters.vat.household_share_of_receipts + ) diff --git a/policyengine_uk/variables/gov/hmrc/baseline_vat.py b/policyengine_uk/variables/gov/hmrc/baseline_vat.py index 2ada621794..4c6f4589b4 100644 --- a/policyengine_uk/variables/gov/hmrc/baseline_vat.py +++ b/policyengine_uk/variables/gov/hmrc/baseline_vat.py @@ -1,4 +1,5 @@ from policyengine_uk.model_api import * +from policyengine_uk.utils.vat import vat_grossing_factor class baseline_vat(Variable): @@ -18,8 +19,5 @@ def formula(household, period, parameters): ) # Survey consumption under-records spending, and households bear only # part of VAT liabilities, so household VAT is grossed up to receipts. - coverage = ( - p.simulation.vat.survey_consumption_coverage - * p.simulation.vat.household_share_of_receipts - ) + coverage = vat_grossing_factor(p.simulation) return raw_vat / coverage diff --git a/policyengine_uk/variables/gov/hmrc/vat.py b/policyengine_uk/variables/gov/hmrc/vat.py index 08410eef36..70012df9c5 100644 --- a/policyengine_uk/variables/gov/hmrc/vat.py +++ b/policyengine_uk/variables/gov/hmrc/vat.py @@ -1,4 +1,5 @@ from policyengine_uk.model_api import * +from policyengine_uk.utils.vat import vat_grossing_factor class vat(Variable): @@ -18,8 +19,5 @@ def formula(household, period, parameters): ) # Survey consumption under-records spending, and households bear only # part of VAT liabilities, so household VAT is grossed up to receipts. - coverage = ( - p.simulation.vat.survey_consumption_coverage - * p.simulation.vat.household_share_of_receipts - ) + coverage = vat_grossing_factor(p.simulation) return raw_vat / coverage From 4054bf17cc65a8c6d666164e245da7b468ac4405 Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Mon, 12 Oct 2026 14:49:17 +0200 Subject: [PATCH 3/3] Document the survey coverage derivation so it can be reproduced or bounded List the dataset revisions and SHA-256 hashes, the OBR and HMRC inputs, the formula and the per-year results behind 0.67, state that the factor is calibrated to Microcosm UK 2024-25 only, and bound it on that release (product 0.459 to 0.481, coverage 0.655 to 0.688). Label the Enhanced FRS figures by release: the earlier 0.927 was policyengine-uk-data 1.56.16; on 1.58.0 the implied coverage is 0.967 and 0.67 puts 2024-25 VAT 44% above receipts. Co-Authored-By: Claude Opus 5.5 --- docs/engineering/vat-coverage.md | 131 +++++++++++++----- .../vat/survey_consumption_coverage.yaml | 27 ++-- 2 files changed, 116 insertions(+), 42 deletions(-) diff --git a/docs/engineering/vat-coverage.md b/docs/engineering/vat-coverage.md index 0cc8fa7f41..49c467898a 100644 --- a/docs/engineering/vat-coverage.md +++ b/docs/engineering/vat-coverage.md @@ -24,7 +24,7 @@ These two factors replace `gov.simulation.microdata_vat_coverage`, a single undated 0.38 added in January 2023 against an older LCFS-based consumption imputation, with no derivation recorded. Its product with today's datasets put 2024-25 VAT 24% above OBR receipts on Microcosm UK 2024-25 and 71% above on -Enhanced FRS 2024-25. +Enhanced FRS 2024-25 (policyengine-uk-data 1.56.16). `gov.simulation.microdata_vat_coverage` is kept as a deprecated parameter so that existing reforms to it still work. Its value is now the product of the two @@ -33,50 +33,119 @@ factors (0.469). If a reform sets it to any other value, `vat` and (`policyengine_uk/utils/vat.py`); otherwise they use the two factors. New reforms should change the two factors instead. +## Which datasets the factor applies to + +The survey coverage of 0.67 is calibrated to one dataset: the certified +Microcosm UK 2024-25 national release. It does not carry over to other +datasets, because their consumption imputations capture a different share of +household VAT. On Enhanced FRS 2024-25 it puts VAT 35% to 44% above receipts, +depending on the release and year (see the tables below). Re-derive the coverage, with +the calculation below, whenever the dataset it should match changes. + ## Derivation (October 2026) -The product of the two factors is the dataset's pre-scaling VAT divided by -receipts for the same year. The survey coverage is that product divided by the -household share. +### Inputs + +| Input | Value | Source | +|---|---|---| +| Microcosm UK 2024-25 national release | `microcosm_uk_2024_25.h5`, Hugging Face `policyengine/populace-uk-private`, tag `microcosm-uk-2024-25-national-20261002T230158Z-5c6b3f68` (revision `f9d1922c`), SHA-256 `aa31bdf67c977927ea2b325567d1cf7a79d94381239bc79918a0a0fc9c9588af` | Private repository | +| Enhanced FRS 2024-25 | `enhanced_frs_2024_25.h5`, Hugging Face `policyengine/policyengine-uk-data-private`, policyengine-uk-data 1.56.16 (revision `a9e52499`, SHA-256 `e433e532b17bd8ce76030156285816e33d44e93edabd2204adbef71d19a68712`) and 1.58.0 (revision `47bbb58d`, SHA-256 `5ba399ea5ab9a6c7d4e9aaf6c623080beb34d9ad5f330f6244dbad72b8d960b6`) | Private repository | +| VAT receipts | £170.994bn (2024-25 outturn), £180.169bn (2025-26), £187.704bn (2026-27) | OBR, Economic and fiscal outlook March 2026, detailed forecast tables, receipts, table 3.8, cash-basis VAT | +| Household share of VAT liabilities | 0.7 | HMRC VAT gap estimates (around 70% of the VAT total theoretical liability); `household_share_of_receipts` | +| Ready reckoner | £8.8bn for one point on the standard rate, 2026-27 | HMRC, Direct effects of illustrative tax changes (June 2025) | +| Statutory rates | 20% standard, 5% reduced | `gov.hmrc.vat` | + +Model year 2024 is the 2024-25 fiscal year. All dataset totals are weighted +sums from `Microsimulation`, computed with policyengine-uk at this branch and +policyengine-core 3.33.2. + +### Formula + +``` +pre-scaling VAT = 0.20 x sum(full_rate_vat_consumption) + + 0.05 x sum(reduced_rate_vat_consumption) +product = pre-scaling VAT / OBR VAT receipts (same year) +survey coverage = product / household share (0.7) +``` + +`vat` divides each household's statutory VAT by the product, so total `vat` +equals receipts when the product matches the dataset. + +### Results + +| Dataset | Year | Standard-rated base | Reduced-rated base | Pre-scaling VAT | Receipts | Product | Survey coverage | +|---|---|---|---|---|---|---|---| +| Microcosm UK 2024-25 (`f9d1922c`) | 2024-25 | £396.15bn | £25.02bn | £80.48bn | £171.0bn | 0.471 | 0.672 | +| Microcosm UK 2024-25 (`f9d1922c`) | 2025-26 | £412.57bn | £26.06bn | £83.82bn | £180.2bn | 0.465 | 0.665 | +| Microcosm UK 2024-25 (`f9d1922c`) | 2026-27 | £423.66bn | £26.76bn | £86.07bn | £187.7bn | 0.459 | 0.655 | +| Enhanced FRS 2024-25, uk-data 1.56.16 | 2024-25 | £546.96bn | £32.21bn | £111.00bn | £171.0bn | 0.649 | 0.927 | +| Enhanced FRS 2024-25, uk-data 1.56.16 | 2025-26 | £569.63bn | £33.55bn | £115.60bn | £180.2bn | 0.642 | 0.917 | +| Enhanced FRS 2024-25, uk-data 1.56.16 | 2026-27 | £584.95bn | £34.45bn | £118.71bn | £187.7bn | 0.632 | 0.904 | +| Enhanced FRS 2024-25, uk-data 1.58.0 | 2024-25 | £570.29bn | £33.17bn | £115.72bn | £171.0bn | 0.677 | 0.967 | +| Enhanced FRS 2024-25, uk-data 1.58.0 | 2025-26 | £593.92bn | £34.54bn | £120.51bn | £180.2bn | 0.669 | 0.956 | +| Enhanced FRS 2024-25, uk-data 1.58.0 | 2026-27 | £609.89bn | £35.47bn | £123.75bn | £187.7bn | 0.659 | 0.942 | + +The parameter takes the Microcosm 2024-25 value, 0.67 (product 0.469), the +latest outturn year. -| Dataset | Year | Pre-scaling VAT | OBR VAT receipts | Product | Survey coverage | -|---|---|---|---|---|---| -| Microcosm UK 2024-25 (populace-uk-private `f9d1922c`) | 2024-25 | £80.5bn | £171.0bn (outturn) | 0.471 | 0.672 | -| Microcosm UK 2024-25 | 2025-26 | £83.8bn | £180.2bn | 0.465 | 0.665 | -| Microcosm UK 2024-25 | 2026-27 | £86.1bn | £187.7bn | 0.459 | 0.655 | -| Enhanced FRS 2024-25 | 2024-25 | £111.0bn | £171.0bn (outturn) | 0.649 | 0.927 | +With that product, total `vat` against receipts is: -Receipts are the cash-basis VAT line of OBR March 2026 EFO detailed forecast -table 3.8. Model year 2024 is the 2024-25 fiscal year. +| Dataset | 2024-25 | 2025-26 | 2026-27 | +|---|---|---|---| +| Microcosm UK 2024-25 (`f9d1922c`) | £171.6bn (+0.4%) | £178.7bn (-0.8%) | £183.5bn (-2.2%) | +| Enhanced FRS 2024-25, uk-data 1.56.16 | £236.7bn (+38.4%) | £246.5bn (+36.8%) | £253.1bn (+34.8%) | +| Enhanced FRS 2024-25, uk-data 1.58.0 | £246.7bn (+44.3%) | £257.0bn (+42.6%) | £263.9bn (+40.6%) | -HMRC's ready reckoner gives an independent check that does not use the level -of receipts: one percentage point on the standard rate raises £8.8bn in -2026-27 (Direct effects of illustrative tax changes, June 2025). On Microcosm -UK 2024-25, one point on the standard rate raises £4.24bn before scaling, so -matching HMRC needs a product of 0.48, within 3% of the level-based 0.469. +### Bound -The value is set against the certified Microcosm UK 2024-25 national release. -On Enhanced FRS 2024-25 it puts 2024-25 VAT about 38% above receipts (it was -71% above with 0.38). The implied product moves by less than 3% between -2024-25 and 2026-27, so the factors are not dated by year. +Total VAT depends only on the product, so the household share being a +rounded "around 70%" changes how the product is split between the two factors, +not the totals. On Microcosm the product is bounded by: -## Reproducing +- the receipts route across 2024-25 to 2026-27: 0.459 to 0.471 (coverage 0.655 + to 0.672); +- the ready reckoner, which does not use the level of receipts: one point on + the standard rate raises £4.24bn before scaling in 2026-27 (1% of + £423.66bn), so matching HMRC's £8.8bn needs a product of 0.481 (coverage + 0.688). + +So on Microcosm the product lies between 0.459 and 0.481, and the coverage +between 0.655 and 0.688 at a household share of 0.7. The value used, 0.469 +(0.67), sits inside that range, and the totals it gives are within 2.2% of +receipts from 2024-25 to 2026-27. The implied product moves by less than 3% +over those years, so the factors are not dated by year. + +### Reproducing + +The two datasets are in private Hugging Face repositories. Anyone with +PolicyEngine access can rerun the calculation; anyone else can check the +arithmetic from the published aggregates in the results table. ```python +from huggingface_hub import hf_hub_download from policyengine_uk import Microsimulation -sim = Microsimulation(dataset="microcosm_uk_2024_25.h5") -year = 2024 -p = sim.tax_benefit_system.parameters(f"{year}-01-01").gov.hmrc.vat -raw = ( - sim.calculate("full_rate_vat_consumption", year).sum() * p.standard_rate - + sim.calculate("reduced_rate_vat_consumption", year).sum() * p.reduced_rate +path = hf_hub_download( + "policyengine/populace-uk-private", + "microcosm_uk_2024_25.h5", + repo_type="dataset", + revision="f9d1922cddab6b54a0dd37794a9bac74e3780c88", ) -obr_receipts = 170.994e9 # OBR March 2026 EFO, table 3.8, 2024-25 -product = raw / obr_receipts -print(product, product / 0.7) +sim = Microsimulation(dataset=path) +# OBR March 2026 EFO, table 3.8, cash-basis VAT receipts +receipts = {2024: 170.994e9, 2025: 180.169e9, 2026: 187.704e9} +for year, obr in receipts.items(): + p = sim.tax_benefit_system.parameters(f"{year}-01-01").gov.hmrc.vat + full = sim.calculate("full_rate_vat_consumption", year).sum() + reduced = sim.calculate("reduced_rate_vat_consumption", year).sum() + raw = full * p.standard_rate + reduced * p.reduced_rate + print(year, raw / 1e9, raw / obr, raw / obr / 0.7) ``` +For Enhanced FRS, download `enhanced_frs_2024_25.h5` from the model repository +`policyengine/policyengine-uk-data-private` at the release tag (for example +`revision="1.58.0"`). + ## Interaction with an explicit energy VAT base A domestic fuel and power VAT base built from `electricity_consumption` and diff --git a/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml b/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml index afdce1f8e6..e5a16038db 100644 --- a/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml +++ b/policyengine_uk/parameters/gov/simulation/vat/survey_consumption_coverage.yaml @@ -4,17 +4,22 @@ metadata: period: year economy: false derivation: >- - Pre-scaling VAT (standard rate times full_rate_vat_consumption plus reduced - rate times reduced_rate_vat_consumption) on the certified Microcosm UK - 2024-25 national release (policyengine/populace-uk-private, revision - f9d1922c) is £80.5bn in 2024. OBR VAT receipts for 2024-25 (outturn, March - 2026 EFO table 3.8) are £171.0bn. 80.5 / 171.0 = 0.471 for the product of - the two factors; divided by the household share of 0.7 gives 0.67. The - HMRC ready reckoner gives a consistent level: £8.8bn for one percentage - point on the standard rate in 2026-27 implies a product of 0.48 on the same - data. On Enhanced FRS 2024-25 the same calculation gives 0.65, so this - value overstates VAT on that dataset by about 38%. Re-derive when the - default dataset changes, with docs/engineering/vat-coverage.md. + Calibrated to the certified Microcosm UK 2024-25 national release only + (policyengine/populace-uk-private, tag + microcosm-uk-2024-25-national-20261002T230158Z-5c6b3f68, revision + f9d1922c). Pre-scaling VAT (standard rate times full_rate_vat_consumption + plus reduced rate times reduced_rate_vat_consumption) on that release is + 0.20 x £396.15bn + 0.05 x £25.02bn = £80.48bn in 2024. OBR VAT receipts + for 2024-25 (outturn, March 2026 EFO table 3.8, cash basis) are £170.994bn. + 80.48 / 170.994 = 0.471 for the product of the two factors; divided by the + household share of 0.7 gives 0.67. Across 2024-25 to 2026-27 the product is + 0.459 to 0.471, and the HMRC ready reckoner (£8.8bn for one point on the + standard rate in 2026-27) implies 0.481, so the coverage lies between 0.655 + and 0.688. On Enhanced FRS 2024-25 the same calculation gives a coverage of + 0.927 (policyengine-uk-data 1.56.16) or 0.967 (1.58.0), so this value puts + VAT on that dataset 38% or 44% above 2024-25 receipts. Re-derive when the + dataset it should match changes, with docs/engineering/vat-coverage.md, + which lists the inputs, file hashes and per-dataset results. reference: - title: OBR, Economic and fiscal outlook March 2026, detailed forecast tables, receipts, table 3.8 href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/