Skip to content

UK domestic electricity and gas: record why ONS 04.5.1/04.5.2 is preferred over DESNZ × QEP, reconcile the two, and declare the price basis PE-UK should project from #1113

Closed

Description

@vahid-ahmadi

Revised after review (6 Oct 2026): following juaristi22's review:


Title

UK domestic electricity and gas: record why ONS 04.5.1/04.5.2 is preferred over DESNZ volume × QEP price, reconcile the two, and declare the price basis PE-UK should project from

Body

Summary

The Microcosm UK national file carries £26.5–26.9bn a year of domestic electricity spending (electricity_consumption, 2024–2027). That matches the bound target, ONS Consumer Trends COICOP 04.5.1 for CY2025 (£26.52bn). It sits about 10% below what the build's own energy stage produces from published volume and price: the DESNZ Energy Trends domestic volume at DESNZ Quarterly Energy Prices (QEP) average prices paid, about £29.4bn. Gas shows the same conflict, and a bigger one (+24.6%).

How the build handles this is already documented (see below). What is missing is the reason ONS is preferred, a reconciliation of the two sources, and a declared price basis that PE-UK can project from.

The figure that prompted this check, about £35bn including VAT, has no support in any official source and is not a valid calibration target.

Already documented: the current implementation contract

The gate notes for the lcfs_consumption energy_rake check in packages/microcosm-build/src/microcosm/build/uk/gates.json already state:

So the precedence (ONS binds; DESNZ × QEP is a stage-time level and diagnostic) and the residual handling are documented. This issue does not ask for them to be re-decided from scratch.

The selected release's producer diagnostics (PE-UK 2.100.0 / core 3.32.5) confirm the fit: electricity expenditure £26.5199bn against the £26.5220bn target (−0.008%). Source: selected-build provenance and diagnostics in PolicyEngine/uk-autumn-budget-dashboard-2026#7 (comment of 6 Oct 2026). This validates the recorded expenditure fit under the producer runtime. It does not show agreement with independent volume measures.

Still to decide or validate

1. A written rationale for the target choice. Record next to price_domestic_energy (in the contract notes or sources.yaml) why ONS 04.5.1/04.5.2 is preferred over DESNZ volume × QEP price, and who approved it. If the preference changes after step 2, state the new choice:

  • (a) keep ONS as the level and record the implied-volume shortfall as a known limitation in the release notes, so per-kWh costings are warned;
  • (b) make DESNZ × QEP the level and keep the ONS rows as diagnostics; or
  • (c) keep both and publish the disagreement with its size so users can bracket.
    Each option should leave the stage and the solver agreeing on the level.

2. Reconcile the two sources before choosing between them. The stage-time DESNZ result and the final weighted ONS result answer different questions. Reconcile:

  • reference period: FY2024-25 against CY2025;
  • coverage: meters against households;
  • regional and payment-method tariffs, and standing charges;
  • the price basis, including VAT treatment.

3. Treat the implied-volume figure as a diagnostic. At QEP FY2024-25 prices, the file's electricity spend implies roughly 83 TWh, against 93.0 TWh in Energy Trends (about −11%). That uses one national-average conversion, which cannot establish the exact size of any shortfall. Before publishing a measured volume shortfall, recompute implied kWh using each household's regional tariff and fixed charge and the final weights. Do the same for gas.

4. Declare the price basis, and have PE-UK project from it. This is the projection-contract defect found in juaristi22's audit (PolicyEngine/uk-autumn-budget-dashboard-2026#7). Microcosm prices energy at QEP FY2024-25 and calibrates to ONS CY2025 expenditure. PE-UK's uprating_indices.yaml leaves energy out of dataset projection because it assumes the data are already at Ofgem Q2 2026 (FY2026-27) prices. That assumption comes from the legacy policyengine-uk-data build. It does not hold for a Microcosm file, which therefore carries 2024-25/2025 prices into every later year. Two deliverables:

  • Producer (Microcosm, this issue): publish with the data the expenditure, price and volume reference years, and the tariff and VAT basis, for electricity and gas.
  • Consumer (PE-UK, needs a named owner and its own issue): project energy from the declared reference years consistently, instead of assuming a fixed later price anchor.

Raising the base to the mock £35bn, or restoring blanket CPI uprating of energy, would not fix this contract. PolicyEngine/policyengine-uk-data#485 is related legacy-pipeline work. A change there does not by itself change Microcosm's exported H5 or PE-UK's projection contract.

Native electricity-VAT policy (the temporary 0% rate from 1 October 2026) is separate PE-UK work in PolicyEngine/policyengine-uk#2166. Choosing an electricity calibration total will not implement it.

5. Rebuild and certify. Keep the source discrepancy and its assumptions recorded until a decision is made. Then rebuild, recalibrate and certify the chosen model/data pair. The selected release's certification covers PE-UK 2.100.0 / core 3.32.5, not the newer drill runtime automatically.

Numbers (weighted aggregates)

£bn Note
Model electricity_consumption, 2025 / 2026 / 2027 26.7 / 26.8 / 26.9 Flat apart from household growth (see item 4)
Producer diagnostic, electricity expenditure vs ONS CY2025 target 26.5199 vs 26.5220 −0.008%, PE-UK 2.100.0 / core 3.32.5
ONS 04.5.1 household electricity, CY2024 / CY2025 26.1 / 26.5 The bound target
DESNZ volume × QEP price, FY2024-25 ~29.1–29.4 93.0 TWh × 24.81p + £209 a year fixed cost × ~29m households. #927 measured £29.36bn at design weights. Not reconciled for period or coverage (item 2)
Ofgem cap × the same volume, FY2026-27 ~30 Direct-debit GB averages: 24.67p / 57.21p a day (Apr–Jun 2026), 26.11p / 57.19p (Jul–Sep), 26.32p / 54.83p (Oct–Dec, 0% VAT on electricity from 1 Oct 2026 to 31 Mar 2027). Assumes Jan–Mar 2027 at the Oct–Dec 2026 rates, weighted by FY2024-25 quarterly volume

Related

Sources

🤖 Generated with Claude Code

Activity

  1. juaristi22 commented on Oct 6, 2026

    @juaristi22
    Collaborator

    A recorded rationale for the electricity/gas target choice would help, but the issue should distinguish the existing implementation contract from the remaining decision and validation work.

    The current gate notes already state that the stage checks DESNZ volumes priced with QEP, the calibrated frame binds ONS 04.5.1/04.5.2 spending, and the difference is a recorded source residual rather than a gate. The request for a formal rationale/approval remains useful. Please link this existing contract and distinguish it from the missing rationale, so the issue does not imply that current precedence or residual handling is undocumented.

    The selected release's producer diagnostics confirm electricity expenditure of £26.5199bn against £26.5220bn (−0.008%). Selected-build provenance and diagnostics. That validates the recorded expenditure fit under the producer runtime, not agreement with every independent volume measure.

    Please retain these requirements:

    1. Reconcile FY2024–25 versus CY2025, meter versus household coverage, regional/payment-method tariffs, standing charges and the price basis before choosing or reconciling the two sources. The stage-time DESNZ result and the final weighted ONS result answer different checks.
    2. Treat the approximate 83TWh calculation as a diagnostic. Recompute implied kWh with each household's actual regional tariff/fixed charge and final weights before publishing a measured volume shortfall. A national average conversion cannot establish its exact size. Include gas in the same exercise.
    3. Replace the suggestion that the price-basis work is fully covered by uk-data#485 with explicit producer and consumer deliverables. uk-data #485 is related legacy-pipeline work; a change there does not by itself alter Microcosm's exported H5 or PE-UK's projection contract.

    Microcosm should publish the expenditure, price and volume reference periods and the applicable tariff/VAT basis with its data. PE-UK should consume that information when projecting energy. The current projection mapping omits energy on a later-price-anchor assumption that does not match Microcosm's QEP FY2024–25 basis. This producer task can live here; the consumer fix needs an explicit PE-UK owner. Link both to PE-UK #2166, whose native electricity-VAT policy is separate work.

    The mock £35bn is not a justified calibration target. Preserve the source discrepancy and assumptions until adjudicated, then rebuild/recalibrate and certify the chosen model/data pair. The selected release's certification applies to UK 2.100.0/core 3.32.5, not automatically to the newer drill runtime.

  2. changed the title [-]Decide the domestic electricity level (ONS 04.5.1 or DESNZ volume × QEP price); the two differ by about 11%[/-] [+]UK domestic electricity and gas: record why ONS 04.5.1/04.5.2 is preferred over DESNZ × QEP, reconcile the two, and declare the price basis PE-UK should project from[/+] on Oct 6, 2026
  3. juaristi22 commented on Oct 8, 2026

    @juaristi22
    Collaborator

    Consumer migration notice for Chronicle PR #323, currently in draft:

    The preserved ONS Consumer Trends workbook is the 30 June 2026 release, acquired on 9 September. Its latest annual expenditure observations remain CY 2025, including electricity of £26.522bn and gas of £15.339bn. The publication date and acquisition date are separate from the expenditure reference year. The 42 annual rows vendored in ons_household_expenditure_facts.json span CY 2020–2025 (seven series across six years); the wider feed also includes quarterly observations through 2026-Q1.

    The 217 existing facts retain their source-record IDs, reference periods and expenditure values. Their aggregate and semantic fact keys change: domestic consumption uses households_on_uk_territory, five source/concept/unit dimensions are added, and lineage retains the value, COICOP header and CDID guard. The corrected vintage also changes provenance and source-cell keys.

    On refresh, update pinned keys and role selectors. The seven existing series retain their specific ons.household_expenditure.* concepts; selecting only the new generic HFCE concept omits them. This notice concerns source metadata and consumer selection; it does not revise Microcosm's target precedence or calibration decisions. See the consumer migration documentation. Local tests remain deferred pending the memory/testing decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions