Skip to content

Restore HMRC salary sacrifice relief calibration targets - #533

Merged
MaxGhenis merged 5 commits into
mainfrom
fix/hmrc-salary-sacrifice-relief-targets
Oct 7, 2026
Merged

MaxGhenis merged 5 commits into
mainfrom
fix/hmrc-salary-sacrifice-relief-targets

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Six HMRC salary sacrifice calibration targets have been missing from every published build since 1.56.16 (18 August 2026). targets/sources/hmrc_salary_sacrifice.py downloads HMRC private pension statistics Tables 6.1 and 6.2. GOV.UK withdrew the July 2025 CSV (HTTP 410 Gone) when HMRC published the July 2026 release on 30 July. The module caught the error, logged it, and returned only the static £24bn contributions target, so the builds kept passing.

This PR restores the targets from the July 2026 release and stops a dead URL from dropping them silently. It also fixes errors that would otherwise come back with them:

  • Rate-band relief was assigned to the wrong band.
  • The employer NICs target ignored the April 2025 rate rise.
  • The NICs relief targets were duplicated with conflicting values.

Five of the six targets return; the income tax relief total is now implied by the bands (change 6).

Merging is a data release. Restoring these targets changes calibration, so this PR waits for Max's go on the uk-data release batch (d833). HMRC's method leaves two readings open (the allowance taper, and the Scottish advanced rate); they are queued for Max as d925, with the sizes in the Readings paragraph.

What was missing

Target 1.56.14 (26 Jul, last build with them) 1.56.16 to 1.57.4
hmrc/salary_sacrifice_it_relief_total present missing
hmrc/salary_sacrifice_it_relief_basic_rate present missing
hmrc/salary_sacrifice_it_relief_higher_rate present missing
hmrc/salary_sacrifice_it_relief_additional_rate present missing
hmrc/salary_sacrifice_employee_nics_relief present missing
hmrc/salary_sacrifice_employer_nics_relief present missing

Sources: the national rows of constituency_calibration_log.csv from push runs 30202165235 (1.56.14, ebf733c) and 36131888928 (1.57.4, b45c373). The 1.57.4 log has 637 national targets and none of these six. The 1.56.16 build log (run 32138754415) records the 410. hmrc/salary_sacrifice_contributions and the obr/salary_sacrifice_* targets stayed in every build.

Changes

  1. Source. sources.yaml now points at the July 2026 CSV (tax year 2024-25). The columns and category labels the parser reads are unchanged from the July 2025 file (compared against the archived copy at the UK Government Web Archive).

  2. Failure behaviour. A failed download falls back to a committed copy, storage/hmrc_pension_relief_tables_6_1_6_2_2024_25.csv (byte-identical to the live asset; sha256 04e0dab6…). This follows the OBR EFO fallback (test_obr_efo_fallback.py). The build now fails instead of returning fewer targets when:

    • a table lacks exactly one positive value for a required row;
    • a table, live or fallback, is for a tax year other than _TAX_YEAR (which also names the committed copy), so pointing sources.yaml at next year's release without refreshing the copy fails rather than giving online and offline builds different targets;
    • the table cannot be parsed. The broad except Exception is gone.

    create_target_matrix still skips, with a warning, any target whose compute raises or returns nothing. A new test checks on the built dataset that every relief target produces a finite, positive column.

  3. Year. The base year now comes from the table's tax_year, mapped to PolicyEngine's year. PolicyEngine UK's year N is tax year N to N+1: a year-2017 simulation reads the personal allowance in force from 6 April 2017 (£11,500), and obr.py and frs.py use the same mapping. So 2024-25 maps to 2024. The old constant put the 2023-24 table at 2024, one year late; hmrc_spi.py and hmrc_cgt.py share that mapping (follow-up below). Growth stays at 3% a year.

  4. Income tax relief by rate. compute_ss_it_relief used to measure relief as the change in total income_tax and put each person's whole relief in one band. It chose the band by comparing counterfactual adjusted net income with the rate thresholds (0, £37,700, £125,140), which apply to taxable income, after the £12,570 personal allowance. So every basic-rate taxpayer with income between £37,700 and £50,270 counted as higher rate.

    HMRC applies income tax rates to employees' pay from ASHE, and models contributions that "straddle" bands (private pension statistics, background and methodology). Relief is now the rise in tax on earned income (earned_taxable_income) when the sacrifice is paid as salary, under the person's own rUK or Scottish schedule. It is allocated bracket by bracket to HMRC's categories:

    • basic: rates below 30% (rUK basic; Scottish starter, basic and intermediate, as HMRC states);
    • additional: the schedule's top rate (rUK additional, Scottish top);
    • higher: the rest (rUK higher; Scottish higher and advanced). HMRC does not say where advanced-rate relief (45%, £75,000–£125,140) goes. Putting it in higher, by income range, is this PR's reading.

    The bands add up exactly to the change in earned-income tax. That leaves out effects a pay-based estimate cannot see, which the change in income_tax includes: savings, dividend and property income tax, the High Income Child Benefit Charge, and the pension annual allowance charge (which falls when the sacrifice is paid as salary). The personal allowance taper stays in, because it follows from pay. On this PR's build the change in income_tax is £9.12bn, against £9.45bn of band relief. Examples, checked in policyengine-uk (2025):

    • £48,000 pay after a £4,000 sacrifice: £454 basic, £692 higher (the old mask put all £1,146 in higher);
    • Scotland, £42,000 after £4,000: £349.02 basic (21%), £981.96 higher (42%);
    • £100,000 after £10,000: £6,000, all higher (60% with the allowance taper).
  5. Duplicate NICs targets. obr/salary_sacrifice_employee_ni_relief (£1.2bn) and obr/salary_sacrifice_employer_ni_relief (£2.9bn) at 2024 equal HMRC's 2023-24 Table 6.2 figures to the £0.1bn. They use the same compute column as the HMRC NICs targets, and in 1.56.14 both pairs had identical targets and estimates. With the 2024-25 table the pairs would disagree (employee £1.03bn vs £1.24bn in 2025, employer £3.81bn vs £2.99bn), which would ask calibration to hit two values for one quantity. This PR removes the OBR-labelled copies. Their cited PDF (2025 SPP Review) returns 404 and has no UK Government Web Archive copy.

  6. No total target. The income tax relief total is the sum of the bands, and HMRC rounds each cell separately: in 2024-25 the bands sum to £8.9bn and the total is £8.8bn. Targeting both would ask calibration for two values, so only the bands are targets.

  7. NICs rates. Each year's NICs relief target is scaled by that year's Class 1 rate over the 2024-25 rate, read from PolicyEngine's parameters and weighted by HMRC's main/additional split. The employer rate rose from 13.8% to 15% on 6 April 2025, so the 2025 employer target is £3.4bn × 1.03 × 15/13.8 = £3.81bn rather than £3.50bn. The employee rates (8% and 2%) did not change. Not adjusted: the secondary threshold cut from £9,100 to £5,000, which affects only sacrifices by people paid under £9,100.

Target values for 2025 (the calibration year)

Target 1.56.14 (2023-24 table) Main (1.57.4) This PR (2024-25 table)
IT relief, total £7.42bn missing not a target (bands sum to £9.17bn)
IT relief, basic rate £1.65bn missing £1.65bn
IT relief, higher rate £4.53bn missing £5.67bn
IT relief, additional rate £1.24bn missing £1.85bn
Employee NICs relief £1.24bn (HMRC and OBR copies) £1.24bn (OBR copy) £1.03bn
Employer NICs relief £2.99bn (HMRC and OBR copies) £2.99bn (OBR copy) £3.81bn

HMRC's 2024-25 table shows £8.8bn of income tax relief on salary-sacrificed contributions, up from £7.2bn for 2023-24. It also shows £1.0bn employee NICs relief (from £1.2bn) and £3.4bn employer NICs relief (from £2.9bn). The July 2026 Table 6 time series leaves the 2023-24 salary sacrifice figures unchanged (£7.2bn, £1.2bn, £2.9bn), although it labels that column "revised".

Calibration fit

Real runs. Two production-settings builds with the same harness (this PR's adds network retries, and routes pandas URL reads through the cache) and seed (512 epochs, PE_UK_DATA_OA_CLONES=1, torch seed 0, frozen HTTP cache):

  • main: b45c373, the code of published 1.57.4.
  • this PR: 138c72a. The head adds only tests and the blank-cell guard, which no real table triggers.

The new downloads are the 2024-25 CSV (byte-identical to the committed copy) and the SLC student support workbook, which both builds read live.

National rows of constituency_calibration_log.csv, last epoch (510). "Before" is each target on main's calibrated data: this PR's code run on main's saved calibration-year dataset with main's final constituency weights. That reconstruction reproduces main's logged estimates for the targets main has.

Target (2025) Target Before (main data) After (this PR)
IT relief, basic rate £1.65bn £3.09bn (+87%) £1.73bn (+4.9%)
IT relief, higher rate £5.67bn £2.14bn (−62%) £5.79bn (+2.2%)
IT relief, additional rate £1.85bn £0.65bn (−65%) £1.93bn (+4.0%)
Employee NICs relief £1.03bn £1.16bn (+12.6%) £0.99bn (−3.4%)
Employer NICs relief £3.81bn £3.28bn (−13.9%) £3.86bn (+1.4%)
hmrc/salary_sacrifice_contributions £24.7bn £27.0bn (+9.4%) £29.8bn (+20.6%)
obr/salary_sacrifice_users_total 7.88m 8.02m (+1.7%) 8.00m (+1.5%)

Other targets. The 635 targets common to both builds barely move:

Measure main this PR
Within 5% 63.9% 63.8%
Within 10% 83.9% 83.8%
Median absolute error 4.10% 4.13%

Five targets worsen by more than a percentage point and four improve. After salary sacrifice contributions, the largest worsenings are:

  • employment income £50k–£70k: +10.6% → +12.4%;
  • council tax Wales band D: +3.1% → +4.3%;
  • council tax London band A: +5.3% → +6.4%;
  • employment income £150k–£200k: +3.8% → +4.8%.

Leaving aside ons/multi_family_households (−2.6 points, but more than 1,000% off in both builds), the largest improvements are dividend income £200k–£300k (+2.0% → +0.2%) and £100k–£150k (+6.3% → +5.1%), and the property income count for £150k–£200k (+18.7% → +17.5%).

The local-authority calibration agrees: 86.0% → 86.1% of common targets within 10%, the restored targets between −2.4% and +5.2%, and salary sacrifice contributions +8.8% → +20.0%.

The cost: salary sacrifice contributions overshoot more. On main's calibration, which does not target relief, the data's relief is mostly basic rate (£3.09bn against £1.65bn), with too little at higher and additional rates. HMRC's ASHE-based figures put most relief at higher and additional rates. To match them, calibration upweights higher-paid users: those with pre-sacrifice pay of £100k or more go from 0.24m people holding £1.92bn to 0.57m holding £6.73bn (review r3's aggregates). Total contributions rise from +9.4% to +20.6% over the £24.7bn target. That target is cited to a 2025 SPP Review PDF that now returns 404; re-sourcing it is a follow-up.

On this build, £1.44bn of the relief comes from people with no pay. Those are the imputed amounts #518 removes. A build with #518 applied, the release configuration, is reported below.

Readings HMRC's method leaves open, sized on this build (relief in each band totals £9.45bn):

  • Personal allowance taper. Relief includes the allowance withdrawn between £100,000 and £125,140, since that follows from pay. With each person's allowance held at its pre-sacrifice value, relief would be £0.43bn lower: higher rate £5.41bn instead of £5.79bn, additional £1.88bn instead of £1.93bn. 0.32m salary sacrificers are in the taper.
  • Scottish advanced rate (45%). It is filed as higher rate. It accounts for £0.20bn of the £0.81bn Scottish relief. policyengine-uk starts the Scottish top rate early (£112,570 of taxable income rather than £125,140; Scottish top rate (48%) starts at £116,760: threshold stored as £112,570 instead of £125,140 policyengine-uk#2130), so some relief that belongs at the advanced rate is filed as additional until that is fixed.
  • Old band rule. The rule this PR replaces gives £1.57bn basic, £4.09bn higher and £0.57bn additional on main's data. That is closer to HMRC only because it counted £37,700–£50,270 earners as higher rate.

With #518, the release configuration. A third build ran on #518's reviewed head at the time (9910ad4) merged with this PR's 138c72a. #518's two later commits (now 14e2c22) change only comments, the changelog and a .copy() with no effect. The comparison is a build of 9910ad4 alone, which used main's harness (without the retries and cache routing); both builds read identical inputs and their pre-calibration datasets are identical in every column, so the two differ only through calibration. "Before" is each target on #518's calibrated data.

Target (2025) Target Before (#518 data) After (#518 + this PR)
IT relief, basic rate £1.65bn £2.50bn (+52%) £1.77bn (+7.2%)
IT relief, higher rate £5.67bn £2.98bn (−47%) £5.89bn (+3.9%)
IT relief, additional rate £1.85bn £2.03bn (+9.7%) £1.92bn (+3.7%)
Employee NICs relief £1.03bn £1.19bn (+15.2%) £1.03bn (−0.3%)
Employer NICs relief £3.81bn £3.46bn (−9.0%) £3.89bn (+2.3%)
hmrc/salary_sacrifice_contributions £24.7bn £24.0bn (−2.8%) £26.8bn (+8.4%)
obr/salary_sacrifice_users_total 7.88m 8.19m (+3.9%) 8.07m (+2.4%)

Common targets:

Measure #518 #518 + this PR
Within 5% 64.3% 65.7%
Within 10% 84.7% 84.6%
Median absolute error 4.15% 4.12%

Six targets improve by more than a point and two worsen: salary sacrifice contributions, and employment income £50k–£70k (+10.0% → +11.3%).

Local authorities:

  • within 10%: 86.9% in both builds;
  • within 5%: 63.1% → 63.6%;
  • restored targets between −0.1% and +6.2%;
  • contributions −2.0% → +8.2%.

Calibration again upweights higher-paid users: those with pre-sacrifice pay of £100k or more go from 0.44m holding £5.93bn to 0.63m holding £8.20bn (review r4's aggregates). With #518, no relief comes from people with no pay. The readings are: band relief £9.58bn, the allowance taper £0.39bn of it (0.37m people), and Scottish advanced-rate relief £0.18bn of £0.74bn.

Invariants

  • tax_by_band: the categories add up to the schedule's tax (checked against PolicyEngine's MarginalRateTaxScale.calc on 5,000 seeded incomes per schedule and year, rUK and Scottish, 2023 to 2025). Each category's tax is non-decreasing in income, so each band's relief is non-negative when the sacrifice is.
  • compute_ss_it_relief: with pay as the only income, the bands add up to the fall in income_tax (a differential check against policyengine-uk on three households).
  • _relief_targets: exactly one positive value per required row, a single tax year equal to _TAX_YEAR, and a main/additional split for each NICs class, or it raises.
  • Offline: the full relief target set builds with the network down or the URL returning 410.

Hypothesis is not a dependency on main (six batch PRs add it), so the property checks use seeded random draws. Each of these mutants of the current code fails at least one test: no NICs rate factor, no tax-year check, advanced rate as additional, Scottish taxpayers on rUK bands, uncapped brackets, relief measured as the change in income_tax, and a blank cell accepted. Review r2 ran 13 mutants of its own; the three that survived were two equivalent under current parameters and the income_tax measure, which the dividend test now catches.

Tests

  • tests/test_hmrc_salary_sacrifice_targets.py (23 tests):
    • the committed table gives five targets with HMRC's values and the employer rate rise;
    • the fallback's tax year matches _TAX_YEAR;
    • a 410 and an offline download both use the committed copy;
    • seven kinds of changed table raise, including a blank cell;
    • tax_by_band against the PolicyEngine scales, monotonicity, and the Scottish categories;
    • compute_ss_it_relief on four real policyengine-uk households, one with dividends that the sacrifice pushes into the higher band (that extra dividend tax is not relief);
    • every relief target produces a loss-matrix column on the built dataset (skips without a build; passed against this branch's production build).
  • tests/test_target_registry.py::test_hmrc_salary_sacrifice_relief_targets: the five targets are in the registry for 2025, and the total and the OBR copies are not.
  • CI's Test job builds the TESTING datasets and runs the whole suite, including the loss-matrix test. At b224171 it gave 655 passed, 3 skipped, 1 xfailed, with every salary sacrifice test run (review r3). Against the first head's production build, the full suite gave 640 passed; the 4 failures were all network timeouts in unrelated sources (explore-education-statistics, ONS).

Reviews

  • r1 (Opus, 40ea945): REQUEST_CHANGES. The findings and the response, finding by finding, are answered in the code at 585402d.
  • r2 (Opus, 585402d, ran the code): APPROVE_WITH_NITS. Its three in-PR nits are fixed in b224171: blank cells raise, a dividend household pins the measure, and the Scottish test reads PolicyEngine's thresholds.
  • r3 (Opus, b224171, ran the code and recomputed every number in this body from the evidence): APPROVE_WITH_NITS. All three r2 nits are fixed and checked with 14 mutants. Its nits were one wrong figure and some wording in this body, fixed here, and a test docstring, fixed in 0863a6a.
  • r4 (Opus, 0863a6a, reran the evidence scripts and checked the docstring in policyengine-uk): APPROVE_WITH_NITS. Its nits were all in this body (one rounding, four phrasings) and are fixed.

Follow-ups (not in this PR)

  • policyengine-uk starts the Scottish 48% rate at £116,760 of income instead of £125,140: Scottish top rate (48%) starts at £116,760: threshold stored as £112,570 instead of £125,140 policyengine-uk#2130.

  • From 2029 the salary sacrifice cap changes the counterfactual in build_loss_matrix._SimContext (review r2: the sacrifice above £2,000 is added back twice), and the 2029–31 NICs relief targets ignore the cap. Calibration runs only at 2025, so current builds are unaffected. This predates this PR.

  • Bands are classified by rate value, so a reform that changed the rates could reclassify relief. Production builds calibrate the baseline only.

  • hmrc_spi.py and hmrc_cgt.py map tax year 2023-24 to year 2024. Under the mapping above that is one year late. Tracked as a separate task.

  • hmrc/salary_sacrifice_contributions (£24bn) cites the same dead SPP Review PDF. Tracked as a separate task.

axiom: n/a: data

🤖 Generated with Claude Code

MaxGhenis and others added 3 commits October 3, 2026 16:38
GOV.UK withdrew the July 2025 Tables 6.1/6.2 CSV (410 Gone) when HMRC
published the July 2026 private pension statistics, and get_targets()
logged the error and returned without the relief targets, so every build
since has calibrated without hmrc/salary_sacrifice_it_relief_* and the
HMRC NICs relief targets.

- Point sources.yaml at the July 2026 CSV (tax year 2024-25) and commit
  a copy in storage; a failed download uses it, and a table without the
  expected rows raises instead of returning fewer targets.
- Map the tax year from the table: PolicyEngine year N is tax year
  N to N+1, so 2024-25 is 2024 (the old comment put 2023-24 at 2024).
- Split IT relief across the bands each person's relief straddles,
  as HMRC does, instead of classifying by adjusted net income against
  taxable-income thresholds (which put every basic-rate taxpayer with
  income between 37,700 and 50,270 in the higher-rate target).
- Drop obr/salary_sacrifice_{employee,employer}_ni_relief: static copies
  of HMRC's 2023-24 Table 6.2 figures computed on the same column as the
  restored HMRC NICs targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ish bands

- Employer NICs relief targets follow the Class 1 rate in PolicyEngine's
  parameters (13.8% in 2024-25, 15% from April 2025), so the 2025 target
  is £3.81bn, not £3.50bn; employee targets weight the main and
  additional rates by HMRC's split (unchanged, 8% and 2%).
- Income tax relief is the rise in tax on earned income, allocated to
  HMRC's basic/higher/additional categories under the person's rUK or
  Scottish rates (starter and intermediate with basic, advanced with
  higher, top with additional). HMRC applies income tax rates to ASHE
  pay; the previous measure (change in income_tax) included the High
  Income Child Benefit Charge and savings and dividend tax, and split
  Scottish relief on rUK boundaries.
- Drop the IT relief total target: it is the sum of the bands, which
  HMRC rounds separately (£8.9bn vs £8.8bn in 2024-25).
- The table's tax year must equal _TAX_YEAR, which names the committed
  copy, so pointing sources.yaml at a new release without refreshing
  the copy fails instead of giving online and offline builds different
  targets.
- Test that every relief target produces a loss-matrix column on the
  built dataset (create_target_matrix skips with a warning otherwise).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits October 4, 2026 01:58
… PE threshold

- _relief rejects a blank (NaN) cell instead of returning a NaN target.
- Test that dividends pushed into a higher band by the sacrifice do not
  count as relief (a revert to the change in income_tax passed every
  test before).
- The Scottish category test reads the thresholds from PolicyEngine's
  schedule instead of stating its top-rate threshold (£112,570) as law
  (policyengine-uk#2130: the statutory threshold is £125,140).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the dividends taxed in the basic rate band move to the higher band,
not the whole £10k.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 4, 2026 07:31
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Hand-off to the UK hub. Head 0863a6a.

  • Reviews:
    • r1: REQUEST_CHANGES at 40ea945, fixed.
    • r2 at 585402d and r3 at b224171: APPROVE_WITH_NITS, nits fixed.
    • r4: APPROVE_WITH_NITS at 0863a6a. Its nits were PR-body only and are fixed.
    • Files: ~/reviews/uk-data-hmrc-ss-relief-2026-10-03/review/review_r{1,2,3,4}.md.
  • CI: gh pr checks 533 exits 0 at 0863a6a; the Test job gave 655 passed, 3 skipped, 1 xfailed. MERGEABLE, out of draft.
  • Held: this is a uk-data merge, so it is a data release. It lands only in the batch on Max's go (d833). Max's decision d925 (two readings HMRC's method leaves open; default park, deadline 10-11) gates it.
  • Depends on: nothing. It merges cleanly with Limit salary sacrifice to the pay it comes out of #518, and the release-configuration fit (Limit salary sacrifice to the pay it comes out of #518 + this PR) is in the body. Batch row added to ~/reviews/uk-data-release-batch-2026-10/BATCH.md.
  • Evidence: seeded production builds, logs and aggregate outputs in ~/reviews/uk-data-hmrc-ss-relief-2026-10-03/ (STATUS.md indexes them). The builds/*.h5 files are licensed FRS derivatives: keep them private.
  • Follow-ups:

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833).

MaxGhenis added a commit that referenced this pull request Oct 7, 2026
…ount (F1)

test_full_target_set_available_offline (from #536) asserted at least 30 OBR
targets offline. The release combines four member PRs that each replace OBR
targets with better sources, which no single PR's CI saw together:
#490 swaps obr/housing_benefit for DWP Housing Benefit targets, #510 swaps
obr/pension_credit for DWP Pension Credit spend and caseload, #533 swaps the
two OBR salary-sacrifice NI relief targets for HMRC's, and #530 merges the
two OBR UC targets into obr/universal_credit. That leaves 29 OBR targets (34
on main), so release CI failed with 29 >= 30.

The test now checks what it was for: offline, the committed workbooks give
exactly the OBR targets that the same workbooks give when served, plus the
receipts and NICs names. A broken fallback still fails it.

Release-only fix for the 10/8 uk-data release (d833).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 5d0d42a into main Oct 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant