Stop calibrating OBR total NICs on ni_employee

[MaxGhenis]

What

_parse_receipts in policyengine_uk_data/targets/sources/obr.py read the Table 3.8 (cash basis) "National insurance contributions" row as obr/ni on the ni_employee variable. That row is total NICs: £200.08bn for 2025-26 in the committed March 2026 workbook. _parse_nics also targets ni_employee, at the Table 3.4 "Class 1 Employee NICs" row (£49.54bn). Two targets on one matrix column with different values cannot both be met. The seeded constituency calibration on main b45c373 ended with obr/ni at 0.242 of target, its estimate equal to the weighted ni_employee total.

Fix: drop the row rather than retarget it. PE-UK 2.93.0 has total_national_insurance (Class 1 employee, Class 2, Class 3, Class 4 and Class 1 employer). Retargeting obr/ni there would still conflict with the class targets. Table 3.4 splits total NICs (£203.99bn accrued) into:

• Class 1 employee £49.54bn, Class 1 employer £145.32bn and Class 4 and 2 self-employed £2.90bn, which are already targeted;
• statutory payment recoveries £3.44bn and "Other NIC" £2.78bn (Class 1A, 1B and 3, settlements, unallocated). PE-UK has no variable for the first three, and no dataset fills Class 3 (Handle unpopulated ni_class_3 target #378).

A total target would therefore demand NICs no household carries, on top of the cash/accrued basis difference (£200.08bn against £203.99bn).

Also: build_loss_matrix.calibration_targets() extracts, unchanged, the target list create_target_matrix built inline, so the new test checks the same set.

Invariants (tested)

1. Shared-column agreement. No two calibration targets whose matrix column is the same plain variable sum or count (and the same countries restriction, if Calibrate Housing Benefit to DWP's GB figures by age group #490/Calibrate GB universal credit to one OBR total; reach the UC payment top band #530 land) disagree on a year's value, after _resolve_value's nearest-year fallback. INTENDED_SHARED_COLUMNS (empty) is where a documented exception would go.
   • The column is taken from _compute_column itself, with every custom compute function swapped for a sentinel.
   • It is checked exhaustively over the OBR targets offline and over all 652 national, regional and country targets.
   • It flags the removed obr/ni mapping.
   • A seeded random differential test (300 target sets) holds the grouped check to a brute-force pairwise comparison.
   • Hypothesis is not a dependency on main, so the random cases use random.Random(0).
2. Mapping pin. Every target parsed from the receipts workbook maps to the listed variable, and obr/ni is absent even though its row exists.
3. Source identity. The Table 3.4 class rows plus the unmodelled rows equal the NICs total (±£0.02bn), and the unmodelled part is positive.
4. The new and changed tests fail on main: 5 failures when run against b45c373's obr.py.

pytest policyengine_uk_data/tests/test_obr_*.py: 33 passed, 5 skipped (the enhanced-FRS signal tests, skipped as on main).

Impact, from real calibrations (2026-10-05)

The setup is the UK hub's #536 harness: one saved input built from main b45c373 (1 output-area clone), and calibrate_local_areas for constituencies with 512 epochs, 650 areas, torch.manual_seed(0) and 8 threads. The committed EFO workbooks are served, and one environment (policyengine-uk 2.93.0) is used for both runs. Base reproduces the hub's run D exactly: same 637 targets, identical totals.

• Target set: base has obr/ni and fix doesn't. Nothing else differs (636 common).

2025 weighted total (£bn)	base (main)	fix	change
Income tax	290.72	289.87	−0.85 (−0.3%)
Employee NI	48.45	47.85	−0.60 (−1.2%)
Employer NI	146.21	144.91	−1.30 (−0.9%)
Self-employed NI	2.77	2.77	0.00
Total NI	197.42	195.53	−1.90 (−1.0%)
VAT	316.82	316.02	−0.80 (−0.3%)
Household net income	1,692.73	1,690.47	−2.26 (−0.1%)

• Fit, estimate/target:
  • obr/ni_employee 0.978 → 0.966: the false pull toward £200bn had been lifting it.
  • obr/ni_employer 1.006 → 0.997.
  • Over the 636 common national targets: 84.12% within 10% in both runs; median |ratio−1| 0.0412 → 0.0415.
• Size of the effect: small. The contradiction is gone, and the dataset barely moves.

VAT finding (reported, not changed here)

obr/vat ends at 1.758× its target (£316.8bn against OBR cash VAT of £180.17bn).

Mechanism (PE-UK 2.93.0, variables/gov/hmrc/vat.py): vat = (full_rate_vat_consumption × 0.20 + reduced_rate_vat_consumption × 0.05) / gov.simulation.microdata_vat_coverage. The parameter is 0.38, its only value, from 2010-01-01. Its description says it scales survey VAT up to HMRC receipts because LCFS under-reports consumption.

Dataset (2025)	Consumption	VAT before scaling	vat (÷0.38)	vs OBR	Coverage that would hit OBR
Pre-calibration input	£1,205.8bn	£112.0bn	£294.9bn	1.64×	0.62
Calibrated base	£1,428.0bn	£120.4bn	£316.8bn	1.76×	0.67

• Consumption coverage: ONS ABJQ (households' individual consumption expenditure, current prices, which includes imputed rent) is £1,782.4bn for 2025. The model's consumption is 80% of that, so the survey no longer under-covers spending by the margin a 0.38 factor assumes.
• impute_vat is not the cause. It imputes a share (full_rate_vat_expenditure_rate, weighted mean 0.349), not a level. One unverified second-order point: the share is taken over VAT-exclusive spending (expdis − totvat), but PE-UK multiplies it by consumption. If that is VAT-inclusive, the base is about 9% high.
• Calibration cannot absorb the gap. Reweighting cannot correct a level that sits on every household. In the hub's run A, with no OBR targets, VAT was £392.9bn; the target pulls it down only to £316.8bn.
• Tracked: Re-derive microdata_vat_coverage: the value has been 0.383 since 2010 and overshoots OBR VAT receipts on current datasets policyengine-uk#1996 proposes re-deriving the constant. Changing it moves every published PE-UK VAT figure, so this PR leaves it alone.
• Pending: a diagnostic calibration with obr/vat excluded from training (fix_novat) is running, to measure what the unattainable target costs the other targets.

Batch interplay (uk-data lands as one release, d833)

• Textual conflicts: none. This branch merges cleanly with Calibrate Housing Benefit to DWP's GB figures by age group #490, Solve Pension Credit take-up over entitled benefit units #510, Calibrate GB universal credit to one OBR total; reach the UC payment top band #530, Restore HMRC salary sacrifice relief calibration targets #533 and Fall back to the committed OBR workbooks when obr.uk answers 200 with a non-workbook page #536, and with all of them together.
• Fall back to the committed OBR workbooks when obr.uk answers 200 with a non-workbook page #536: its new _RECEIPTS_AND_NICS_TARGETS in test_obr_efo_fallback.py lists "obr/ni". Whichever of Fall back to the committed OBR workbooks when obr.uk answers 200 with a non-workbook page #536 and this PR lands second must drop that entry (one line).
• Offline OBR target count: main's test_full_target_set_available_offline asserts len(names) >= 30. On main there are 34 OBR targets. This PR takes them to 33 (−1). The rest of the batch removes 5 more: Calibrate Housing Benefit to DWP's GB figures by age group #490 −1, Solve Pension Credit take-up over entitled benefit units #510 another −1 (stacked on Calibrate Housing Benefit to DWP's GB figures by age group #490), Calibrate GB universal credit to one OBR total; reach the UC payment top band #530 −1 and Restore HMRC salary sacrifice relief calibration targets #533 −2. The combined tree has 28, so the last of these to land trips the threshold.
• Everything else in test_obr_* passes on the combined tree, except Calibrate GB universal credit to one OBR total; reach the UC payment top band #530's new hypothesis-based test, which needs the hypothesis dev dependency that Calibrate Housing Benefit to DWP's GB figures by age group #490/Solve Pension Credit take-up over entitled benefit units #510 add.

The FRS-derived inputs, outputs and harness stay local in ~/reviews/uk-hub/jobs/obr-ni-target/. Only aggregates appear here.

axiom: n/a: calibration-target mapping in the data build; no policy rule changes.

🤖 Generated with Claude Code