Put FRS self-employment losses in trading_loss

[MaxGhenis]

Summary

FRS 2024-25 records self-employment losses as negative SEINCAM2 ("Any losses are recorded as such", FRS methodology glossary). The build floored them at zero, so about 74,000 people (weighted) and £1.28bn a year of losses (2024-25 prices, before uprating) disappeared from the data.

This PR splits SEINCAM2 into two non-negative inputs:

• self_employment_income: the profit, unchanged (stays non-negative, as test_non_negative_incomes requires);
• trading_loss: the loss as a positive amount.

Their difference is the reported profit. policyengine-uk needs them apart because its programmes treat a loss differently: Income Tax and tax credits set it against other income, Class 4 against trading profits only, means-tested benefits never offset it against other earnings, and HBAI counts it as negative income (PolicyEngine/policyengine-uk#2085).

Checked before relying on the sign: none of the negative values is a survey missing-value code. The five small whole-number values between −£9 and −£1 a week are all different, and each of those adults reports a loss on a self-employed job (PROFIT2).

SPI-donor rows get their losses from the second-stage QRF. Their incomes are imputed from the SPI, which has no current-year loss field (only LOSSBF, losses brought forward). So trading_loss joins the FRS-only variables that impute_frs_only_variables rewrites on those rows, drawn from FRS respondents with similar demographics and imputed incomes. It is appended last: microimpute imputes the outputs in order, each with its own seeded generator, so every earlier imputation and NumPy's global random state are unchanged. (Python's random module state does change; no later build step uses it.) As in the FRS, where SEINCAM2 nets a person's trades, a row never keeps both a profit and a loss.

Why not zero them: the first build zeroed the SPI-donor losses, and the calibrated enhanced FRS then carried £0.29bn of losses on about 23,000 weighted people, against £1.28bn on 74,000 in the FRS. SPI-donor rows hold 28% of household weight.

trading_loss also gets the self-employment rows in uprating_factors.csv and uprating_growth_factors.csv: the table generator produces those once policyengine-uk#2085 gives the variable that uprating.

Nothing else in the build changes. The split adds no random draws, and the QRF output is appended after the others. policyengine-uk releases before #2085 read trading_loss only as a deduction from self-employment income, which is zero for every FRS loss-maker, so calibration targets are unchanged.

Tests

• test_trading_loss.py: Hypothesis properties of split_self_employment_profit for any weekly profits, including missing values: both outputs non-negative, never both positive, profit less loss equals the annualised reported profit, and monotone in the reported profit. Data checks on the built FRS and on the enhanced FRS's SPI-donor rows (no negative loss, never a profit and a loss together), asserting on counts only so a failure never prints a record's amounts.
• test_non_negative_incomes.py covers trading_loss.
• Adds hypothesis to the dev extras (the same line as Say which joint occupiers each FRS non-dependant family resides with #522, Supply is_claimant_or_partner from the FRS adult table #524 and Set UC gainful self-employment from the FRS main-job status #525; whichever lands second relocks).

Impact

Real runs on a seeded build of 94bd3d7 (cf7042b changes only the changelog), against policyengine-uk main 84f5ad43a and #2085 at 905b41aef. Aggregates only; any cell resting on fewer than 10 FRS households is suppressed.

Losses in the data. The calibrated enhanced FRS (2024 data year) carries £314m of losses on 27,000 weighted people: £290m on FRS rows and £25m imputed onto SPI-donor rows. The raw FRS has £1.28bn on 74,000 people. Calibration gives FRS loss-makers about a third of their survey weight (no calibration target covers trading losses), and SPI-donor rows hold 28% of household weight. With losses zeroed on SPI rows (the first version) the total was £290m.

With policyengine-uk main (no rule reads the losses yet): every output is identical to the same build with trading_loss zeroed, in each year 2025-26 to 2030-31 (37 arrays of taxes, benefits, household incomes and poverty flags; only the input itself differs). The data change is neutral until #2085 is released.

With #2085, against the same build with losses zeroed (#2085 is itself bitwise neutral without losses):

	2025-26	2026-27	2030-31
Trading losses	£319m	£330m	£387m
Trade loss relief against general income	£77m	£80m	£102m
Income Tax	−£13.1m	−£14.0m	−£18.4m
NICs, means-tested benefits, tax credits	£0	£0	£0
HBAI household net income	−£306m	−£316m	−£368m
Household market income	−£319m	−£330m	−£387m

In 2026-27, 27,000 people (61 FRS households) have a loss, 23,500 (50 households) get relief and 9,100 (35 households) pay less Income Tax. Fewer than 10 FRS households move across a poverty line, so those counts are not published.

Merge

Part of the batched uk-data release (d833): approved on gates, then merged with the batch on Max's go. Lands after PolicyEngine/policyengine-uk#2085 is released, so the data's losses are read by the new rules.

Review

• Round 1 (independent Opus review of c3bc616): REQUEST CHANGES. It asked for count-only assertions and uprating rows, and measured that zeroing left £0.29bn of calibrated losses, which led to the QRF change. All addressed in 94bd3d7.
• Round 2 (independent Opus review of 94bd3d7): APPROVE, with one nit (the changelog still described zeroing), fixed in cf7042b. A delta review of cf7042b is pending.

🤖 Generated with Claude Code