Map FRS EMPSTATI 11 to OTHER_INACTIVE, not LONG_TERM_DISABLED

[MaxGhenis]

What was wrong

create_frs set employment_status with categorical(person.empstati, 1, range(12), EMPLOYMENTS).fillna("LONG_TERM_DISABLED"). EMPLOYMENTS has 11 entries, so zip(range(12), EMPLOYMENTS) stops at code 10 and EMPSTATI 11 had no mapping. The fillna then made every code-11 adult LONG_TERM_DISABLED.

In the UKDS FRS 2024-25 data dictionary (SN 9563, adult table, EMPSTATI "Adult - Employment Status - ILO definition"), 11 is "Other Inactive" and 9 is "Permanently sick/disabled". policyengine-uk's EmploymentStatus already has OTHER_INACTIVE.

In the raw FRS 2024-25 adult table, 916 adults (2.05m by FRS grossing weight) have code 11, beside 1,678 (3.24m) with code 9. All of them came out LONG_TERM_DISABLED.

The fix

• FRS_EMPSTATI_EMPLOYMENT_STATUS maps codes 1-11 explicitly, one data-dictionary label per line, using EmploymentStatus member names (a rename in policyengine-uk breaks loudly).
• derive_employment_status_from_frs(empstati, is_adult_record): child-table rows are CHILD. An adult whose code is missing or not in the table raises a ValueError naming the codes, instead of falling back to a guessed status.
  • CI build logs are public, and adults are not survey households, so the message names the unknown codes once each, in numeric order with "blank" last (for example "0, 12, blank"). It prints no count of any kind.
  • Within create_frs, a blank adult code arrives as 0, because the person table fills blanks, so it is listed as "0". "blank" appears only when the function is called directly.
  • The codes are formatted from the numeric values rather than Series.astype(str), whose NaN handling changed in pandas 3.
  • Why raise rather than keep a default: the old default is how code 11 became "long-term disabled". LONG_TERM_DISABLED is a health status that feeds the ESA proxies, and OTHER_INACTIVE would equally be a guess for a code nobody has read.
  • This cannot break a current build. Every adult in the 2020-21, 2022-23, 2023-24 and 2024-25 releases on disk has a code from 1 to 11; no adult is blank.
  • Children: the child table has no EMPSTATI, and the person table fills the blank with 0. The old code reached CHILD through code 0; the new code uses the row's table. categorical's NaN default of 1 (FT_EMPLOYED) never applied, because the combined table is already filled.

Who reads employment_status

In uk-data (git grep at origin/main b45c373, and the diffs of all 30 open uk-data PRs):

• derive_esa_health_condition_proxy reads ESA_HEALTH_EMPLOYMENT_STATUSES = (LONG_TERM_DISABLED, SHORT_TERM_DISABLED). Code-11 adults of working age with a reported status leave this proxy; that is the intended change. The tuple itself stays as it is.
• derive_esa_support_group_proxy reads LONG_TERM_DISABLED with no hours worked, as a subset of the health proxy. Code-11 adults leave it too.
• derive_legacy_jobseeker_proxy reads UNEMPLOYED only (code 5), so it is unchanged.
• On main, no calibration target, imputation or loss matrix reads employment_status or the three proxies.
• Open PRs that read it are all unaffected:
  • Set UC gainful self-employment from the FRS main-job status #525 reads only FT_SELF_EMPLOYED and PT_SELF_EMPLOYED.
  • Set the UC start-up period from the FRS claim and business start dates #527 reads raw EMPSTATI codes 3 and 4.
  • Draw SPI incomes within earnings groups set by FRS employment status #529 reads it in two places, and codes 9 and 11 are treated identically in both:
    • Its SPI earnings groups (imputations/income.py) test only the employee, self-employed and CHILD statuses, so both codes follow the same path.
    • Its new ONS LFS calibration targets (targets/sources/ons_labour_market.py) count only employee and self-employed statuses, so both codes count zero.

In policyengine-uk (git grep of origin/main at 3c48247eb and at 84f5ad43a, and of the heads of all 129 open PRs):

• No variable formula reads employment_status.
• dynamics/labour_supply.py calculate_excluded_from_labour_supply_responses reads it only for self-employment and STUDENT, so it is unchanged.
• No variable is defined for esa_health_condition_proxy, esa_support_group_proxy or legacy_jobseeker_proxy. They are saved in the dataset but nothing in the model reads them.
• In open PRs, none is affected:
  • #2007 sets STUDENT as a test input.
  • #1915 passes a year to the labour-supply exclusion reader and adds tests for it, but its predicate is still self-employment and STUDENT.

Impact (real build and microsimulation)

Build. I built main b45c373 and this branch at d398400 with the same settings: 512 calibration epochs, torch seed 0, web targets from one shared HTTP cache, and PE_UK_DATA_OA_CLONES=1. The release default is 10 clones. Two builds of main are output-identical, so seeded builds reproduce.

• Compared: the saved FRS and enhanced FRS. Every column of the household and benefit-unit tables is identical between main and the branch, including weights, so national calibration did not move.
• In the person table, only employment_status, esa_health_condition_proxy and esa_support_group_proxy differ.
• Not compared: the local authority and constituency weight files. On main, no calibration target reads employment_status.
• The later commits change only the text of an error that no current FRS release reaches.

Dataset	Change	Person records	Survey households	Weighted (m)
FRS	LONG_TERM_DISABLED → OTHER_INACTIVE	916	876	2.05
FRS	leave the ESA health-condition proxy	839	803	1.94
FRS	leave the ESA support-group proxy	833	797	1.93
Enhanced FRS	LONG_TERM_DISABLED → OTHER_INACTIVE	3,006	876	2.30
Enhanced FRS	leave the ESA health-condition proxy	2,778	803	2.10
Enhanced FRS	leave the ESA support-group proxy	2,754	797	2.09

In the enhanced FRS, the 3,006 records are the 916 FRS rows plus their SPI copies, capital-gains clones and CGT band donors. No record enters either proxy, and the JSA proxy is unchanged. In the FRS, LONG_TERM_DISABLED falls from 5.30m to 3.24m.

Microsimulation. For each enhanced FRS, I ran one policyengine-uk Microsimulation that calculated every variable for 2025 and 2026 and fingerprinted each array. I did this twice: on main 3c48247eb (976 variables) and again on main 84f5ad43a (2.109.1, 979 variables). Both times:

• Every array is bit-identical between main and the branch except employment_status itself.
• Neither run had a calculation error.
• The labour-supply exclusion mask is unchanged.
• So the change moves no tax, benefit, income, poverty or budget figure.

In the 2025 microsimulation, LONG_TERM_DISABLED goes from 5.81m to 3.50m and OTHER_INACTIVE from 0 to 2.31m.

Tests

• test_frs_employment_status.py:
  • Codes 0-11 checked exhaustively, as adult and as child rows.
  • The code table pinned to the data dictionary's labels.
  • Adult codes plus CHILD are a bijection onto EmploymentStatus.
  • Unknown adult codes (0, negative, 12+, non-integer, NaN) raise.
• Hypothesis properties:
  • The mapping is row-wise and commutes with row order.
  • Any unknown adult code anywhere fails.
  • For any ages, State Pension ages, hours and reported flags, the ESA health proxy is exactly reported ∧ working age ∧ code ∈ {9, 10}.
  • The support group is a subset of the health proxy and only code 9.
  • Code 11 never sets either proxy.
• test_legacy_benefit_proxies.py: create_frs runs end to end for every code 1-11 (status and both ESA proxies), for a child row next to a code-11 adult, and for code 12, which is rejected.
• Built-dataset checks (CI's make data output) on the FRS and enhanced FRS:
  • Every status is a valid EmploymentStatus.
  • OTHER_INACTIVE is present.
  • Neither ESA proxy is set for anyone outside the two sick/disabled statuses.
• Failure-message tests:
  • The message depends only on which codes occur and contains no count.
  • Codes are listed once each, in numeric order, with "blank" last. Hypothesis covers this, plus a fixed case where a string sort would put 100 before 11.5.
• Mutation check: 10 of 10 mutations are killed. They include:
  • the old behaviour, and 11 → LONG_TERM_DISABLED;
  • unknown codes → a default;
  • 9 and 10 swapped;
  • OTHER_INACTIVE added to the ESA health statuses;
  • child rows treated as adults;
  • codes listed unsorted or in string order;
  • the adult count printed.

Adds hypothesis as a dev dependency. The pyproject.toml and uv.lock change is identical to #522's and #525's, so whichever lands second merges cleanly.

axiom: n/a: data mapping of an FRS survey code to an input variable; no policy rule changes.

microcosm's UK runtime (uk_runtime/frs_employment.py) keeps the old map on purpose, for parity. A separate session is porting this fix there, with a differential test against this PR's function.

Reviews:

• r1 approved d398400, with low findings that 5f9912d addresses.
• r2 (GPT-6.1 Sol) requested changes on 5f9912d: a count of adults isn't safe to print when the threshold is 10 survey households, the ordering tests were too weak, and the build scope and two open-PR readers needed stating. 1832ade addresses all four.
• r3 approved 1832ade, with no blocking findings.

Part of the batched uk-data release (d833); not to be merged on its own.

🤖 Generated with Claude Code

[MaxGhenis]

Hand-off to the UK hub. This lands with the d833 batch. Don't merge it on its own.

• Head: 1832ade
• Review: r3 approves 1832ade with no blocking findings (Subfleet 20261003-072847-ukdata526-review-r3, ~/reviews/frs-empstati-other-inactive-2026-10-02/review/review-r3.md). Earlier rounds:
  • r1 approved d398400.
  • r2 (GPT-6.1 Sol) requested changes on 5f9912d: no count in the unknown-code error, ordering tests, build scope, and two open-PR readers. All four are fixed in 1832ade.
• Checks at 1832ade: all 4 pass (changelog, lint, bundle manifest, Test with the full data build), verified 2026-10-03 22:47Z. The PR is MERGEABLE, out of draft, with no change requests.
• Impact: zero on model outputs.
  • Seeded builds of main b45c373 and the branch differ only in employment_status and the two ESA proxy columns, and national weights are identical.
  • On policyengine-uk main (3c48247eb, 976 variables, and 84f5ad43a, 979 variables), every array for 2025 and 2026 is bit-identical apart from employment_status.
  • The PR body has the detail.
• Depends on: nothing. The pyproject.toml and uv.lock hypothesis change is identical to Say which joint occupiers each FRS non-dependant family resides with #522's and Set UC gainful self-employment from the FRS main-job status #525's; whichever lands second relocks if needed.
• Hold: uk-data lands as one batch on Max's go (d833).
• Follow-ups owed: none here. microcosm#1097 ports the same mapping and has its own owner.

[MaxGhenis]

Queued in release PR #544 for the 10/8 uk-data batch. It lands only on Max's go (d833).