Run the ESI premium stage in the stacked pool (#454)

[MaxGhenis]

Summary

The stacked ASEC + ACS pool now carries employer_sponsored_insurance_premiums on every row, so a pool-lineage (--exact-k) release has the input that us_release_input_coverage_gate requires and us_esi_premiums_anchor_gate grades. Follow-up to #1189 for #454; docs/us-esi-employer-premiums.md named this gap under "What this does not do".

Stacked on #1189 (esi-employer-premiums-454). This repo's CI runs only on PRs that target main, so checks start when #1189 merges and this PR is retargeted. Until then the test evidence is local; see the test plan.

axiom: n/a: data/microsim pipeline

The two design questions

1. Scaling. A cps_source operator sees only the rows with raw CPS evidence, at pool weights. assemble_stacked_spine gives each source half of ASEC's household mass, so those rows hold half the population. Run unchanged, the stage would hold half the policyholders to the whole anchor and double every premium.

The pool passes the stage anchor_share: the household mass of the rows it runs on over the pool's household mass, read from the live frame (us_esi_premiums_household_mass_share). The factor is anchor x share / sum(w x raw). Assembly multiplies every ASEC household weight by that same share, so the factor equals the single-source factor and each ASEC person carries the dollars a single-source build assigns. anchor_share defaults to 1, which leaves the base build as it was.

2. ACS rows. ACS has no policyholder flag, payment status, tier or firm size, so the stage cannot run there. The column enters the pool's QRF transfer as family source_operator_esi_premiums, on the transfer's fixed predictor surface (age, sex and State always; wages, self-employment, Social Security, retirement and investment income, household head and tenure where both sources observe them).

A QRF draw keeps no total and can put a premium on a non-worker. with_us_esi_premium_pool_anchor runs in the derive stage, once every row is filled. On transferred rows only, it:

1. gives every support clone its clone-0 source record's draw, so a person carries one premium whether the pool transferred before or after it cloned;
2. clears the premium where that source record reports zero wages; then
3. scales the rest by one factor so those rows carry the same weighted premium per unit of household mass as the source-derived rows.

A transferred person's wage must be evidence. The anchor resolves the ACS earnings universe over the transferred rows. An eligible person (15 or older) needs a reported raw WAGP, even if a producer filled the mapped wage. A person below 15 needs the named universe zero. A missing or non-numeric mapped wage is refused too. In the stacked pool nothing real is refused: the late ACS earnings-universe producer writes the zeros below 15 before the derive stage, and Census reports WAGP for everyone else.

It tells the two kinds of row apart by whether the raw ASEC coverage columns are present, corroborated by the CPS record id (PERIDNUM), not by a source channel. So esi_premiums.py stays spine-blind and moves to the registered operator set. A CPS record that lost its coverage codes is refused by the anchor and fails both gates; it is never graded as a transfer.

Where this departs from the brief

The brief suggested a post-clone operator, since the draw is keyed by source identity. I placed the stage pre-clone:

• A pre-clone operator's outputs are early gap-fill targets. The donors are ASEC rows before the PUF clone, so the fill sees the measured CPS wage on donors and the measured ACS wage on recipients. A post-clone operator would fill from PUF-detail clones, whose wages are PUF-imputed.
• The clone copies one value per source person, so clone agreement holds by construction.
• The late producer DAG is unchanged: 38 producers, 19 transfer groups, 70 targets, 71 edges, and the same schedule and payload digests.

Invariants

For every valid pool, stated in the module and property-tested with Hypothesis in tests/engine_free/us/test_us_esi_premiums_pool.py:

• Conservation. The pool-wide weighted column equals the source-derived rows' total divided by their household mass share, which is the single-source column total.
• Equal intensity. Source-derived and transferred rows carry the same weighted premium per unit of household mass.
• Source rows are immutable. No source-derived cell changes by a byte.
• One premium per person. Every support clone of a transferred person carries its source record's draw.
• Structural zeros. A transferred person whose source record has zero wages carries nothing, on every support clone.
• Proportionality. Every other transferred value is one common multiple of what the transfer drew for the source record.
• Idempotence. A pool already on that scale is returned as the same object.
• Weight homogeneity. Rescaling every household weight moves no value.

The strategies generate zero-weight households and support clones (index 1 and the tail's index 2), with per-clone draws on the transferred side.

Differential tests: the pool's CPS rows equal a single-source build's dollars row for row, and the pool total equals the single-source total for any mass split. The anchor-universe total the release gate recomputes equals the anchor on the pooled fixture, before and after the PUF clone.

Assumption. The anchor (BEA NIPA 7.8 line 17 since #1189's d1317 change) counts every policyholder, and ACS rows have no policyholder flag to price the non-employed ones from. The gates take the pool's anchor-universe total as the ASEC rows' total plus the transferred column over the ASEC rows' employed share. That assumes employed policyholders carry the same share of employer premiums on the ACS half as on the ASEC half. The anchor gate records the assumption and fails if the transferred rows' column per unit of household mass leaves 0.8 to 1.25 times the source rows'.

Measured on the pinned inputs

experiments/us-esi-454/run_pool_fill_on_pinned_inputs.py runs the part of the pool that decides the column on the real inputs (pooled 2023-2025 ASEC, ACS 2024 1-year PUMS, the real assembly, stage, early gap-fill QRF, anchor operator, by-origin battery and both gates) on a household sample. It is not a pool build, and no artifact was built or published. Receipts hold aggregates only.

	1% of households	5% of households
Person rows, ASEC / ACS	4,177 / 34,293	21,133 / 171,381
ACS rows filled / left null	34,293 / 0	171,381 / 0
ACS-over-ASEC incidence, after the fill / after the anchor (band 0.8 to 1.25)	0.933 / 0.930	0.975 / 0.969
Conditional quantile distance, after the fill / after the anchor (limit 0.25)	0.072 / 0.172	0.043 / 0.084
Transferred rows cleared for zero wages	34 ($1.7B)	255 ($2.8B)
Scale factor on the other transferred rows	1.105	1.046
Mean per holder after the anchor, ASEC / ACS	$10,956 / $12,071	$11,766 / $12,325
Pool column total (single-source: $863.9B)	$884.0B	$863.0B
Transferred wages that were not evidence	0	0
By-origin battery, signal gate, anchor gate	pass	pass

Figures are at #1189's BEA anchor ($977.0B for 2024).

The fill reproduces ASEC's premium per holder and gives slightly fewer ACS people one, and ACS rows carry fewer weighted people at equal household mass, so the factor is above one and ACS holders end about 5% above ASEC holders in the 5% sample. The factor fell from 1.10 to 1.05 as the donor sample grew fivefold. A full build was not run. In the 5% sample the earnings-universe producer wrote the named zero wage on the 25,511 ACS records below 15, and Census reported WAGP for all 145,870 others.

Sealed pools and pinned surfaces

• Legacy v11 authority. Adding a family changes the live stacked authority. _legacy_stacked_authority_receipt rebuilds the version-11 authority from live surfaces to score attested schema-9 pools, so a new family would have changed it. The reconstruction now excludes families added after v11 (_POST_V11_AUTHORITY_TARGET_FAMILIES), and a new test pins the legacy authority digest and its components to their previous values.
• Legacy publication golden. test_legacy_entrypoint_publication_matches_origin_main_golden moves only in checkpoint identity digests. Run against the base branch with the same fixture, the pool H5 and agreement bytes are identical and the manifest differs only in stage_checkpoints.*identity_sha256.
• H3 graph. us_runtime/graph.py declares the premium, the seven raw coverage columns, the CPS record id (a string cell), the mapped and raw ACS wages as source cells, and the premium as a derive rewrite. The derive kernel's implementation hash now covers esi_premiums.py. The parity fixture was regenerated with tools/graph_us_post_transfer_fixture.py, whose helper path was stale since the test split. Every pre-existing source and expected cell is byte-identical. Writing a real-shaped 21-digit PERIDNUM into the fixture exposed a defect in the generic csv-tables source codec. It inferred numbers before casting to the declared dtype, so a declared string column of long numeric ids came back rounded. It now reads declared string columns as strings, and a codec test covers long ids, leading zeros, blanks and whitespace. The new expected column shows the transferred rows moving from the drawn $6,000 to $8,000, the ASEC row's premium at equal household mass.
• Re-pins. Counts and digests the new early family and derive consumer move are re-pinned at computed values: 135 battery targets, 49 early targets in 14 families, 119 pool transfer targets, a 1,079-row remaining-stage manifest, the stacked authority and its components, the gap-fill schedule, the checkpoint identity, the spec bundle, field-usage claims and the coverage evidence. The seed protocol and seed map digests do not move.
• No calibration spec. No post-transfer calibration spec is added for this column. Terminal validation replays a calibrated target's bytes, which would refuse the later rescale.

Independent review

GPT-6.1 Sol reviewed this twice, and both reviews asked for changes. The last commit answers the second.

Round 1

Finding	Answer
A missing clone-0 wage was read as no wages and the premium cleared	Wages must be evidence (round 2 tightened this: see below).
A CPS record with every coverage field null passed as a transferred row, in a pool or a single-source frame	Row kind is corroborated by PERIDNUM. The anchor refuses such a record and both gates fail on it.
The legacy two-spine path transfers after it clones, so support clones could draw different premiums	Every transferred clone takes its clone-0 record's draw, and the signal gate fails on transferred clone disagreement. I kept the stage on the legacy path rather than freezing separate legacy surfaces: every pool operator is shared with it, and a pool lineage without the input fails the release gates.
A pool whose transferred rows hold no household weight failed the gates and the anchor	The anchor keeps the factor at one, and the gates skip the two-sided comparison.

Round 2

Finding	Answer
The legacy PUF pass fills missing mapped wages with zero, so the round-1 wage check could not see missing evidence there	The anchor resolves the ACS earnings universe over the transferred rows. An eligible person with a blank raw WAGP is refused whatever the mapped column holds, and a person below 15 must carry the named universe zero. WAGP and the universe inputs are declared anchor inputs.
The target used 1 - share, which loses a tiny side to rounding, so the factor depended on weight scale	Each side's household mass is summed on its own. The target is source total x transferred mass / source mass, and the gates compare each side against its own mass. The property strategy now spans weights from 1e-9 to 1e10.
The H3 graph declared PERIDNUM as a float	It is a string cell, and the graph's CSV source codec now reads declared string columns as strings (it rounded long numeric ids).

My own widened property tests found one more defect in round 1: the mass share depended on floating-point summation order. Round 2's separate masses replace that code.

Not in this PR

• ACS local lane. declared_acs_transfer_target_families still does not name the column, and fill_reviewed_nulls refuses that frame. A separate branch, esi-premiums-acs-local-lane-454, is stacked on this one.
• ACS-native predictors (HINS1, COW, ESR) for the fill.
• ACS usual hours. The pool's operator boundary refuses the ACS-native usual-hours column on main; the experiment drops it. A separate session is fixing that.

Test plan

Local, on the pushed head:

• uv run ruff check . and python3 tools/ci_test_plan.py verify
• tools/generate_us_bundle_from_constants.py --check, tools/spec_engine_coverage.py --check, tools/repin_us_pool_engine_contracts.py --check
• the test files the change touches, across the engine-free, engine-contract and engine-workflow tiers and the H3 graph parity test (rerun on the final head)
• full engine-free and engine-US jobs on the final head (running)
• third-round independent review

CI, once this targets main:

• engine-free and engine-US jobs green on both Python versions
• lint and wheels

🤖 Generated with Claude Code

[MaxGhenis]

The base of this PR, #1189, moved to head 3cba55254. Decision d1317 (2026-10-11) set the stage's scale anchor to BEA NIPA 7.8 line 17 (B4923C) instead of CMS NHE Table 24, and the US feed is re-pinned. What this PR needs on its next rebase:

• esi_premiums.py. EMPLOYER_PREMIUM_ANCHOR is now BEA, with values for 2023, 2024 and 2025 and a new feed_record_set key. EMPLOYER_PREMIUM_CROSS_CHECK is now NHE, with values for 2023 and 2024. _anchor keeps its signature and returns BEA values, and 2025 is now a valid build year. The anchor gate's cross_check_ratio is the total over NHE. There is a new us_esi_premiums_anchor_feed_mismatches(facts), which the release tool calls after the feed pin check. Beyond the constants, messages and docstrings, _person_raw_shares, the handler's scale line, us_esi_premiums_summary and the gate structure are unchanged.
• Numbers. The single-source column is $863.9B on the default pool (scale factor 0.9708), not $925.7B. Pool fill receipts move by the same factor, 977.034 / 1,047.0.
• Feed. us/chronicle_feed.json pins Chronicle cf9ece2 (40,181 rows, facts_sha256 054e5684…). The scope has 640 pairs and 66 runs, and target_parity_manifest.json has 82 feed families.
• Spec bundle. spec_sha256 is b7311566… and coverage is 42,217 fields with 41 inventory checks. The source_stages claim is 1,749 / d0979722…, and the inventory digests changed. Seed sites are unchanged at 53. The digest re-pins in this PR need one more pass.
• Main. It was merged in at 909586128, which brings in Bind state × AGI-band SOI targets and re-pin the US feed for TY2023 state bands (#940) #1102.

A round-4 independent review is running on #1189.

[MaxGhenis]

One more commit on the base, #1189, now at head ebfb05582 (answers to its fourth review). On top of the notice above, a rebase needs these:

• Release tool. The feed check is now _check_us_esi_anchor_against_feed(facts, allow_unpinned_feed=..., allow_esi_premium_gaps=...) in tools/build_us_fiscal_refresh_release.py. esi_premiums.json is schema_version 3, with an anchor_feed block from _esi_anchor_feed_evidence.
• Spec bundle. spec_sha256 is now 0a5ecc7f… and the source-stage manifest digest is 559062a1…: a note in us/source_stages.json changed wording. Three inventory digests in spec_engine/inventory_coverage.py moved again (full_checkpoint, late_resource_semantics, source_manifest). Field counts and the source_stages claim are unchanged from the notice above.
• Target parity. tools/build_us_target_parity_manifest.py has a new _ESI_NHE_ORIGIN, and the manifest is regenerated.

esi_premiums.py itself did not change in this commit.

[MaxGhenis]

The base, #1189, is now at head dc8578d05 and is no longer conflicting with main. It merges main at 123927679 and answers its fifth review's nits. A rebase onto it needs these:

• Spec bundle. spec_sha256 is 41d89d1b…. Main's PUF-support channel erases CPS-reported Social Security for 11.6M people, inflating SSI eligibility #1178 and this branch both move the seed digests, so seed_protocol is 303da988… and seed_map is 08698f44…. source_manifest stays 629a0712…, and the golden semantic hash is 707b3049…. Field counts are unchanged (42,217; 41 inventory checks).
• Release tool. The dry-run report's inputs now carries esi_anchor_feed, and its waivers list allow_unpinned_feed.

esi_premiums.py is unchanged since 3cba55254. A round-6 delta review is running on the merge.