Produce ESI employer premiums scaled to BEA B4923C from restored ASEC policyholder fields (#454)

changelog.d/us-esi-employer-premiums-454.added.md

@@ -0,0 +1 @@
+The US base build produces `employer_sponsored_insurance_premiums` (microcosm #454). The `meps_esi_premiums` stage, declared but unable to run because the pinned ASEC inputs lack the Census policyholder fields, now reads `NOW_OWNGRP`, `NOW_HIPAID`, `NOW_GRPFTYP2`, `PEMLR` and `NOEMP` restored from the pinned Census person files. Every current ESI policyholder takes the MEPS-IC 2025 employer share for their coverage tier, firm size and State (census division for government employers), and one factor scales that whole universe to BEA NIPA Table 7.8 line 17 (B4923C, $977.0B for 2024), which counts contributions for active and retired employees (decision d1317). The column carries the employed policyholders' part ($863.9B on the default pool, before calibration); dependents, non-employed policyholders and the unincorporated self-employed carry zero. The US Chronicle feed is re-pinned to banked B4923C facts (Chronicle `cf9ece2`, three facts added, every other fact byte-identical), and the release tool refuses a feed whose values differ from the stage's. A release must carry the input with signal and its raw ASEC coverage columns, and hold the anchor-universe total within 5% of the anchor on the dense and the sparse export; the verdict ships as the `esi_premiums` gate evidence, with CMS NHE Table 24 and MEPS-IC's active-employee total as recorded cross-checks. The multispine pool does not run the stage yet, and an ACS local build refuses a frame whose input is null on its ACS rows. `pre_tax_health_insurance_premiums` is not produced: it waits for the engine's disjoint premium contract (policyengine-us #10046).

docs/evidence/spec-engine/us-f0-coverage.json

@@ -4,9 +4,9 @@
     "version": 1
   },
   "country": "us",
-  "documentation_sha256": "0931cbd3aae2abcd8c072b78eba40c1c006b1bf7fa23c6a6681f9a895e47070e",
+  "documentation_sha256": "44f40b4e8aaa2e6be67a7bc9680cc29f88e185c10573ffb9c3f9dfbd3ad1081f",
   "field_usage": {
-    "authored_normative_field_count": 32404,
+    "authored_normative_field_count": 32437,
     "claim_count": 49,
     "claims": [
       {
@@ -514,8 +514,8 @@
         ],
         "mode": "legacy_behavior",
         "pointer_class": "all",
-        "pointer_count": 1716,
-        "pointer_sha256": "3e808ed9f5a791c0dfb85be0c59adf803366bf6e8c7a9f5a58f92974e044b607",
+        "pointer_count": 1749,
+        "pointer_sha256": "d0979722ea5153bd152e5e1c1201f5d530d11b21aecad18aca38473962caa062",
         "rationale": null,
         "relative_sink_prefix": "/source_manifest/stages",
         "source_prefix": "/authored/spec~1sources.yaml/stages",
@@ -772,20 +772,20 @@
         "verifier": "vintages"
       }
     ],
-    "configuration_field_count": 42184,
-    "consumed_field_count": 42184,
+    "configuration_field_count": 42217,
+    "consumed_field_count": 42217,
     "generation0_effect_counts": {
-      "legacy_behavior": 38498,
+      "legacy_behavior": 38531,
       "no_generation0_effect": 3686
     },
     "mode_counts": {
       "compiler_semantic": 27717,
       "front_end_validation": 354,
       "identity_only": 103,
-      "legacy_behavior": 14010
+      "legacy_behavior": 14043
     },
     "multiple_primary_use_field_count": 0,
-    "pointer_inventory_sha256": "8f655670b9794adae905ede64bdceba6395eeb8d642b6c5896d5c516f1f6a6a7",
+    "pointer_inventory_sha256": "c1e2a118a0d53db0bb628b14357a2c86db2956332f5b359c9092764f9c451a95",
     "resolved_binding_field_count": 9780,
     "unused_field_count": 0
   },
@@ -1361,12 +1361,12 @@
         ],
         "expected": {
           "producer_count": 38,
-          "sha256": "cc34ba11288d01145fc2dc1a669df3f7c4ef2019d73afb02dd8eeee852b16d20"
+          "sha256": "cbac7c8fbdbb82f0f06ab356d0c9f0d3645ca930550b44af2272d93315f2e934"
         },
         "failures": [],
         "observed": {
           "producer_count": 38,
-          "sha256": "cc34ba11288d01145fc2dc1a669df3f7c4ef2019d73afb02dd8eeee852b16d20"
+          "sha256": "cbac7c8fbdbb82f0f06ab356d0c9f0d3645ca930550b44af2272d93315f2e934"
         },
         "status": "covered"
       },
@@ -1656,13 +1656,13 @@
           "compiler_ir.node_slices"
         ],
         "expected": {
-          "map_sha256": "7d03cc07e973165f38781d5932d5fe6d16ca0b1e4dc854e7841460737fd043fb",
-          "protocol_sha256": "37da8f68444677850a97bdf958b7f62b426a60da570b80a002eeba8af4861f68"
+          "map_sha256": "08698f4440eb31d1e2886036e7fd0ab86d4c6a007deaa650127dc4346a2151e0",
+          "protocol_sha256": "303da98866edb2a87b1dc36d9ff5a0146deb3a68a4f5361416b5794c13ae4cfb"
         },
         "failures": [],
         "observed": {
-          "map_sha256": "7d03cc07e973165f38781d5932d5fe6d16ca0b1e4dc854e7841460737fd043fb",
-          "protocol_sha256": "37da8f68444677850a97bdf958b7f62b426a60da570b80a002eeba8af4861f68"
+          "map_sha256": "08698f4440eb31d1e2886036e7fd0ab86d4c6a007deaa650127dc4346a2151e0",
+          "protocol_sha256": "303da98866edb2a87b1dc36d9ff5a0146deb3a68a4f5361416b5794c13ae4cfb"
         },
         "status": "covered"
       },
@@ -1677,7 +1677,7 @@
           "compiler_ir.seed_stream_map"
         ],
         "expected": {
-          "implementation_sha256": "37da8f68444677850a97bdf958b7f62b426a60da570b80a002eeba8af4861f68",
+          "implementation_sha256": "303da98866edb2a87b1dc36d9ff5a0146deb3a68a4f5361416b5794c13ae4cfb",
           "protocol": "legacy-v1",
           "streams": [
             "build_model",
@@ -1698,7 +1698,7 @@
         },
         "failures": [],
         "observed": {
-          "implementation_sha256": "37da8f68444677850a97bdf958b7f62b426a60da570b80a002eeba8af4861f68",
+          "implementation_sha256": "303da98866edb2a87b1dc36d9ff5a0146deb3a68a4f5361416b5794c13ae4cfb",
           "protocol": "legacy-v1",
           "streams": [
             "build_model",
@@ -1775,12 +1775,12 @@
           "legacy_adapter.imputation.late_producer_resource_semantics"
         ],
         "expected": {
-          "sha256": "f4f8bfeb79268a4043a843b47cec1d1c0e58e9f5dd6bc23417ada95446dd7b8e",
+          "sha256": "629a0712659a26b2d6aa6c0e5672421f415d625e0599d6baa0a834d118953c6a",
           "stages": 37
         },
         "failures": [],
         "observed": {
-          "sha256": "f4f8bfeb79268a4043a843b47cec1d1c0e58e9f5dd6bc23417ada95446dd7b8e",
+          "sha256": "629a0712659a26b2d6aa6c0e5672421f415d625e0599d6baa0a834d118953c6a",
           "stages": 38
         },
         "status": "covered"
@@ -1887,7 +1887,7 @@
             "alpha",
             "zeta"
           ],
-          "sha256": "3db095ce2d1ef89028d3862f0edc827d704f099c5b3323f9b115642a3c27e007"
+          "sha256": "e5da85693fab1f1cba9c37beba5cdf91477962d466bc21e9f29125f2f90b7e79"
         },
         "failures": [],
         "observed": {
@@ -1911,7 +1911,7 @@
             "alpha",
             "zeta"
           ],
-          "sha256": "3db095ce2d1ef89028d3862f0edc827d704f099c5b3323f9b115642a3c27e007"
+          "sha256": "e5da85693fab1f1cba9c37beba5cdf91477962d466bc21e9f29125f2f90b7e79"
         },
         "status": "covered"
       },
@@ -1975,7 +1975,7 @@
             "take_up_contract",
             "us_qbi_reconciliation_contract"
           ],
-          "sha256": "61a5484eab9fae1d6651b3ae15f1793d4934ea2f6ed2c08d05716b745c221014"
+          "sha256": "fb863f18a79610dc1005089a32e1535c502c5b9f7e5ba589db0d6025b01e3c82"
         },
         "status": "covered"
       },
@@ -2599,7 +2599,7 @@
       "country": "us",
       "schema_id": "country_spec",
       "schema_version": 1,
-      "spec_sha256": "fbf25dbae68242198972f5bef4f1fc20f65ee0769615b8d94639e6a80121f31f"
+      "spec_sha256": "41d89d1b74bc3aec02b8dd9c3b3004928bfb0eface7bb670827289e75ab31ab8"
     }
   },
   "report_schema_version": 3,
@@ -2609,7 +2609,7 @@
     "country": "us",
     "schema_id": "country_spec",
     "schema_version": 1,
-    "spec_sha256": "fbf25dbae68242198972f5bef4f1fc20f65ee0769615b8d94639e6a80121f31f"
+    "spec_sha256": "41d89d1b74bc3aec02b8dd9c3b3004928bfb0eface7bb670827289e75ab31ab8"
   },
   "status": "pass"
 }

docs/us-asec-census-person-columns.md

@@ -198,3 +198,43 @@ the Census recode.
   Census recode restored, `A_EXPRRP == 13` now appears in every vintage, but
   it still conflates partners and roommates, so the exclusion's conclusion
   stands.
+
+## Columns added for the ESI premium stage (#454)
+
+Added 2026-10-10. The `meps_esi_premiums` stage
+(`docs/us-esi-employer-premiums.md`) reads six Census person columns that
+**none** of the four pinned inputs carries, the 2024 and 2025 files included,
+so the restoration appends them for every pooled vintage and never has an
+input copy to verify against.
+
+| column | restored | reader |
+|---|---|---|
+| `NOW_OWNGRP` | yes | `esi_premiums._esi_person_codes` → the current employment-based policyholder |
+| `NOW_HIPAID` | yes | same → employer paid all, some or none of the premium |
+| `NOW_GRPFTYP2` | yes | same → coverage tier (family, self plus one, self-only) |
+| `NOW_GRPFTYP` | yes | same → cross-checked against `NOW_GRPFTYP2` |
+| `PEMLR` | yes | same → employed at interview |
+| `NOEMP` | yes | same → employer size band |
+
+The reviewed domains are the Census API codebooks for survey years 2023 to
+2026 (`api.census.gov/data/<year>/cps/asec/mar/variables/<column>.json`),
+which are identical across the four years for each column. The data also
+carry `PEMLR == 0` (children and Armed Forces), which the codebooks omit, so
+the `PEMLR` domain is 0 to 7.
+
+Observed in the pinned person members (rows, by exact `PERIDNUM` join):
+
+| income year | member | persons | policyholders (`NOW_OWNGRP` 1) | `NOW_HIPAID` all / some / none | `NOW_GRPFTYP2` family / self plus one / self-only | employed (`PEMLR` 1-2) |
+|---|---|---:|---:|---|---|---:|
+| 2022 | `pppub23.csv` | 146,133 | 39,838 | 8,470 / 27,889 / 3,479 | 10,677 / 7,652 / 21,509 | 68,695 |
+| 2023 | `pppub24.csv` | 144,265 | 39,161 | 7,843 / 28,047 / 3,271 | 10,393 / 7,386 / 21,382 | 68,511 |
+| 2024 | `pppub25.csv` | 142,125 | 38,554 | 7,748 / 27,378 / 3,428 | 10,176 / 7,228 / 21,150 | 67,436 |
+| 2025 | `pppub26.csv` | 134,729 | 36,185 | 7,449 / 25,717 / 3,019 | 9,623 / 6,806 / 19,756 | 63,429 |
+
+In every member the payment-status and tier counts sum to the policyholder
+count, and everyone else carries 0 in both. The stage refuses a frame where
+that does not hold.
+
+The older note that the 2024 and 2025 inputs "carry all 18" `NOW_*` recodes
+counts the coverage-type recodes reviewed for #720. `NOW_OWNGRP`,
+`NOW_HIPAID`, `NOW_GRPFTYP` and `NOW_GRPFTYP2` were never among them.

docs/us-chronicle-feed-repin.md

@@ -13,6 +13,53 @@ any other feed, and `test_us_chronicle_feed.py` fails if the parity resources,
 the generator and the pin disagree, or if the scope file changes without a
 new pin.
 
+## The 2026-10-11 re-pin: BEA private group health insurance (microcosm#454)
+
+Decision d1317 made BEA NIPA Table 7.8 line 17 (series B4923C) the scale
+anchor of the `meps_esi_premiums` stage (`docs/us-esi-employer-premiums.md`)
+and asked for the feed to carry the banked fact. The pin moved from `51aa40f`
+to `cf9ece2`, a commit that is `51aa40f` plus two commits (Chronicle branch
+`feed/us-51aa40f-esi-b4923c`, tag `microcosm-us-feed-454-v1`), not Chronicle
+main:
+
+- The digit-text guard match from Chronicle main's `aa3e4de3`, and nothing
+  else from that commit. BEA's Section 7 workbook writes its year headers as
+  text (`'2024'`), and at `51aa40f` no guard could match them. Only a guard
+  that used to fail can pass, so every package that built before builds the
+  same cells.
+- The `bea-nipa-private-group-health-insurance` package as
+  PolicyEngine/chronicle#331 merged it to main, with its concept evidence
+  (BEA's estimates cover active and retired employees). The test file drops
+  main's artifact-year restamp test, whose guard (chronicle#292) postdates
+  this base, and the package is not added to the US poverty hard-target
+  coverage contract, as on main.
+
+The scope adds three pairs, `bea_nipa.cy<year>.private_group_health_insurance`
+for 2023, 2024 and 2025, each built at its own year (640 pairs, 66 package
+runs). Built at `cf9ece2`:
+
+- the feed keeps all 40,178 `51aa40f` rows byte for byte (joined on
+  `aggregate_fact_key`) and adds exactly the three B4923C facts: $923,195M,
+  $977,034M and $1,027,929M, each from `Section7All_xls.xlsx` (sha256
+  `de1c34e3…`);
+- compiled at 2024 with the packaged CD crosswalk and the Medicaid
+  substitutions, with target aging on and again with it off, the registry is
+  the same ordered list of targets on both feeds, name, value and metadata
+  (33,250 targets with aging on). B4923C is not compiled into a target: the stage scales to it,
+  and whether to calibrate the column is decision d1264. The target-parity
+  manifest lists it as a `deferred` reviewed exclusion (82 feed families, 32
+  compiled, 53 reviewed exclusions; it was a source-absent family before);
+- two independent rebuilds gave the same `facts_sha256`.
+
+The comparison scripts and their reports are in
+`experiments/us-esi-454/feed-repin/`. The release tool now refuses a feed
+whose B4923C values differ from the stage's pinned anchor
+(`us_esi_premiums_anchor_feed_mismatches`); a diagnostic run under
+`--allow-unpinned-feed` or `--allow-esi-premium-gaps` continues and records
+the difference in `esi_premiums.json` (in the report, for a dry run). A later re-pin that changes other rows
+(microcosm#1192 re-dates the W-2 item facts) must carry the three B4923C
+pairs and redo the row comparison against its own previous feed.
+
 ## The 2026-10-04 re-pin: TY2023 state AGI bands (microcosm#940)
 
 The state x AGI-band targets of #940 need the TY2023 Historic Table 2 state
@@ -54,20 +101,22 @@ the first labelled US export.
 
 | Field | Value |
 |---|---|
-| Chronicle commit | `51aa40fd28e2ca81937752515e773d123fecfb71` (`c5e5bf8` plus the TY2023 state AGI package; tag `microcosm-us-feed-940-v1`) |
-| Feed file | `consumer_facts_us_51aa40f.jsonl`, 40,178 rows, 169,038,757 bytes |
-| `facts_sha256` | `965a29ac9458edb58a05184917ee4a19772fd3dc7b4588db6eb6da5803641aac` |
+| Chronicle commit | `cf9ece2e7be11158c21fdc4dd38b44d92db552f6` (`51aa40f` plus the digit-text guard and the B4923C package; tag `microcosm-us-feed-454-v1`) |
+| Feed file | `consumer_facts_us_cf9ece2.jsonl`, 40,181 rows, 169,055,585 bytes |
+| `facts_sha256` | `054e5684005a7b143dcb84eabac634135ce9b2b33bb57b540602e474a4ba7c57` |
 | Consumer fact schema | `chronicle.consumer_fact.v3`, schema file sha256 `bdb51e2a…` (unchanged from the UK pin) |
-| Scope | 637 (record set, period) pairs; 63 package runs over build years 2020 to 2029 |
+| Scope | 640 (record set, period) pairs; 66 package runs over build years 2020 to 2029 |
 | Consumer artifact | none: refused at this commit, see below |
 
-The previous pin was `c5e5bf8aa84960c1a200ee47303b19c953092d0f`
-(`consumer_facts_us_c5e5bf8.jsonl`, 39,158 rows, `facts_sha256`
-`b8543739…`, 586 pairs, 62 runs); the rest of this document describes its
-2026-09-18 rebuild unless it says otherwise.
+The previous pins were `51aa40fd28e2ca81937752515e773d123fecfb71`
+(`consumer_facts_us_51aa40f.jsonl`, 40,178 rows, `facts_sha256`
+`965a29ac…`, 637 pairs, 63 runs) and, before it,
+`c5e5bf8aa84960c1a200ee47303b19c953092d0f` (`consumer_facts_us_c5e5bf8.jsonl`,
+39,158 rows, `facts_sha256` `b8543739…`, 586 pairs, 62 runs); the rest of this
+document describes the 2026-09-18 rebuild unless it says otherwise.
 
 The feed is too large for the repository. Its home on the build machine is
-`~/PolicyEngine/_buildh-runtime/inputs/consumer_facts_us_51aa40f.jsonl`,
+`~/PolicyEngine/_buildh-runtime/inputs/consumer_facts_us_cf9ece2.jsonl`,
 beside the previous pins; `tools/build_us_target_parity_manifest.py` reads it
 there by default. A holder of the Chronicle commit regenerates it byte for
 byte with the commands below.