Triage policyengine-us-data backlog for Populace migration #46

Context

policyengine-us-data is no longer the live build surface for replacement microdata. Populace supersedes it, so open policyengine-us-data issues/PRs should be triaged into Populace work only when the underlying data-quality, target, export, or release-contract concern is still relevant.

This issue records a pass over the live open PolicyEngine/policyengine-us-data issues and PRs on 2026-06-14, after Populace PRs #44 and #45 merged.

Already covered or mostly covered in Populace

• policyengine-us-data Generate complete ACS WIC participation and reject missing release inputs #1156 / PR Say what #720 changed in the unmarried-partner gap's reason #1157: microunit tax-unit construction. Populace already delegates US tax units and SPM units to microunit through populace-frame.
• policyengine-us-data Score head-to-head household slices in parallel with --workers, bit-identical to sequential #1171 / PR Score head-to-head household slices in parallel with --workers, bit-identical to sequential #1171: negative auto_loan_interest from SCF sentinels. Populace already has manifest-declared sentinel handling plus nonnegative source/export gates for auto_loan_interest.
• policyengine-us-data ACS local λ frontier on the weighted loss: re-pick λ and measure the district population cost #1105, Align Child Benefit claim exports and opt-out draws with the UK model #1107, Seed protocol: declare draw_condition for zero-target humanitarian sites #872, Bind SSA SSI age-band recipient counts as ordinary indicator targets (#470) #476, CHIP enrollment targets are concept-mismatched: CMS total_chip_enrollment includes M-CHIP; model materializes separate CHIP only (20 states zero-support) #321, UK: household-wealth imputation stage with cash / S&S ISA split #181, social_security stored input contradicts its four components for ~2k records (decomposition gap) #183, Add proximal L1 solver path #184, Add fiscal diagnostics comparison tool #190, Remove count aggregation from requirements example #192: fiscal/SOI/JCT/state-tax target surface. Populace US calibration does not pin federal income tax or AGI; base undershoots income tax ~18% — add SOI fiscal targets + a macro-realism gate #40 / PR Restore US fiscal calibration targets #44 added federal income tax, JCT tax-expenditure reform contracts, state income tax, SOI rows, and macro-realism gates. Keep remaining target refreshes as future target-vintage work, not old-repo work.
• policyengine-us-data Make zero-valued calibration targets intentionally nonzero-weighted #104, Doctrine: survey-derived statistics (poverty above all) are permanent calibration holdouts #596 and PE-US #8614 follow-up: partnership/S-corp formula aggregate vs leaves. Populace Propagate partnership/S-corp income split (policyengine-us#8614) into the US build #35 / PR Enforce partnership S-corp leaf exports #45 now exports the PE-US leaves and blocks partnership_s_corp_income as a formula-owned compatibility column.
• policyengine-us-data Stop telemetry test servers when a test body fails #1165 and PR Load native ACS disability and weeks-worked inputs in the ACS local lane (#1021) #1054 concepts: target coverage / target-policy reporting. Populace Use Arch US poverty/nonfiler coverage contract in Populace target gates #29 and Add NIPA and program-admin nonfiler controls to the US target profile #30 already track source-family coverage and hard-target inventory. Copy only the useful fail / warn / diagnostic_only policy concept if Populace needs more granular enforcement.

Copy into existing Populace issues

• Populace Add NIPA and program-admin nonfiler controls to the US target profile #30: program-admin hard targets and nonfiler controls.
  • Copy/source-link policyengine-us-data Publish UK execution graphs through a general build emitter service #1148 / PR Stamp migrated-spool fixture rows inside the retention window #1162 for CHIP enrollment controls.
  • Copy/source-link Make UK build outcomes honest in staging, telemetry and the Logbook #1147, Import-entry unit family P1: ledger ingest of official trade margins (#615) #620 / PR Weight UK BRMA assignment by census private-rented households #1087, Align UK CGT calibration with FY2024-25 individual observations #889, Restore the SIPP donor half of the financial-asset blend (50/50 SIPP+SCF per household) (#374) #510 for ACA marketplace/APTC take-up and state targets.
  • Copy/source-link Count U.S. savings bonds in the SCF half of bond_assets #1136, Retirement surfaces are owned by the base builder: no release-time refit #557 for Medicaid category/enrollment reconciliation.
  • Copy/source-link US pool: persist weeks_worked now that policyengine-us has no formula for it (SNAP work tests read it since pe-us#9716) #1094, Assign US congressional districts from Ledger facts #188, Add opt-in ACS congressional district age targets #189, [corrected] populace_us_2024 itemized-deduction report retracted (measurement artifact); residual = state-level SOI income targets #122, ACS local lane: load native WKWN (weeks worked), MIL (veteran) and disability items for SNAP work-requirement tests #1021 for SSI/OASDI/disability support and take-up.
  • Copy/source-link Migrate the US stacked driver onto microcosm.build.logbook_adoption #689, Standing per-component repeal-revenue validation surface (#298) #535, Port Section 199A simulation and audit processed PUF inputs #532 for LIHEAP/TANF/WIC/SNAP/take-up details where they map to source-backed targets.
• Populace Track SPM-specific input gaps: housing, WIC, school meals, child support, workers comp, and expense deductions #32/Add a Populace US poverty/SPM diagnostic gate for resource distribution and negative-income tails #33: SPM/resource diagnostics.
  • Copy/source-link UK domestic electricity and gas: record why ONS 04.5.1/04.5.2 is preferred over DESNZ × QEP, reconcile the two, and declare the price basis PE-UK should project from #1113, WS-C deferred adjudications and the reconciliation layer #736, Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #733, Complete #622: compile the UK target surface from Ledger facts at the 2024-25 line #735, Add Logbook family model and typed exact-k cardinality #739, Port the CGT structure, salary sacrifice, and student loan stages onto the UK spine at FRS 2024-25 #740, Repair the stale lock: microcosm-frame's declared uk extra #732, Treat PE-US aggregate outputs as formula-owned #182, Refit crushes keogh carrier weights to $6.1M of $148.97M — smoke floor unreachable on the sparse arm (attempt 15) #445, Bind every blocking gate verdict into release certification: waived gates vanish from manifests and publication accepts any nonempty gates mapping #381 for housing assistance, SPM expenses, child support, CE/MEPS/AHS-style source gaps and validation-only diagnostics.
• Populace Populate remaining US tax-input and reported-observation layers #38: remaining US tax-input and reported-observation layers.
  • Copy/source-link UK: extended-hours usage draw (mean ~15 of 30) leaves free early-years spending at £5.8bn vs DfE £9.9bn #1126, UK: source the extended-hours usage draw from DfE additional-hours PTEs (#1126) #1127 / PR Refuse scale, sum, share and fraction on concepts that are not annual flows #1128, Bind UK census households from Chronicle and map NI Data Zones with NISRA's published lookup (#887) #900 for capital-gains basis / holding period / SOCA-style imputation.
  • Copy/source-link qualified_tuition_expenses essentially unimputed (641 records, $4.5B) — education credits understated ≥40% #253, Run ACA source refresh in fiscal release builder #85, Country-agnostic gate battery core: phased evaluation, unconditional reporting, per-country gate selection (#611) #619, populace-us-2024 c86a631 (2026-06-19) build stores weekly_hours_worked_before_lsr as constant 40 — SNAP work-requirement analyses silently return zero effect #613 for section 179, Alaska PFD separation, liability pruning, and top-tail AGI support.
  • Copy/source-link Draw only valid node prose in the G5a A4 property test #1160, Add calibration warm-start and target-frame checkpoints #227, Carve the PUF donor's E19200 lineage to the mortgage-only concept (#515 interim) #525, Two-signal FLSA overtime incidence: usual-hours leg joins the reference-week snapshot (#451 item 4 residual) #526, Screen aggregate-scale mortgage outlier rows out of the PUF donor (#516) #527, ORG hourly-wage QRF imputes sub-minimum wages for part-year workers (donor income is annualized-weekly; recipient income is actual-annual) #529 for tips and Stage-1 source quality failures.
  • Copy/source-link Close the schema-3 terminal-report path for non-grandfathered exact-k UK releases #712, Score legacy flat PE H5s #219, Add direct CHIP enrollment targets to the US fiscal surface #139, Allow staging telemetry repo config from environment #138, UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation #180, Remove aggregation method from Ledger target selectors #193, Score OBBBA reform validation stacked (match JCT) #185 for TIN/SSN/pay-period/pregnancy/widowship support if those remain PE-US input needs.
• Populace Drop raw-source and build-intermediate columns from the published populace-us artifact #27/Fix in_nyc before enforcing no-formula exports #34: export-surface and formula-owned storage.
  • Copy/source-link Restore the release H5 write that #437 dropped #444, UK spine: preserve FRS benefit-unit capital for UC, re-anchor take-up after SPI (#828) #829, Add Census state SPM poverty-rate backtests (104 out-of-sample validation rows) #348, UK output-area-anchored geography ladder (US block-ladder pattern) #349, EPIC: populace-am — Armenia: closing the EG DNA macro–micro gap on public data #814, farm_income is an unpinned export dimension (109 selected carriers) — parity gate fails Build M attempt 12 #441, Add UK SPI support rows #149 where they identify formula-owned exports, entity-level ID propagation, filing-status, or naming issues still relevant to Populace.

New Populace follow-up issues to consider

• PUF real-half / clone support and release gates.
  • Source: policyengine-us-data Harden Axiom adapter references and kernel hash coverage (NZ v0, G1b) #1139, Refuse unsigned ids int64 cannot hold; match concept ids as int64 #1158 / PR Add population-side transport graph kernels (NZ graph G5a) #1159, PR Reconstruct Orrery graphs from existing build output artifacts #1170, Delivery gate can prescribe a provably identical retry when the basis is already true (post-#549/#550 residual) #555, Impute capital gains amounts from the published joint distribution #560, Transfer targets must be engine input leaves: drop formula-owned weeks_worked, fix medicare premiums leaf, add ownership guard (#578 inc 2) #590, ACS transfer predictors accept semantic booleans; adjudicated null handling; full predictor sweep (#578 inc 2) #595, Rowwise local solve surface under the doctrine (#495 increment 6b) #541.
  • Populace should decide whether it needs explicit real-half vs clone-half support diagnostics, a PUF-clone weight-share floor/cap, and smoke tests for PUF-imputed charitable/business/tax-detail variables. This should stay spec-driven: country manifests can declare the support contract, but shared runtime code should implement the operation.
• Geography and local-area support.
  • Source: Give each country its own policyengine-core pin (#1086) #1143, Add country-neutral Orrery execution evidence and UK contracts #1144, UK landlords: bind the SPI net property amounts, leave property_wealth to the engine, add receipts and finance costs #1145, Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146, PUF stage manifest declares a support_clip op that no execution path honors (dead donor_realized contract) #482, ESI employer premium contributions as a first-class, validated resource component #454, Materialize Marketplace selected-plan benchmark ratios and metal-level diagnostics #75, Bind SOI Table 2.1 mortgage targets to the capped deductible concept (#511 target-side leg) #517, SSA SSI recipients by age band: mint as ledger facts, bind as registry count targets #470, Restore strike_benefits from the OI_VAL decomposition (#591 — ECPS parity) #598, PR Gate the ACS local release on district ESS collapse (report-only by default) #1088.
  • Populace should track boundary vintage, district geojson provenance, county/rating-area assignment, local H5/local-area build contracts, and matrix scaling separately from the old staged pipeline.
• Release manifest / publication contract.
  • Source: Confirm whether count/amount categories should affect US fiscal target loss weights #724, Prior-year income availability 0.0428 vs release band floor 0.05 on the stacked pool #799 / PR Set UK road-fuel and energy levels in the LCFS stage on a calendar-2024 basis (#1113) #1167, Use target-period Medicaid facts in the 2024 US fiscal refresh #91, Populate reported TANF, SNAP, and WIC enrollment from CPS admin-linkage fields (#591) #600, Logbook: record requested and realized exact-k cardinality #641, The stacked spine: one origin-labeled spine, banked gap-fill, null-vs-zero doctrine, by-origin battery (#578 revision pilot) #616, Add the L2 sweep harness over exported production calibration inputs #310, Tracked publish wrapper to wire release Slack alerts #179, Port UK compact source stages and 2024-25 refresh from uk-data #145, Cache US target reform materialization #201, Fill missing current CD targets from state totals #209, PR Gate reviewed default fills on their means-tested consumer notes (#1022) #1071, PR Derive years_since_us_entry from CPS PEINUSYR (and ACS YOEP) so policyengine-us can drop its default #1077.
  • Populace needs the useful release-manifest ideas, compatible model version metadata, diagnostics summaries, and published-run indices, but not the old eCPS Stage 5 implementation shape.
• Wealth/property/asset imputation.
  • Source: Export country-agnostic Microcosm graphs for Orrery, with the UK as the first consumer #1079, Overlay UK graph execution results in Orrery exports #1080, UK FRS spine: stop counting lodgers' rent and property losses as property income; sub-letting rent for every tenure #1081 / PR US: store years_since_us_entry from PEINUSYR in the national pool #1085, Add coverage-universe targets for uninsured and private coverage #230, Calibrate the undocumented population count as a Ledger fact (weights lane follow-up to #266) #267, Polish follow-ups from the target-aging review (#280) #281, UK spine: benefit-unit-grain SPI UC reporter draw with pre-take-up screen (#832); exact capital sentinel (#833) #835, Populate passive_partnership_s_corp_income: SCF non-active-business shape, Form 8960 level #734.
  • Populace already declares scf_primary_residence_value; still verify whether PE-US-facing primary_residence_value, home equity, vehicle variables, auto-loan eligibility, net-worth decomposition, and Fed Financial Accounts benchmark replacement need dedicated Populace tasks.

Move outside the live Populace repo

• policyengine-us-data Add UK target-weight rules, size-stage L2 and a selection experiment harness (#1124) #1163 / PR UK: move the LCFS stage to the Living Costs and Food Survey 2024-25 #1164: production eCPS baseline resolver and integrity gate. The lesson is important, but incumbent/eCPS comparison should live in PolicyEngine/populace-benchmarks or another benchmark repo, not in live Populace build code.
• Any remaining eCPS/local-baseline promotion comparisons should be benchmark-repo work only. Populace release gates can expose diagnostics; they should not depend on a stale incumbent artifact.

Do not migrate directly

• Microplex-specific issues such as Cite code by symbol, not line number, in the NZ benefit-unit rule #1142 and populace_us_2024 under-1 geography: CA carries 21.7% of US infants (~1.9x its population share), MI 1.0% (a third of its share) #520.
• L0/national-penalty work such as PR Keep the telemetry emitter service off the modelling stack #1166 and PR Net-new UK stages: UC deduction attributes, bus fares, WAS debt #685, except for high-level calibration-policy lessons already captured above. Populace should stay on its current calibration stack and spec-driven country content.
• Old staged-pipeline service-boundary PRs/issues (Add Axiom graph adapter surfaces and simulate.rules_by_ref@1 (NZ v0, G1) #1118, reform_validation registry: four state rows measure the wrong quantity at policyengine-us 2.2.1 (IA HF1020, MN WFC, ID CTC, CO CTC), and state SPM rate rows need a sampling SE #1116, Weight the ACS local release's calibration targets with the national release's loss formula, from one shared implementation #1104, UK: port the unblocked uk-data fixes tracked in #1095 #1100, US export household subsampler and post-export probe #1098, policyengine-uk 2.102.4+ needs policyengine-core 3.32.9+, which the certified US default refuses #1086, Record process telemetry for US release builds #1083, Chi-square design-weight penalty and softmax mass parametrization for calibration; ACS local ESS frontier #1078, Calibrate US federal student loan balances to Federal Student Aid portfolio targets and show them on the dashboard #1074, Carry SCF education debt into the US build and allocate federal student loan balances to borrowers #1073, Run Route A's base stage on Modal (us-puf-support-base) #1066, UK student-loans realization gate refuses every spine since #1006: the PLAN_5 top-up shortfall is below one person's weight #1049 and related PRs Add the New Zealand spec package for the graph build (NZ v0, G2) #1119, Tax-input coverage gaps surfaced by the reform-validation registry on us-6.2.1 (tracker) #1117, UK: model landlord property income on the taxable (net of expenses) concept, not rent received #1106, Record the export-probe tools' source at load; add the Route A p = 0.05 probe record #1101, Add always-on build telemetry delivery #1099, UK pensions: resident State Pension, Pension Credit, contributions and salary-sacrifice relief (#1069) #1084, Keep US staging telemetry on for every build #1082, UK State Pension projection bundle: entitlement sidecar, abroad dataset, manifest and scorecard for PolicyEngine UK #1076, UK static aging: ONS 2024-based projection reader and weights-only annual projections to 2050-51 #1075, Seed ACS-row SNAP/TANF take-up in the ACS local lane (#1019) #1051, US pensions: every CPS/PUF/ACS pension dollar lands on the private leaves, so public-pension state exemptions never apply #1068) unless a specific contract concept is missing in Populace.
• Temporary threshold loosening / broken eCPS build patches (Draw only valid node prose in the G5a A4 property test #1160 / PR Add the transport skeleton composer, rules registry and local build driver (NZ graph G6) #1161) except for the underlying target or source-quality causes listed above.

Acceptance criteria

• Existing Populace issues above are updated or linked with the relevant source backlog.
• Any missing Populace issues from the follow-up section are opened or explicitly rejected.
• No old policyengine-us-data, Microplex, L0, or eCPS comparison implementation is copied into live Populace without translating it into Populace's spec-driven architecture.
• Benchmark-only eCPS comparison work is tracked outside live Populace.