Repository navigation
Export CPS parent pointers as person-level parent ids - #1032
Merged
Merged
Conversation
Resolve PEPAR1/PEPAR2 against (PH_SEQ, A_LINENO) to the co-resident member's person_id and export them as parent_1_id / parent_2_id, the relation behind own_children_in_household rather than only its cardinality (microcosm#884, consumed by policyengine-us#9404). - eligibility_inputs: one shared pointer normalization behind both the count and the ids, int64 storage (a clone-remapped person_id runs past 2**53), and gate invariants asserting co-residence, no self-parenting, and that own_children_in_household equals the number of people whose parent ids name that person. - acs_inputs / operator_boundary: the ACS spine declares the pointers unknown (0) natively. Its synthesized PEPAR values are a reference-person link, not the ASEC measured pointer, and an ACS-minted person_id is renumbered by the assembly offset after that mapping runs. - multispine_pool: the parent ids are native-complete on both arms, so they never enter the QRF transfer plan. They are not PolicyEngine-US variables, so _target_encoding would treat an id as continuous and hand an ACS child a fractional interpolation between two ASEC person ids. The transfer surface is unchanged at 118 targets. - puf_support: the support clone now shifts person-referencing columns with person_id, preserving the 0 sentinel; without it a clone's child pointed at the other arm's parent. - source_stages.json + regenerated sources.yaml, with the frozen generation-0 digest re-pinned. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Stage tests: two-parent, single-parent, three-generation (the grandchild names only the parent), unresolved and absent pointers, int64 dtype, person_id-not-line-number, missing person_id, and a repeated (PH_SEQ, A_LINENO). - Clone tests: each support arm's child names that arm's parent, and the 0 sentinel is not shifted into a valid-looking id. - Gate tests: a pointer naming no person, one outside the household, self-parenting, a count disagreeing with the pointers, a null id, and the person_id-0 case, which cannot be named by a 0-means-unknown pointer and is reported rather than failed. - ACS tests: the native mapping writes the declared unknown sentinel, the receipt satisfies the operator boundary, and the columns stay absent when the raw pointers are. - us_input_mass_totals now drops the person-referencing id columns: the weighted sum of an arbitrary integer label is not a mass, and it moves whenever person ids are renumbered, so comparing it against a reference release on a different id space would fail for no reason. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…sign The new eligibility-stage outputs move the source_stages asset digest, the bundle spec_sha256 (now d3469f47...), and, because puf_support is an attested kernel module, the seed-protocol, seed-map, resource-semantics, source-manifest and full-checkpoint identity digests. Re-pinned each at its observed value, bumped the field-usage counts by the four new authored fields, re-pinned the source_stages pointer claim and pointer inventory, and regenerated docs/evidence/spec-engine/us-f0-coverage.json with tools/spec_engine_coverage.py. The operator-ordering doc's eligibility row now says why the stage is pre-clone and CPS-scoped, and that its outputs name a person_id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sumers Independent review (2026-09-25) returned request-changes with seven findings and the rebased branch failed CI in four lanes. Changes: - Capital-gains tail clone shifts parent ids with person_id, keeping the 0 sentinel (_shift_person_references); the preassembled PUF tax-detail rows, which never had a pointer, carry 0 rather than null through the support clone's reference remap. - The release gate no longer demands own_children_in_household equal the pointers naming a person. ACS rows impute the count with pointers declared unknown, and a clone moves the unnameable id-0 parent, so the gate asserts pointed_at_by <= count; the exact equality is asserted where the relationships are measured, when the stage derives them. - A frame that already carries the pre-#884 outputs with signal (a --base-h5 input) gets only parent_1_id/parent_2_id appended, resolved from the raw pointers when present and 0 otherwise; no other column is recomputed. Parent ids leave the non-constant release set: all-zero means unknown. - Co-residence compares exact int64 household ids (float64 conflated ids past 2**53), and a new check requires every person_id in a household carrying a parent link to fit PolicyEngine-US's int32 storage. - Pool resume ignores a transferred/simulated checkpoint whose recorded person schema lacks an eligibility output, since the byte-stable legacy identity cannot see new outputs. - The input-mass exclusion moves into the shared helper as PERSON_REFERENCE_COLUMNS, restoring main's exact US alias. - Re-pinned at observed values after the rebase and these edits: spec_sha256 1cf1f44d..., the five inventory digests, field-usage mode/effect counts, and the loader golden vector (attested kernel sources moved). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
export-parent-ids-884
branch
from
September 27, 2026 06:19
11cad61 to
808ee8f
Compare
…ional Round-2 review of #1032: - Main's stored-input contract (#1031) refuses a stored column that is neither a variable of the certified engine (policyengine-us 2.2.1) nor registered. parent_1_id/parent_2_id now have register entries naming their producer and the removal condition (the pinned engine defining them, which policyengine-us#9406 does); the producer-binding tests cover them. - Existing frames with the pre-#884 outputs (a --base-h5 input, an existing pool) pass through untouched instead of having ids appended: resolving raw pointers there keyed on (PH_SEQ, A_LINENO) pairs that repeat after cloning and could export ACS reference-person links. The release gate treats the ids as optional and asserts their identities only when both are present, so existing pools still feed releases. - A vacuous test assertion now asserts the gate passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d file Round-3 review of #1032: two microcosm-data tests require every register entry to be stored by an H5 file examined for #1026, and no examined file stores the parent ids yet. Following the *_spine precedent, the tests name them as producer-bound entries (bound to eligibility_inputs by test_us_stored_input_register.py), and the register docstring says so. Also fixes a gate assertion that filtered on text no message contains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard treated any transferred/simulated checkpoint lacking an eligibility output as stale, so the checkpoint round-trip tests' synthetic fixtures (which carry no eligibility columns) were rebuilt instead of resumed, and it still required the parent ids after they became optional. It now marks stale only a checkpoint that carries the pre-#884 eligibility outputs but not the parent ids: the shape main writes, where resuming would build without ids that a fresh run produces. Anything else is left to the existing identity and schema checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes microcosm#884.
What this does
Exports the CPS ASEC parent pointers as two person columns,
parent_1_idandparent_2_id: each is theperson_idof the co-resident member that the person'sPEPAR1/PEPAR2pointer names withinPH_SEQ, and 0 when the pointer is absent or names no co-resident line. Until now Microcosm exported only the count behind them (own_children_in_household), so a rules engine could ask how many children an adult has but not whose child a person is. policyengine-us#9406 consumes the ids for Medicaid MAGI household composition (42 CFR 435.603(f)(3)).eligibility_inputs.py): one pointer normalization behind both the count and the ids; int64 storage. Where relationships are measured (the stage's own derivation) the count must equal the number of ids naming each person, exactly.own_children_in_householdcounts (ACS rows impute the count with pointers declared unknown, so the gate checks this bound rather than equality), and everyperson_idin a household carrying a link fits PolicyEngine-US's int32 storage. Parent ids may be all zero ("unknown"), so they are not in the non-constant release set.--base-h5input, an existing pool) passes through untouched, without parent ids. The ids are optional end to end: the release gate asserts their identities only when both are present, so existing pools still feed releases, and PolicyEngine-US uses its count-based proxy where they are absent.microcosm-datastored_inputs.py):parent_1_id/parent_2_idare registered stored non-variable columns (the pinned engine, policyengine-us 2.2.1, does not define them), naming their producer and the removal condition: drop the entries when the pin moves to a policyengine-us release with #9406. Without this, main's stored-input contract (Refuse US releases that store model inputs the certified engine does not define (#1026) #1031) refuses every release that carries them.acs_inputs.py,operator_boundary.py): declares the pointers unknown (0) natively.multispine_pool.py): the ids are native-complete on both arms and never enter the QRF transfer plan.puf_support.py,puf_capital_gains_tail.py): the support clone and the capital-gains tail clone shift parent ids withperson_id, leaving 0 alone; rows that never had a pointer carry 0.input_mass.py, shared):PERSON_REFERENCE_COLUMNSare skipped as structural, like entity ids.tools/build_us_multispine_pool.py): a transferred/simulated checkpoint whose recorded person schema lacks an eligibility output is ignored as stale, since the byte-stable legacy identity cannot see new outputs.How this branch was built
The work was done on 9/7 on
parent-ids-export(pushed unchanged as a backup at7ec9eafc1), which fell ~797 commits behind main. This branch is fresh fromorigin/main(31315687d) with the three substantive commits cherry-picked (ccce48cc9,806f780a4,29a7b71fd); the lane-journal commits (ade4b5cb4,2361549cb,7ec9eafc1) and the journal/checkpoint files are dropped.Conflicts were only digest pins. Re-pinned at observed values after the rebuild, each for the stated reason:
source_stages.jsonasset sha256 (sources.yaml,generate_us_bundle_from_constants.py,test_us_bundle_core_contracts.py,test_us_spec_bundle.py)spec_sha256→d3469f47…(test_us_multispine_pool_tool.py,us-f0-coverage.json)source_stagespointer claim; pointer inventoryinventory_coverage.py); the loader golden vectorpuf_supportandpuf_capital_gains_tailare attested kernel modules (spec_engine/seeds.py_DIRECT_KERNEL_MODULES)tools/generate_us_bundle_from_constants.py --checkandtools/spec_engine_coverage.py --checkpass (42,184/42,184 fields; 41/41 inventory checks).docs/evidence/spec-engine/us-f0-coverage.jsonwas regenerated by the tool, not edited.Effect on certified builds
A build from this code gains two person columns. For existing columns the intent is no change, and the round-1 review's one counterexample (a
--base-h5frame whose carriedveterans_benefitswas overwritten by re-derivation) is fixed by passing such frames through untouched (asmaindoes), covered bytest_legacy_frame_passes_through_untouched. Draw sites seed from seed/salt/stable-key material; the moved digests attest source and are not seed inputs. The QRF transfer targets are unchanged.A PolicyEngine-US version without these variables loads such a build unchanged:
policyengine_coreskips dataset columns that match no variable and logs a warning. The latest publishedpopulace_us_2024.h5hasperson_id4–1,432,502, inside int32; the gate now enforces that for linked households instead of assuming it.Publishing a build with these columns is a separate, deliberate release step and is not part of this PR.
Verification
origin/main2bce66156.tools/generate_us_bundle_from_constants.py --checkandtools/spec_engine_coverage.py --checkpass (42,184/42,184 fields; 41/41 inventory checks); every pin was recomputed from the tools, not hand-edited.microcosm-datatest package, pass;tools/ci_test_groups.py --verifyok;ruff checkclean. The full suite runs in CI.808ee8feb. Round 2 confirmed those resolved, answered "no" on whether a completing build changes existing columns, and found the stored-input register gap and the existing-frame key problem, addressed in813978feb. Round 3 confirmed those and asked that the register entries rest on their producer in the microcosm-data tests (the parent ids postdate the H5 files the register was built from), addressed indd372e109. Round 4 approved (no new correctness bug; existing output columns unchanged), with nits carried under known limits.🤖 Generated with Claude Code