Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/884-parent-ids-export.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
US person input columns `parent_1_id` and `parent_2_id`: the CPS ASEC `PEPAR1`/`PEPAR2` pointers resolved to a co-resident parent's `person_id` (0 when unknown or absent), so downstream rules can ask whose child a person is rather than only how many children an adult has.
48 changes: 24 additions & 24 deletions docs/evidence/spec-engine/us-f0-coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
"version": 1
},
"country": "us",
"documentation_sha256": "911b21d13cc214601cfb537c538fed2851e5166e8f72de5ebfce5f2918af43d3",
"documentation_sha256": "0931cbd3aae2abcd8c072b78eba40c1c006b1bf7fa23c6a6681f9a895e47070e",
"field_usage": {
"authored_normative_field_count": 32400,
"authored_normative_field_count": 32404,
"claim_count": 49,
"claims": [
{
Expand Down Expand Up @@ -514,8 +514,8 @@
],
"mode": "legacy_behavior",
"pointer_class": "all",
"pointer_count": 1712,
"pointer_sha256": "a46d5e657d919db23ee7752d844bd8bbe59c15321dec07102837452d7994d8dd",
"pointer_count": 1716,
"pointer_sha256": "3e808ed9f5a791c0dfb85be0c59adf803366bf6e8c7a9f5a58f92974e044b607",
"rationale": null,
"relative_sink_prefix": "/source_manifest/stages",
"source_prefix": "/authored/spec~1sources.yaml/stages",
Expand Down Expand Up @@ -772,20 +772,20 @@
"verifier": "vintages"
}
],
"configuration_field_count": 42180,
"consumed_field_count": 42180,
"configuration_field_count": 42184,
"consumed_field_count": 42184,
"generation0_effect_counts": {
"legacy_behavior": 38494,
"legacy_behavior": 38498,
"no_generation0_effect": 3686
},
"mode_counts": {
"compiler_semantic": 27717,
"front_end_validation": 354,
"identity_only": 103,
"legacy_behavior": 14006
"legacy_behavior": 14010
},
"multiple_primary_use_field_count": 0,
"pointer_inventory_sha256": "21b6fe2f6b96307f8ec7d20d4162ee2e0702f363bb28ebd3fa47a4cf7fc12ed4",
"pointer_inventory_sha256": "8f655670b9794adae905ede64bdceba6395eeb8d642b6c5896d5c516f1f6a6a7",
"resolved_binding_field_count": 9780,
"unused_field_count": 0
},
Expand Down Expand Up @@ -1361,12 +1361,12 @@
],
"expected": {
"producer_count": 38,
"sha256": "6abbfad73014ce3bac9fbd549f16e4a5e645a3b0fbd450b56e186876f3f0dd95"
"sha256": "cc34ba11288d01145fc2dc1a669df3f7c4ef2019d73afb02dd8eeee852b16d20"
},
"failures": [],
"observed": {
"producer_count": 38,
"sha256": "6abbfad73014ce3bac9fbd549f16e4a5e645a3b0fbd450b56e186876f3f0dd95"
"sha256": "cc34ba11288d01145fc2dc1a669df3f7c4ef2019d73afb02dd8eeee852b16d20"
},
"status": "covered"
},
Expand Down Expand Up @@ -1656,13 +1656,13 @@
"compiler_ir.node_slices"
],
"expected": {
"map_sha256": "dad44808366198b8826823b837c7b592c9ecee2299adbd9f8a20251dd6d0b00a",
"protocol_sha256": "5f6d3b527060bc6f15f1871c64a42994d4882ee5962f3350ea8a71a83ffcd6bf"
"map_sha256": "3cf10a524310ea0ada18ddc65cfa585d1d0652d23e7d7a180c050ac686224e28",
"protocol_sha256": "91989ed5e366e19598f86335b2e66723dcb2b6481a78d0dbeb83183a1aa09bd4"
},
"failures": [],
"observed": {
"map_sha256": "dad44808366198b8826823b837c7b592c9ecee2299adbd9f8a20251dd6d0b00a",
"protocol_sha256": "5f6d3b527060bc6f15f1871c64a42994d4882ee5962f3350ea8a71a83ffcd6bf"
"map_sha256": "3cf10a524310ea0ada18ddc65cfa585d1d0652d23e7d7a180c050ac686224e28",
"protocol_sha256": "91989ed5e366e19598f86335b2e66723dcb2b6481a78d0dbeb83183a1aa09bd4"
},
"status": "covered"
},
Expand All @@ -1677,7 +1677,7 @@
"compiler_ir.seed_stream_map"
],
"expected": {
"implementation_sha256": "5f6d3b527060bc6f15f1871c64a42994d4882ee5962f3350ea8a71a83ffcd6bf",
"implementation_sha256": "91989ed5e366e19598f86335b2e66723dcb2b6481a78d0dbeb83183a1aa09bd4",
"protocol": "legacy-v1",
"streams": [
"build_model",
Expand All @@ -1698,7 +1698,7 @@
},
"failures": [],
"observed": {
"implementation_sha256": "5f6d3b527060bc6f15f1871c64a42994d4882ee5962f3350ea8a71a83ffcd6bf",
"implementation_sha256": "91989ed5e366e19598f86335b2e66723dcb2b6481a78d0dbeb83183a1aa09bd4",
"protocol": "legacy-v1",
"streams": [
"build_model",
Expand Down Expand Up @@ -1775,12 +1775,12 @@
"legacy_adapter.imputation.late_producer_resource_semantics"
],
"expected": {
"sha256": "a87278445d928ea20e15935a8cc07312d6e819e34409b04e8c814f17910d828f",
"sha256": "f4f8bfeb79268a4043a843b47cec1d1c0e58e9f5dd6bc23417ada95446dd7b8e",
"stages": 37
},
"failures": [],
"observed": {
"sha256": "a87278445d928ea20e15935a8cc07312d6e819e34409b04e8c814f17910d828f",
"sha256": "f4f8bfeb79268a4043a843b47cec1d1c0e58e9f5dd6bc23417ada95446dd7b8e",
"stages": 38
},
"status": "covered"
Expand Down Expand Up @@ -1887,7 +1887,7 @@
"alpha",
"zeta"
],
"sha256": "b1d5c65254b51897b27f0f5e44e344781ce9e01e305aaf1bfc042cc32bc417dc"
"sha256": "b7a336af7cb300905d09ad9a33aa1a624f878c2471352d4e252fa69db55aee17"
},
"failures": [],
"observed": {
Expand All @@ -1911,7 +1911,7 @@
"alpha",
"zeta"
],
"sha256": "b1d5c65254b51897b27f0f5e44e344781ce9e01e305aaf1bfc042cc32bc417dc"
"sha256": "b7a336af7cb300905d09ad9a33aa1a624f878c2471352d4e252fa69db55aee17"
},
"status": "covered"
},
Expand Down Expand Up @@ -1975,7 +1975,7 @@
"take_up_contract",
"us_qbi_reconciliation_contract"
],
"sha256": "5b65545ed6f351b30e94ab1405e13d2c7f692ae0c8d4959badcfe9bb9d52d455"
"sha256": "61a5484eab9fae1d6651b3ae15f1793d4934ea2f6ed2c08d05716b745c221014"
},
"status": "covered"
},
Expand Down Expand Up @@ -2599,7 +2599,7 @@
"country": "us",
"schema_id": "country_spec",
"schema_version": 1,
"spec_sha256": "f830589331dad583398485de5724dace0714d8526fb63b37f81cd783b1cb8f30"
"spec_sha256": "1cf1f44d9a11458f59ea7d75dce68303816cab3e66668a6c6f6ecfe62d56f8dc"
}
},
"report_schema_version": 3,
Expand All @@ -2609,7 +2609,7 @@
"country": "us",
"schema_id": "country_spec",
"schema_version": 1,
"spec_sha256": "f830589331dad583398485de5724dace0714d8526fb63b37f81cd783b1cb8f30"
"spec_sha256": "1cf1f44d9a11458f59ea7d75dce68303816cab3e66668a6c6f6ecfe62d56f8dc"
},
"status": "pass"
}
2 changes: 1 addition & 1 deletion docs/us-multispine-operator-ordering.md
Original file line number Diff line number Diff line change
Expand Up @@ -1066,7 +1066,7 @@ runs:
| Relationships | pre | Household-head status is a rent-recipient predictor. |
| Medicare take-up | post | Rowwise carry/completion is clone-safe. |
| Housing inputs | pre | Rent is drawn once per source household and then cloned unchanged. |
| Eligibility inputs | pre | Raw `PH_SEQ`/`A_LINENO` parent pointers would count every cloned child twice. |
| Eligibility inputs | pre | Raw `PH_SEQ`/`A_LINENO` parent pointers would count every cloned child twice, and `PH_SEQ` collides between the ASEC and ACS spines, so the stage runs on the CPS-evidence projection. Its `parent_1_id`/`parent_2_id` outputs name a `person_id`, so the support clone shifts them with the rest of the person id surface ([microcosm#884](https://github.com/PolicyEngine/microcosm/issues/884)). |
| Pregnancy | post | Stable source-identity hashes share draws across clones. |
| WIC | post | Remapped family grouping and source-identity draws are clone-safe. |
| Housing assistance | post | The QRF intentionally replaces only the PUF support role. |
Expand Down
16 changes: 12 additions & 4 deletions packages/microcosm-build/src/microcosm/build/input_mass.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,15 @@

from microcosm.frame import Frame

__all__ = ["input_mass_totals"]
__all__ = ["PERSON_REFERENCE_COLUMNS", "input_mass_totals"]

#: Person columns whose values name another person (a co-resident parent's
#: ``person_id``, ``0`` = unknown; microcosm#884). Like entity ids they are
#: labels, not measured quantities: their weighted sum means nothing and moves
#: whenever person ids are renumbered, so a parity comparison against a release
#: on a different id space would fail for no reason. They are skipped as
#: structural in every schema that carries them.
PERSON_REFERENCE_COLUMNS: frozenset[str] = frozenset({"parent_1_id", "parent_2_id"})


def input_mass_totals(
Expand All @@ -35,8 +43,8 @@ def input_mass_totals(
Every non-structural numeric column is summed under the owning entity's
effective weights (household weights broadcast through membership for
entities without their own vector); boolean columns total their weighted
``True`` mass. String/enum columns and structural columns (entity ids and
person membership ids) are skipped.
``True`` mass. String/enum columns and structural columns (entity ids,
person membership ids, and :data:`PERSON_REFERENCE_COLUMNS`) are skipped.

Args:
frame: A microcosm frame in any country schema.
Expand All @@ -56,7 +64,7 @@ def input_mass_totals(
"""

schema = frame.schema
structural = {schema.person_id_column}
structural = {schema.person_id_column, *PERSON_REFERENCE_COLUMNS}
for group in schema.group_entities:
structural.add(schema.id_column(group))
structural.add(schema.membership_column(group))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@
)
from .schemas import load_schema_registry

EXPECTED_AUTHORED_FIELD_COUNT = 32_400
EXPECTED_AUTHORED_FIELD_COUNT = 32_404
EXPECTED_RESOLVED_BINDING_FIELD_COUNT = 9_780
EXPECTED_CONFIGURATION_FIELD_COUNT = 42_180
EXPECTED_CONFIGURATION_FIELD_COUNT = 42_184


class FieldUsageError(AssertionError):
Expand Down Expand Up @@ -485,8 +485,8 @@ def _path_inventory(rows: Sequence[tuple[str, object]]) -> tuple[int, str]:
"d6782c5de5bbed1bdc6bf653c4a6d4aadcad4ccc72d35e1092e130fcb04680a3",
),
"source_stages": (
1_712,
"a46d5e657d919db23ee7752d844bd8bbe59c15321dec07102837452d7994d8dd",
1_716,
"3e808ed9f5a791c0dfb85be0c59adf803366bf6e8c7a9f5a58f92974e044b607",
),
"spine_assembly_household_mass_shares": (
2,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -350,18 +350,18 @@
"acs_person_predictors": "878c788a6f037d7aca12b3586ea034eff04f3034ffa11935a736493042551f25",
"authority": "9d4a9672a0f03039b1fe874b9fe21ed575be0d29f14afc396d03cdf5c809bdd2",
"early_families": "4aa9f736fd76e83955477ad1667e58f48f264783f05bdc7f0102cd32d61323bd",
"full_checkpoint": "b1d5c65254b51897b27f0f5e44e344781ce9e01e305aaf1bfc042cc32bc417dc",
"full_checkpoint": "b7a336af7cb300905d09ad9a33aa1a624f878c2471352d4e252fa69db55aee17",
"gap_fill_schedule": "1c31f9868f7884347cc19cf1ff65da43f950b9114941a715bab168246db414a7",
"graph_nodes": "40cd51ffdfe2e9d9d08d48c08e8ded9de1e4b134783bab05c4abc6ad5c72ca1e",
"geography_assignment": "f49425ca8734ac559c73cf44f6458d86d3162a48956b98a27e6e758959361585",
"late_families": "d91f9ff0eb52f43e7b6eed3d5c58c37abe1620c3a11021da15dae9c10e16d382",
"late_resource_semantics": "6abbfad73014ce3bac9fbd549f16e4a5e645a3b0fbd450b56e186876f3f0dd95",
"late_resource_semantics": "cc34ba11288d01145fc2dc1a669df3f7c4ef2019d73afb02dd8eeee852b16d20",
"late_schedule": "e59c019d3d454eac99ac0ac209b6c5b6faaf9bdfcaeee18c36a25be19bf7da2f",
"ownership": "5f64f0aac49e2313177564f71876bffc8c81b3ded4df701e70930e60e9c98356",
"primary_tuples": "fdf23da429f8c501e198a2f5b719763d523a41565c2153fdf17b0344f21bcd7c",
"seed_map": "dad44808366198b8826823b837c7b592c9ecee2299adbd9f8a20251dd6d0b00a",
"seed_protocol": "5f6d3b527060bc6f15f1871c64a42994d4882ee5962f3350ea8a71a83ffcd6bf",
"source_manifest": "a87278445d928ea20e15935a8cc07312d6e819e34409b04e8c814f17910d828f",
"seed_map": "3cf10a524310ea0ada18ddc65cfa585d1d0652d23e7d7a180c050ac686224e28",
"seed_protocol": "91989ed5e366e19598f86335b2e66723dcb2b6481a78d0dbeb83183a1aa09bd4",
"source_manifest": "f4f8bfeb79268a4043a843b47cec1d1c0e58e9f5dd6bc23417ada95446dd7b8e",
"take_up": "9522ce40f7dea569312dd7a7beb474e5a5afadd2f1e10cb5876534c3ef623d35",
"tail": "ac92829c88a1a4fb6460d61190918d5d99c6c377fc8dd8f62f02b332d09bf59c",
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2227,13 +2227,17 @@
"is_blind",
"is_full_time_college_student",
"own_children_in_household",
"parent_1_id",
"parent_2_id",
"veterans_benefits"
],
"nonnegative_outputs": [
"own_children_in_household",
"parent_1_id",
"parent_2_id",
"veterans_benefits"
],
"notes": "Direct mappings of measured ASEC demographic variables reproducing the retired enhanced-CPS pipeline's constructions (each cited to the retired pipeline's source files in the stage module): is_disabled <- any PEDIS* difficulty item == 1, aligned with reported SSI (under-65 SSI_VAL > 0 implies disabled \u2014 the retired align_reported_ssi_disability rule from datasets/cps/takeup.py, applied to is_disabled in cps.py); is_blind <- PEDISEYE == 1; is_full_time_college_student <- A_HSCOL == 2 & A_FTPT == 1 (the retired primary branch; census_cps.py loads A_FTPT, so the published surface ran the strict form \u2014 the A_HSCOL-only fallback for loads without A_FTPT is not reproduced); own_children_in_household <- count of household members whose PEPAR1/PEPAR2 parent pointers resolve to the person within PH_SEQ; veterans_benefits <- VET_VAL. Nothing is imputed. Without this stage every input takes its engine default (False/0), erasing SNAP disability, student, parent/child, and veteran exemption channels (microcosm #244/#248). is_veteran is formula-owned in PolicyEngine-US (derived from veterans_benefits) and is deliberately not exported (microcosm #24)."
"notes": "Direct mappings of measured ASEC demographic variables reproducing the retired enhanced-CPS pipeline's constructions (each cited to the retired pipeline's source files in the stage module): is_disabled <- any PEDIS* difficulty item == 1, aligned with reported SSI (under-65 SSI_VAL > 0 implies disabled \u2014 the retired align_reported_ssi_disability rule from datasets/cps/takeup.py, applied to is_disabled in cps.py); is_blind <- PEDISEYE == 1; is_full_time_college_student <- A_HSCOL == 2 & A_FTPT == 1 (the retired primary branch; census_cps.py loads A_FTPT, so the published surface ran the strict form \u2014 the A_HSCOL-only fallback for loads without A_FTPT is not reproduced); own_children_in_household <- count of household members whose PEPAR1/PEPAR2 parent pointers resolve to the person within PH_SEQ; parent_1_id/parent_2_id <- the person_id of the co-resident member the person's PEPAR1/PEPAR2 pointer names within PH_SEQ, 0 when the pointer is absent or resolves to no co-resident line (microcosm#884; the relation behind the count, consumed by policyengine-us#9404); veterans_benefits <- VET_VAL. Nothing is imputed. Without this stage every input takes its engine default (False/0), erasing SNAP disability, student, parent/child, and veteran exemption channels (microcosm #244/#248). is_veteran is formula-owned in PolicyEngine-US (derived from veterans_benefits) and is deliberately not exported (microcosm #24)."
},
{
"stage": "pregnancy",
Expand Down
16 changes: 11 additions & 5 deletions packages/microcosm-build/src/microcosm/build/us/spec/sources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ sources:
stage_asset:
id: source_stages
path: microcosm.build.us/source_stages.json
sha256: 9f4f983091fb18dea8f4524cc100be20f9fca2b006435461592f8edf53c9a5d8
sha256: e75217c0db3075bf2664492167d31ce7ee82c2c5f6a70e3b67e8bb6f4ddd7ecf
stage_manifest:
version: 1
country: us
Expand Down Expand Up @@ -2222,9 +2222,13 @@ stages:
- is_blind
- is_full_time_college_student
- own_children_in_household
- parent_1_id
- parent_2_id
- veterans_benefits
nonnegative_outputs:
- own_children_in_household
- parent_1_id
- parent_2_id
- veterans_benefits
notes: 'Direct mappings of measured ASEC demographic variables reproducing the retired enhanced-CPS
pipeline''s constructions (each cited to the retired pipeline''s source files in the stage module):
Expand All @@ -2234,10 +2238,12 @@ stages:
A_FTPT == 1 (the retired primary branch; census_cps.py loads A_FTPT, so the published surface ran
the strict form — the A_HSCOL-only fallback for loads without A_FTPT is not reproduced); own_children_in_household
<- count of household members whose PEPAR1/PEPAR2 parent pointers resolve to the person within PH_SEQ;
veterans_benefits <- VET_VAL. Nothing is imputed. Without this stage every input takes its engine
default (False/0), erasing SNAP disability, student, parent/child, and veteran exemption channels
(microcosm #244/#248). is_veteran is formula-owned in PolicyEngine-US (derived from veterans_benefits)
and is deliberately not exported (microcosm #24).'
parent_1_id/parent_2_id <- the person_id of the co-resident member the person''s PEPAR1/PEPAR2 pointer
names within PH_SEQ, 0 when the pointer is absent or resolves to no co-resident line (microcosm#884;
the relation behind the count, consumed by policyengine-us#9404); veterans_benefits <- VET_VAL. Nothing
is imputed. Without this stage every input takes its engine default (False/0), erasing SNAP disability,
student, parent/child, and veteran exemption channels (microcosm #244/#248). is_veteran is formula-owned
in PolicyEngine-US (derived from veterans_benefits) and is deliberately not exported (microcosm #24).'
- stage: pregnancy
survey: Census CPS ASEC + CDC natality-derived national pregnancy rate
source: https://www.cdc.gov/nchs/nvss/births.htm
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@
from microcosm.build.us_runtime.eligibility_inputs import (
US_ELIGIBILITY_INPUTS_NONCONSTANT_PERSON_COLUMNS,
US_ELIGIBILITY_INPUTS_OUTPUT_COLUMNS,
US_ELIGIBILITY_INPUTS_PARENT_ID_COLUMNS,
US_ELIGIBILITY_INPUTS_REQUIRED_SOURCE_COLUMNS,
US_ELIGIBILITY_INPUTS_STAGE_NAME,
derive_us_eligibility_inputs_from_manifest,
Expand Down Expand Up @@ -1249,6 +1250,7 @@
"us_pregnancy_summary",
"with_us_pregnancy_inputs",
"US_ELIGIBILITY_INPUTS_NONCONSTANT_PERSON_COLUMNS",
"US_ELIGIBILITY_INPUTS_PARENT_ID_COLUMNS",
"US_ELIGIBILITY_INPUTS_OUTPUT_COLUMNS",
"US_ELIGIBILITY_INPUTS_REQUIRED_SOURCE_COLUMNS",
"US_ELIGIBILITY_INPUTS_STAGE_NAME",
Expand Down
Loading
Loading