Repository navigation
Conversation
This was referenced Sep 14, 2026
MaxGhenis
added a commit
that referenced
this pull request
Sep 15, 2026
The review's F2. Two of the carried changes move published numbers, and both fragments described only the mechanism. The member-share allocation shifts SPM resources between co-resident SPM units, so SPM poverty and deep poverty change for multi-unit households where any unit receives housing assistance - and only for those; a single-unit household's allocation is its own award (`test_single_unit_allocation_preserves_program_amount`). An awarded family's tenant payment can now zero a co-resident unit's housing resource (`test_independent_unit_caps_do_not_redistribute_unused_housing_value`), which the fragment names as an assumption. Counting actual `housing_assistance` in `household_benefits` and `cbo_means_tested_transfers` raises household benefits, household net income, marginal tax rates and CBO transfer aggregates for every assisted household by whatever the SPM cap withheld, without moving SPM poverty, and CARE/FERA countable income loses the housing entry from 2015 for annual calculations. Both are retyped `changed` rather than `fixed`: they are not defect repairs to a caller, they are moved numbers. No `breaking` fragment is added. `spm-supplied-policy-caches.fixed.md` returns, dropped from #9448 while `test_supplied_prepared_system_retains_warm_policy_children` and its neighbours were carried. It describes the behaviour this branch ships, not #9448's mechanism for it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 15, 2026
The review's F5. Both were run against this branch's mechanism before deciding:
test_cached_formula_trace_binds_replacement_tracer[False] FAILED
test_cached_formula_trace_binds_replacement_tracer[True] FAILED
test_system_parameter_lookup_does_not_cache_tracer FAILED
One root cause, observed directly. Core memoises the
`TracingParameterNodeAtInstant` in the root parameter node's
`_at_instant_cache`, bound to whichever tracer built it. #9463 isolates a
traced request by giving it its own root with an empty at-instant cache, keyed
on the `trace` setter; replacing a tracer in place afterwards - a bare
`simulation.tracer = FullTracer()`, or a `parameters.tracer` write on a system -
reaches none of that, and core's per-formula soft recast in
`Simulation._run_formula` rewrites only the root's own `trace`/`tracer` fields,
not the memo. So the second tracer records nothing: for the system-lookup case,
iteration 0 records one parameter and iteration 1 records zero, with the access
going to the first tracer's closed tree.
Passing them means rebinding that memo at lookup, which is #9448's mechanism
rather than this one, so they are not forced. The boundary is recorded on
`test_traced_simulations_record_their_own_parameter_accesses`, which covers the
supported path, with the note that the only bare tracer write left in this
package is `tools/branched_simulation.py`, which nothing constructs.
The policy-family docstring said #9463 reverses child-to-parent propagation,
full stop. It reverses it for parameter reforms; a variable-only reform on a
shared-policy branch still reaches its parent and siblings, and
`test_a_branch_variable_reform_is_not_isolated_from_its_parent` pins that. The
correction also records what the dropped cases actually carried: #9448's
`test_child_reform_updates_warm_parent_and_sibling` parametrizes a parameter
dictionary against a root-replacing `modify_parameters` reform, so those cases
are contradicted rather than merely duplicated, and the drop stands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
to DTrim99/policyengine-us
that referenced
this pull request
Sep 15, 2026
From PolicyEngine#9448, reduced. General household benefits, CBO means-tested transfers and the three Tlaib household-benefit overrides now add actual `housing_assistance` instead of the Census SPM's capped housing subsidy, so a change to an SPM threshold no longer moves a non-SPM result. SPM resources keep the capped valuation, now allocated across SPM units by member share. CARE/FERA countable income excludes housing subsidies from the August 14, 2014 parameter date of CPUC D.14-08-030; annual values resolve at January 1, so calendar 2014 retains the earlier modelled inclusion (using the actual benefit) and calendar 2015 first applies the exclusion. Also carries PolicyEngine#9448's release-candidate wheel tooling and the isolated build backend pin. Dropped from PolicyEngine#9448: its `spm.py` and `system.py` isolation hunks (PolicyEngine#9463 owns the mechanism), its default-dataset hash and batch-runner hunks (PolicyEngine#9464 owns those), its `docs/usage/microsimulation.md` edit (PolicyEngine#9463's is more specific), its `reforms/reforms.py` change (no consumer) and its `tests/core/test_spm_system.py` edits (they invert PolicyEngine#9463's contract). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
to DTrim99/policyengine-us
that referenced
this pull request
Sep 15, 2026
…tatus From PolicyEngine#9464, reduced. `REJECTED_DATASET_INPUTS` extends the calculator's `FORMULA_OWNED_INPUTS` with the country-side poverty aliases (`in_poverty`, `in_deep_poverty`, `deep_poverty_line`, `deep_poverty_gap`, `person_in_poverty`) and with the two SPM housing-allocation outputs PolicyEngine#9448 adds, so a dataset can no longer store a formula-owned measurement under a name the calculator does not police. `DATASET_SOURCE_INPUTS` keeps `is_spm_independent_minor_role` declarable by a producer, and the two sets are asserted disjoint. `DEFAULT_DATASET_SHA256` pins the certified build the mutable Hugging Face tag must resolve to, verified once per process and only for the country default; an explicit `dataset=` argument remains the caller's own artifact. The batch runner now decides a batch on the child's real exit status, reported through a marker written before interpreter teardown, and enumerates `.yml` alongside `.yaml` everywhere it lists files. Dropped from PolicyEngine#9464: `reported_housing_assistance` and the CPUC entry that counted it, plus the three tests that pinned housing assistance as countable - CPUC D.14-08-030 excludes housing subsidies outright, so there is nothing to gate on receipt and the variable had no other consumer. Its citation research is folded into the CPUC parameter instead. Adaptations: the dataset guard is grafted onto PolicyEngine#9463's `set_input`; `test_spm_system.py`'s import block is the union of both PRs' needs; `test_care_eligibility_is_the_same_under_every_spm_geography` expects `(40_700, True)` because $40,700 of countable income is inside the 2024 one-person CARE limit of $40,880 once the subsidy is excluded; and the two counting non-vacuity tests are replaced by their exclusion analogues so the suite still fails if the housing entry is silently dropped rather than excluded. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
to DTrim99/policyengine-us
that referenced
this pull request
Sep 15, 2026
Folds PolicyEngine#9464's citation research into PolicyEngine#9448's dated exclusion list. The ~25 non-housing entries regain a source: PG&E Form 01-9077 (Rev. 6.26) at PG&E's own URL, plus Revised Cal. P.U.C. Sheet No. 61313-E (Electric Sample Form 79-1055, Advice 7905-E, effective June 1, 2026), which carries the same income footnote as a CPUC filing. Both are annotated as being corrected: PG&E's June 5, 2026 reply brief in A.26-01-003 states it does not use housing and military subsidies in its income analysis and will revise the application. The dead bha.berkeleyca.gov href is gone; all nine references return HTTP 200. Records review item R8: FERA coverage of the exclusion rests on the 2026 brief, not on D.14-08-030. Verified first-hand against the decision PDF - it contains zero case-sensitive occurrences of "FERA", "Family Electric" or "739.12", while the brief's Section B answers the income definition under Public Utilities Code 739.1(a) and 739.12(a) for CARE and FERA alike. Verified against the live parameter tree: 2013-01-01, 2014-01-01 and 2014-08-13 resolve to 27 entries including housing_assistance; 2014-08-14, 2015-01-01 and 2026-01-01 resolve to 26 without it, the only set difference being housing_assistance, and all 27 names resolve to model variables. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
to DTrim99/policyengine-us
that referenced
this pull request
Sep 15, 2026
Found while adjudicating whether PolicyEngine#9448's `test_spm_policy_family.py` could be carried onto PolicyEngine#9463's mechanism, and reproduced here with public-API calls. `clone_spm_system` hands core an ordinary instance to clone through - `plain_policy_copy` -> `SharedParameterPolicy.unshared_copy()` resets `__class__` to the base class - and never put the barrier back. So `Simulation.clone()` returned a plain `CountryTaxBenefitSystem`, and `apply_reform` on anything reading that tree took the unguarded path: clone = Simulation(situation=...).clone() clone.get_branch("probe").apply_reform({SINGLE standard deduction: 100_000}) # clone's own SINGLE standard deduction moved 14,600 -> 100,000 That is the defect `SharedParameterPolicy` exists to prevent, and it contradicts `test_a_branch_reform_does_not_rewrite_its_parents_policy`, which only passed because it never cloned. `with_parameter_barrier`'s own docstring already states the rule: every simulation's system carries the barrier, because a tree it starts out owning can acquire readers later. Bounded: a clone gets its own tree, so neither the shipped module-level system nor the simulation the clone came from was reachable this way; the leak stayed inside one cloned simulation's branch family. Verified before and after. `test_cloning_a_shared_policy_system_stops_sharing_its_tree` asserted `"shared_parameters" not in cloned.__dict__`, which pinned the defective implementation rather than the invariant. It now asserts that the clone holds its own tree behind the barrier; its two substantive assertions are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
to DTrim99/policyengine-us
that referenced
this pull request
Sep 15, 2026
…mily tests PolicyEngine#9448's `test_spm_policy_family.py` cannot be carried whole: run unchanged against PolicyEngine#9463's mechanism, 16 of its 27 cases fail. They split four ways. Nine assert PolicyEngine#9448's own internals - the `_parameter_view` / `_parameter_source` / `parameters.modified` provenance protocol and `get_parameters_at_instant` - which this branch does not implement, because PolicyEngine#9463 owns the mechanism. Six turn on `Simulation.tracer` being a plain attribute in core, so a bare `simulation.tracer = X` re-primes nothing and X records no parameter receipts. PolicyEngine#9463 hooks the `trace` setter, `__init__`, `get_branch`, `apply_reform`, `_adopt_detached_parameters` and `clone`, but not a bare `tracer` write. No production caller does one that matters, so these are dropped rather than fixed here. Four assert that a child branch's reform propagates up to its parent and siblings. Stated plainly: that is stock policyengine-core behaviour - `Simulation.clone` hands a `clone_system=False` branch the SAME tax-benefit system object and `apply_reform` mutates it in place - so PolicyEngine#9463 is a deliberate deviation from core here, not a restoration of it. It is still the right deviation to ship: every branch in this repository that actually mutates policy already passes `clone_system=True` (`tob_revenue_oasdi.py:42`, `behavioral_response_measurements.py:49`), while the `clone_system=False` branches only set inputs, and a formula-internal branch that silently moved its parent's answer would be order-dependent. But it is a design decision, not something core's contract compels. One depends on a `reforms/reforms.py` hunk this branch drops, which returns `None` from `create_structural_reforms_from_parameters` when no structural reform is active. That case is unreachable - `create_mi_surtax_reform` always returns a reform, so the filtered tuple is never empty - and carrying the hunk verbatim is unsafe: `CountryTaxBenefitSystem.__init__` packs the result into a reform tuple, and `CountryTaxBenefitSystem(reform=((), None))` raises `AttributeError: 'NoneType' object has no attribute 'apply'` (verified). The eleven cases kept here are the mechanism-independent remainder: a nested branch clone's inherited inputs and private reform, a construction-time structural reform detaching before it mutates parameters, the warm-cache contract for a caller-supplied system across clone hops and a reform wrapper, an at-instant view refreshing after an in-place leaf update, and a reform reading back its own edit before it returns. All eleven pass. Also pins the boundary of the barrier in `test_spm_simulation_isolation.py`: it covers the parameter tree, not the variable registry, so a variable-only reform on a shared-policy branch does still reach its parent and siblings. PolicyEngine#9463 never claimed otherwise, but nothing recorded the limit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Sep 22, 2026
Re-levels #9462 (branch point 218 commits behind) onto 2.7.0. Ten content conflicts; main wins everywhere its change is not the SPM scope separation, and the scope separation is reapplied on top. Drops this PR's `spm_unit_ordinary_housing_subsidy` bridge as superseded. Main's d1251c4 ("Keep SPM valuation out of program benefits and CARE/FERA income", the reduced #9448) already moved general household benefits, CBO means-tested transfers, CARE/FERA countable income and the three Tlaib household-benefit overrides onto the modelled `housing_assistance` award. That is the same separation this PR's bridge was built for, reached without a new variable: `housing_assistance` is a program amount for every unit, so those aggregates never consult the measurement universe. Keeping the bridge would have put the SPM-capped valuation back into ordinary income for included units and made household net income itself depend on a scope declaration. The independently qualified 2.2.1 backport (9b26e9b) reached the same conclusion on a base that already contained d1251c4. Conflict resolutions: - parameters/gov/household/cbo_means_tested_transfers.yaml: main (`housing_assistance`). - parameters/gov/household/household_benefits.yaml: main (`housing_assistance`) in both dated lists. - parameters/gov/states/ca/cpuc/income_sources.yaml: main, including its new 2014-08-14 value dropping housing subsidies per D.14-08-030. - variables/household/income/household/household_benefits.py: main. - reforms/congress/tlaib/{boost/boost_middle_class_tax_credit, economic_dignity_for_all_agenda/edaa_end_child_poverty_act, end_child_poverty_act}.py: main's lists, including the added `trump_dividend` entry and `housing_assistance`; the SPM-resource `spm_unit_benefits` overrides keep the scope mask. - spm.py: main's structure, with the scope helpers added. - variables/.../spm_unit_capped_housing_subsidy.py: main's allocated assistance and allocated tenant payment, with the scope mask. - tests/microsimulation/test_microsim.py: reconciled to the post-2.2.1 contract. Also carries the four items the independently qualified 2.2.1 backport (9b26e9b) has and this PR lacked, with their tests: - `set_input` invalidates dependent caches when a `DATASET_SOURCE_INPUTS` name is set, and scope joins that set. Without it a scope declaration set after a dependent result was cached left the stale result in place. - `CountyRequiringSPMProvider.with_county_input_types`, applied to the selected units in `masked_policyengine_amount`, so an excluded numeric county cannot fail a measurement that never used it. - `spm_unit_oecd_equiv_net_income` and `spm_unit_income_decile` inherit the universe. Ranking over a population containing outside NaN income would otherwise corrupt every included unit's decile. - `SPM_DISTRIBUTION_OUTPUTS` joins `REJECTED_DATASET_INPUTS`. Test files taken from that backport: test_spm_universe.py, test_income_decile_ties.py, test_spm_production_parity.py, and its edits to test_spm_system.py, test_spm_source_input_contract.py, test_spm_measurement_boundary.py and test_microsim.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
Closing as superseded by #9467 (merged 2026-09-15 as 5115fb8), on the US + core hub's PR sweep. #9467's source table names this PR's current head,
Its "What was dropped, and why" section explains the remaining hunks: the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SPM housing valuation currently leaks into general household benefits, CBO transfer income and CARE/FERA income. A change to an SPM threshold can therefore change a non-SPM result without changing the household's assistance. Use the modeled housing award for general benefits and CBO income, apply the CPUC housing-income exclusion from its effective date, and keep SPM caps in SPM resource calculations. A dependency guard covers both housing and work/childcare caps.
Independent simulations currently share policy and cache state. Give them their own policy ownership and tracing state, preserve deliberate branch-family sharing when a reform replaces parameters or variables, and reject saved poverty outputs as population inputs. Retain warm caches when reusing a prepared root, clone or variable-only reform wrapper at the same structural start. Detach before applying newly detected structural reforms at a different start so arbitrary callbacks cannot mutate the shared source. Verify the bytes of the documented default dataset after download. Fix the test batch runner so the child exit code, complete YAML discovery and a real deadline determine the result.
For SPM resource valuation, sum modeled housing awards within each household, allocate them to SPM units by member share, and apply each unit's housing cap. Allocate tenant payments only from units with positive awards using the same shares. Preserve multiple actual awards, exclude nonrecipients' hypothetical payments, and require SPM geography/composition only where there is an allocated award. Document the producer's assisted-family and timing assumptions and distinguish the tenant-payment allocation assumption from verified Census parity. Reject both new calculated allocation quantities as dataset inputs.
Declare the observed independent-minor role as a native dataset source input even though household examples have a fallback formula. Keep it disjoint from calculated SPM outputs; the producer must carry the observed boolean rather than synthesize a default. Four source-role controls pass in both Core configurations.
The nonpublishing CI matrix builds distinct rc1 and final candidate wheels from clean PR-head checkouts using the real version-bump and guarded lock-refresh helpers. Match the publishing toolchain, pin the isolated build backend closure, and retain actual wheel and source receipts. This keeps tests that mutate an itemization parameter out of the release build.
Validation:
Fable approved the exact source revision
8f74a5152d1e52c33bd7a7cba6340346bd7fe004against base2ba7daa30e0e79a40b0597c123e4df76e09b6101, with no actionable findings. Actual dedicated Ubuntu rc1/final CI wheel artifacts were authenticated, including all installed rc1 members; these prebinding artifacts do not qualify the future dataset.Draft release hold: the associated Microcosm receipt/immigration producer changes, new dataset build, final default-data binding, exact model/data qualification and topline comparison are still in progress. Existing published data certificates do not certify changed producer outputs. This PR does not claim the corrected stack is live or change dependency pins on downstream services.