Skip to content

Keep SPM valuation out of program benefits and isolate simulation policy - #9448

Closed
MaxGhenis wants to merge 9 commits into
mainfrom
max/spm-isolation-fixes-20260912
Closed

MaxGhenis wants to merge 9 commits into
mainfrom
max/spm-isolation-fixes-20260912

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

SPM housing valuation currently leaks into general household benefits, CBO transfer income and CARE/FERA income. A change to an SPM threshold can therefore change a non-SPM result without changing the household's assistance. Use the modeled housing award for general benefits and CBO income, apply the CPUC housing-income exclusion from its effective date, and keep SPM caps in SPM resource calculations. A dependency guard covers both housing and work/childcare caps.

Independent simulations currently share policy and cache state. Give them their own policy ownership and tracing state, preserve deliberate branch-family sharing when a reform replaces parameters or variables, and reject saved poverty outputs as population inputs. Retain warm caches when reusing a prepared root, clone or variable-only reform wrapper at the same structural start. Detach before applying newly detected structural reforms at a different start so arbitrary callbacks cannot mutate the shared source. Verify the bytes of the documented default dataset after download. Fix the test batch runner so the child exit code, complete YAML discovery and a real deadline determine the result.

For SPM resource valuation, sum modeled housing awards within each household, allocate them to SPM units by member share, and apply each unit's housing cap. Allocate tenant payments only from units with positive awards using the same shares. Preserve multiple actual awards, exclude nonrecipients' hypothetical payments, and require SPM geography/composition only where there is an allocated award. Document the producer's assisted-family and timing assumptions and distinguish the tenant-payment allocation assumption from verified Census parity. Reject both new calculated allocation quantities as dataset inputs.

Declare the observed independent-minor role as a native dataset source input even though household examples have a fallback formula. Keep it disjoint from calculated SPM outputs; the producer must carry the observed boolean rather than synthesize a default. Four source-role controls pass in both Core configurations.

The nonpublishing CI matrix builds distinct rc1 and final candidate wheels from clean PR-head checkouts using the real version-bump and guarded lock-refresh helpers. Match the publishing toolchain, pin the isolated build backend closure, and retain actual wheel and source receipts. This keeps tests that mutate an itemization parameter out of the release build.

Validation:

  • Final policy-family tests pass 27 cases on each Core version, plus eight existing ownership controls on Core 3.32.5. The tests include real-root reform wrappers and prove detachment occurs before both direct parameter updates and modifier callbacks.
  • The affected policy-family and SPM-system modules pass 69 tests with each of Core 3.30.2 and 3.32.5. New regressions first reproduced clone/nested-clone/reform-wrapper cache rebuilding and stale public parameter-lookup tracing; unchanged and structurally modified roots now take the correct paths.
  • The 33 release-lock and build-artifact guard tests pass, including the real uv resolver probe.
  • 79 runtime/SPM tests pass with Core 3.30.2 and the selected release dependencies (Core 3.32.5); two behavioral-response end-to-end tests pass.
  • 115 benefit, CARE/FERA and reform tests pass in both dependency configurations, including unchanged partner fixtures.
  • Independent reviewers approved the policy ownership, benefit boundary and batch-runner changes. The ownership review added six adversarial probes; batch review exercised four real subprocess cases.
  • The combined allocation/integration suite passes 108 Python tests and three existing single-unit YAML cases in each dependency configuration. Independent allocation review adds four probes per configuration, covering interleaved and permuted mixed households, actual HUD awards, nonrecipient payments, and single-unit parity; no actionable findings remain.

Fable approved the exact source revision 8f74a5152d1e52c33bd7a7cba6340346bd7fe004 against base 2ba7daa30e0e79a40b0597c123e4df76e09b6101, with no actionable findings. Actual dedicated Ubuntu rc1/final CI wheel artifacts were authenticated, including all installed rc1 members; these prebinding artifacts do not qualify the future dataset.

Draft release hold: the associated Microcosm receipt/immigration producer changes, new dataset build, final default-data binding, exact model/data qualification and topline comparison are still in progress. Existing published data certificates do not certify changed producer outputs. This PR does not claim the corrected stack is live or change dependency pins on downstream services.

MaxGhenis added a commit that referenced this pull request Sep 15, 2026
The review's F2. Two of the carried changes move published numbers, and both
fragments described only the mechanism.

The member-share allocation shifts SPM resources between co-resident SPM units,
so SPM poverty and deep poverty change for multi-unit households where any unit
receives housing assistance - and only for those; a single-unit household's
allocation is its own award
(`test_single_unit_allocation_preserves_program_amount`). An awarded family's
tenant payment can now zero a co-resident unit's housing resource
(`test_independent_unit_caps_do_not_redistribute_unused_housing_value`), which
the fragment names as an assumption.

Counting actual `housing_assistance` in `household_benefits` and
`cbo_means_tested_transfers` raises household benefits, household net income,
marginal tax rates and CBO transfer aggregates for every assisted household by
whatever the SPM cap withheld, without moving SPM poverty, and CARE/FERA
countable income loses the housing entry from 2015 for annual calculations.

Both are retyped `changed` rather than `fixed`: they are not defect repairs to
a caller, they are moved numbers. No `breaking` fragment is added.

`spm-supplied-policy-caches.fixed.md` returns, dropped from #9448 while
`test_supplied_prepared_system_retains_warm_policy_children` and its
neighbours were carried. It describes the behaviour this branch ships, not
#9448's mechanism for it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Sep 15, 2026
The review's F5. Both were run against this branch's mechanism before deciding:

    test_cached_formula_trace_binds_replacement_tracer[False]  FAILED
    test_cached_formula_trace_binds_replacement_tracer[True]   FAILED
    test_system_parameter_lookup_does_not_cache_tracer         FAILED

One root cause, observed directly. Core memoises the
`TracingParameterNodeAtInstant` in the root parameter node's
`_at_instant_cache`, bound to whichever tracer built it. #9463 isolates a
traced request by giving it its own root with an empty at-instant cache, keyed
on the `trace` setter; replacing a tracer in place afterwards - a bare
`simulation.tracer = FullTracer()`, or a `parameters.tracer` write on a system -
reaches none of that, and core's per-formula soft recast in
`Simulation._run_formula` rewrites only the root's own `trace`/`tracer` fields,
not the memo. So the second tracer records nothing: for the system-lookup case,
iteration 0 records one parameter and iteration 1 records zero, with the access
going to the first tracer's closed tree.

Passing them means rebinding that memo at lookup, which is #9448's mechanism
rather than this one, so they are not forced. The boundary is recorded on
`test_traced_simulations_record_their_own_parameter_accesses`, which covers the
supported path, with the note that the only bare tracer write left in this
package is `tools/branched_simulation.py`, which nothing constructs.

The policy-family docstring said #9463 reverses child-to-parent propagation,
full stop. It reverses it for parameter reforms; a variable-only reform on a
shared-policy branch still reaches its parent and siblings, and
`test_a_branch_variable_reform_is_not_isolated_from_its_parent` pins that. The
correction also records what the dropped cases actually carried: #9448's
`test_child_reform_updates_warm_parent_and_sibling` parametrizes a parameter
dictionary against a root-replacing `modify_parameters` reform, so those cases
are contradicted rather than merely duplicated, and the drop stands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit to DTrim99/policyengine-us that referenced this pull request Sep 15, 2026
From PolicyEngine#9448, reduced. General household benefits, CBO means-tested transfers
and the three Tlaib household-benefit overrides now add actual
`housing_assistance` instead of the Census SPM's capped housing subsidy, so a
change to an SPM threshold no longer moves a non-SPM result. SPM resources keep
the capped valuation, now allocated across SPM units by member share.

CARE/FERA countable income excludes housing subsidies from the August 14, 2014
parameter date of CPUC D.14-08-030; annual values resolve at January 1, so
calendar 2014 retains the earlier modelled inclusion (using the actual benefit)
and calendar 2015 first applies the exclusion.

Also carries PolicyEngine#9448's release-candidate wheel tooling and the isolated build
backend pin.

Dropped from PolicyEngine#9448: its `spm.py` and `system.py` isolation hunks (PolicyEngine#9463 owns
the mechanism), its default-dataset hash and batch-runner hunks (PolicyEngine#9464 owns
those), its `docs/usage/microsimulation.md` edit (PolicyEngine#9463's is more specific),
its `reforms/reforms.py` change (no consumer) and its
`tests/core/test_spm_system.py` edits (they invert PolicyEngine#9463's contract).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit to DTrim99/policyengine-us that referenced this pull request Sep 15, 2026
…tatus

From PolicyEngine#9464, reduced.

`REJECTED_DATASET_INPUTS` extends the calculator's `FORMULA_OWNED_INPUTS` with
the country-side poverty aliases (`in_poverty`, `in_deep_poverty`,
`deep_poverty_line`, `deep_poverty_gap`, `person_in_poverty`) and with the two
SPM housing-allocation outputs PolicyEngine#9448 adds, so a dataset can no longer store a
formula-owned measurement under a name the calculator does not police.
`DATASET_SOURCE_INPUTS` keeps `is_spm_independent_minor_role` declarable by a
producer, and the two sets are asserted disjoint.

`DEFAULT_DATASET_SHA256` pins the certified build the mutable Hugging Face tag
must resolve to, verified once per process and only for the country default; an
explicit `dataset=` argument remains the caller's own artifact.

The batch runner now decides a batch on the child's real exit status, reported
through a marker written before interpreter teardown, and enumerates `.yml`
alongside `.yaml` everywhere it lists files.

Dropped from PolicyEngine#9464: `reported_housing_assistance` and the CPUC entry that
counted it, plus the three tests that pinned housing assistance as countable -
CPUC D.14-08-030 excludes housing subsidies outright, so there is nothing to
gate on receipt and the variable had no other consumer. Its citation research
is folded into the CPUC parameter instead.

Adaptations: the dataset guard is grafted onto PolicyEngine#9463's `set_input`;
`test_spm_system.py`'s import block is the union of both PRs' needs;
`test_care_eligibility_is_the_same_under_every_spm_geography` expects
`(40_700, True)` because $40,700 of countable income is inside the 2024
one-person CARE limit of $40,880 once the subsidy is excluded; and the two
counting non-vacuity tests are replaced by their exclusion analogues so the
suite still fails if the housing entry is silently dropped rather than excluded.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit to DTrim99/policyengine-us that referenced this pull request Sep 15, 2026
Folds PolicyEngine#9464's citation research into PolicyEngine#9448's dated exclusion list. The ~25
non-housing entries regain a source: PG&E Form 01-9077 (Rev. 6.26) at PG&E's
own URL, plus Revised Cal. P.U.C. Sheet No. 61313-E (Electric Sample Form
79-1055, Advice 7905-E, effective June 1, 2026), which carries the same income
footnote as a CPUC filing. Both are annotated as being corrected: PG&E's
June 5, 2026 reply brief in A.26-01-003 states it does not use housing and
military subsidies in its income analysis and will revise the application. The
dead bha.berkeleyca.gov href is gone; all nine references return HTTP 200.

Records review item R8: FERA coverage of the exclusion rests on the 2026 brief,
not on D.14-08-030. Verified first-hand against the decision PDF - it contains
zero case-sensitive occurrences of "FERA", "Family Electric" or "739.12",
while the brief's Section B answers the income definition under Public
Utilities Code 739.1(a) and 739.12(a) for CARE and FERA alike.

Verified against the live parameter tree: 2013-01-01, 2014-01-01 and
2014-08-13 resolve to 27 entries including housing_assistance; 2014-08-14,
2015-01-01 and 2026-01-01 resolve to 26 without it, the only set difference
being housing_assistance, and all 27 names resolve to model variables.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit to DTrim99/policyengine-us that referenced this pull request Sep 15, 2026
Found while adjudicating whether PolicyEngine#9448's `test_spm_policy_family.py` could be
carried onto PolicyEngine#9463's mechanism, and reproduced here with public-API calls.

`clone_spm_system` hands core an ordinary instance to clone through -
`plain_policy_copy` -> `SharedParameterPolicy.unshared_copy()` resets
`__class__` to the base class - and never put the barrier back. So
`Simulation.clone()` returned a plain `CountryTaxBenefitSystem`, and
`apply_reform` on anything reading that tree took the unguarded path:

    clone = Simulation(situation=...).clone()
    clone.get_branch("probe").apply_reform({SINGLE standard deduction: 100_000})
    # clone's own SINGLE standard deduction moved 14,600 -> 100,000

That is the defect `SharedParameterPolicy` exists to prevent, and it
contradicts `test_a_branch_reform_does_not_rewrite_its_parents_policy`, which
only passed because it never cloned. `with_parameter_barrier`'s own docstring
already states the rule: every simulation's system carries the barrier, because
a tree it starts out owning can acquire readers later.

Bounded: a clone gets its own tree, so neither the shipped module-level system
nor the simulation the clone came from was reachable this way; the leak stayed
inside one cloned simulation's branch family. Verified before and after.

`test_cloning_a_shared_policy_system_stops_sharing_its_tree` asserted
`"shared_parameters" not in cloned.__dict__`, which pinned the defective
implementation rather than the invariant. It now asserts that the clone holds
its own tree behind the barrier; its two substantive assertions are unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit to DTrim99/policyengine-us that referenced this pull request Sep 15, 2026
…mily tests

PolicyEngine#9448's `test_spm_policy_family.py` cannot be carried whole: run unchanged
against PolicyEngine#9463's mechanism, 16 of its 27 cases fail. They split four ways.

Nine assert PolicyEngine#9448's own internals - the `_parameter_view` /
`_parameter_source` / `parameters.modified` provenance protocol and
`get_parameters_at_instant` - which this branch does not implement, because
PolicyEngine#9463 owns the mechanism.

Six turn on `Simulation.tracer` being a plain attribute in core, so a bare
`simulation.tracer = X` re-primes nothing and X records no parameter
receipts. PolicyEngine#9463 hooks the `trace` setter, `__init__`, `get_branch`,
`apply_reform`, `_adopt_detached_parameters` and `clone`, but not a bare
`tracer` write. No production caller does one that matters, so these are
dropped rather than fixed here.

Four assert that a child branch's reform propagates up to its parent and
siblings. Stated plainly: that is stock policyengine-core behaviour -
`Simulation.clone` hands a `clone_system=False` branch the SAME
tax-benefit system object and `apply_reform` mutates it in place - so PolicyEngine#9463
is a deliberate deviation from core here, not a restoration of it. It is still
the right deviation to ship: every branch in this repository that actually
mutates policy already passes `clone_system=True`
(`tob_revenue_oasdi.py:42`, `behavioral_response_measurements.py:49`), while
the `clone_system=False` branches only set inputs, and a formula-internal
branch that silently moved its parent's answer would be order-dependent. But
it is a design decision, not something core's contract compels.

One depends on a `reforms/reforms.py` hunk this branch drops, which returns
`None` from `create_structural_reforms_from_parameters` when no structural
reform is active. That case is unreachable - `create_mi_surtax_reform` always
returns a reform, so the filtered tuple is never empty - and carrying the hunk
verbatim is unsafe: `CountryTaxBenefitSystem.__init__` packs the result into a
reform tuple, and `CountryTaxBenefitSystem(reform=((), None))` raises
`AttributeError: 'NoneType' object has no attribute 'apply'` (verified).

The eleven cases kept here are the mechanism-independent remainder: a nested
branch clone's inherited inputs and private reform, a construction-time
structural reform detaching before it mutates parameters, the warm-cache
contract for a caller-supplied system across clone hops and a reform wrapper,
an at-instant view refreshing after an in-place leaf update, and a reform
reading back its own edit before it returns. All eleven pass.

Also pins the boundary of the barrier in `test_spm_simulation_isolation.py`:
it covers the parameter tree, not the variable registry, so a variable-only
reform on a shared-policy branch does still reach its parent and siblings.
PolicyEngine#9463 never claimed otherwise, but nothing recorded the limit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Sep 22, 2026
Re-levels #9462 (branch point 218 commits behind) onto 2.7.0. Ten content
conflicts; main wins everywhere its change is not the SPM scope separation,
and the scope separation is reapplied on top.

Drops this PR's `spm_unit_ordinary_housing_subsidy` bridge as superseded.
Main's d1251c4 ("Keep SPM valuation out of program benefits and CARE/FERA
income", the reduced #9448) already moved general household benefits, CBO
means-tested transfers, CARE/FERA countable income and the three Tlaib
household-benefit overrides onto the modelled `housing_assistance` award. That
is the same separation this PR's bridge was built for, reached without a new
variable: `housing_assistance` is a program amount for every unit, so those
aggregates never consult the measurement universe. Keeping the bridge would
have put the SPM-capped valuation back into ordinary income for included units
and made household net income itself depend on a scope declaration. The
independently qualified 2.2.1 backport (9b26e9b) reached the same conclusion
on a base that already contained d1251c4.

Conflict resolutions:

- parameters/gov/household/cbo_means_tested_transfers.yaml: main
  (`housing_assistance`).
- parameters/gov/household/household_benefits.yaml: main
  (`housing_assistance`) in both dated lists.
- parameters/gov/states/ca/cpuc/income_sources.yaml: main, including its new
  2014-08-14 value dropping housing subsidies per D.14-08-030.
- variables/household/income/household/household_benefits.py: main.
- reforms/congress/tlaib/{boost/boost_middle_class_tax_credit,
  economic_dignity_for_all_agenda/edaa_end_child_poverty_act,
  end_child_poverty_act}.py: main's lists, including the added
  `trump_dividend` entry and `housing_assistance`; the SPM-resource
  `spm_unit_benefits` overrides keep the scope mask.
- spm.py: main's structure, with the scope helpers added.
- variables/.../spm_unit_capped_housing_subsidy.py: main's allocated
  assistance and allocated tenant payment, with the scope mask.
- tests/microsimulation/test_microsim.py: reconciled to the post-2.2.1
  contract.

Also carries the four items the independently qualified 2.2.1 backport
(9b26e9b) has and this PR lacked, with their tests:

- `set_input` invalidates dependent caches when a `DATASET_SOURCE_INPUTS`
  name is set, and scope joins that set. Without it a scope declaration set
  after a dependent result was cached left the stale result in place.
- `CountyRequiringSPMProvider.with_county_input_types`, applied to the
  selected units in `masked_policyengine_amount`, so an excluded numeric
  county cannot fail a measurement that never used it.
- `spm_unit_oecd_equiv_net_income` and `spm_unit_income_decile` inherit the
  universe. Ranking over a population containing outside NaN income would
  otherwise corrupt every included unit's decile.
- `SPM_DISTRIBUTION_OUTPUTS` joins `REJECTED_DATASET_INPUTS`.

Test files taken from that backport: test_spm_universe.py,
test_income_decile_ties.py, test_spm_production_parity.py, and its edits to
test_spm_system.py, test_spm_source_input_contract.py,
test_spm_measurement_boundary.py and test_microsim.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #9467 (merged 2026-09-15 as 5115fb8), on the US + core hub's PR sweep.

#9467's source table names this PR's current head, 8f74a5152d, as "carried in part", and says it supersedes "the reduced parts of #9448". It carries:

  • the CPUC housing exclusion;
  • the household benefits and CBO transfer switch to actual housing_assistance;
  • the SPM household-to-unit housing allocation;
  • the release-build tooling;
  • the auto-routing test coverage.

Its "What was dropped, and why" section explains the remaining hunks: the spm.py and system.py isolation (owned by #9463) and the batch-runner implementation. No unique change from this branch is left unaccounted for. Reopen it if one turns up.

@MaxGhenis MaxGhenis closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant