Unify the SPM isolation, CPUC and dataset guard fixes

[MaxGhenis]

One candidate composed from four overlapping SPM drafts, so the pieces that
belong together can be reviewed and released once instead of four times.

Base: upstream/main @ 2cc6f045ab (version 2.1.0). Source heads, each read
with gh pr view <n> --json headRefOid:

PR	head	verdict
#9466	9f9510b735b3b9b2e375854d66fd315202e31d35	carried whole
#9463	d8cbd4b7a0b292beafad37cfc72813e1d710fc3e	carried whole, plus one bug fix
#9448	8f74a5152d1e52c33bd7a7cba6340346bd7fe004	carried in part
#9464	cafcd9443df7f638a5ad547a92c6e0bcfd784fbb	carried in part
#9462	47e94fa491bbf65ae4b6bd5667f054445b26de36	deliberately excluded

Supersedes #9463, #9464 and the reduced parts of #9448; #9462 (measurement universe) is deliberately not included and stays gated on Microcosm #925.

Where each hunk came from

Area	From	What
spm-calculator version range, uv.lock, fragment	#9466	whole PR, unchanged
Simulation isolation in spm.py (detach_parameters, _rebind_method_aliases, bind_private_entities, isolate_parameter_tracing)	#9463	whole mechanism
County-FIPS text contract	#9463
tools/pinned_tbs.py cache key, docs/usage/microsimulation.md, test_microsim.py	#9463
test_spm_simulation_isolation.py (22 tests) + test_spm_system.py isolation additions	#9463
Clone keeps the copy-on-write barrier	new	bug found here in #9463; see below
CPUC housing exclusion, dated at 2014-08-14	#9448
Household benefits / CBO transfers / three Tlaib reforms switch to actual housing_assistance	#9448
SPM household-to-unit housing allocation (2 new variables)	#9448	included; see below
test_cpuc_spm_independence.py, test_spm_measurement_boundary.py, test_spm_housing_allocation.py, test_spm_source_input_contract.py, CPUC YAML cases, Tlaib/household fixtures	#9448
.github/release_build.py, release-lock.md, tests/test_release_build.py, pr.yaml, push.yaml, [build-system] pins	#9448
test_spm_policy_family.py	#9448	11 of 27 cases; see below
REJECTED_DATASET_INPUTS derived-poverty-alias guard in set_input	#9464
DEFAULT_DATASET_SHA256 + _verify_default_dataset and its four tests	#9464
Batch exit-status shim (BATCH_EXIT_MARKER, batch_status) and its suite	#9464
.yml enumeration	#9464	#9448 converts the same 14 sites, so nothing extra to take
test_batched_per_subdir_budget.py (incl. test_auto_path_routes_keep_root_yml_files)	#9448	unique auto-routing coverage
CPUC citations for the ~25 non-housing sources	#9464	folded into #9448's list

What was dropped, and why

• Keep SPM valuation out of program benefits and isolate simulation policy #9448's spm.py and system.py isolation hunks — Isolate shared policy from later reforms and repair clone routing #9463 owns the
  mechanism. Keep SPM valuation out of program benefits and isolate simulation policy #9448's system.py contains nothing else; its default-dataset
  hash is superseded by Count housing subsidies as CPUC does, and tighten the dataset, default and batch guards #9464's.
• Keep SPM valuation out of program benefits and isolate simulation policy #9448's batch-runner implementation and test_batched_process_results.py
  — written against a timeout_seconds= API Count housing subsidies as CPUC does, and tighten the dataset, default and batch guards #9464 does not have, and one test
  asserts behaviour Count housing subsidies as CPUC does, and tighten the dataset, default and batch guards #9464 deliberately inverts.
• Keep SPM valuation out of program benefits and isolate simulation policy #9448's test_spm_system.py edits — three invert Isolate shared policy from later reforms and repair clone routing #9463's contract
  (policy.parameters is not system.parameters where Isolate shared policy from later reforms and repair clone routing #9463 pins is), five
  duplicate test_spm_simulation_isolation.py, one pins the dropped hash path.
• Keep SPM valuation out of program benefits and isolate simulation policy #9448's docs/usage/microsimulation.md edit — Isolate shared policy from later reforms and repair clone routing #9463's is more specific
  (names the test, the 18 units, and use_weights=False).
• Keep SPM valuation out of program benefits and isolate simulation policy #9448's reforms/reforms.py hunk — returns None from
  create_structural_reforms_from_parameters when no structural reform is
  active. Unreachable (create_mi_surtax_reform always returns one), and
  unsafe as written: CountryTaxBenefitSystem(reform=((), None)) raises
  AttributeError: 'NoneType' object has no attribute 'apply' (verified).
• Count housing subsidies as CPUC does, and tighten the dataset, default and batch guards #9464's reported_housing_assistance, its CPUC entry, its YAML test, its
  changelog fragment and the three tests that pin housing assistance as
  countable — the exclusion verdict below leaves nothing to gate on receipt, and
  the variable had no other consumer anywhere in the tree.
• Separate SPM measurement scope from ordinary income #9462 entirely.

The CPUC contradiction: exclude

#9448 and #9464 disagreed. #9464 counted housing subsidies, citing PG&E Form
01-9077 and the CPUC-filed tariff sheet Revised Cal. P.U.C. Sheet No. 61313-E
(Advice 7905-E, effective 1 June 2026). Both documents are genuine and both say
what #9464 said they say. They are also outranked.

CPUC D.14-08-030 §6.2 (physical p.75), read from the PDF:

Until such resolution, the income definition remains unchanged with one
exception. As discussed in Section 6.1.1.3 of this decision, housing
subsidies will not be considered income.

Ordering Paragraph 40(3) (physical p.124):

(3) Housing subsidies are not counted as income;

The decision is effective 14 August 2014 ("Dated August 14, 2014, San
Francisco, California"). A Commission decision defining income outranks a
utility sample form filed by advice letter — and the filing utility has since
told the Commission the form is wrong. PG&E's reply brief of 5 June 2026 in
A.26-01-003:

While PG&E's enrollment application lists housing and military subsidies as
part of income, in practice PG&E does not use housing and military subsidies
as part of its income analysis because those funds are designated for
housing, and not energy utility expenses. PG&E plans to revise its enrollment
application to remove the reference to housing and military subsidies.

So the entry is excluded, not revalued. #9464's citation research is kept: the
~25 non-housing sources now cite Form 01-9077 (Rev. 6.26) and Sheet 61313-E,
annotated that their housing-and-military line is the line being corrected.

FERA rests on the 2026 brief, not on the decision (review item R8). Verified
directly against the PDF: D.14-08-030 contains zero case-sensitive occurrences
of "FERA", "Family Electric" or "739.12". The brief's §B answers the income
definition under Public Utilities Code 739.1(a) and 739.12(a) — 739.12(a) is
FERA — for CARE and FERA alike. That reasoning is now recorded in the parameter.

The break is keyed at 2014-08-14. Annual values resolve at January 1, so
calendar 2014 keeps the earlier inclusion (using the actual benefit, not the SPM
cap) and calendar 2015 first applies the exclusion. Verified against the live
parameter tree: 2013/2014/2014-08-13 → 27 entries including
housing_assistance; 2014-08-14/2015/2026 → 26 without it, the only set
difference being housing_assistance.

The pre-2014 inclusion is carried forward from the existing model and was not
revalidated as historical law.

The housing allocation: included

It needs no new dataset input, so it goes in. Both new variables are pure
formulas over housing_assistance, hud_ttp and SPM-unit membership, and both
are added to REJECTED_DATASET_INPUTS so a dataset cannot supply either. I read
the released default H5 directly: its spm_unit table has 11 columns, and the
only housing-related ones are the booleans receives_housing_assistance and
takes_up_housing_assistance_if_eligible. There is no housing-dollar input, so
the dollars stay country-formula-owned.

A bug found in #9463, and fixed here

clone_spm_system hands core an ordinary instance to clone through —
plain_policy_copy → SharedParameterPolicy.unshared_copy() resets
__class__ to the base — and never put the barrier back, so
Simulation.clone() returned a plain CountryTaxBenefitSystem:

clone = Simulation(situation=...).clone()
clone.get_branch("probe").apply_reform({SINGLE standard deduction: 100_000})
# the clone's own SINGLE standard deduction moved 14,600 -> 100,000

That is the defect SharedParameterPolicy exists to prevent, and it contradicts
#9463's own test_a_branch_reform_does_not_rewrite_its_parents_policy, which
passed only because it never cloned. Live callers reach it:
tools/branched_simulation.py:19 and every get_branch(..., clone_system=True).
Bounded: a clone gets its own tree, so neither the shipped module-level system
nor the simulation the clone came from was reachable this way.

Fixed by ending clone_spm_system with with_parameter_barrier(cloned), which
is what that function's own docstring already prescribes. Two regression tests
added. test_cloning_a_shared_policy_system_stops_sharing_its_tree asserted
"shared_parameters" not in cloned.__dict__, pinning the defective
implementation rather than the invariant; it now asserts the clone holds its own
tree behind the barrier, with its two substantive assertions unchanged.

test_spm_policy_family.py: 11 of 27 cases

Run unchanged against #9463's mechanism, 16 of #9448's 27 cases fail. Nine
assert #9448's own internals; six turn on Simulation.tracer being a plain core
attribute that nothing re-primes; one needs the dropped reforms.py hunk; and
four assert that a child branch's reform propagates up to its parent and
siblings.

On that last group, stated plainly: that is stock policyengine-core
behaviour. Simulation.clone hands a clone_system=False branch the same
tax-benefit system object and apply_reform mutates it in place. #9463 is a
deliberate deviation here, not a restoration. It is still the right deviation —
every branch in this repository that actually mutates policy already passes
clone_system=True, the clone_system=False branches only set inputs, and a
formula-internal branch that silently moved its parent's answer would be
order-dependent — but it is a design decision, and reviewers should take it as
one.

The 11 kept cases are the mechanism-independent remainder and cover axes #9463's
suite leaves open: a nested branch clone's inherited inputs and private reform, a
construction-time structural reform detaching before it mutates parameters, the
warm-cache contract for a caller-supplied system across clone hops and a reform
wrapper, an at-instant view refreshing after an in-place leaf update, and a
reform reading back its own edit before returning.

Boundary now pinned: the barrier covers the parameter tree, not the variable
registry. A variable-only reform on a clone_system=False branch does still
reach its parent and siblings, because they share one variables dict. #9463
never claimed otherwise, but nothing recorded it; a new test does.

Review items

R1 (fourth ECPA fixture) and R2 (the three Tlaib fixtures discriminating actual
12,000 from capped 1,000) were already fixed in #9448's head — verified, not
assumed. R8 is recorded in the parameter. R9 is covered by #9448's CPUC Cases 4
and 5 plus test_cpuc_housing_exclusion_effective_date.

Verification

Python 3.14.4, policyengine-core 3.30.2, spm-calculator 1.0.0.

Suite	Result	Exit
ruff format --check .	6445 files already formatted	0
ruff check .	All checks passed	0
pytest tests/core tests/unit tests/code_health	see below
CPUC YAML (policy/baseline/gov/states/ca/cpuc)	36 passed	0
Tlaib YAML (policy/contrib/congress/tlaib)	22 passed	0
Reform YAML (policy/reform)	51 passed	0
.github/tests/test_release_*.py	33 tests, 1 skipped	0
.github/release_lock.py --committed	resolved 149 packages	0

No population simulation was run here; the microsimulation suites run in CI.

Please look at

CandidateWheel in pr.yaml runs release_lock.py --refresh on every pull
request, twice (rc1 and final), and that step runs a live uv lock against PyPI
and fails closed on any root-version drift. Today that resolver step runs only at
release time in push.yaml. This is #9448's design and is carried as-is, but it
means a PyPI hiccup can turn every open PR red, including PRs that touch only a
YAML fixture. Worth a maintainer's call before this merges.

Review, second pass

An adversarial review of this branch requested changes on six findings. Each is
addressed below, one commit per finding. Nothing in the first-pass body above is
retracted.

F1 (high) — the housing allocation extends SPM_COMPOSITION_REQUIRED to
co-resident non-recipient units. The allocation prorates a household's modelled
housing_assistance to every SPM unit by member share, and the capped subsidy
masks on a positive allocated share, so masked_policyengine_amount now demands
a classified measurement adult from units that held zero housing resource under
2.0.1. The only default-dataset microsimulation test subsampled 1,000 units, so
nothing ruled the pattern out. test_microsim.py now carries a full-population
probe with no subsample over the certified default build: it computes
spm_unit_allocated_housing_subsidy, spm_measurement_adults and
spm_unit_net_income for 2024, asserts the resource chain completes without
SPMInputError, and asserts that no unit holds an allocated share while
classifying no measurement adult - with the count in the message, so a nonzero
result names how many units are affected. The same allocation probe runs over
enhanced_cps_2024, which this suite already exercises and which is the one
population file known to carry units that classify no adult; each assertion
names its dataset. No guard was added. Whether the allocation should be
restricted to units that classify an adult, leaving the remainder with the
awarded unit, is a decision for a reviewer holding the count.

F2 (high) — two changes move published numbers, and no fragment said so.
spm-housing-allocation and care-housing-income are retyped changed and
rewritten around consequence rather than mechanism. The allocation moves SPM
resources between co-resident units, so SPM poverty and deep poverty change for
multi-unit households where any unit receives housing assistance and only for
those; an awarded family's tenant payment can zero a co-resident unit's housing
resource, named as an assumption. Counting actual housing_assistance in
household_benefits and cbo_means_tested_transfers raises household benefits,
household net income, marginal tax rates and CBO transfer aggregates for every
assisted household by whatever the SPM cap withheld, without moving SPM poverty,
and CARE/FERA countable income loses the housing entry from 2015 for annual
calculations. No breaking fragment. #9448's spm-supplied-policy-caches
fragment is restored, describing the behaviour this branch ships rather than
#9448's mechanism for it.

F3 (medium) — DEFAULT_DATASET_SHA256 was unverifiable inside the repo. It
was the only occurrence of that digest in the tree, and both tests that exercise
the check substitute their own value for it. docs/spm.md now has a Default
population build section carrying the hash beside the URI, with the date it was
read from the producer's build, and stating where the documented value meets the
published bytes: Microsimulation() digests whatever the tag resolves to, so the
microsimulation CI job that downloads the tag and constructs the default is the
check against the Hub. test_documented_default_build_matches_the_shipped_constants
reads the URI and the hash back out of that section and compares them with the
shipped constants, with no monkeypatching. The artifact was not downloaded here.

F4 (low/medium) — docs contradicted shipped metadata. docs/spm.md said the
published wheel requires exactly spm-calculator==1.0.0; pyproject.toml
requires >=1.0.0,<=1.0.0.post1, with uv.lock pinning 1.0.0.
.github/release-lock.md said the country lock was blocked until
spm-calculator==1.0.0 reached PyPI; it reached PyPI on 2026-09-11 and the
committed lock has resolved it from https://pypi.org/simple with sdist and
wheel hashes since. Both passages now describe what ships, and the release-lock
note records that the guard validates sources, URLs and hashes and never a
version string - .github/release_lock.py contains no calculator reference.

F5 (low) — carried-test accounting. #9448's two tracer tests were run against
this branch's mechanism before deciding, and all three cases fail:

test_cached_formula_trace_binds_replacement_tracer[False]  FAILED
test_cached_formula_trace_binds_replacement_tracer[True]   FAILED
test_system_parameter_lookup_does_not_cache_tracer         FAILED

One root cause, observed directly: core memoises the
TracingParameterNodeAtInstant in the root parameter node's
_at_instant_cache, bound to whichever tracer built it. #9463 isolates a traced
request by giving it its own root with an empty at-instant cache, keyed on the
trace setter, so replacing a tracer in place afterwards reaches none of that,
and core's per-formula soft recast in Simulation._run_formula rewrites only the
root's own trace/tracer fields rather than the memo. In the system-lookup
case the first tracer records one parameter access and the second records zero.
Passing them means rebinding that memo at lookup, which is #9448's mechanism
rather than this one, so they are not forced; the boundary is recorded on
test_traced_simulations_record_their_own_parameter_accesses.

The policy-family docstring is corrected: #9463 reverses child-to-parent
propagation for parameter reforms, and a variable-only reform on a
shared-policy branch still reaches its parent and siblings, pinned by
test_a_branch_variable_reform_is_not_isolated_from_its_parent. The correction
also records that every propagation case dropped from #9448 carried a parameter
reform - test_child_reform_updates_warm_parent_and_sibling parametrizes a
parameter dictionary against a root-replacing modify_parameters reform - so
those cases are contradicted rather than merely duplicated, and the drop stands.

F6 (low) — the retained pre-2014 CPUC inclusion counts a modelled benefit.
housing_assistance is HUD's payment where the unit is eligible
(receives_housing_assistance | (is_renter & is_income_eligible)) times a
take-up flag defaulting to true, and it is at least the capped valuation 2.0.1
counted in this list, so countable income rises for years before the decision and
a household near the limit can lose CARE or FERA eligibility there. Both the
parameter comment and the fragment now say so.

🤖 Generated with Claude Code

Full-population probe result

The whole-population probe over the certified default build (populace_us_2024) passed in CI at head 6c9d4ab5: zero SPM units hold an allocated housing share while classifying no measurement adult, so the member-share allocation raises no SPM_COMPOSITION_REQUIRED on the shipped data, and the resource chain completes on the subsample check. The same probe over enhanced_cps_2024 also passed. The hosted runner was killed twice while the probe computed the full-population resource chain on top of the allocation arrays; the count is still taken at full population and only the completion check runs on a 10,000-unit subsample. All 35 checks pass.