Count housing subsidies as CPUC does, and tighten the dataset, default and batch guards

changelog.d/batched-exit-status.fixed.md

@@ -0,0 +1 @@
+Decide a test batch by the child process exit status rather than its printed summary, and enumerate `.yml` test files alongside `.yaml`.

changelog.d/cpuc-housing-subsidy-source.changed.md

@@ -0,0 +1 @@
+Count the housing subsidy a household reports receiving toward CA CPUC CARE/FERA income, in place of the SPM's capped housing subsidy. PG&E Form 01-9077 defines total gross annual household income as "all taxable and non-taxable revenues from all people living in the home, from whatever sources derived, including, but not limited to, ... housing and military subsidies", with no netting against rent and no cap; the SPM cap made CARE eligibility depend on which SPM geography a request selected, and the ungated `housing_assistance` would have counted a voucher the model imputes to any income-eligible renter.

changelog.d/reported-housing-assistance.added.md

@@ -0,0 +1 @@
+Add `reported_housing_assistance`, the housing assistance of a unit that reports receiving it, for income definitions that count subsidies received rather than modelled take-up.

changelog.d/spm-dataset-poverty-alias-guard.fixed.md

@@ -0,0 +1 @@
+Reject derived poverty aliases such as `in_poverty` from stored datasets.

changelog.d/spm-default-dataset-hash.fixed.md

@@ -0,0 +1 @@
+Verify the default dataset content hash so a moved build id cannot silently change every standalone-country result.

policyengine_us/parameters/gov/states/ca/cpuc/income_sources.yaml

@@ -1,13 +1,16 @@
 description: >-
   Income sources counted toward CA CPUC CARE/FERA eligibility.
   Total gross annual household income includes all taxable and
-  nontaxable revenues from all people living in the home, from
+  non-taxable revenues from all people living in the home, from
   whatever sources derived, including, but not limited to, wages,
   salaries, interest, dividends, spousal and child support payments,
   public assistance payments, Social Security and pensions, housing
   and military subsidies, rental income, income from self-employment
   and all employment-related, non-cash income.
-# PG&E Form 01-9077, CARE/FERA Program Application.
+# Quoted verbatim from the footnote to the CARE and FERA Income Guidelines
+# tables on PG&E Form 01-9077, CARE/FERA Program Application (Rev. 6.26); the
+# same footnote appears on the CPUC-filed tariff sheet. Each entry below names
+# the phrase it implements.
 values:
   0000-01-01:
   # Wages, salaries
@@ -36,7 +39,35 @@ values:
   - pension_income
   - retirement_distributions
   # Housing subsidies
-  - spm_unit_capped_housing_subsidy
+  #
+  # The form lists "housing and military subsidies" among the revenues that
+  # make up total gross annual household income, with no netting against rent,
+  # no cap and no poverty-measurement valuation anywhere on it - so the entry
+  # is the housing subsidy itself, at the amount the household receives.
+  #
+  # reported_housing_assistance is housing_assistance gated on
+  # receives_housing_assistance, an input defaulting to false. The ungated
+  # variable is not a report of receipt: is_eligible_for_housing_assistance is
+  # receives_housing_assistance | (is_renter & is_income_eligible) and
+  # takes_up_housing_assistance_if_eligible defaults to true, so counting it
+  # would attach an imputed voucher to every income-eligible renter who
+  # supplies a rent: a Los Angeles renter earning $35,000 and paying $24,000
+  # of rent, sending receives_housing_assistance: false, carries $11,580 of
+  # modelled assistance against the payment standard the model derives for the
+  # county, and $13,500 against a supplied $36,000 standard - either way out
+  # of CARE, for a subsidy nobody receives. The form asks an applicant what
+  # the household receives, so the entry counts receipt.
+  #
+  # It was spm_unit_capped_housing_subsidy, the Census SPM's housing subsidy
+  # capped at the housing portion of the SPM threshold less the tenant
+  # payment. That cap belongs to the poverty measure, not to this form, and it
+  # made a utility discount depend on which SPM geography the request
+  # selected: a Los Angeles renter earning $40,700 with $23,790 of housing
+  # assistance had a $0 capped subsidy under national selection and $310.88
+  # under county selection, moving countable income across the CARE limit and
+  # flipping ca_care_eligible from true to false. Requesting CARE with no SPM
+  # configuration at all raised SPM_GEOGRAPHY_REQUIRED.
+  - reported_housing_assistance
   # Military subsidies
   - veterans_benefits
   - military_service_income
@@ -57,8 +88,14 @@ metadata:
   period: year
   label: CA CPUC CARE/FERA income sources
   reference:
-    - title: PG&E Form 01-9077, CARE/FERA Program Application
-      href: https://bha.berkeleyca.gov/sites/default/files/2022-04/PGE%20CARE%20FERA%20Application.pdf
+    - title: PG&E Form 01-9077, CARE/FERA Program Application (Rev. 6.26)
+      href: https://www.pge.com/assets/pge/localized/en/docs/account/billing-and-assistance/care-fera-application.pdf
+    - title: >-
+        Revised Cal. P.U.C. Sheet No. 61313-E, PG&E Electric Sample Form
+        No. 79-1055, CARE/FERA Program Application for Sub-Metered
+        Residential Customers (Large Print), Advice 7905-E, effective
+        June 1, 2026 - carries the same income footnote as a CPUC filing
+      href: https://www.pge.com/tariffs/assets/pdf/tariffbook/ELEC_FORMS_79-1055.pdf
     - title: CA PUC 739.1 (CARE)
       href: https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=PUC&sectionNum=739.1
     - title: CA PUC 739.12 (FERA)

policyengine_us/spm.py

@@ -34,6 +34,35 @@
 
 COUNTY_FIPS_PATTERN = re.compile(r"[0-9]{5}")
 
+# Poverty outputs this country derives from the calculator's measurement.
+#
+# ``FORMULA_OWNED_INPUTS`` names what the calculator owns: the thresholds, the
+# SPM resource total, and the canonical indicators ``poverty_line``,
+# ``poverty_gap``, ``spm_unit_is_in_spm_poverty`` and
+# ``spm_unit_is_in_deep_spm_poverty``. These names are country-side functions
+# of those same values - ``in_poverty`` of ``poverty_gap``,
+# ``deep_poverty_line`` of ``poverty_line``, ``deep_poverty_gap`` of
+# ``deep_poverty_line`` and ``spm_unit_net_income``, ``in_deep_poverty`` of
+# ``deep_poverty_gap``, and ``person_in_poverty`` of ``in_poverty`` - so a
+# dataset column for any of them is an alias for a formula-owned output under a
+# name the calculator does not police. Accepting one lets the poverty
+# calculation documented in ``docs/usage/microsimulation.md`` and the canonical
+# indicator publish different poverty rates for the same population: a dataset
+# storing ``in_poverty=[False, False]`` alongside a population every unit of
+# which is below its threshold reports 0% and 100% at once.
+DERIVED_POVERTY_OUTPUTS = frozenset(
+    {
+        "deep_poverty_gap",
+        "deep_poverty_line",
+        "in_deep_poverty",
+        "in_poverty",
+        "person_in_poverty",
+    }
+)
+
+# Every name a dataset must not store, from either side of the boundary.
+REJECTED_DATASET_INPUTS = frozenset(FORMULA_OWNED_INPUTS) | DERIVED_POVERTY_OUTPUTS
+
 COUNTY_INPUT_FIX = (
     'send county_fips as a five-digit string (for example "06037"), or select '
     'geography_kind="national" in the spm configuration'
@@ -347,7 +376,7 @@ def set_input(self, variable_name, period, value):
         # public consumers validate their household input contract separately.
         if (
             getattr(self, "is_over_dataset", False)
-            and variable_name in FORMULA_OWNED_INPUTS
+            and variable_name in REJECTED_DATASET_INPUTS
         ):
             raise ValueError(
                 f"Dataset supplies formula-owned SPM output {variable_name}. "

policyengine_us/system.py

@@ -1,3 +1,5 @@
+import hashlib
+from functools import lru_cache
 from pathlib import Path
 from policyengine_core.taxbenefitsystems import TaxBenefitSystem
 from policyengine_us.entities import *
@@ -53,6 +55,16 @@
 # Populace ships from a Hugging Face *dataset* repo, hence the `hf://datasets/`
 # prefix handled in `_resolve_dataset_path`.
 DEFAULT_DATASET = "hf://datasets/policyengine/populace-us/populace_us_2024.h5@populace-us-2024-spm-20260909"
+# Content hash of the certified build the default URI must resolve to.
+#
+# A Hugging Face tag is a mutable pointer: re-tagging the repository, or a
+# truncated or substituted download, would hand the model another schema-valid
+# H5 and change every standalone-country result with no rejection anywhere.
+# Only the country's own default is checked - an explicit `dataset=` argument
+# names the caller's own artifact and remains the caller's responsibility.
+DEFAULT_DATASET_SHA256 = (
+    "6496cc4393d4d3c6574f76eca231de5898c803b9067645591fd5c4d3e65aee84"
+)
 
 
 class CountryTaxBenefitSystem(TaxBenefitSystem):
@@ -312,6 +324,44 @@ def _resolve_dataset_path(dataset_str):
         raise FileNotFoundError(f"Dataset file not found: {dataset_str}")
 
 
+def _file_sha256(file_path):
+    digest = hashlib.sha256()
+    with open(file_path, "rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+@lru_cache(maxsize=None)
+def _checked_default_dataset_digest(file_path, size, mtime_ns):
+    """Digest the resolved default build once per process.
+
+    The certified build is ~830 MB, so a per-construction digest would add
+    about a third of a second to every default `Microsimulation`. Key the
+    digest on the resolved path together with its size and modification time,
+    so a replaced or re-downloaded cache entry is digested again rather than
+    inheriting the previous file's verdict.
+    """
+    return _file_sha256(file_path)
+
+
+def _verify_default_dataset(file_path):
+    """Reject a default build whose bytes are not the certified ones."""
+    stat = Path(file_path).stat()
+    actual = _checked_default_dataset_digest(
+        str(file_path), stat.st_size, stat.st_mtime_ns
+    )
+    if actual != DEFAULT_DATASET_SHA256:
+        raise ValueError(
+            f"The default dataset {DEFAULT_DATASET} resolved to {file_path}, "
+            f"whose content is not the certified build: expected sha256 "
+            f"{DEFAULT_DATASET_SHA256}, got {actual}. The build id in the "
+            "default URI points at different bytes than this model version "
+            "was certified against; upgrade policyengine-us, or pass an "
+            "explicit dataset= argument to use that artifact deliberately."
+        )
+
+
 def _is_hdfstore_format(file_path):
     """Check if an HDF5 file uses entity-level HDFStore format.
 
@@ -362,6 +412,9 @@ def __init__(self, *args, **kwargs):
         )
 
         dataset = kwargs.get("dataset")
+        # Only the build this model version chose for itself is content-checked
+        # below; a caller-supplied dataset= is the caller's own artifact.
+        uses_country_default = dataset is None
         if dataset is None:
             # Route the class default through the same interception below as an
             # explicit dataset, so an entity-level (HDFStore) default such as
@@ -381,6 +434,10 @@ def __init__(self, *args, **kwargs):
         # entity-level datasets, making this interception unnecessary.
         if dataset is not None and isinstance(dataset, str):
             local_path = _resolve_dataset_path(dataset)
+            if uses_country_default and dataset == DEFAULT_DATASET:
+                # A subclass may point `default_dataset` elsewhere; only the
+                # shipped default is certified by DEFAULT_DATASET_SHA256.
+                _verify_default_dataset(local_path)
             if _is_hdfstore_format(local_path):
                 from policyengine_us.data.economic_assumptions import (
                     extend_single_year_dataset,