Score the post-export release gates over household batches (Route A PR-4)

CLAUDE.md

@@ -118,6 +118,12 @@ to the private repository; when you run `tools/build_uk_rowwise_candidate.py`
 yourself, pass `--staging-local-only` unless the operator asked for a staged
 upload.
 
+The US fiscal-refresh builder scores its written H5 in household batches.
+Before a release rerun, run the small-H5 guard sweep described in
+[the release build rule](docs/us-release-build-rule.md#post-export-scoring).
+That fixture check is separate from full-export timing and release
+certification.
+
 The US native-SPM-role source-enrichment lane is a separate release type:
 `tools/build_us_spm_role_enrichment.py` creates a local candidate from the exact
 reviewed BuildP parent, preserving original variables and inherited schema-5

changelog.d/us-batched-post-export-scoring.fixed.md

@@ -0,0 +1 @@
+The US release's reform-coverage smoke, reform validation and demographics score the written H5 in household batches, bind its SHA-256 to the release manifest, and evaluate each baseline plan in ascending period order. Each reform shares one tax-benefit system across its batch engines. Guards refuse watched population-aggregate formulas in multi-batch passes, watched baseline-branch readers on reform engines, and changes to behavioral-response parameters. Real-engine tests exercise the refusals and compare shipped baseline plans and selected reform totals with whole-file engines; a separate sequential-worker sweep checks every shipped smoke and out-of-sample validation reform, including OBBBA stacks and in-sample fallback simulations. The MD CCS request-order premise is observed with a warning if the upstream error disappears, not asserted to persist. Writer tests pin export-before-scorer ordering and calibrated in-sample JCT estimates. After constructing the export frame, the build drops calibrated target frames; exact-k retains the clean export frame for its receipt's household-count check. Poverty and SPM outcomes remain record-only.

docs/us-release-build-rule.md

@@ -79,16 +79,19 @@ waits on that fix.
 
 In policyengine-us 2.2.1 one SPM unit with no classified adult (age 18 or over,
 or age 15 or over with `is_spm_independent_minor_role`) refuses the SPM
-measurement for the whole population. The first whole-dataset simulation of the
-written release file is the reform-coverage smoke gate
-(`tools/build_us_fiscal_refresh_release.py:11657`), which runs after the export
-write (`:11641`) and before the calibration NPZ (`:11723`); its LIHEAP probe
-reaches SPM composition through `spm_unit_benefits`
-(`tools/build_us_release_input_coverage_manifest.py:513-520`), so a population
-with such a unit refuses there. The reform-validation diagnostics that follow
-the NPZ (`_write_reform_validation`, near `:11725`; 104 `in_poverty` rows from
-`us/state_spm_poverty_levels.json` on one whole-dataset simulation) would
-refuse the same way; they record, they do not gate.
+measurement for every unit in the simulation that holds it. The first
+simulation of the written release file is the reform-coverage smoke gate
+(`us_reform_coverage_smoke_gate` in `tools/build_us_fiscal_refresh_release.py`),
+which runs after the export write and before the calibration NPZ; its LIHEAP
+probe reaches SPM composition through `spm_unit_benefits` (the probe list in
+`tools/build_us_release_input_coverage_manifest.py`), so a population with such
+a unit refuses there. The reform-validation diagnostics that follow the NPZ
+(`_write_reform_validation`; 104 `in_poverty` rows from
+`us/state_spm_poverty_levels.json`) would refuse the same way; they record, they
+do not gate. These stages now score the written file in household batches, not
+one whole-dataset simulation each (see
+[Post-export scoring](#post-export-scoring)); the batch that holds such a unit
+refuses, and the refusal still aborts the stage.
 
 The certified default passes because `tools/build_us_spm_role_enrichment.py`
 added the role afterwards, and that tool accepts only Build P's exact bytes
@@ -176,8 +179,14 @@ stable source IDs, and their outputs are compared with or merged into the
 original assembled table. This lets historical source kernels consume the
 projection without treating intentionally removed provenance as corruption.
 
-`--dense-default-dataset` is diagnostic only. A release build leaves it unset,
-so the default is the sparse dataset that runs on standard machines.
+Until 23 September 2026, `--dense-default-dataset` was diagnostic only and a
+release build left it unset, so the default was the sparse dataset that runs on
+standard machines. Decided 23 September 2026 (microcosm#956): route A's first
+certified release, which is not the default, calibrates the whole base dense
+(`--dense-default-dataset`) on the national and state target surface
+(`--target-surface national_state`), and sparse L0 tuning is pursued after it.
+The standing principle for the graph era is the same target surface for every
+dataset size, with L0 the only difference across sizes.
 
 ### 4. Three raw inputs exist in one untracked directory
 
@@ -194,6 +203,99 @@ and size and fetches them as it fetches the SIPP donor; the base stage verifies
 the digest it is given. `us/spec/sources.yaml` is generated from a sealed
 six-role receipt and does not change until a real build re-cuts it.
 
+## Post-export scoring
+
+The reform-coverage smoke, reform validation and demographics score the written
+release H5 through one household-batched scorer
+(`_HouseholdBatchedPostExportScorer` in
+`tools/build_us_fiscal_refresh_release.py`, microcosm#956). Each used to build
+one Microsimulation over the whole file.
+
+- The scorer loads the written H5 once, binds its SHA-256, and refuses to let
+  the release manifest pin different bytes.
+- It refuses a file whose group units do not each nest in one household, or
+  whose batches do not partition every entity, before any engine exists.
+- Batches hold at most `--maximum-microsim-batch-size` households (default
+  5,000; route A runs 2,000). Every engine declares the release SPM selection
+  (`US_RELEASE_SPM_SELECTION`, county geography) and is released before the
+  next batch builds.
+- Each stage's baseline requests are learned before export by an engine-free
+  dry run and scored in one pass over the batches, in ascending period order.
+  One engine refuses a period earlier than one it has already computed.
+  The real-engine test probes the MD CCS request-order premise by requesting
+  2025 state taxes after 2027 keys. It checks for the
+  `md.msde.ccs.payment.informal.rates` error and warns if that error is absent;
+  it does not require the upstream bug to persist.
+- Each reform builds one tax-benefit system and scores its measure batch by
+  batch. Each batch engine gets that system alone, without `reform=`: given
+  both, policyengine-us creates a baseline branch, which clones the system.
+- Every statistic reads full-length values with their engine weights.
+
+The totals equal a whole-file simulation's up to floating-point summation order
+only if every scored measure is additive across households and never reads the
+`baseline` branch that a batch reform engine lacks. The scorer rejects these
+known violations at run time:
+
+- In a multi-batch pass, an engine refuses once it has computed a formula that
+  aggregates over its whole simulation. In policyengine-us 2.2.1 these are the
+  Medicaid SLCSP state sums behind `medicaid_cost` and the weighted household
+  and SPM-unit income deciles (`POST_EXPORT_POPULATION_AGGREGATE_VARIABLES`).
+  Each batch would compute them over itself alone.
+- A reform engine refuses once it has computed a formula that reads the
+  baseline branch (`POST_EXPORT_BASELINE_BRANCH_READERS`). These are the
+  Medicaid denominator a reform holds at baseline, and the behavioral-response
+  measurements.
+- A reform pass refuses a reform that sets the labor-supply or capital-gains
+  response parameters off their baseline values. Without a baseline branch,
+  those responses score 0. A whole-file engine also scores 0 at the engine's
+  zero default elasticities.
+
+Tests pin the watched lists against the installed engine's formula sources,
+exercise each guard on a real engine, and compare the shipped baseline plans
+(18 smoke keys, 75 validation keys and 1 demographics key) against whole-file
+values on a small written H5 in three batches.
+
+On 2026-09-24, the sweep below completed on a written fixture containing eight
+households across five states, with Medicaid enrollees, under policyengine-us
+2.2.1. All 107 reform passes cleared the guards in three batches: 41 smoke
+passes and 66 validation passes, covering 52 out-of-sample specs, 12 in-sample
+fallbacks and two OBBBA pre-baselines. The 18 OBBBA stacked provision states
+were included. All three baseline plans and four selected reform passes
+matched whole-file values at `rtol=1e-12, atol=1e-6`; their weights matched
+exactly. This is evidence for the fixture's calculation paths.
+
+The complete reform sweep is a separate pre-rerun step:
+
+```bash
+uv run --no-sync python tools/sweep_us_post_export_scoring.py \
+  --dataset-path /path/to/small-written-export.h5 --batch-size 3 \
+  --output /path/to/guard-sweep.json
+```
+
+The input must contain enough households for at least two batches. The tool
+records the shipped consumers' requests, including every out-of-sample
+validation reform and the OBBBA pre-baseline and stacked states. It also
+exercises the fallback simulations used when an in-sample estimate is absent.
+It scores at most four reform passes per worker, serially, and compares all
+baseline values and four reform passes with whole-file engines. Workers stop
+at a sampled 7 GiB RSS threshold and report their peak RSS, elapsed time,
+counts and the scored H5 digest.
+Run this fixture sweep before R4's full-export timing steps, and repeat it
+after a probe, spec or engine-lock change before starting a release.
+R4's existing full-export rehearsal times the smoke and validation
+baseline plans plus three selected reform passes; it does not cover every
+reform and does not replace this sweep.
+
+`calibration_diagnostics.json` records the plan under `build.post_export_scoring`
+(method, batch size, batch count, each stage's baseline keys and periods), and
+`reform_coverage_smoke.json` records what the smoke scored under
+`post_export_scoring`. Poverty rows stay record-only. Before the export write
+the build frees the target frame and the calibration result's target tables:
+`Frame.with_weights` copies every table, target columns included. Dense and
+L0 results drop those frames; exact-k substitutes the clean export frame for
+its receipt's later household-count check. The fixture sweep does not measure
+memory or wall time of the batched stages at full size.
+
 ## Measured stage times
 
 From the September attempt, on one 128 GiB machine, under policyengine-us

packages/microcosm-build/src/microcosm/build/us_runtime/reform_coverage_smoke.py

@@ -16,8 +16,10 @@
 
 The gate takes an injected ``simulate(reform) -> simulation`` (the same seam as
 :mod:`microcosm.build.us_runtime.reform_validation`), so it unit-tests without
-policyengine-us and runs live against the written release H5 in the build via
-:func:`microcosm.build.us_runtime.reform_validation.default_simulate_factory`.
+policyengine-us. In the build it runs live against the written release H5
+through the release tool's household-batched post-export scorer
+(``tools/build_us_fiscal_refresh_release.py``, microcosm#956), which serves the
+same seam from one engine per household batch.
 """
 
 from __future__ import annotations

packages/microcosm-build/src/microcosm/build/us_runtime/reform_validation.py

@@ -684,6 +684,19 @@ def _capped_weighted_total(
 US_RELEASE_SPM_SELECTION: dict[str, object] = {"geography_kind": "county"}
 
 
+def _released_engine_state(simulation: Any) -> bool:
+    """Release a finished simulation; True if it held engine state to sweep.
+
+    Callers run a full collection after releasing an object with a populations
+    dict (microcosm#456). The release tool's household-batched adapter releases
+    and collects its own batch engines; it and the engine-free recorder expose
+    no populations dict requiring another collection here.
+    """
+    holds_engine_state = isinstance(getattr(simulation, "populations", None), dict)
+    release_engine_simulation(simulation)
+    return holds_engine_state
+
+
 def default_simulate_factory(dataset_path: Path) -> SimulateFn:
     """Build a simulate() that runs a Microsimulation over the release H5."""
 
@@ -768,8 +781,8 @@ def released_weighted_total(
         try:
             return _weighted_total(transient_simulation, measure, at_period)
         finally:
-            release_engine_simulation(transient_simulation)
-            gc.collect()
+            if _released_engine_state(transient_simulation):
+                gc.collect()
 
     def stacked_obbba_effects() -> dict[str, tuple[float, float, float]]:
         """Score the OBBBA provisions *stacked* in their JCX-35-25 order.
@@ -1023,9 +1036,10 @@ def _level_total(level: BaselineLevelSpec) -> float | None:
     # microcosm#456: the shared baseline simulation has served every reform row
     # and baseline-level row by now; release it before assembling the payload.
     if baseline is not None:
-        release_engine_simulation(baseline)
+        sweep = _released_engine_state(baseline)
         baseline = None
-        gc.collect()
+        if sweep:
+            gc.collect()
 
     has_out_of_sample = any(not spec.in_sample for spec in specs) or bool(
         baseline_levels

packages/microcosm-build/src/microcosm/build/us_runtime/release_gate_preflight.py

@@ -49,14 +49,14 @@
 5. **SPM measurement composition** — does every SPM unit have a classified
    adult? In ``spm-calculator`` 1.0.0 a *single* SPM unit whose members are all
    under 18, with none aged 15-17 carrying an SPM independence role, raises
-   ``SPMInputError("SPM_COMPOSITION_REQUIRED")`` for the **whole population's**
-   SPM measurement (``spm_calculator/policyengine_adapter.py``
+   ``SPMInputError("SPM_COMPOSITION_REQUIRED")`` for the SPM measurement of
+   **every unit in its simulation** (``spm_calculator/policyengine_adapter.py``
    ``policyengine_amount``: ``if np.any(adults < 1): raise``). The release's 104
-   state SPM poverty levels run on one whole-dataset ``Microsimulation``
-   (:func:`~microcosm.build.us_runtime.reform_validation.default_simulate_factory`)
+   state SPM poverty levels are measured on the written H5, one engine per
+   household batch (the release tool's post-export scorer, microcosm#956),
    *after* calibration, export and the NPZ write, so that engine traceback —
-   which names neither the unit nor a remedy — is what an operator gets hours
-   into a build. This reproduces the engine rule
+   which names neither the unit nor a remedy, and aborts the stage — is what an
+   operator gets hours into a build. This reproduces the engine rule
    (``adult = (age >= 18) | ((age >= 15) & role)``) over frame columns and names
    the offending units and the remedy in seconds.
 

packages/microcosm-build/src/microcosm/build/us_runtime/spm_composition.py

@@ -366,16 +366,18 @@ def check_spm_composition(
 ) -> CheckResult:
     """Does every SPM unit have a classified adult (no solve, no engine)?
 
-    The release's 104 state SPM poverty levels are measured on one whole-dataset
-    ``Microsimulation`` (``reform_validation.default_simulate_factory``), and in
-    ``spm-calculator`` 1.0.0 a *single* SPM unit with no classified adult raises
-    ``SPMInputError("SPM_COMPOSITION_REQUIRED")`` for the **whole population's**
-    measurement (``spm_calculator/policyengine_adapter.py`` ``policyengine_amount``:
+    The release's 104 state SPM poverty levels are measured on the written H5,
+    one engine per household batch (the release tool's post-export scorer,
+    microcosm#956), and in ``spm-calculator`` 1.0.0 a *single* SPM unit with no
+    classified adult raises ``SPMInputError("SPM_COMPOSITION_REQUIRED")`` for the
+    measurement of **every unit in its simulation**
+    (``spm_calculator/policyengine_adapter.py`` ``policyengine_amount``:
     ``if np.any(adults < 1): raise``), naming neither the offending unit nor a
-    remedy. The release tool runs this same classification on its calibrated
-    export frame as a batched pre-export gate, so a build refuses by name before
-    the H5 and NPZ writes — but only after paying for the calibration that frame
-    comes from. Run pre-solve, this is the same verdict in seconds.
+    remedy, and aborting the stage. The release tool runs this same
+    classification on its calibrated export frame as a batched pre-export gate,
+    so a build refuses by name before the H5 and NPZ writes — but only after
+    paying for the calibration that frame comes from. Run pre-solve, this is the
+    same verdict in seconds.
 
     This reproduces the engine's classification exactly —
     ``adult = (age >= 18) | ((age >= 15) & role)`` with ``role`` resolved by

packages/microcosm-build/tests/conftest.py

@@ -9,7 +9,6 @@
 import pandas as pd
 import pytest
 
-from microcosm.build.uk_runtime.ods_tables import ODS_MIME_TYPE
 from microcosm.frame import EntitySchema, Frame, WeightKind, Weights
 
 
@@ -77,10 +76,14 @@ def _write_ods(
     path: Path,
     tables_xml: str,
     *,
-    mimetype: str = ODS_MIME_TYPE,
+    mimetype: str | None = None,
     store_mimetype: bool = True,
     include_content: bool = True,
 ) -> Path:
+    if mimetype is None:
+        from microcosm.build.uk_runtime.ods_tables import ODS_MIME_TYPE
+
+        mimetype = ODS_MIME_TYPE
     with ZipFile(path, "w") as archive:
         archive.writestr(
             "mimetype",