Gate the ACS local release on district ESS collapse (report-only by default)

[MaxGhenis]

Why

#1078 measured Kish ESS by state and district for the ACS local release ("ESS floor as a release gate" in experiments/us-acs-local-l2-basis-20260928/README.md). At the release's ACS share of 0.5:

• the release's three solves (full surface and two holdout folds) leave 15, 22 and 24 congressional districts below 25% of their design-weight ESS;
• every chi-square-penalized solve leaves none;
• an absolute district floor of 15 also separates them: the release's smallest district ESS is 7.8-11.7, and the penalized solves' smallest is at least 18.6.

Whether the gate should block a release is still open for review, so this PR builds the gate and records its result, but does not block by default. The tool already records the evidence: calibration_summary.json carries weight_origin.{design,calibrated}.effective_sample_size_by_district.

What changes

tools/build_us_acs_local_release.py, finalize stage:

• district_ess_collapse_gate reads both per-district ESS maps from the summary's weight_origin. It counts:

  • the districts that collapse: calibrated ESS below --district-ess-relative-floor (default 0.25) × design-weight ESS;
  • the districts below the floor: calibrated ESS below --district-ess-floor (default 15).

  A floor of 0 turns its check off. Each district list is sorted worst first. The gate also records the smallest district ESS and the smallest ratio.

• The result is recorded in three places:

  • gate_summary.json under gates.district_ess_collapse;
  • build_manifest.json, which embeds gates unchanged;
  • a reviewed-limitations entry, district_effective_sample_size_gate, which reaches release_manifest.json. Its text states the counts, the smallest district and whether the gate blocked.

• Blocking is opt-in (--district-ess-gate-blocking, default off).

  • Off: the gate is report-only. It passes and carries its verdict in criteria_met. This is required, not cosmetic: _check_local_area_gates in microcosm.data.contract refuses to publish a local-area release with any gate whose passed is not true, so a report-only gate recording passed: false would block publication anyway.
  • On: finalize adds the gate to simulation_readiness_blockers when any district collapses or falls below the floor. It does the same when the summary records no per-district ESS (fails closed).
  • Package under the flag refuses, before any release directory exists, a gate report that finalize did not evaluate as blocking at the same floors. Otherwise running --stage package --district-ess-gate-blocking after a report-only finalize would ship a collapse the flag exists to stop.

• The default solve is unchanged. The tool edit adds one self-contained block (constants, the gate, its limitation, the package check), five lines in do_finalize, one call in do_package, three flags with range checks, and a docstring line. Other open PRs edit this tool, so it touches no shared hunk. In particular it leaves Chi-square design-weight penalty and softmax mass parametrization for calibration; ACS local ESS frontier #1078's _ess_concentration_limitation and penalty flags alone.

The README and a changelog fragment document the flags.

Evidence: the gate on #1078's solves

I replayed district_ess_collapse_gate on the per-district ESS in #1078's 59 run receipts (results/runs/*.json, metrics.concentration.per_cd, kish_ess and kish_ess_design). On every one of the 57 solves in results/frontier.csv it reproduces exactly:

• cds_below_0.25_of_prior;
• cd_ess_min;
• n_cds (436).

There were 0 mismatches. The replay ran this session; it isn't committed, because the receipts live on #1078's branch. At share 0.5:

Solve	Collapsed (full / fold 0 / fold 1)	Below 15	Smallest district ESS
Release (projection, λ 0)	15 / 22 / 24	9 / 15 / 10	11.7 / 10.7 / 7.8
Softmax, λ 0	8 / 5 / –	2 / 1 / –	13.1 / 14.8 / –
Projection, chi-square λ 0.03	0 / 0 / 0	0 / 0 / 0	22.4 / 22.9 / 24.2
Softmax, chi-square λ 0.03	0 / 0 / 0	0 / 0 / 0	23.0 / 23.0 / 24.1

#1078's rerun of the release solve (dup_release_repro) gives 16 collapsed, not 15, so the count moves by at least 1 between reruns. At ACS share 0.9 the unpenalized solves leave 261-328 districts collapsed and none below 15. So with share 0.9, the relative gate is the one that binds.

Invariants

These are property-tested with Hypothesis in packages/microcosm-build/tests/engine_free/us/test_us_acs_local_district_ess_gate.py. The weight-origin evidence is built by cd_surface.weight_origin_summary, the function calibration_evidence uses.

1. Monotone thresholds. The collapsed districts are nested in the relative floor, and the below-floor districts in the absolute floor. Both counts are therefore monotone in their threshold.
2. A design-weight solve collapses nothing. With calibrated = design weights, no district collapses at any relative floor in [0, 1]. The same holds for a uniform rescaling of the design weights at any floor up to 0.99. The absolute floor is not relative: in a design-weight solve it flags exactly the design's own districts below the floor.
3. Bounds. Both counts are at most the number of districts, which equals the number of distinct district codes. criteria_met holds exactly when both counts are 0. The entry survives a JSON round trip unchanged.
4. Blocking changes only the pass flag. Blocking alters only passed, blocking and report_only. A report-only gate always passes. A blocking gate passes exactly when criteria_met, and the limitation's calibration_blocker is not passed.
5. Differential. The collapsed districts read from the summary equal those an independent np.bincount Kish ESS finds from the raw weights. Districts within 1e-9 relative of the threshold are excluded as floating-point ties, because the two compute sums in a different order.

The replay above is a sixth, differential check against #1078's analysis code. It was executed, not committed.

Intended violation. A report-only gate records passed: true even when districts collapse. This is pinned by test_the_release_contract_publishes_a_report_only_gate_whatever_it_measured, because the local-area contract publishes only passing gates.

Tests

• Example tests in the new module: the strict < at both floors, worst-first ordering, zero floors, a district with zero design ESS, integer ESS from JSON, 15 kinds of missing or malformed evidence, out-of-range floors (in the function and the parser), the limitation text, and the contract interaction.
• End-to-end tests in test_us_acs_local_release_tool.py:
  • report-only finalize records the gate without blocking;
  • blocking finalize fails on a collapse;
  • blocking finalize fails closed without evidence;
  • blocking package refuses five kinds of non-blocking or mismatched report;
  • package accepts a passing blocking gate and ignores a report-only one;
  • finalize to package ships the gate in gate_summary.json and build_manifest.json, with its limitation in release_manifest.json, accepted by the contract.

axiom: n/a: ACS local release tool gate; no policy rules change.

🤖 Generated with Claude Code