Pin .py 6.4.1 instead of the deleted policyengine-us 1.764.6

[anth-volk]

Fixes #221

Summary

policyengine-us==1.764.6 was deleted from PyPI. Since then the hosted runtime's hashed lock and the root evaluation project's microcosm extra can't be installed. CI's hosted Python job fails on every PR, and the Modal backend image can't be rebuilt. No .py release can bring the old pair back, because .py 5.0.0–5.3.1 all pin the deleted version.

This PR makes .py (the PyPI package policyengine) the only declared model dependency, policyengine[us]==6.4.1, for both the hosted runtime and the root harness. Its us extra pins policyengine-us 2.2.1, policyengine-core 3.33.0 and spm-calculator 1.0.0, and those versions are derived and enforced rather than pinned directly.

Both environments move to populace-us-2024-spm-20260915, the US release .py 6.4.1 certifies:

Field	Value
Repository	public policyengine/populace-us
Commit	8ab57ffc2ca41d8af631ff62ebbce95e966299f3
H5 SHA-256	6496cc4393d4d3c6574f76eca231de5898c803b9067645591fd5c4d3e65aee84

The new release is a source-enrichment of the Build P release served until now: Build P's tables, weights and calibration diagnostics, plus is_spm_independent_minor_role.

The rule, documented in frontend/HOSTED_RUNTIME.md and the backend/requirements.txt header: upgrade by bumping .py; the dataset must be one that version of .py certifies.

What changes

• Runtime check (validate_runtime):
  • requires the installed .py to be 6.4.1;
  • derives the exact -us, -core and spm-calculator versions from its us extra;
  • requires hosted_release.json's packages block and the installed versions to equal them.
• Data certification (validate_manifests):
  • Unchanged: the manifest digest and release-identity checks.
  • New: the installed .py's bundle manifest must certify the release. That covers the release ID, revision, dataset URI and SHA-256, manifest paths and model version, plus every release file digest it records, including the parent release and build manifests.
  • No longer the identity, by design: the release's own built-with Core (3.32.5) and the build manifest's runtime block.
  • Only through the .py identity, and nowhere else, the build manifest's exported-household count is checked only when the build records one.
  • The strictness test is reworked: a dataset, release, parent manifest, model or .py version that the pinned .py doesn't certify is refused.
• Build-time verifier: verify_hosted_runtime.py runs during the Modal image build. Its SPM checks now bind the spm-calculator 1.0.0 and policyengine-us 2.2.1 APIs (the old spm_calculator.forecast, .geoadj and spm_unit.* modules are gone). The new checks are:
  • the SPM variables and the dataset source input are defined on the country system;
  • both SPM providers load the bundled, hash-verified forecast without downloading;
  • a real national 2024 reference threshold is computed (two adults, two children, renting: $39,219.89 from forecast spm-rolling-2026-09-09).
• Variable catalog:
  • The generator now verifies the reviewed H5 by SHA-256 and builds from a simulation over it. It no longer calls Microsimulation(), which under 2.2.1 downloads the country package's default dataset.
  • The regenerated catalog has 6,167 variables and records its release and package versions.
  • The 22 removed names are gone, including would_claim_wic, which 2.x renamed takes_up_wic_if_eligible.
  • The catalog was regenerated from the SHA-verified H5 copy already downloaded during the compatibility investigation, not a new download.
  • public/model-coverage.json also names some removed variables, but it is a labelled snapshot of policyengine-us 1.755.1 that the coverage page attributes to that version, so it is unchanged.
• Guard test: frontend/tests/test_model_dependency_pins.py runs in CI. It fails if a bare policyengine-us, policyengine-core or spm-calculator pin reappears in backend/requirements.txt or pyproject.toml. It also asserts the policyengine[us]==6.4.1 pin, and that both locks resolve exactly the tuple .py pins.
• Root harness:
  • The adapter, overview.yaml and the integration docs move to the same pair. Results report policyengine[us]==6.4.1 as their model version.
  • policyengine-us 2.2.1 requires pandas 3, so the microcosm extra's pandas bound moves to >=3,<4.
  • uv.lock was regenerated with uv 0.12, which also writes lockfile revision 5.

Commits

Red runs used a venv with the .py 6.4.1 stack against the unchanged implementation. The old lock can no longer be installed, so no 1.764.6 environment exists to test against.

1. Pin .py 6.4.1 as the hosted runtime's model and data identity (22a0d0e)
   • Red: 19 failed (20 passed) across test_model_dependency_pins.py, test_hosted_release.py, test_modal_endpoint.py and test_hosted_runtime_verification.py. All failed for the intended reasons:
     • bare -us, -core and SPM pins;
     • the Build P revision and manifest digests;
     • no certifying_package and no policyengine entry in packages;
     • tuple and .py-version refusals that didn't raise;
     • no .py bundle check;
     • the verifier refusing policyengine-us 2.2.1.
   • Green: 39 passed.
   • Gates: all passed.
     • Fresh venv from the new lock: uv pip install --require-hashes ✓, uv pip check ✓, verify_hosted_runtime.py ✓, pytest frontend/tests 82 passed.
     • The Modal image's build-time command, python /root/scripts/verify_hosted_runtime.py, run locally in the image's /root layout with its environment variables ✓ (nothing deployed).
     • Every pin in backend/requirements.lock exists on PyPI (87/87).
     • bun test 603 pass, tsc --noEmit ✓, next build ✓.
     • Root pytest 440 passed, 1 skipped.
2. Build the variable catalog from the pinned .py and reviewed dataset (92e31a5)
   • Red: 4 failed (the generator took no dataset ×2, the catalog had no provenance, would_claim_wic was still listed).
   • Green: 4 passed.
   • Gates: hosted verifier ✓, pytest frontend/tests 86 passed, bun test 603 pass, tsc ✓, next build ✓, root pytest 440 passed, 1 skipped.
3. Move the root evaluation harness to .py 6.4.1 and its certified release (5a609d1)
   • Red: 6 failed. Four root tests failed on the overview's dataset and model, the download URL, the shared hosted/harness pair, and the reported model version. Two guard tests failed on the pyproject.toml pin and the uv.lock tuple.
   • Green: 6 passed.
   • Gates:
     • Root venv from the new uv.lock with all extras (uv sync --frozen --all-extras ✓, uv pip check ✓): pytest 443 passed. The previously skipped huggingface_hub test now runs.
     • Hosted verifier ✓, pytest frontend/tests 88 passed.
     • Every pin exists on PyPI: uv.lock 100/100, requirements.lock 87/87.

Engine-compatibility evidence

Collected locally before the commits, on the real H5s (SHA-256 verified).

• Build P cannot be used under 2.2.1.
  • Its release certifies only policyengine-us==1.764.6 and policyengine-core==3.26.11.
  • Running the hosted calculation path on it, 2.2.1 refuses spm_unit_spm_threshold, the deploy workflow's national check: SPMInputError: SPM unit has no classified adult: supply source-backed independence or household head/spouse structure.
  • Its stored would_claim_wic take-up is silently ignored, because the variable was renamed.
• The new release is Build P plus one input.
  • All six tables are value-identical to Build P's, plus is_spm_independent_minor_role.
  • Its parent release and build manifests hash to exactly the Build P digests this repository pinned (ecda19d9…, f503fc88…), and its calibration diagnostics file is the same (870449b4…).
• Real calculations under .py 6.4.1 on the new release, using the hosted path (single_year_simulation_dataset + Microsimulation):
  • spm_unit_spm_threshold: $3.888T weighted over 59,900 SPM units;
  • ca_income_tax: $121.198B over 3,876 California tax units;
  • a national → CA → national sequence reproduced identical values;
  • peak memory about 1 GB (local macOS; 1.764.6 needed about 12 GB). The 16 GiB Modal allocation is kept until a hosted measurement exists.
• The release's own certificate says core ==3.32.5 and model >=2.0.0,<2.3 (a publisher claim), and its build manifest has no runtime block or household count. That is why the identity is .py's certification, which pins core 3.33.0.

Deploy implications

• Merging to main runs CI. When CI passes, deploy.yml runs:
  1. It deploys a commit-specific Modal backend, calibration-diagnostics-<12-character-commit>, in the main environment. The image installs the new lock and runs verify_hosted_runtime.py while it builds.
  2. It verifies the backend metadata.
  3. It builds an unaliased Vercel candidate.
  4. It runs a genuine national spm_unit_spm_threshold calculation and a California ca_income_tax calculation on the new backend, plus the calibration smoke checks.
  5. Only then does it promote the Vercel deployment and recheck the public routes.
• Production currently still serves the backend deployed for c630c91 on 2026-10-13. Its image was built while 1.764.6 existed. Until this merges, any push to main fails CI, so deploy.yml doesn't run.
• The first cold request downloads the new 827 MB H5. Build P's was 463 MB.
• Rollback with vercel rollback PREVIOUS_DEPLOYMENT keeps working as long as c630c91's backend app is retained. The workflow doesn't delete earlier apps.
• Published cross-dataset and Chronicle results were produced with Build P and 1.764.6. The full evaluation has not been re-run on the new pair, and the integration docs say so.

Environment variables on the deploy path

backend/modal_app.py, .github/workflows/deploy.yml and .github/scripts/deploy-dashboard.sh read or declare these. This PR adds no new variables. The new engine packages read none at runtime on the hosted path; the ones they read are only used by tooling and download helpers. I checked names only, never values.

Variable	Provider	Wiring	Status
MODAL_TOKEN_ID, MODAL_TOKEN_SECRET	GitHub Actions organization secrets visible to this repository	Step env: of the configuration check, Modal deploy and backend-URL steps	Names confirmed. Unverified: that they are valid for the PolicyEngine Modal workspace's main environment.
MICROCOSM_MODAL_KEY, MICROCOSM_MODAL_SECRET	Production environment secrets	Step env: → smoke-test arguments, and Vercel --env for the candidate	Confirmed
VERCEL_TOKEN	Production environment secret	Step env: of the configuration check, the Vercel candidate build and the promotion	Confirmed
VERCEL_AUTOMATION_BYPASS_SECRET	Production environment secret	Step env: of the configuration check and the staged smoke steps → --vercel-bypass-secret	Confirmed
HF_RELEASE_READ_TOKEN	Repository secret	Step env: of the configuration check and the Vercel candidate build → Vercel runtime HF_TOKEN; not given to Modal	Confirmed
TELEMETRY_GCP_WIF_PROVIDER, TELEMETRY_GCP_DEPLOY_SERVICE_ACCOUNT	Production environment secrets	google-github-actions/auth inputs (id-token: write)	Confirmed
MICROCOSM_TELEMETRY_COLLECTOR_URL	Production environment variable	Step env: → Vercel --env	Confirmed
PROJECT_ID, READ_SECRET_NAME	Production environment variables TELEMETRY_GCP_PROJECT_ID and TELEMETRY_READ_SECRET_NAME	Step env: of the collector-secret check	Variables confirmed. Unverified: that the Google Secret Manager secret they name exists; the workflow step checks this.
MODAL_ENVIRONMENT, VERCEL_CLI_VERSION, VERCEL_ORG_ID, VERCEL_PROJECT_ID	Literals in the workflow's top-level env:	Available to every step	Confirmed
SOURCE_COMMIT, CALIBRATION_MODAL_APP_NAME, MICROCOSM_CALCULATION_URL, MICROCOSM_BACKEND_SOURCE_COMMIT, GITHUB_OUTPUT	Workflow-run SHA, step outputs, runner	Step env:	Confirmed
CALIBRATION_SOURCE_COMMIT, CALIBRATION_SOURCE_TREE_SHA256, CALIBRATION_MODAL_APP_NAME, CALIBRATION_REQUIREMENTS, PYTHONPATH	Baked into the Modal image by Image.env at build time	Present for the build-time verify_hosted_runtime.py step and at runtime. The function attaches no Modal secrets.	Confirmed
POPULACE_HF_REPO, POPULACE_HF_REVISION	None, intentionally	Optional overrides that must stay absent; a conflicting value is refused with 503 or 409	—
HF_HOME, HF_HUB_CACHE, XDG_CACHE_HOME, HF_HUB_DISABLE_XET	Defaults set in code	—	—

The Modal backend downloads the dataset anonymously, which works because policyengine/populace-us is public.

Overlap with in-flight PRs

This PR changes none of the files changed by #220, #219 or #216, so I expect no textual conflicts. #220 changes frontend/lib/microcosm/production-release.test.ts and use-microcosm.ts, which read hosted_release.json and variable-catalog.json. This PR keeps both files' shape: packages is still a flat name-to-version map, and the catalog still has variables.

Open questions

• Re-run the full Chronicle evaluation on the new pair?
• Regenerate model-coverage.json against policyengine-us 2.2.1? That needs Microcosm and policyengine-us checkouts and a live release API.
• Record the hosted memory and latency of the first deploy, and revisit the 16 GiB allocation.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
calibration-diagnostics	Ready	Preview	Oct 15, 2026 9:21pm UTC