diff --git a/changelog.d/uk-cut-modal.added.md b/changelog.d/uk-cut-modal.added.md new file mode 100644 index 0000000000..266cc5d453 --- /dev/null +++ b/changelog.d/uk-cut-modal.added.md @@ -0,0 +1 @@ +`tools/modal_uk_cut.py` runs the UK cut runner in a Modal container, with its work directory on a Modal volume so a rerun resumes, for when the build machine is busy or asleep. diff --git a/changelog.d/uk-cut-pepy-certification.added.md b/changelog.d/uk-cut-pepy-certification.added.md new file mode 100644 index 0000000000..26c39703d7 --- /dev/null +++ b/changelog.d/uk-cut-pepy-certification.added.md @@ -0,0 +1 @@ +`tools/certify_uk_cut_in_policyengine_py.py` certifies a published UK cut into policyengine.py on a fresh branch: it refuses when another pull request or branch is already certifying the cut, checks the regional route before writing anything, runs certify-data, the lock, TRO, constant and snapshot regeneration and the strict bundle check, measures the UC floor and runs the tests, then drafts the pull request with every judgment call left as a checklist. An accepted regional model mismatch certifies with policyengine.py's audited input-surface evidence (policyengine.py#699). Its load limit, commit co-authors and pull request footer are the operator's to set. diff --git a/changelog.d/uk-cut-runner.added.md b/changelog.d/uk-cut-runner.added.md new file mode 100644 index 0000000000..4f5d61c565 --- /dev/null +++ b/changelog.d/uk-cut-runner.added.md @@ -0,0 +1 @@ +`tools/run_uk_cut.py` builds, signs and assembles a Microcosm UK national cut in one resumable command, fetching every input from Hugging Face by the pins the build already enforces (`tools/uk_cut_inputs.json` says only where each input lives; the Chronicle artifact's path follows `chronicle_feed.json`). It resumes a cut only under the key that signed it, chains the certification's Logbook row on the build's own row, and never submits an Orrery publication. `tools/modal_rebuild_uk_chronicle_artifact.py` rebuilds the pinned Chronicle UK consumer artifact on Modal and hosts it privately. diff --git a/changelog.d/uk-gate-signing-key-registry.added.md b/changelog.d/uk-gate-signing-key-registry.added.md new file mode 100644 index 0000000000..1aa86c968c --- /dev/null +++ b/changelog.d/uk-gate-signing-key-registry.added.md @@ -0,0 +1 @@ +UK gate reports, release-cut certifications and dense gate reports now verify only under a signing key registered in `microcosm.data.gate_signing_keys`, and every UK signer refuses an unregistered or retired key. The national build checks its key before loading any input, instead of failing after the solve. `python -m microcosm.data.gate_signing_keys check` reports whether the key in the environment may sign. diff --git a/changelog.d/uk-gate-team-signing-key.added.md b/changelog.d/uk-gate-team-signing-key.added.md new file mode 100644 index 0000000000..608de22d50 --- /dev/null +++ b/changelog.d/uk-gate-team-signing-key.added.md @@ -0,0 +1 @@ +Register a second UK gate signing key, `uk-gate-team-2026`, held by Max Ghenis in the agent store on his build Mac, so cuts can be built and signed without the first builder's key. The key that signed every earlier cut stays active. diff --git a/changelog.d/uk-gate-vahid-signing-key.added.md b/changelog.d/uk-gate-vahid-signing-key.added.md new file mode 100644 index 0000000000..c2cb386405 --- /dev/null +++ b/changelog.d/uk-gate-vahid-signing-key.added.md @@ -0,0 +1 @@ +Register a third UK gate signing key, `uk-gate-vahid-ahmadi-2026`, held by Vahid Ahmadi in his laptop's login keychain, so he can build and sign a cut. diff --git a/conftest.py b/conftest.py index f434d79435..7a7c44ebf3 100644 --- a/conftest.py +++ b/conftest.py @@ -130,3 +130,44 @@ def unexpected_start(*args, **options): yield emitters for emitter in emitters: emitter.close() + + +@pytest.fixture(autouse=True) +def _admit_test_gate_signing_keys(request, monkeypatch): + """Let tests sign with their own UK gate keys. + + Production verifies only keys registered in + ``microcosm.data.gate_signing_keys``. Tests sign with throwaway keys, so + every unregistered key counts as an active test key here, while the + committed entries keep their real status. A test marked + ``committed_gate_key_registry`` sees the committed registry unchanged. + """ + + if request.node.get_closest_marker("committed_gate_key_registry"): + return + try: + from microcosm.data import gate_signing_keys + except ImportError: + return + committed = gate_signing_keys._registry + + class _AdmitTestKeys(dict): + def __missing__(self, key_sha256): + return gate_signing_keys.GateSigningKey( + key_sha256=key_sha256, + label="test-key", + status="active", + custodian="the test suite", + held_in="the test", + registered="test", + evidence="admitted by the root conftest", + ) + + def get(self, key_sha256, default=None): + return self[key_sha256] + + def registry(country): + entries = committed(country) + return None if entries is None else _AdmitTestKeys(entries) + + monkeypatch.setattr(gate_signing_keys, "_registry", registry) diff --git a/docs/agent-guide.md b/docs/agent-guide.md index 455aad6cbb..47ddbc1411 100644 --- a/docs/agent-guide.md +++ b/docs/agent-guide.md @@ -232,7 +232,10 @@ grades every waiver register on the staged frame before the solve; see README "Dry-running the release's registers". Publication also refuses a release whose build recorded staging telemetry that never reached its repo (`--allow-missing-staging` overrides); a build that declared `--no-staging` -publishes without the flag. Never publish or promote artifacts as a side +publishes without the flag. A UK national cut runs end to end with +`tools/run_uk_cut.py` ([uk-cut-anyone.md](uk-cut-anyone.md)): inputs fetched +by pin, a signing key registered in `microcosm.data.gate_signing_keys`, and +an inspection-only `--publish`. Never publish or promote artifacts as a side effect of another task. A UK rowwise run's **staged** bundle (`staged//` in the private repository, written by the build itself) is inspection evidence, not a release: it never moves `releases/` or diff --git a/docs/uk-cut-anyone.md b/docs/uk-cut-anyone.md new file mode 100644 index 0000000000..7edd3efdb3 --- /dev/null +++ b/docs/uk-cut-anyone.md @@ -0,0 +1,438 @@ +# Building a UK national cut without the original builder's machine + +Any team member or agent session with the credentials below can build, sign +and publish a Microcosm UK national cut, then open its policyengine.py +certification pull request. Each input comes from Hugging Face and is checked +against the pins the build itself enforces. + +This is the short path. The step-by-step reference behind it stays in +[uk-national-release-assembly-runbook-806.md](uk-national-release-assembly-runbook-806.md). + +**Status (2026-10-26).** The runner's steps have so far run end to end only +against stand-in tools. Its first real run, by Vahid Ahmadi from his own +laptop with his own registered key, is the last check before it is relied +on for a re-cut; until then the hand-run launchers stay the fallback. + +## What you need + +| Need | Where it comes from | Check | +|---|---|---| +| A clean checkout of `main` locked at the policyengine-uk you want | a "Lock policyengine-uk X" PR merged to main | `tools/run_uk_cut.py preflight` | +| `HF_TOKEN` | write access to `policyengine/populace-uk-private` and `policyengine/populace-uk-staging`, and read access to the input repos below | preflight | +| `MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY` | a key registered **active** in `microcosm.data.gate_signing_keys` (see [Signing keys](#signing-keys)) | `python -m microcosm.data.gate_signing_keys check` | +| Cover under the UK Data Service licences for the FRS, SPI, WAS, LCFS, NTS and ETB | the licence holders | access to the input repositories (below) | +| About 60 GB of free disk, AC power, a quiet host | the machine | preflight | + +## One command + +```bash +cd +uv sync --all-packages --locked --extra uk +export HF_TOKEN=… +# The key goes to this one command, never to the whole shell: +MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(security find-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w)" \ + uv run --no-sync python tools/run_uk_cut.py all --label sat24 # build, certify, assemble +# add --publish to publish for inspection +``` + +The runner passes the key on only to the steps that sign (spine, national +build, certification) or verify (assembly, publication); the dry run, the +scoring, the stage-1 comparison and the preflight tests never see it. + +Agent sessions on Max's build Mac read the team key, whose custodian is Max, +with `agent-secret get microcosm/uk-gate-signing-key-team maxghenis`, and +`HF_TOKEN` with `agent-secret get HUGGING_FACE_TOKEN_MAX maxghenis`. + +`all` runs these steps in order: + +1. `fetch` +2. `preflight` +3. spine +4. national dry run, then the national calibration +5. re-score, only when the in-build evaluation did not pass +6. certify +7. stage-1 comparison (with `--previous-h5`) +8. assemble +9. publish preflight +10. publish (with `--publish`) + +Every step first checks whether its outputs already exist and verify, and +skips itself if they do. So when a run stops, fix the cause and run the same +command again. + +Nothing is overwritten: + +- An unfinished spine attempt is left for you to inspect. The builder + deletes a spine's sidecars when it starts, including a gate-blocked + attempt's signed report, so a retry needs `--retry-spine`, which moves the + whole attempt to `spine.failed-