Make UK build outcomes honest in staging, telemetry and the Logbook

changelog.d/build-outcomes-blocked-status.changed.md

@@ -0,0 +1 @@
+UK builds now record how they ended the same way everywhere. Staging run documents move to schema version 3: a build whose gates refused its candidate closes with the new `blocked` status and a `block` record (gate phase, blocking gate ids, count) instead of `completed`, and a build that raises closes `failed` with a classified error code and `failure_class` (`INTERRUPTED`, `OUT_OF_MEMORY`, `GRAPH_NODE_FAILED`, `BUILD_FAILED`) instead of a uniform `BUILD_FAILED`. A dense build refused at the preflight gates is now recorded as blocked at phase `preflight` (it used to close as a pass), with Logbook receipts that resolve in the preflight gate document. The hosted emitter gains the same `blocked` run event and error codes. One classification (`microcosm.build.run_outcome`) drives the staging bundle, the emitter and the Logbook disposition, so they cannot disagree. The delivery summary keeps contract version 2; version 2 documents stay readable, and the version 2 fixtures are frozen beside the new version 3 set. Readers must accept version 3 first (PolicyEngine/calibration-diagnostics#206).

changelog.d/build-outcomes-dense-gate-signing.fixed.md

@@ -0,0 +1 @@
+A graph-built dense candidate can now be assembled. Its `*.local_gates.json` used to hold the graph's unsigned 26-gate document, which the dense release contract, the release preflight and the size evaluation could not read. The build now writes the signed six-gate local battery report there: the local outcomes are projected from the graph's terminal document (nothing is re-evaluated), replayed through the gate battery and signed with the attempt's Logbook build id as `release_id`. The report is written before any refusal, so a blocked candidate leaves it too, and it stays unsigned (never shippable) without the signing key or a Logbook attempt. The full document keeps its graph evidence name (`uk.full.gates.calibrated.gate_report.json`, `outputs.full_gate_report`) and is what the package binds. Gate receipts now resolve: local gates in the signed report, the rest in the full document. `--release-candidate` now refuses to start without a 32-byte signing key or with `--target-geographies` that select no local targets; a filtered build writes no local report and says why. The release preflight requires exactly 32 bytes of key.

changelog.d/build-outcomes-resume-lineage.added.md

@@ -0,0 +1 @@
+UK full builds (dense and national) now record how a resumed attempt built on earlier ones. Each attempt's `request.json` names its Logbook build id and staging run id; the rowwise manifest gains an `execution` block (graph store, attempt directory, nodes reused from the store versus computed, and the earlier attempt directories on the same store with their ids); and the staging run gets the counts as a `graph_execution` stage event. The block sits outside the run parameters, so the candidate identity is unchanged.

changelog.d/build-outcomes-sigterm.changed.md

@@ -0,0 +1 @@
+UK spine, dense and national builds now record a SIGTERM (a supervisor, a budget stop, `kill`) instead of dying silently: the staging run closes `failed` with `TERMINATED`, the attempt gets a `discarded` Logbook row like Ctrl-C, and the command exits with status 143. The spine build now also records Ctrl-C (it used to leave the run `running` with no row). A second SIGTERM still kills the process at once; SIGKILL and out-of-memory kills stay covered only by the hosted emitter's heartbeat.

changelog.d/build-outcomes-spine-output-sha256.added.md

@@ -0,0 +1 @@
+The UK spine build now records its output H5's file sha256, measured after the last write (smoke marking included): in the build sidecar (`output {filename, sha256, size_bytes}`), in a `<h5>.sha256` file beside the H5, in the `spine_h5_creation` stage event and in the Logbook row's `pipeline` verdict (`artifact_sha256`). A full build binding that spine refuses an input H5 whose digest differs from the sidecar's `output.sha256`; sidecars written before carry no such key and still bind by content identity.

docs/uk-full-build-graph.md

@@ -24,7 +24,7 @@ uv run --no-sync python tools/build_uk_full.py --release-role dense \
 
 The three atomic-area supports are the artifacts pinned in `uk/uk_atomic_area_supports.provenance.json` and `uk/spec/sources.yaml` (built by `tools/build_uk_atomic_area_supports.py` from the published ONS, NRS and NISRA lookups; publisher registration PolicyEngine/chronicle#269). `--geography-assignment atomic` is the default and requires all three; `--atomic-support-sha256-{ew,scotland,ni}` pin them like `--ladder-sha256`, and a release candidate requires all four pins. `--geography-assignment legacy` keeps the previous sequential ladder draw for measurement builds, takes no supports, and is refused by `--release-candidate`.
 
-The checkpoint must bind the exact frame content, the current spine stage roster and the gate-report bytes. The bound-spine node compares the checkpoint's gate report digests with the branch's own gate declarations and refuses a spine whose gate manifest differs from them, so `--input-h5` needs a spine built by a branch with the same declarations; every acceptance spine on disk when this registration landed predates them and is not admitted. Historical candidate H5 files and reviewed-bypass sidecars are not alternate build sources. Chronicle facts and manifest must match the independently reviewed national and local feed declarations; filtering targets does not relax source validation.
+The checkpoint must bind the exact frame content, the current spine stage roster and the gate-report bytes. A spine build records its H5's file digest in the sidecar (`output {filename, sha256, size_bytes}`), in a `<h5>.sha256` file beside it, in its `spine_h5_creation` stage event and in its Logbook `pipeline` verdict (`artifact_sha256`); when the sidecar carries `output.sha256`, the full build refuses an `--input-h5` whose measured digest differs (older sidecars without the key bind by content identity alone). The bound-spine node compares the checkpoint's gate report digests with the branch's own gate declarations and refuses a spine whose gate manifest differs from them, so `--input-h5` needs a spine built by a branch with the same declarations; every acceptance spine on disk when this registration landed predates them and is not admitted. Historical candidate H5 files and reviewed-bypass sidecars are not alternate build sources. Chronicle facts and manifest must match the independently reviewed national and local feed declarations; filtering targets does not relax source validation.
 
 The target registry binds Census household and demographic rows from the reviewed Chronicle feed, including the approved Northern Ireland constituency geography. Geography is assigned after expansion by the shared atomic-geography operators (microcosm#931): `uk.full.identity` keys every household with `household_draw_key` from the spine's explicit lineage (source household id, SPI support channel and clone index, CGT clone and donor flags) and the pool clone index; `uk.full.geography.assign` draws one atomic area per household (E&W 2021 Output Area, Scotland 2022 Output Area, NI 2021 Data Zone) by census household count within the household's FRS region, with a keyed `sha256-u53-v1` stream so a household's draw never depends on row order, K or any other household; `uk.full.geography.derive` reads every larger geography off the support's versioned mappings; `uk.full.geography.local_authority` resolves the engine's `local_authority` key from the derived authority code (the same resolver the ladder path uses); the shared `uk.full.geography.gate` and the UK distribution gate `uk.full.geography_gate` follow, and `uk.full.pool` refuses unless the shared gate passed. The OA ladder still supplies the constituency and local-authority rosters, household dispersion and lookup support for target compilation; its household counts are not a second source of calibration targets. Source receipts retain the Chronicle identity, the paired ladder digest and the geography binding (assignment mode, definition sha256, support pins, identity column, stream), so target values and the geography used to assign households can be audited separately.
 
@@ -99,15 +99,17 @@ The full target compiler preserves the unreduced band-edge register, reference-p
 
 The shared store defaults to `<out>/.graph-store`. `--graph-store` can reuse another store. `--resume require` requires completed numerical nodes and evidence to be available; output materialization and byte readback still verify the recreated files. `--resume-size-checkpoint` imports a legacy size-search checkpoint only after validating invocation identity, ordered target/household axes, initial weights and recomputed losses. It skips the saved dense solve and search. New runs persist their intermediates as graph artifacts before drawing.
 
-Each attempt also keeps its checkpoint manifests and small stage, gate and provenance reports under `<graph-store>/uk-full-attempts/<attempt-id>`. A failed run's `failure.json` links to this evidence, including a persisted spine gate verdict before downstream admission stops execution. A previously completed output bundle remains intact.
+Each attempt also keeps its checkpoint manifests and small stage, gate and provenance reports under `<graph-store>/uk-full-attempts/<attempt-id>`. Its `request.json` carries the request bindings and the attempt's Logbook build id and staging run id (`attempt`). The rowwise manifest's `execution` block records the graph store, the attempt directory, how many graph nodes the attempt reused from the store rather than computed (the first run to reach a node decides), and the earlier attempt directories on the same store with their ids, so a resumed build names the attempts whose work it reused; the same counts reach the staging run as a `graph_execution` stage event. The block sits outside the run parameters, so it does not change the candidate identity. A failed run's `failure.json` links to this evidence, including a persisted spine gate verdict before downstream admission stops execution. A previously completed output bundle remains intact.
 
-The output bundle is named from the role's posture and the FRS release vintage: `microcosm_uk_2024_25_local.h5`, its signed gate report `microcosm_uk_2024_25_local.local_gates.json`, and the `.diagnostics.json`, `.targets.csv`, `.area_support.csv`, `.holdout.json` and `.target_selection.json` siblings on the same stem, beside `graph.json`, `operations.json`, the stored source/stage/gate evidence and the graph manifests. `candidate.json` is the immutable graph package inventory: it binds the dataset, its evidence, the target selector and the independent K/k request. Comparison inputs refer to this candidate identity. Adding comparison evidence does not rewrite the candidate package. `rowwise_candidate_manifest.json` is projected from the stored artifacts in the schema-4 shape the rowwise tool wrote (`graph_terminal.rowwise_candidate_manifest_from_graph`), so the dense release pre-flight and assembler read a graph build as they read a rowwise-tool build; it records the release role, the release verdict, `staging_delivery` and `staged_dataset`.
+The output bundle is named from the role's posture and the FRS release vintage: `microcosm_uk_2024_25_local.h5`, its local gate report `microcosm_uk_2024_25_local.local_gates.json`, and the `.diagnostics.json`, `.targets.csv`, `.area_support.csv`, `.holdout.json` and `.target_selection.json` siblings on the same stem, beside `graph.json`, `operations.json`, the stored source/stage/gate evidence and the graph manifests. `candidate.json` is the immutable graph package inventory: it binds the dataset, its evidence, the target selector and the independent K/k request. Comparison inputs refer to this candidate identity. Adding comparison evidence does not rewrite the candidate package. `rowwise_candidate_manifest.json` is projected from the stored artifacts in the schema-4 shape the rowwise tool wrote (`graph_terminal.rowwise_candidate_manifest_from_graph`), so the dense release pre-flight and assembler read a graph build as they read a rowwise-tool build; it records the release role, the release verdict, `staging_delivery` and `staged_dataset`.
+
+The graph evaluates every declared UK gate once; `uk.full.gates.calibrated.gate_report.json` is that full terminal document, the one the package inventory binds (`outputs.full_gate_report`). The local gate report is the six-gate local battery the dense release contract verifies: the driver projects the six `UK_LOCAL_GATE_SCOPE` outcomes from the full document (nothing is re-evaluated; the projection checks the declarations are the same), replays them through the gate battery without raising (the graph's own enforcement still sets the exit status), and writes the report before any refusal, so a blocked candidate leaves it too. Its `release_id` is the Logbook attempt's build id. It is signed when the attempt is bound and `MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY` holds a 32-byte key; otherwise it is written unsigned, with `signing_error`, and is never shippable. A build whose `--target-geographies` select no local targets drops the local fit claim: it writes no local gate report (`outputs.local_gate_report` is null and `local_gate_report_absence` says why) rather than recording the excluded local gates as not applicable. The Logbook receipts of the six local gates resolve in the local report (`#/gates/<id>`); the other gates' receipts resolve in the full document (`#/report/outcomes/<index>`).
 
 The terminal graph node writes unsigned `certification.json` from those identified artifacts and any declared native or matched-size comparisons. `build.json` is the completion marker that binds both `candidate.json` and the certification artifact. Physical output bytes are checked against their declared artifacts. Bundle publication writes the completion marker last and rolls back handled failures or interrupts. A process kill or power loss can leave an absent completion marker; a directory without a valid bound marker is not a completed build.
 
-A non-dry dense run is wrapped in the rowwise tool's operational envelope: the Logbook attempt (a `uk-local-candidate` row spooled under `<out>/logbook-spool` on every terminal outcome, chained through `--logbook-prev-row-digest` or `POPULACE_LOGBOOK_PREV_ROW_DIGEST`, with an error receipt on failure), version 2 staging telemetry with the run's sampling evidence (`sample: {"mode": "full"}` on the f100 rung, null below it, judged on the effective fraction), stage events around each graph phase and per-epoch `calibration_progress` rows from the dense solve and, with `--dataset-households`, the size search and refit (rows tagged with their `phase`), and the staged-dataset delivery of the published bundle under `staged/<run_id>/` in the private repository. `--staging-local-only`, `--no-staging`, `--staging-read-back` and `--no-staged-dataset` behave as in [UK staging operations](uk-staging-operations.md). Dry runs plan without solving or writing and record no Logbook row.
+A non-dry dense run is wrapped in the rowwise tool's operational envelope: the Logbook attempt (a `uk-local-candidate` row spooled under `<out>/logbook-spool` on every terminal outcome, chained through `--logbook-prev-row-digest` or `POPULACE_LOGBOOK_PREV_ROW_DIGEST`, with an error receipt on failure), version 3 staging telemetry with the run's sampling evidence (`sample: {"mode": "full"}` on the f100 rung, null below it, judged on the effective fraction), stage events around each graph phase and per-epoch `calibration_progress` rows from the dense solve and, with `--dataset-households`, the size search and refit (rows tagged with their `phase`), and the staged-dataset delivery of the published bundle under `staged/<run_id>/` in the private repository. `--staging-local-only`, `--no-staging`, `--staging-read-back` and `--no-staged-dataset` behave as in [UK staging operations](uk-staging-operations.md). Dry runs plan without solving or writing and record no Logbook row.
 
-Structural failures stop export. The maintained local statistical failure policy may still export an unreleasable diagnostic candidate with a nonzero process status. Missing evidence remains explicit. `--release-candidate` applies the maintained strictness and solve settings; it does not publish, sign or authorize a release. Fixture acceptance proves graph behavior. Native certification additionally requires measured incumbent comparison evidence supplied with `--native-scorecard`; exact-count promotion also requires the measured comparison at the requested k through `--matched-size-scorecard`. The certification node verifies their candidate and output identities before assessing readiness.
+Structural failures stop export. The maintained local statistical failure policy may still export an unreleasable diagnostic candidate with a nonzero process status. Missing evidence remains explicit. `--release-candidate` applies the maintained strictness and solve settings and requires the UK gate signing key and local targets, because it signs the local gate report; it does not publish or authorize a release. Fixture acceptance proves graph behavior. Native certification additionally requires measured incumbent comparison evidence supplied with `--native-scorecard`; exact-count promotion also requires the measured comparison at the requested k through `--matched-size-scorecard`. The certification node verifies their candidate and output identities before assessing readiness.
 
 `tools/build_uk_rowwise_dataset.py` stays the separate driver it was (its tests load it by path, and it still serves `--candidate-clone-counts`). `tools/calibrate_uk_national_dataset.py` was retired by microcosm#823 and does not forward.