Register one UK full-build graph with all-geography calibration

CLAUDE.md

@@ -129,7 +129,8 @@ effect of another task. A UK rowwise run's **staged** bundle
 is inspection evidence, not a release: it never moves `releases/` or
 `latest.json` and is not loadable through the certified loader. The build's
 default is to upload that bundle (hundreds of megabytes of licensed microdata)
-to the private repository; when you run `tools/build_uk_rowwise_candidate.py`
+to the private repository; when you run `microcosm-build-uk`
+(`tools/build_uk_full.py`, or its stub `tools/build_uk_rowwise_candidate.py`)
 yourself, pass `--staging-local-only` unless the operator asked for a staged
 upload.
 
@@ -238,7 +239,8 @@ Update this guide in the same PR whenever the workspace layout, test
 commands, or release flow change. If you find it contradicting the repo,
 trust the repo and fix this file.
 
-UK size experiments use `tools/build_uk_rowwise_candidate.py --release-role dense --dataset-households`
+UK size experiments use `microcosm-build-uk --release-role dense --dataset-households`
+(`tools/build_uk_full.py`; `tools/build_uk_rowwise_candidate.py` is a stub over it)
 with the same pool inputs as the dense candidate. The flag changes exported
 support, not clone K. Sizes remain candidate-only until their matched comparison
 and promotion scorecard are adjudicated; see

README.md

@@ -71,11 +71,12 @@ This writes `progress.json`, `events.ndjson`, `calibration_progress.json`, and
 final candidate diagnostics under `runs/<run_id>/` without updating production
 `latest.json`.
 
-The UK commands (`tools/build_uk_frs_spine.py` and
-`tools/build_uk_rowwise_candidate.py`, whose `--release-role` builds either
-the national or the dense line) stage version 2 telemetry to
-`policyengine/populace-uk-staging` under the same switch. The rowwise
-candidate command also **stages the finished dataset bundle** it built,
+The UK commands (`tools/build_uk_frs_spine.py`, a shim over the package's
+`uk_runtime.spine_build`, and `microcosm-build-uk` / `tools/build_uk_full.py`,
+whose `--release-role` builds either the national or the dense line;
+`tools/build_uk_rowwise_candidate.py` is a stub over the same driver) stage
+version 2 telemetry to `policyengine/populace-uk-staging` under the same
+switch. The build command also **stages the finished dataset bundle** it built,
 national, dense or exact-count, under `staged/<run_id>/` in the
 private `policyengine/populace-uk-private` repository so the team can inspect
 it without publishing it: `releases/` and `latest.json` are untouched, the

changelog.d/901-uk-graph-interface-amendments-26-27.changed.md

@@ -0,0 +1 @@
+The two shared graph-contract amendments of microcosm#918, which that proposal recorded as 25 (a same-kind `WeightUpdate` is declarable) and 26 (the kernel context carries the version's metadata, mass log and column order), are numbered 26 and 27 on main, because main recorded the live-population observer opt-in (microcosm#950 and #951) as amendment 25 while #918 was open. Every code comment, docstring, test docstring, changelog fragment, acceptance heading and receipt moves in lockstep, and `docs/graph-interface.lock` is re-recorded because the renumbered comments live in `decl.py` and `kernel.py` (the owner sign-off rule of the acceptance record applies). The `_project_context` docstring now points at `_execute_graph`, where microcosm#938 moved the boundary selection, and amendment 27 states that the executor's boundary mass logs are live references under amendment 25's opt-in. The three root-level review artefacts of the shared-contract lane are dropped; the receipts under `experiments/` stay, with a note on the renumbering.

changelog.d/901-uk-hmrc-tail-retired.removed.md

@@ -0,0 +1 @@
+Removed the two frozen HMRC tail stages `frs_hmrc_retained_leaves` and `hmrc_spi_income` from the UK spine manifest (35 to 33 stages) together with their transform module `uk_runtime/frs_hmrc_leaves.py`, their source-stage resource `uk/hmrc_income_source_stages.json` (and their entries in `uk/source_stages.json` and `uk/spec/sources.yaml`) and the `UK_SPINE_EXCLUSIONS` constant that hid them from every consumer, so the manifest roster is the graph roster. The FRS HMRC leaf columns now come from `uk_runtime/frs_hmrc_source`, which the HMRC source contract, the SPI spine and income stages, the source runtime and the graph kernels read; `release_input_coverage` refuses `superseded_by` and parses `required_predecessor_stages`, and the coverage-manifest tool loses its HMRC path. Because microcosm#1006 placed `spi_income_band_donors` between the support channel and the income spine, the audited HMRC family names it as a predecessor and the contract admits its two operation kinds. The release-input coverage manifest and the H2 spine parity fixture were regenerated with their tools; the gate-register digests did not move.

changelog.d/uk-full-build.changed.md

@@ -0,0 +1 @@
+The UK full build is one executable graph served by one driver, `microcosm-build-uk` (`tools/build_uk_full.py`), which carries the release roles of microcosm#823. `--release-role dense` builds the K-clone joint national and local surface through the graph: source spine, geographic cloning, target compilation and selection, calibration, exact-count sizing, gates, diagnostics and a checked export, with all applicable geographies calibrated by default and `--target-geographies country` as an explicit filter in the same build. `--release-role national` is parsed and validated by the same posture-aware validator and then dispatched, before any graph is prepared, to the retained calibration seam through `uk_runtime.national_role`, so the national line is built by the same engine as before. The dense role keeps the posture's solve defaults and refusal tables, the `microcosm_uk_2024_25_local.*` output names, a schema-4 `rowwise_candidate_manifest.json` projected from the stored graph artifacts (which the dense release pre-flight and assembler accept), the Logbook row, staging telemetry around each graph phase and the staged bundle, and gains `--baseline-pi-floor`, `--no-size-checkpoint`, `--candidate-clone-counts` and `--households-only`. `tools/build_uk_rowwise_candidate.py` is now a stub over the driver and `tools/build_uk_frs_spine.py` a shim over `uk_runtime.spine_build`, into which the spine tool moved with its gate batteries as graph nodes and its stage evidence read back from the content store; every runbook command keeps working through them. Bound stage, gate and diagnostic artifacts are restored on replay, exported bytes are checked against their declared artifacts, and the graph's terminal node reports unsigned certification readiness.

changelog.d/uk-full-graph-contracts.added.md

@@ -0,0 +1 @@
+Added the country-agnostic pieces the UK full-build graph binds: `microcosm.build.artifact_files` (file artifacts, byte materialisation and staged-bundle publication), `microcosm.build.stage_evidence` (the typed stage-evidence artifact and its codec, which walks the observation and graph-source proxies the way the UK driver's collector did), the gate-battery phase-report payload codec with `GateBatteryRun.record_phase`, `microcosm.calibrate.artifacts` (ordered problem, solution and calibration-result artifacts), `microcosm.calibrate.target_selection` (ordered target-selection receipts) and the `TargetSpec` `to_dict`/`from_dict` codec on the registry. On the UK side, dense calibration, the informed size search, the exact-count draw and the refit are separate resumable graph nodes over the original pool. The same-kind weight update and the frame-context kernel fields land through the shared graph amendments 26 and 27.

changelog.d/uk-shared-graph-contracts-frame-context.added.md

@@ -0,0 +1 @@
+Added `KernelContext.frame_metadata`, `frame_mass_log` and `frame_column_order`, so a kernel can reconstruct its population version's declared slices without seeing any column it did not declare. The mass log a node receives is the one its key binds: its version's structural boundary for an ordinary node, the base version's cumulative log for a structural one. All three are detached from the live population before a kernel sees them, and all three enter the executor's input-mutation check (graph amendment 27).

changelog.d/uk-shared-graph-contracts-weight-update.added.md

@@ -0,0 +1 @@
+Added `WeightUpdate`, a declared same-kind replacement of an entity's weight values, with `weight_update_receipt` binding the ordered entity axis the replacement values are positional against, checked against the incumbent axis on cold execution and on every replay. An update replaces weight values only: design anchors, and so any `max_weight_ratio` declared against them, are unchanged (graph amendment 26).

docs/graph-acceptance.md

@@ -526,6 +526,156 @@ lock unchanged:
     lock is unchanged. Adopted 2026-09-18 for the retention seal's verifier,
     which reads the live population and seals its content (#950, #951).
 
+26. **A weight update that keeps its kind is declarable.**
+    `WeightTransition` only ever moves a kind forward, so a stage that
+    recomputes weights it already holds — a sampling normalization is the
+    case this was extracted for — could not be declared at all, and the
+    only way to express it was to misdeclare a transition.
+    `WeightUpdate(entity, kind, reason, mass)` is that declaration and is
+    deliberately narrower than a transition: the incumbent kind, the
+    declared kind and the returned weights' kind must all be the same one;
+    `mass` is `conserve` or `declared` (`WEIGHT_UPDATE_MASS_POLICIES`),
+    because an update that neither changes kind nor bounds mass records
+    nothing a reader could check it against; and `reason` is required,
+    non-empty and normative.
+
+    Positional replacement values are not self-describing: the same vector
+    is correct against one row order and silently wrong against another.
+    A kernel therefore binds its ordered entity axis with
+    `microcosm.graph.weight_update.weight_update_receipt` under
+    `receipt['weight_update']`, and the executor recomputes that binding
+    from the incumbent axis it is about to apply the values to. This is
+    checked on replay by construction rather than by a parallel rule: a
+    cache hit reconstructs the `KernelResult` with its restored weights
+    and receipt and re-applies the REWEIGHT to the current base, so it
+    re-enters the same function. A count mismatch, a missing binding and a
+    binding against a different axis are each a rejection.
+
+    An update replaces weight *values* and does not re-anchor design
+    ancestry. `Population.design_weights` is captured once, at `CREATE`
+    (`Population.from_frame`), and afterwards only carried by stable entity
+    id (`_carry_design_weights`), which `patch` passes on explicitly so the
+    re-derive-from-the-frame default is never taken. A design-kind update
+    therefore leaves every existing row's anchor where it was; a later
+    `EXPAND`'s copied rows still inherit the anchor of the row they copy
+    rather than that row's current value; and `max_weight_ratio` with
+    `weight_anchor='design'` keeps the denominator it was written against —
+    the error text has always said "original design weight". A row admitted
+    with no ancestor is anchored on whatever design weight the `EXPAND`
+    installs for it, because it has no earlier weight to be anchored on;
+    that is the anchor definition applied to a row with no ancestry, not a
+    mixture. Re-anchoring on a same-kind update would instead let an
+    unrelated normalization silently widen every cap declared upstream of
+    it by that normalization's factor, which is a non-local change to an
+    already-declared contract. A stage that wants a cap against normalized
+    weights states the ratio it means.
+
+    The shared calibration kernel does **not** consume this yet:
+    `calibrate.adam@1` emits no `receipt['weight_update']`
+    (`packages/microcosm-calibrate/src/microcosm/calibrate/kernels.py`), so
+    declaring a re-solve of an existing calibration as a `WeightUpdate`
+    would be refused by the axis check as unverifiable, and its own guard
+    still asks for a `WeightTransition` by name. Re-solving an existing
+    calibration through the shared kernel is a **future consumer
+    adaptation**, not a case this amendment already covers.
+
+    `WeightUpdate.to_kind` is a property, not a field, so the two
+    declarations have disjoint field sets (`{entity, to_kind, mass}` and
+    `{entity, kind, reason, mass}`) and a `WeightUpdate` can never
+    canonicalize, or serialize, to the same bytes as a
+    `WeightTransition`. Declaration JSON discriminates on those names, and
+    a transition's payload is byte-for-byte what it was before this
+    amendment, so every declaration written earlier restores unchanged.
+    Existing `to_kind` readers — the design-weight cap and the calibration
+    view — keep working through the property; the view drops the arrow
+    that would claim a kind moved. `Node` gains no field, so no existing
+    node key moves. `decl.py` is re-locked. Raised by the source review of
+    the UK full-build graph (#901, head `051fb972`), whose
+    `uk.full.normalize` node is the first consumer; its UK graph stages
+    and calibration science stay in that branch.
+
+27. **The context carries the version's metadata, mass log and column
+    order.** The executor projects each entity table in *declaration*
+    order, so `KernelContext.tables` is not the population version's
+    layout, and the version's `Frame` metadata and mass log were not
+    reachable from a kernel at all. A kernel that has to hand a declared
+    projection back to a legacy function as a `Frame` therefore could not
+    reconstruct one without inventing the parts it could not see.
+    `kernel.py` gains three read-only fields:
+
+    - `frame_metadata` — the version's own metadata. The executor passes
+      `Frame.metadata`, which `Frame` has already deeply frozen;
+      `KernelContext` adds a read-only view over it and does **not** itself
+      deep-freeze a mapping built some other way. (This is a deliberate
+      difference from the UK branch, which imports
+      `microcosm.frame.bundle._freeze_metadata` into the frozen interface:
+      a frozen contract should not depend on another shard's private name.)
+    - `frame_mass_log` — the `Frame` mass records the node's *key* binds,
+      in order. An ordinary node's key binds its version's structural
+      boundary (`population_input`) and the owners of the columns it
+      declared (`input_artifacts`); it does not bind the other ordinary
+      members of its version. Its log is therefore the version's
+      **boundary** log, captured when the structural node was admitted, and
+      a record another member appends afterwards is not visible to it. The
+      alternative — the cumulative log the version carries at the moment the
+      node runs — would be a kernel input no key binds: adding or
+      re-parameterising an unrelated sibling would change what the node
+      sees while its key, and so its cache entry, stayed put, and a hit
+      would replay output computed against a different log. A structural
+      node is given its base version's cumulative log instead, because its
+      key does bind it: `keys.py` binds the base's frame identity *and*
+      every ordinary member of that version through `members`, which
+      `compile_graph` fills with `members.get(base, ())`. Boundaries are
+      captured where the version is admitted, so cold execution and a
+      restored cache hit record the same one. A node needing a stage's
+      completed records therefore runs after that stage's structural
+      boundary or reads its predecessor's evidence; incidental node order is
+      not authority. The write side already existed
+      (`receipt['frame_mass_log_append']`); only the read side was missing.
+    - `frame_column_order` — entity to the version's own column order,
+      restricted to the columns projected into `tables`. An entry that is
+      not exactly an ordering of that table's columns is refused, so an
+      order can neither hide a column the node was given nor name one it
+      was not: a column *name* is itself information about the version, and
+      B1's "nothing else is visible" covers names as well as values.
+
+    All three are detached before a kernel sees them and are covered by
+    B4's before/after mutation check. Detachment is not redundant with
+    `Frame`'s freezing: a frozen dataclass still yields to
+    `object.__setattr__`, so passing the version's own `_FrozenMapping`
+    leaves and `MassChangeRecord`s by reference would make every kernel — and
+    anything that retains a context past its own mutation check — a live
+    handle on the population. The executor therefore hands out a deep copy
+    of the metadata and rebuilt mass records (the rule `_observer_snapshot`
+    already followed, now shared), and `_context_digest` binds the metadata
+    through the frame format's own store codec, the mass records field by
+    field, and the projected column order. Neither is a substitute for the
+    other: the digest catches a kernel whose output stops being a function
+    of its declared inputs, while detachment is what stops a retained view
+    from rewriting the live version after that check has passed.
+
+    The three ride after `artifacts` and before `tolerances`, so amendment
+    17's statement that `numerics` rides at the end of the context stays
+    literally true and amendment 19's that `artifacts` rides before the
+    pair does too — the unit assertion of *adjacency* becomes the ordering
+    amendment 19 actually claimed. The acceptance suite's B2 field set
+    gains the three in its own commit, as amendment 19's did. Nothing here
+    is normative: `Node` is untouched, no canonical projection changes, and
+    no node key moves. The fields are rebuilt from a restored `Frame` on a
+    cache hit exactly as they are from a computed one, which is what
+    amendment 22's metadata-preserving Frame format makes possible.
+    `kernel.py` is re-locked. Raised by the same source review as amendment
+    26; the UK full-build graph's `context_frame` helper (#901, head
+    `051fb972`) is the first consumer. The executor keeps live references in its
+    boundary mass logs, so under amendment 25's opt-in
+    (`_population_observer_detach=False`) a mutating observer can change
+    what a later node sees as `frame_mass_log`; that lies inside the
+    guarantees amendment 25 already withdraws and adds no new one.
+    Renumbered from 26 on the rebase onto main (2026-09-25), because main had
+    recorded the observer opt-in as amendment 25 in the meantime; amendment 26
+    above was 25 in the same lane.
+
+
 Adding a normative field with a default changes the canonical projection
 of every node that carries it, so node keys moved with amendments 11 and
 13's sibling field `entrants`; no released artifact pins a graph key yet.

docs/graph-interface.lock

@@ -1,2 +1,2 @@
-ed0a859adcae12510d5ba74d51c694617201f7b448b108a3f602410f5da44876  decl.py
-dbf57c137330f0f12744c557ee594586a1b308b6d6adaba1938e2b6efded21ca  kernel.py
+b25ae4a62fd2777a5dffa30ecbba7d345a804b74b9d6b6e68c2a5d38f3e0a129  decl.py
+24045ab2a62295874520d7940c7f7b59be9d909a3cfbf7e111a734e369b0815b  kernel.py

docs/uk-chronicle-feed-repin.md

@@ -52,9 +52,12 @@ refuses. The cross-grain legs of English constituencies and authorities come
 from `region_code_by_area` in `local_area_crosswalk.json`, regenerated from
 the sha-pinned ladder with `tools/generate_uk_local_area_crosswalk.py`.
 
-The national calibration runner refuses a feed whose facts or manifest digest
-differs from the committed pin. `--allow-unpinned-feed` is an explicit
-diagnostic override recorded in the run manifest; it is not a re-pin procedure.
+Both release roles of `microcosm-build-uk` refuse a feed whose facts or
+manifest digest differs from the committed pin. On the national role
+`--allow-unpinned-feed` is an explicit diagnostic override recorded in the run
+manifest; it is not a re-pin procedure. The dense role refuses that flag: the
+graph's target compilation checks the supplied hashes and the artifact against
+the committed pin and has no override.
 
 History: the `ec7169b` re-pin (#887/#900) moved census household targets onto
 the same Chronicle compile path as every other bound UK local family; the