Build the UK national line through the rowwise driver's release role (#823)

CLAUDE.md

@@ -171,7 +171,7 @@ Update this guide in the same PR whenever the workspace layout, test
 commands, or release flow change. If you find it contradicting the repo,
 trust the repo and fix this file.
 
-UK size experiments use `tools/build_uk_rowwise_candidate.py --dataset-households`
+UK size experiments use `tools/build_uk_rowwise_candidate.py --release-role dense --dataset-households`
 with the same pool inputs as the dense candidate. The flag changes exported
 support, not clone K. Sizes remain candidate-only until their matched comparison
 and promotion scorecard are adjudicated; see

README.md

@@ -63,11 +63,12 @@ This writes `progress.json`, `events.ndjson`, `calibration_progress.json`, and
 final candidate diagnostics under `runs/<run_id>/` without updating production
 `latest.json`.
 
-The UK commands (`tools/build_uk_frs_spine.py`,
-`tools/calibrate_uk_national_dataset.py`, `tools/build_uk_rowwise_candidate.py`)
-stage version 2 telemetry to `policyengine/populace-uk-staging` under the same
-switch. The rowwise candidate command also **stages the finished dataset
-bundle** it built, dense or exact-count, under `staged/<run_id>/` in the
+The UK commands (`tools/build_uk_frs_spine.py` and
+`tools/build_uk_rowwise_candidate.py`, whose `--release-role` builds either
+the national or the dense line) stage version 2 telemetry to
+`policyengine/populace-uk-staging` under the same switch. The rowwise
+candidate command also **stages the finished dataset bundle** it built,
+national, dense or exact-count, under `staged/<run_id>/` in the
 private `policyengine/populace-uk-private` repository so the team can inspect
 it without publishing it: `releases/` and `latest.json` are untouched, the
 release contract is not consulted, and a `releasable: false` size run stages

changelog.d/uk-dense-filename-823.changed.md

@@ -0,0 +1 @@
+The UK dense line's published microdata is `microcosm_uk_2024_25_dense.h5` (the FRS release vintage, matching the `microcosm-uk-2024-25-dense` release id) and the driver's joint-surface outputs are `microcosm_uk_2024_25_local.h5` and `microcosm_uk_2024_25_local.local_gates.json`; the contract and the data registry's dense variant mirror the published filename under a three-legged lockstep pin.

changelog.d/uk-national-doctrine-family-equal-823.changed.md

@@ -0,0 +1 @@
+The UK national calibration doctrine is now the certified-cut posture every campaign run had overridden to: `UK_NATIONAL_SOLVE_EPOCHS` is 1500 and `UK_NATIONAL_TARGET_WEIGHT_RULE` is `family_equal` (ruling 2026-09-20), so a doctrine national run records no overrides; `uniform` remains a receipted per-run override.

changelog.d/uk-national-role-incumbent-evaluation-823.added.md

@@ -0,0 +1 @@
+The rowwise driver's national role evaluates the finished candidate against an incumbent (`--incumbent-h5`, `--incumbent-sha256`, `--incumbent-label`) at the end of the build, after the bundle is staged and before the telemetry completes: `score_vs_incumbent.json` beside the outputs and as a reviewed telemetry artifact, rows the incumbent cannot materialize pruned from both arms and named on stderr, and a manifest `evaluation` block with the rule-1 verdict the release-cut certifier requires. The evaluation never blocks the build; an error is recorded and warned. The full compiled register is frozen beside the solve register as `national_contract_registry.json`. The dense role refuses the flags.\n

changelog.d/uk-retire-national-seam-cli-823.changed.md

@@ -0,0 +1 @@
+`tools/calibrate_uk_national_dataset.py` is retired: the national line is built by `tools/build_uk_rowwise_candidate.py --release-role national`, which delegates to the same seam library (`run_uk_calibration`) and was shown bit-for-bit equal to the command on the synthetic parity twin (weights, diagnostics rows, build record). The runbooks, the staging operations guide and the README now name the one driver.

changelog.d/uk-rowwise-national-role-823.added.md

@@ -0,0 +1 @@
+`tools/build_uk_rowwise_candidate.py --release-role national` now builds the certified national line through the calibration seam library: no cloning, national targets only, the seam doctrine (1,500 epochs, `family_equal`, learning rate 0.02, seed 0) with explicit flags as receipted overrides, the six calibration-seam gates, the seam-shaped `build_record.json` that `tools/certify_uk_release_cut.py` certifies, a frozen `national_target_registry.json` for the scorer, and the same staging telemetry (operation `uk_national_calibration`, run id = the attempt id) and staged bundle as the dense role. `run_uk_calibration` accepts a caller-minted attempt id and the committed Chronicle feed-pin check moved to `uk_runtime.chronicle_feed`.

changelog.d/uk-rowwise-release-role-823.added.md

@@ -0,0 +1 @@
+`tools/build_uk_rowwise_candidate.py` now requires `--release-role {national,dense}`: a frozen posture record (`uk_runtime/rowwise_posture.py`) supplies each role's solve defaults, gate scope, Logbook pipeline, staging ids, release id and output filenames, the manifest (schema 4) records `release_role` and `release_id`, and each role refuses the other's flags. The dense pre-flight and assembler accept only a `dense` manifest; the national role is declared and validated here and gains its build flow in the next increment. The driver's default telemetry upload cadence is now 300 s, and a refused stage event is reported and dropped instead of aborting the build.

changelog.d/uk-staging-terminal-flush-823.fixed.md

@@ -0,0 +1 @@
+Staging telemetry now delivers its terminal state even when mid-run upload failures had paused remote writes: completion and failure re-open the best-effort session for a bounded, backed-off final flush, so a run's remote `progress.json` no longer stays `running` after a rate-limited Hub (the v20 national run's last-mile gap).

docs/uk-dataset-size-plan-355.md

@@ -76,7 +76,7 @@ Use the inputs and environment from the existing
 Pass the same pinned source arguments to the existing driver and add:
 
 ```bash
-uv run python tools/build_uk_rowwise_candidate.py \
+uv run python tools/build_uk_rowwise_candidate.py --release-role dense \
   --input-h5 "$UK_SPINE_H5" --input-sha256 "$UK_SPINE_SHA256" \
   --ladder "$UK_LADDER_NPZ" --ladder-sha256 "$UK_LADDER_SHA256" \
   --ledger-facts "$UK_LEDGER_FACTS" \

docs/uk-dense-release-assembly-runbook-762.md

@@ -36,16 +36,23 @@ digest mismatch, or doctrine constants that are not the ruled ones.
 ## 2. Run the release candidate
 
 ```bash
-uv run --no-sync python tools/build_uk_rowwise_candidate.py --release-candidate \
-  --input-h5 <spine.h5> --input-sha256 <spine> \
+uv run --no-sync python tools/build_uk_rowwise_candidate.py --release-role dense \
+  --release-candidate --input-h5 <spine.h5> --input-sha256 <spine> \
   --ladder build/uk/uk_oa_ladder_2021.npz --ladder-sha256 <ladder> \
   --ledger-facts <chronicle-uk-artifact-dir> --ledger-facts-sha256 <facts> \
   --ledger-manifest-sha256 <manifest> --seed 42 \
   --logbook-prev-row-digest <previous row> --out <candidate-dir>
 ```
 
-`--release-candidate` pins the doctrine (bound 10, `grain_equal`, K=15, 1500
-epochs), resolves the engine in a single block, and runs the rotated holdout.
+`--release-role dense` is required (microcosm#823): the role supplies the
+solve defaults (K=15, seed 42, 1,500 epochs, learning rate 0.15,
+`grain_equal`), names the outputs `microcosm_uk_2024_25_local.h5` and
+`microcosm_uk_2024_25_local.local_gates.json`, records itself in the
+manifest, and refuses the national role's flags. `--release-candidate` pins
+the doctrine (bound 10, `grain_equal`, K=15, 1500 epochs), resolves the
+engine in a single block, and runs the rotated holdout. Best-effort staging
+telemetry uploads every 300 s by default on this driver (the Hub allows about
+128 commits per hour per repository).
 Expect about 3.5 hours and 10 GB at K=15.
 
 ## 3. Pre-flight the finished run, then score it
@@ -66,7 +73,7 @@ Measure the full pinned incumbent surface before assembly:
 
 ```bash
 uv run --no-sync python tools/evaluate_uk_incumbent_surface.py \
-  --candidate-h5 <candidate-dir>/microcosm_uk_2025_local.h5 \
+  --candidate-h5 <candidate-dir>/microcosm_uk_2024_25_local.h5 \
   --candidate-manifest <candidate-dir>/rowwise_candidate_manifest.json \
   --ledger-facts <chronicle-uk-artifact-dir> --ledger-facts-sha256 <facts> \
   --ledger-manifest-sha256 <manifest> --engine-blocks 1 \
@@ -104,14 +111,17 @@ uv run --no-sync python tools/assemble_uk_dense_release_dir.py \
   --out-dir releases
 ```
 
-Assembly verifies the hash join (every manifest output against its bytes, the
-spine against its pin, the gate report against the Logbook build id), requires
+Assembly requires `release_role: "dense"` in the candidate manifest (as does
+the pre-flight; a candidate built before the role existed is rebuilt, never
+grandfathered), verifies the hash join (every manifest output against its
+bytes, the spine against its pin, the gate report against the Logbook build
+id), requires
 the manifest's `staging_delivery` receipt (the run's version 2 staging
 telemetry evidence, copied into `build_manifest.json` as `staging` so
 publication can apply its undelivered-staging refusal, as on the national
 lane), re-runs the candidate pre-flight, mints the cut tag
 `microcosm-uk-2024-25-dense-<YYYYMMDDTHHMMSSZ>-<uuid8>` from the run's attempt
-id, clones the H5 beside itself as `microcosm_uk_2025_dense.h5`, stages
+id, clones the H5 beside itself as `microcosm_uk_2024_25_dense.h5`, stages
 `build_manifest.json`, `release_manifest.json`, `calibration_diagnostics.json`,
 `gate_summary.json`, `uk_source_coverage.json`, the signed `uk_local_gates.json`,
 `score_vs_incumbent.json`, `incumbent_surface_evaluation.json`, the original

docs/uk-national-calibration-runbook-623.md

@@ -17,40 +17,53 @@ is the release-cut producer's job, not calibration's.
 
 ## Command shape
 
-Calibration runs through `tools/calibrate_uk_national_dataset.py`. That driver
-is the only one that builds the measure resolver and applies the committed
+Calibration runs through the rowwise driver's national release role,
+`tools/build_uk_rowwise_candidate.py --release-role national` (microcosm#823;
+it replaced the retired `tools/calibrate_uk_national_dataset.py`). The role
+delegates the build to the calibration seam library
+(`uk_runtime.calibration_run.run_uk_calibration`): it is the only path that
+builds the measure resolver from the input file and applies the committed
 measure-exclusion register, and 187 of the activated references bind model
 outputs that no frame carries — so it is the only path on which this target
-surface materializes. The June builder
-(`tools/build_uk_national_dataset.py`) constructs the calibration stage without
-either and aborts on the first unmaterializable reference; it also rebuilds SPI
-income onto its input, which a spine artifact already carries.
+surface materializes. No cloning, no ladder, national targets only. The June
+builder (`tools/build_uk_national_dataset.py`) constructs the calibration
+stage without either and aborts on the first unmaterializable reference; it
+also rebuilds SPI income onto its input, which a spine artifact already
+carries.
 
 ```bash
-uv run --no-sync python tools/calibrate_uk_national_dataset.py \
+uv run --no-sync python tools/build_uk_rowwise_candidate.py --release-role national \
   --input-h5 data/ukds/acceptance/623-first-calibrated-candidate/input-spine.h5 \
   --input-sha256 <sha256-of-input-spine-h5> \
   --ledger-facts <ledger-consumer-artifact> \
   --ledger-facts-sha256 <sha256> \
   --ledger-manifest-sha256 <sha256> \
-  --staging-h5 data/ukds/acceptance/623-first-calibrated-candidate/populace_uk_2023.h5 \
-  --diagnostics-json data/ukds/acceptance/623-first-calibrated-candidate/calibration_diagnostics.json \
-  --build-record-json data/ukds/acceptance/623-first-calibrated-candidate/build_record.json \
-  --terminal-gate-json data/ukds/acceptance/623-first-calibrated-candidate/terminal_gates.json \
-  --release-id dev-623-first-calibrated-candidate
+  --out data/ukds/acceptance/623-first-calibrated-candidate
 ```
 
+The role names every output after the pinned FRS release vintage:
+`microcosm_uk_2024_25.h5`, `calibration_diagnostics.json`,
+`build_record.json`, `microcosm_uk_2024_25.terminal_gates.json`, the frozen
+`national_target_registry.json` (the scorer's register) and
+`rowwise_candidate_manifest.json`; `--dry-run` compiles the register and
+prints the plan without solving. Staging telemetry and the staged bundle
+follow the dense role's switches (`--staging-local-only`, `--no-staging`,
+`--staging-read-back`, `--no-staged-dataset`); the run id is the attempt id.
+
 The diagnostics digest is measured, not declared: the seam writes the
 diagnostics file, hashes its actual bytes, and only then constructs and signs
 the terminal gate evidence. There is no `--calibration-diagnostics-sha256` to
 supply, and no way for the receipt to claim an identity the file does not have.
 
-Solve parameters are per-run overrides, not defaults. The campaign settings are
-`--epochs 1500 --target-weight-rule family_equal`; each override is validated
+Solve parameters are per-run overrides of the declared doctrine. Since the
+2026-09-20 ruling (microcosm#823) the doctrine is the campaign posture itself:
+1,500 epochs, `family_equal`, learning rate 0.02, seed 0, so a doctrine run
+passes no solve flags and records no overrides; `--target-weight-rule uniform`,
+another `--epochs`, `--learning-rate` or `--target-loss-cap` is validated
 through the doctrine dataclass and echoed as an explicit deviation in the
-manifest, diagnostics and build record. `--release-candidate` refuses every
-override flag, and refuses `--measure-exclusions`, so a release candidate is
-always solved under declared doctrine against the committed target surface.
+manifest, diagnostics and build record. The national role refuses
+`--release-candidate` outright: its six-gate battery cannot sign
+shippability, which comes only from `tools/certify_uk_release_cut.py`.
 
 Signing the terminal gate report needs
 `MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY` in the environment.
@@ -62,11 +75,11 @@ identity never depends on the incumbent's bytes:
 
 ```bash
 uv run --no-sync python tools/score_uk_national_candidate.py \
-  --candidate-h5 data/ukds/acceptance/623-first-calibrated-candidate/populace_uk_2023.h5 \
+  --candidate-h5 data/ukds/acceptance/623-first-calibrated-candidate/microcosm_uk_2024_25.h5 \
   --candidate-sha256 <sha256-from-the-build-record> \
   --incumbent-h5 <enhanced-frs-2024-25-h5> \
   --incumbent-sha256 <sha256> \
-  --registry-json data/ukds/acceptance/623-first-calibrated-candidate/frozen-register.json \
+  --registry-json data/ukds/acceptance/623-first-calibrated-candidate/national_target_registry.json \
   --output-json data/ukds/acceptance/623-first-calibrated-candidate/score_vs_enhanced_frs.json
 ```
 
@@ -77,10 +90,15 @@ and both sides are scored on the same frozen register.
 
 `data/ukds/acceptance/623-first-calibrated-candidate/` should contain:
 
+- `microcosm_uk_2024_25.h5`
 - `build_record.json`
-- `terminal_gates.json`
+- `microcosm_uk_2024_25.terminal_gates.json`
 - `calibration_diagnostics.json`
-- `score_vs_enhanced_frs.json`
+- `national_target_registry.json`
+- `national_contract_registry.json`
+- `rowwise_candidate_manifest.json`
+- `score_vs_incumbent.json` (the build's own when run with `--incumbent-h5`;
+  the manifest's `evaluation` block records its verdict)
 - `logbook-spool/` (one row for the attempt, whatever its disposition)
 
 Acceptance follows #578: the candidate must not regress incumbent battery

docs/uk-national-release-assembly-runbook-806.md

@@ -27,25 +27,42 @@ reference together through the run.
 
 ## 1. Calibrate the national candidate
 
-Use the national calibration driver and record the input digest rather than
-relying on a mutable path:
+Use the rowwise driver's national release role and record the input digest
+rather than relying on a mutable path:
 
 ```bash
-uv run --no-sync python tools/calibrate_uk_national_dataset.py \
+uv run --no-sync python tools/build_uk_rowwise_candidate.py --release-role national \
   --input-h5 <spine-h5> \
   --input-sha256 <spine-h5-sha256> \
   --ledger-facts <ledger-consumer-facts> \
   --ledger-facts-sha256 <ledger-facts-sha256> \
   --ledger-manifest-sha256 <ledger-manifest-sha256> \
-  --staging-h5 <candidate-dir>/microcosm_uk_2024.h5 \
-  --diagnostics-json <candidate-dir>/calibration_diagnostics.json \
-  --build-record-json <candidate-dir>/build_record.json \
-  --terminal-gate-json <candidate-dir>/terminal_gates.json \
-  --release-id dev-uk-national-calibration
+  --incumbent-h5 <incumbent-h5> \
+  --incumbent-sha256 <incumbent-h5-sha256> \
+  --out <candidate-dir>
 ```
 
-Use only the campaign doctrine overrides that were separately adjudicated.
-The build record's id has the form
+`--incumbent-h5` (with its digest; `--incumbent-label` names it, default
+`enhanced_frs_2024_25`) makes the build evaluate the finished candidate
+against the incumbent after the bundle is staged: `score_vs_incumbent.json`
+lands beside the outputs with the rule-1 verdict, rows the incumbent cannot
+materialize are pruned from both arms and named on stderr, the receipt rides
+the staging telemetry as `artifacts/score_vs_incumbent.json`, and the manifest
+records `evaluation` (`status`, `verdict`, `rule_1`, `scored_surface`,
+`pruned_measures`). The evaluation never blocks the build: an error is
+recorded as `status: error` and warned. Without the flags the manifest says
+`not_requested`, and the certifier has no receipt to read until the candidate
+is scored by hand.
+
+The role's doctrine is the campaign posture (1,500 epochs, `family_equal`,
+learning rate 0.02, seed 0): a certified cut passes no solve flags and records
+no overrides. It writes `microcosm_uk_2024_25.h5`,
+`calibration_diagnostics.json`, `build_record.json`,
+`microcosm_uk_2024_25.terminal_gates.json`, `national_target_registry.json`,
+`national_contract_registry.json` (the full compiled register the scoring
+surface takes its band edges from) and `rowwise_candidate_manifest.json`
+into `<candidate-dir>`. The build
+record's id has the form
 `uk-frs-calibration-attempt-<YYYYMMDDTHHMMSSZ>-<uuid8>`; assembly derives the
 per-cut tag from that suffix.
 
@@ -58,13 +75,13 @@ and compose the signed certification:
 
 ```bash
 uv run --no-sync python tools/certify_uk_release_cut.py \
-  --candidate-h5 <candidate-dir>/microcosm_uk_2024.h5 \
+  --candidate-h5 <candidate-dir>/microcosm_uk_2024_25.h5 \
   --candidate-sha256 <candidate-sha256-from-build-record> \
-  --candidate-name microcosm_uk_2024 \
+  --candidate-name microcosm_uk_2024_25 \
   --spine-h5 <spine-h5> \
   --diagnostics-json <candidate-dir>/calibration_diagnostics.json \
   --build-record-json <candidate-dir>/build_record.json \
-  --seam-gate-report <candidate-dir>/terminal_gates.json \
+  --seam-gate-report <candidate-dir>/microcosm_uk_2024_25.terminal_gates.json \
   --ledger-facts <ledger-consumer-facts> \
   --ledger-facts-sha256 <ledger-facts-sha256> \
   --ledger-manifest-sha256 <ledger-manifest-sha256> \
@@ -74,8 +91,8 @@ uv run --no-sync python tools/certify_uk_release_cut.py \
 ```
 
 With the default paths, this writes
-`microcosm_uk_2024.release_cut_gates.json` and
-`microcosm_uk_2024.release_certification.json` next to the candidate. Continue
+`microcosm_uk_2024_25.release_cut_gates.json` and
+`microcosm_uk_2024_25.release_certification.json` next to the candidate. Continue
 only when the certification says `shippable: true`.
 
 ## 3. Assemble the release directory
@@ -86,13 +103,13 @@ and validates the finished directory:
 
 ```bash
 uv run --no-sync python tools/assemble_uk_release_dir.py \
-  --candidate-h5 <candidate-dir>/microcosm_uk_2024.h5 \
+  --candidate-h5 <candidate-dir>/microcosm_uk_2024_25.h5 \
   --spine-h5 <spine-h5> \
-  --certification-json <candidate-dir>/microcosm_uk_2024.release_certification.json \
+  --certification-json <candidate-dir>/microcosm_uk_2024_25.release_certification.json \
   --build-record-json <candidate-dir>/build_record.json \
   --diagnostics-json <candidate-dir>/calibration_diagnostics.json \
-  --seam-gate-report <candidate-dir>/terminal_gates.json \
-  --release-cut-gate-json <candidate-dir>/microcosm_uk_2024.release_cut_gates.json \
+  --seam-gate-report <candidate-dir>/microcosm_uk_2024_25.terminal_gates.json \
+  --release-cut-gate-json <candidate-dir>/microcosm_uk_2024_25.release_cut_gates.json \
   --score-receipt <candidate-dir>/score_vs_enhanced_frs.json \
   --out-dir releases
 ```