Per-line release pointers, the UK 2024-25 national default and its promotion command (#823)

CLAUDE.md

@@ -96,8 +96,9 @@ Builds, calibrations, and releases run outside PR CI, need gated Hugging Face
 data and credentials, and cannot run from forks. Release publication is a
 deliberate human step (`tools/publish_release.sh` →
 `microcosm-publish-release`), gated by `tools/preflight_us_release_gates.py`;
-see README "Releasing & alerts". Publication also refuses a release whose
-build recorded staging telemetry that never reached its repo
+reviewed line promotion is a separate deliberate call to the same CLI with
+`--promote-line`. See README "Releasing & alerts". Publication also refuses a
+release whose build recorded staging telemetry that never reached its repo
 (`--allow-missing-staging` overrides); a build that declared `--no-staging`
 publishes without the flag. Never publish or promote artifacts as a side
 effect of another task. A UK rowwise run's **staged** bundle

README.md

@@ -175,6 +175,12 @@ tools/publish_release.sh releases/<id> --repo-id policyengine/populace-us
 publish CLI posts a release alert to Slack — `#populace-us` or `#populace-uk`,
 chosen from the repo id.
 
+Promotable UK release lines use pointers named `latest-<line>.json`. Publish a
+cut for inspection with `--no-latest --tag-name <cut-tag>`, then promote the
+reviewed cut with `--promote-line <line> --tag-name <cut-tag>`. Promotion moves
+only that line pointer; the UK repository-global `latest.json` remains frozen
+on the June 2023 release. Promotion reuses the immutable cut tag the inspect publication created (it checks the tagged manifest is byte-identical) and writes only the pointer commit, so the two-step sequence and a retry after a failed pointer commit both work. A line's registry entry is registered off the default variant until its first promotion; the default flips in a follow-up after the pointer exists.
+
 US exact-k ladder candidates use a tag-only lane. Run
 `tools/build_us_exact_k_ladder_release.py`, then execute the `publish_command`
 recorded in `package_result.json`. That command includes `--create-tag`,

changelog.d/823-line-pointers.added.md

@@ -0,0 +1 @@
+Added schema-1 per-line release pointers, publisher and CLI guards, and line-specific alerts so UK national and local cuts can be promoted without changing `latest.json`.

changelog.d/823-line-role-rule.fixed.md

@@ -0,0 +1 @@
+A UK publication line carries one dataset role: the publisher refuses a line promotion whose manifest declares another role (`national` publishes `national_default` releases, `local-k<N>` lines `non_default_local_area`), and the loader refuses a manifest that reached a line pointer with the wrong role. The release-line grammar refuses `evidence` (it would collide with the evidence-tier pointer), the loader states why every artifact must pin the pointer's revision, and the national assembler again binds `certification.candidate.filename` to the line's dataset filename.

changelog.d/823-national-promotion.added.md

@@ -0,0 +1 @@
+Add separate inspect and `latest-national.json` promotion commands to UK national release assembly, pin its candidate to `microcosm_uk_2024_25.h5`, and document the reviewed-cut workflow.

changelog.d/823-national-variant-until-promoted.changed.md

@@ -0,0 +1 @@
+The UK 2024-25 national line is registered as variant `national` (`microcosm.data.load("uk", 2025, variant="national")`), not as the 2025 default: `resolve("uk")` stays on the June 2023 line until the line's first cut is promoted and a follow-up flips the entry to the default variant, so a default load never chases a pointer that does not exist yet (review of #966; Max's #823 ruling keys the line as a variant).

changelog.d/823-promotion-reuses-the-cut-tag.fixed.md

@@ -0,0 +1 @@
+Promoting a UK release line reuses the immutable cut tag an inspect publication already created: the publisher recognises an existing tag whose `release_manifest.json` is byte-identical to the local release, creates no second immutable revision, and writes only the main commit carrying the line pointer; a tag that describes another release refuses. The documented inspect-then-promote sequence and a retry after a failure between tag creation and the pointer commit both work (review of #966).

changelog.d/823-uk-2025-registry.changed.md

@@ -0,0 +1 @@
+Registered the UK 2024-25 national line in the data registry as variant `national` on `microcosm_uk_2024_25.h5`, followed through `latest-national.json`, and taught the certified loader to follow per-line pointers at immutable revisions; non-default local-area manifests load only through their line pointers. `available()` now lists default-variant `(country, year)` pairs only (`available_variants()` lists every registered variant, the dense line included). The UK default stays on the June 2023 line until the national line's first promotion and the follow-up flip of its entry to the default variant.

docs/uk-national-release-assembly-runbook-806.md

@@ -138,7 +138,7 @@ as an operator cross-check but refuses to replace one.
 
 ## 4. Publish for inspection
 
-Run the command printed by the assembler. Its shape is:
+Run the assembler's `publish_command` first. Its shape is:
 
 ```bash
 uv run python -m microcosm.data.publish_cli \
@@ -149,17 +149,37 @@ uv run python -m microcosm.data.publish_cli \
   --tag-name microcosm-uk-2024-25-national-<timestamp>-<uuid8>
 ```
 
+The assembler also prints this `promote_command`; keep it for §6 and run it
+only after the cut passes the review in §5:
+
+```bash
+uv run python -m microcosm.data.publish_cli \
+  releases/microcosm-uk-2024-25-national \
+  --repo-id policyengine/populace-uk-private \
+  --artifact-root <candidate-dir> \
+  --promote-line national \
+  --tag-name microcosm-uk-2024-25-national-<timestamp>-<uuid8>
+```
+
 Do not omit `--tag-name`, and do not pass `--no-create-tag`: every artifact in
 the manifest is pinned to that immutable per-cut tag. `--no-latest` is
-mandatory for this inspect lane — and enforced: publication refuses to move
-`latest.json` for any tag that is not the release id itself, so omitting the
-flag fails closed instead of promoting an inspect cut.
-
-If tag creation returns HTTP 409 after the staging commit, publication can
-leave the constant branch
-`release-staging/microcosm-uk-2024-25-national` behind. Delete that branch
-manually in the private Hugging Face repository before retrying the same cut.
-Do not delete the immutable cut tag.
+mandatory for the inspect publication. The promotion command replaces it with
+`--promote-line national`; the CLI requires that flag to accompany
+`--tag-name` and refuses to combine it with `--no-latest`. A per-cut tag without
+either flag cannot move the repository-global `latest.json`.
+
+The two commands share the cut tag by design: the promotion recognises the
+immutable tag the inspect publication created, checks that the tagged
+`release_manifest.json` is byte-identical to the local one, creates no second
+immutable revision, and writes only the main commit carrying
+`latest-national.json`. The same recognition makes a retry safe after a
+failure between tag creation and the pointer commit: rerun the same command.
+A tag that describes another release refuses; publish under a fresh cut tag.
+
+Only a failure before the tag exists (the staging commit itself) can leave
+the constant branch `release-staging/microcosm-uk-2024-25-national` behind.
+Delete that branch manually in the private Hugging Face repository before
+retrying. Never delete an immutable cut tag.
 
 ## 5. Inspect on the dashboard
 
@@ -173,12 +193,21 @@ Adjudicate the release using the copied certification, scoped gate reports,
 calibration diagnostics, and score receipt. The release remains inspect-only
 until that review is complete.
 
-## Promotion is a separate change
+## 6. Promote the reviewed cut
+
+After completing the review in §5, run the assembler's `promote_command` shown
+in §4. The promotion moves only `latest-national.json`, recording `national` as
+the line and the reviewed cut tag as its `revision`. It never moves the
+repository-global `latest.json`, which remains on the June 2023 release.
+
+The 2025 national entry is registered off the default variant until this
+first promotion: `microcosm.data.load("uk", 2025, variant="national")` follows
+`latest-national.json` as soon as the pointer exists, while
+`microcosm.data.resolve("uk")` stays on the June 2023 line. The default moves
+in a one-line follow-up PR after promotion (the registry entry's `variant`
+becomes the default), with its test; sequencing it this way means a default
+load never chases a pointer that does not exist yet. An explicit
+`microcosm.data.load("uk", 2023)` continues to follow `latest.json`.
 
-Do not write `latest.json` for this line yet. The current pointer cannot name a
-per-cut tag, while the certified loader expects the artifact revision to equal
-the release id and fetches the manifest from a tag named by that id. Promotion
-needs the loader/pointer design tracked in microcosm#823 before a reviewed cut
-can become the default; publication enforces this by refusing a pointer move
-for any per-cut tag. Until then, publish every national cut with `--no-latest`
-and its explicit per-cut tag.
+With `SLACK_WEBHOOK_POPULACE_UK` configured, successful promotion sends an
+alert naming the `national` line and the promoted revision.

packages/microcosm-build/tests/test_uk_release_assembler.py

@@ -22,6 +22,7 @@
 )
 from microcosm.build.uk_runtime.release_identity import UK_NATIONAL_RELEASE_ID
 from microcosm.data.contract import validate_release_dir
+from microcosm.data.registry import REGISTRY
 from microcosm.frame import WeightKind
 
 
@@ -69,6 +70,14 @@ def test_uk_release_publisher_labels_come_from_target_contract() -> None:
     assert driver._uk_publisher_labels() == expected
 
 
+def test_national_dataset_filename_mirrors_registry() -> None:
+    driver = _load_driver_module()
+
+    assert driver._DATASET_KEY == "microcosm_uk_2024_25"
+    assert driver._DATASET_FILENAME == f"{driver._DATASET_KEY}.h5"
+    assert REGISTRY[("uk", 2025, "national")].filename == driver._DATASET_FILENAME
+
+
 def _frame(weights: list[float], *, weight_kind: WeightKind):
     ids = np.arange(len(weights), dtype="int64")
     return uk_national_frame(
@@ -330,7 +339,7 @@ def test_assemble_green_release_dir(assembler_inputs, capsys) -> None:
 
     validate_release_dir(release_dir)
     calibration_path = assembler_inputs["candidate"].with_name(
-        "microcosm_uk_2024_calibration.npz"
+        "microcosm_uk_2024_25_calibration.npz"
     )
     with np.load(calibration_path) as calibration:
         np.testing.assert_array_equal(
@@ -348,6 +357,7 @@ def test_assemble_green_release_dir(assembler_inputs, capsys) -> None:
 
     build_manifest = json.loads((release_dir / "build_manifest.json").read_text())
     release_manifest = json.loads((release_dir / "release_manifest.json").read_text())
+    assert build_manifest["dataset"]["filename"] == driver._DATASET_FILENAME
     assert build_manifest["staging"] == {
         "contract_version": 2,
         "enabled": False,
@@ -362,6 +372,14 @@ def test_assemble_green_release_dir(assembler_inputs, capsys) -> None:
     }
     assert build_manifest["attempt_id"] == _ATTEMPT_ID
     assert build_manifest["cut_tag"] == _CUT_TAG
+    assert release_manifest["default_datasets"] == {"national": driver._DATASET_KEY}
+    assert release_manifest["artifacts"][driver._DATASET_KEY]["path"] == (
+        driver._DATASET_FILENAME
+    )
+    assert (
+        release_manifest["artifacts"][f"{driver._DATASET_KEY}_calibration"]["path"]
+        == f"{driver._DATASET_KEY}_calibration.npz"
+    )
     assert {entry["revision"] for entry in release_manifest["artifacts"].values()} == {
         _CUT_TAG
     }
@@ -391,6 +409,24 @@ def test_assemble_green_release_dir(assembler_inputs, capsys) -> None:
         "--tag-name",
         _CUT_TAG,
     ]
+    promote_command = shlex.split(summary["promote_command"])
+    assert promote_command == [
+        "uv",
+        "run",
+        "python",
+        "-m",
+        "microcosm.data.publish_cli",
+        str(release_dir),
+        "--repo-id",
+        "policyengine/populace-uk-private",
+        "--artifact-root",
+        str(assembler_inputs["candidate"].parent),
+        "--promote-line",
+        "national",
+        "--tag-name",
+        _CUT_TAG,
+    ]
+    assert "--no-latest" not in promote_command
 
 
 def test_assemble_preserves_successful_version_2_delivery(
@@ -432,6 +468,23 @@ def test_assemble_refuses_candidate_sha_mismatch(assembler_inputs) -> None:
         _load_driver_module().main(assembler_inputs["argv"])
 
 
+def test_assemble_refuses_candidate_with_wrong_filename(assembler_inputs) -> None:
+    candidate = assembler_inputs["candidate"]
+    wrongly_named_candidate = candidate.with_name("microcosm_uk_2024.h5")
+    wrongly_named_candidate.write_bytes(candidate.read_bytes())
+    argv = list(assembler_inputs["argv"])
+    argv[argv.index("--candidate-h5") + 1] = str(wrongly_named_candidate)
+
+    with pytest.raises(
+        SystemExit,
+        match=(
+            r"national line's candidate is microcosm_uk_2024_25\.h5 as written by "
+            r"tools/build_uk_rowwise_candidate\.py --release-role national"
+        ),
+    ):
+        _load_driver_module().main(argv)
+
+
 def test_assemble_refuses_unshippable_certification(assembler_inputs) -> None:
     certification = json.loads(assembler_inputs["certification"].read_text())
     certification["shippable"] = False

packages/microcosm-build/tests/test_uk_staging_integration.py

@@ -147,7 +147,10 @@ def test_synthetic_smoke_command_refuses_release_options(tmp_path: Path) -> None
         cwd=ROOT,
         text=True,
         capture_output=True,
-        timeout=30,
+        # The refusal comes from argparse, but reaching it imports the build
+        # package and the engine; the 3.14 CI runner has taken over 30 s for
+        # that alone (a flake seen on #966), so allow what a cold import needs.
+        timeout=120,
     )
 
     assert result.returncode == 2

packages/microcosm-build/tests/uk_certification_fixtures.py

@@ -167,7 +167,7 @@ def green_score_receipt(candidate_sha: str) -> dict:
 def green_certification_inputs(tmp_path: Path):
     """Three green signed parts plus a closed identity join."""
 
-    candidate = tmp_path / "microcosm_uk_2024.h5"
+    candidate = tmp_path / "microcosm_uk_2024_25.h5"
     candidate.write_bytes(b"candidate-bytes")
     candidate_sha = sha256(candidate)
     diagnostics = tmp_path / "calibration_diagnostics.json"
@@ -248,7 +248,7 @@ def green_certification_inputs(tmp_path: Path):
     )
     return {
         "release_id": "uk-757-first-certified-cut",
-        "candidate_name": "microcosm_uk_2024",
+        "candidate_name": "microcosm_uk_2024_25",
         "candidate_path": candidate,
         "candidate_sha256": candidate_sha,
         "spine_sha256": GREEN_PARENT_SPINE_SHA256,

packages/microcosm-data/README.md

@@ -23,8 +23,9 @@ sim.calculate("household_net_income", 2024).sum()
 `latest.json`, reads the selected release manifest at the immutable release
 tag, verifies the artifact SHA-256, and refuses model or Core versions outside
 the release's certified compatibility specifiers. `available()` lists
-published `(country, year)` pairs; `available_variants()` lists every published
-`(country, year, variant)`.
+the default-variant `(country, year)` pairs (the ones a bare ``load(country, year)``
+resolves); `available_variants()` lists every published
+`(country, year, variant)`, non-default lines included.
 
 The old mutable-root behavior is available only as an unsafe escape hatch:
 `load("us", 2024, unverified_root=True)`. It emits a runtime warning because it

packages/microcosm-data/src/microcosm/data/contract.py

@@ -71,6 +71,7 @@
     "US_SOURCE_COVERAGE_DIAGNOSTICS_FILE",
     "ReleaseContractError",
     "compatibility_claim_declarer_error",
+    "line_for_release_id",
     "release_dataset_role",
     "required_release_files",
     "validate_evidence_release_dir",
@@ -615,6 +616,38 @@ def compatibility_claim_declarer_error(declared_by: object) -> str | None:
 # microcosm.build.uk_runtime.release_identity.UK_NATIONAL_RELEASE_ID (the
 # data shard cannot import the build shard); lockstep-tested.
 _UK_NATIONAL_RELEASE_ID = "microcosm-uk-2024-25-national"
+_UK_LOCAL_LINE_RELEASE_ID_RE = re.compile(
+    r"^microcosm-uk-2024-25-local-k(?P<n>[1-9][0-9]*)$"
+)
+
+
+def line_for_release_id(release_id: str) -> str | None:
+    """Return the promotable UK publication line for ``release_id``."""
+    if release_id == _UK_NATIONAL_RELEASE_ID:
+        return "national"
+    match = _UK_LOCAL_LINE_RELEASE_ID_RE.fullmatch(release_id)
+    if match is not None:
+        return f"local-k{match.group('n')}"
+    return None
+
+
+def dataset_role_for_line(line: str) -> str:
+    """The one dataset role a publication line may carry.
+
+    The national line publishes national-default releases; every local-area
+    line publishes non-default local-area releases. The publisher refuses a
+    line promotion whose manifest declares another role, and the loader
+    refuses a manifest that reached a line pointer with the wrong role, so a
+    local-area release can never ride the national pointer (or the reverse)
+    on the strength of its release id alone.
+    """
+    if line == "national":
+        return NATIONAL_DEFAULT_DATASET_ROLE
+    if re.fullmatch(r"local-k[1-9][0-9]*", line):
+        return NON_DEFAULT_LOCAL_AREA_DATASET_ROLE
+    raise ValueError(f"{line!r} is not a publication line.")
+
+
 # The dense joint national + local line (microcosm#762 A18, ruling
 # 2026-09-03): the same constant-id approach, published on the inspect lane
 # under the non-default local-area role. Mirrored from
@@ -682,6 +715,17 @@ def compatibility_claim_declarer_error(declared_by: object) -> str | None:
 # a hand-edited or stale revision cannot claim a cut the attempt chain never
 # produced.
 _UK_NATIONAL_REVISION_SUFFIX_RE = re.compile(r"[0-9]{8}T[0-9]{6}Z-[0-9a-f]{8}")
+
+
+def _uk_line_cut_tag_re(release_id: str) -> re.Pattern[str] | None:
+    """Return the immutable cut-tag grammar for a promotable UK line."""
+    if line_for_release_id(release_id) is None:
+        return None
+    return re.compile(
+        re.escape(release_id) + "-" + _UK_NATIONAL_REVISION_SUFFIX_RE.pattern
+    )
+
+
 # The canonical release-dir filenames of the evidence the certification signs
 # (tools/assemble_uk_release_dir.py copies each byte-for-byte). Validation
 # binds every local file to its signed digest: a certification whose evidence
@@ -1276,6 +1320,7 @@ def _check_release_manifest(
             "release_manifest.json must declare a non-empty 'artifacts' mapping."
         )
     else:
+        line_cut_tag_re = _uk_line_cut_tag_re(release_id)
         diagnostics_artifact = artifacts.get("calibration_diagnostics")
         if not isinstance(diagnostics_artifact, Mapping):
             failures.append(
@@ -1303,13 +1348,9 @@ def _check_release_manifest(
                 revision == release_id
                 or (annual_revision is not None and revision == annual_revision)
                 or (
-                    release_id == _UK_NATIONAL_RELEASE_ID
+                    line_cut_tag_re is not None
                     and isinstance(revision, str)
-                    and revision.startswith(release_id + "-")
-                    and _UK_NATIONAL_REVISION_SUFFIX_RE.fullmatch(
-                        revision[len(release_id) + 1 :]
-                    )
-                    is not None
+                    and line_cut_tag_re.fullmatch(revision) is not None
                 )
             )
             # A present-but-non-string revision must fail here rather than
@@ -1322,7 +1363,7 @@ def _check_release_manifest(
                 expected = (
                     f"the release id {release_id!r} or a "
                     f"'{release_id}-<YYYYMMDDTHHMMSSZ>-<uuid8>' per-cut tag"
-                    if release_id == _UK_NATIONAL_RELEASE_ID
+                    if line_cut_tag_re is not None
                     else f"the release id {release_id!r}"
                 )
                 failures.append(