UK: make the promoted national line the default load (#823)

changelog.d/823-national-line-default-load.changed.md

@@ -0,0 +1 @@
+The UK 2024-25 national line is the default UK load: `microcosm.data.load("uk")` and `load("uk", 2025)` follow `latest-national.json`, which has named the certified cut microcosm-uk-2024-25-national-20261025T091414Z-a6299504 (built on policyengine-uk 2.151.0) since 26 October 2026, instead of the June 2023 `populace_uk_2023` artifact, which stays at `load("uk", 2023)`. The line's own `national` variant key retires (microcosm#823, Max's ruling).

changelog.d/823-uk-2025-registry.changed.md

@@ -1 +1 @@
-Registered the UK 2024-25 national line in the data registry as variant `national` on `microcosm_uk_2024_25.h5`, followed through `latest-national.json`, and taught the certified loader to follow per-line pointers at immutable revisions; non-default local-area manifests load only through their line pointers. `available()` now lists default-variant `(country, year)` pairs only (`available_variants()` lists every registered variant, the dense line included). The UK default stays on the June 2023 line until the national line's first promotion and the follow-up flip of its entry to the default variant.
+Registered the UK 2024-25 national line in the data registry as variant `national` on `microcosm_uk_2024_25.h5`, followed through `latest-national.json`, and taught the certified loader to follow per-line pointers at immutable revisions; non-default local-area manifests load only through their line pointers. `available()` now lists default-variant `(country, year)` pairs only (`available_variants()` lists every registered variant, the dense line included).

docs/uk-national-release-assembly-runbook-806.md

@@ -222,13 +222,11 @@ in §4. The promotion moves only `latest-national.json`, recording `national` as
 the line and the reviewed cut tag as its `revision`. It never moves the
 repository-global `latest.json`, which remains on the June 2023 release.
 
-The 2025 national entry is registered off the default variant until this
-first promotion: `microcosm.data.load("uk", 2025, variant="national")` follows
-`latest-national.json` as soon as the pointer exists, while
-`microcosm.data.resolve("uk")` stays on the June 2023 line. The default moves
-in a one-line follow-up PR after promotion (the registry entry's `variant`
-becomes the default), with its test; sequencing it this way means a default
-load never chases a pointer that does not exist yet. An explicit
+The 2025 national entry was registered off the default variant until the
+first promotion, so that a default load never chased a pointer that did not
+exist yet. The first certified cut was promoted on 2026-10-26, and the entry
+is now the default variant (microcosm#823): `microcosm.data.load("uk")` and
+`microcosm.data.load("uk", 2025)` follow `latest-national.json`. An explicit
 `microcosm.data.load("uk", 2023)` continues to follow `latest.json`.
 
 With `SLACK_WEBHOOK_POPULACE_UK` configured, successful promotion sends an

packages/microcosm-build/tests/engine_free/uk/test_uk_release_assembler.py

@@ -28,7 +28,7 @@
     uk_weight_projection_factor,
 )
 from microcosm.data.contract import validate_release_dir
-from microcosm.data.registry import REGISTRY
+from microcosm.data.registry import DEFAULT_VARIANT, REGISTRY
 from microcosm.frame import WeightKind
 from test_support.paths import paths_for
 
@@ -87,7 +87,7 @@ def test_national_dataset_filename_mirrors_registry() -> None:
 
     assert driver._DATASET_KEY == "microcosm_uk_2024_25"
     assert driver._DATASET_FILENAME == f"{driver._DATASET_KEY}.h5"
-    assert REGISTRY[("uk", 2025, "national")].filename == driver._DATASET_FILENAME
+    assert REGISTRY[("uk", 2025, DEFAULT_VARIANT)].filename == driver._DATASET_FILENAME
 
 
 def _frame(weights: list[float], *, weight_kind: WeightKind):

packages/microcosm-data/src/microcosm/data/registry.py

@@ -114,15 +114,15 @@ def register(spec: DatasetSpec) -> DatasetSpec:
 register(
     DatasetSpec(
         # microcosm#823: the UK 2024-25 national line, followed through its
-        # own pointer. Registered OFF the default variant until the first
-        # promoted cut exists on the Hub: resolve("uk") picks the latest
-        # default-variant year at import time, so a default entry here would
-        # send every default load to a pointer that does not exist yet. The
-        # flip to DEFAULT_VARIANT is a one-line follow-up after promotion
-        # (review of #966; Max's #823 ruling keys the line as a variant).
+        # own pointer, is the default UK load. It stayed off the default
+        # variant until its first certified cut was promoted (review of #966;
+        # Max's #823 ruling), and latest-national.json has named
+        # microcosm-uk-2024-25-national-20261025T091414Z-a6299504 since
+        # 2026-10-26. resolve("uk") picks the latest default-variant year, so
+        # it loads this line; the June 2023 artifact stays at ("uk", 2023).
         country="uk",
         year=2025,
-        variant="national",
+        variant=DEFAULT_VARIANT,
         hf_repo="policyengine/populace-uk-private",
         filename="microcosm_uk_2024_25.h5",
         engine_module="policyengine_uk.data",

packages/microcosm-data/tests/engine_free/shared/test_loader.py

@@ -345,23 +345,22 @@ def __init__(self, *, file_path: str) -> None:
     return constructed
 
 
-def test_resolve_uk_default_stays_on_2023_until_the_national_line_is_promoted():
-    """The national line is registered off the default variant until its
-    pointer exists on the Hub, so a default load never chases an absent
-    pointer (review of #966); the flip is a one-line follow-up."""
-    assert latest_year("uk") == 2023
-    assert resolve("uk").key == ("uk", 2023, DEFAULT_VARIANT)
-    assert resolve("uk", 2025, variant="national").pointer_path == (
-        "latest-national.json"
-    )
-    with pytest.raises(ValueError, match="published years"):
-        resolve("uk", 2025)
+def test_resolve_uk_default_is_the_promoted_national_line():
+    """microcosm#823: once the national line's first certified cut was
+    promoted, the line became the default UK load; the June 2023 line stays
+    addressable at its year and its own pointer (Max's #823 ruling)."""
+    assert latest_year("uk") == 2025
+    assert resolve("uk").key == ("uk", 2025, DEFAULT_VARIANT)
+    assert resolve("uk").pointer_path == "latest-national.json"
+    assert resolve("uk", 2023).pointer_path == "latest.json"
+    with pytest.raises(ValueError, match="variant 'national'"):
+        resolve("uk", 2025, variant="national")
 
 
 def test_resolve_uk_national_variant_follows_the_line_pointer(
     tmp_path: Path,
 ) -> None:
-    spec = resolve("uk", 2025, variant="national")
+    spec = resolve("uk", 2025)
     manifest = _uk_release_manifest()
     hub = _line_pointer_hub(
         tmp_path,
@@ -374,7 +373,7 @@ def test_resolve_uk_national_variant_follows_the_line_pointer(
     certified = loader._resolve_certified_release(spec, hub_download=hub)
 
     manifest_path = f"releases/{UK_NATIONAL_RELEASE_ID}/release_manifest.json"
-    assert spec.key == ("uk", 2025, "national")
+    assert spec.key == ("uk", 2025, DEFAULT_VARIANT)
     assert spec.pointer_path == "latest-national.json"
     assert certified.release_id == UK_NATIONAL_RELEASE_ID
     assert certified.artifact_revision == UK_NATIONAL_CUT_TAG
@@ -449,9 +448,7 @@ def test_line_pointer_refuses_any_artifact_revision_mismatch(tmp_path: Path) ->
         ValueError,
         match=r"artifacts not pinned to expected revision.*calibration_report",
     ):
-        loader._resolve_certified_release(
-            resolve("uk", 2025, variant="national"), hub_download=hub
-        )
+        loader._resolve_certified_release(resolve("uk", 2025), hub_download=hub)
 
 
 def test_local_area_line_selects_its_only_microdata_artifact(tmp_path: Path) -> None:
@@ -568,9 +565,7 @@ def test_loader_refuses_an_unknown_dataset_role(tmp_path: Path) -> None:
     )
 
     with pytest.raises(ValueError, match="unknown dataset_role 'research_preview'"):
-        loader._resolve_certified_release(
-            resolve("uk", 2025, variant="national"), hub_download=hub
-        )
+        loader._resolve_certified_release(resolve("uk", 2025), hub_download=hub)
 
 
 def test_download_uses_pointer_manifest_and_release_revision(
@@ -804,7 +799,7 @@ def _is_missing_pointer_error(exc: BaseException, pointer_path: str) -> bool:
     [
         ("us", 2024, DEFAULT_VARIANT),
         ("uk", 2023, DEFAULT_VARIANT),
-        ("uk", 2025, "national"),
+        ("uk", 2025, DEFAULT_VARIANT),
     ],
 )
 def test_live_latest_pointer_resolves_to_a_coherent_certified_release(
@@ -823,6 +818,9 @@ def test_live_latest_pointer_resolves_to_a_coherent_certified_release(
     which are updated atomically with each certification PR.
     """
     spec = resolve(country, year, variant=variant)
+    # The UK 2025 default follows its line's pointer since microcosm#823, so
+    # the pointer, not the variant, says how the release is named.
+    line = loader._line_from_pointer_path(spec.pointer_path)
     try:
         certified = loader._resolve_certified_release(spec)
     except Exception as exc:
@@ -834,26 +832,26 @@ def test_live_latest_pointer_resolves_to_a_coherent_certified_release(
             pytest.skip(
                 f"repo requires credentials this environment lacks (private): {exc}"
             )
-        if variant != DEFAULT_VARIANT and _is_missing_pointer_error(
-            exc, spec.pointer_path
-        ):
+        if line is not None and _is_missing_pointer_error(exc, spec.pointer_path):
             # A line pointer exists only once its first cut is promoted;
             # until then the line is registered but not yet live. Anything
             # else on a promoted line (a manifest 404, a bad pin) fails.
             pytest.skip(f"line pointer {spec.pointer_path} not yet promoted: {exc}")
         raise
 
-    assert certified.release_id.startswith(
-        # Live certified releases still carry the pre-rename id prefix; new
-        # builds may adopt the microcosm- prefix at the next versioned release.
-        (f"microcosm-{country}-{spec.year}-", f"populace-{country}-{spec.year}-")
-    )
     assert certified.artifact_repo_id == spec.hf_repo
     assert certified.artifact_path == spec.filename
-    if variant == DEFAULT_VARIANT:
+    if line is None:
+        assert certified.release_id.startswith(
+            # Live certified releases still carry the pre-rename id prefix; new
+            # builds may adopt the microcosm- prefix at the next versioned
+            # release.
+            (f"microcosm-{country}-{spec.year}-", f"populace-{country}-{spec.year}-")
+        )
         assert certified.artifact_revision == certified.release_id
     else:
         # A line pointer names the immutable cut tag of its constant id.
+        assert certified.release_id.startswith(f"microcosm-{country}-")
         assert certified.artifact_revision.startswith(f"{certified.release_id}-")
     assert re.fullmatch(r"[0-9a-f]{64}", certified.artifact_sha256)
     assert certified.model.name == spec.engine_package
@@ -862,7 +860,7 @@ def test_live_latest_pointer_resolves_to_a_coherent_certified_release(
 def test_line_pointer_refuses_a_manifest_with_another_lines_role(tmp_path: Path):
     """A line pointer carries one role: the national pointer cannot select a
     local-area manifest, nor a local-area pointer a national one."""
-    spec = resolve("uk", 2025, variant="national")
+    spec = resolve("uk", 2025)
     manifest = _uk_release_manifest()
     manifest["dataset_role"] = "non_default_local_area"
     manifest["default_datasets"] = {}

packages/microcosm-data/tests/engine_free/shared/test_registry.py

@@ -69,8 +69,8 @@ def test_available_lists_default_variant_keys_only_sorted(monkeypatch) -> None:
     keys = available()
     assert ("us", 2024) in keys
     assert ("uk", 2023) in keys
-    # The national line is off the default variant until promoted.
-    assert ("uk", 2025) not in keys
+    # The national line is the default variant since its promotion (#823).
+    assert ("uk", 2025) in keys
     assert ("zz", 2099) not in keys
     assert keys == sorted(keys)
 
@@ -146,16 +146,16 @@ def test_uk_2025_dense_is_registered_off_the_default_variant() -> None:
     assert spec.filename == "microcosm_uk_2024_25_dense.h5"
     assert spec.engine_class == "UKSingleYearDataset"
     assert spec.engine_package == "policyengine-uk"
-    assert ("uk", 2025, "national") in REGISTRY
+    assert ("uk", 2025, DEFAULT_VARIANT) in REGISTRY
 
 
-def test_uk_2025_national_is_registered_off_the_default_variant_until_promoted():
-    """resolve("uk") must not reach a pointer that does not exist yet: the
-    line's pointer appears on the Hub at its first promotion, and only the
-    follow-up flip to the default variant moves the default (review of
-    #966; Max's #823 ruling keys the line as a variant)."""
-    spec = REGISTRY[("uk", 2025, "national")]
-    assert spec.variant == "national" != DEFAULT_VARIANT
+def test_uk_2025_national_line_is_the_default_variant_after_promotion():
+    """microcosm#823: the line's first certified cut was promoted on
+    2026-10-26, so the line moved to the default variant (Max's #823
+    ruling) and its own variant key retired; the June 2023 line stays
+    addressable at its year."""
+    spec = REGISTRY[("uk", 2025, DEFAULT_VARIANT)]
     assert spec.pointer_path == "latest-national.json"
     assert spec.filename == "microcosm_uk_2024_25.h5"
-    assert ("uk", 2025, DEFAULT_VARIANT) not in REGISTRY
+    assert ("uk", 2025, "national") not in REGISTRY
+    assert REGISTRY[("uk", 2023, DEFAULT_VARIANT)].filename == "populace_uk_2023.h5"