Publish calibration diagnostics as schema 9, each hierarchy label declared once

README.md

@@ -367,8 +367,9 @@ the historical `record` and `projection`) penalize distance from the design
 weights; see [penalized calibration toward the design weights](docs/calibration-l2-basis.md)
 for the algebra, the evidence and the measured frontier.
 
-National and ACS local-area builds now use the same typed schema-8 calibration
-diagnostics writer. The local builder adds its Census population marginals to a
+National and ACS local-area builds now use the same typed calibration
+diagnostics writer, which publishes schema 9: the schema-8 document builders
+assemble, with each hierarchy label declared once in `hierarchy_catalogs`. The local builder adds its Census population marginals to a
 versioned `TargetRegistry`, including provider, category, geography, and target
 hierarchy, before calibration. Both builders always attempt diagnostics after
 the calibrated dataset exists. If construction, validation, serialization, or
@@ -383,11 +384,11 @@ UK release workflows stop when diagnostics are unavailable because their later
 release checks require that evidence; historical schema-6 and schema-7 UK
 artifacts remain readable through isolated compatibility validation.
 
-A Draft 2020-12 JSON Schema for schema 8, generated from those models, is
-committed at
-`packages/microcosm-diagnostics/src/microcosm/diagnostics/schemas/calibration_diagnostics.v8.schema.json`
-for consumers that check the file without importing them, such as the
-collector. Its description lists the model rules a JSON Schema cannot express;
+Draft 2020-12 JSON Schemas for schema 9 and schema 8, generated from those
+models, are committed at
+`packages/microcosm-diagnostics/src/microcosm/diagnostics/schemas/calibration_diagnostics.v9.schema.json`
+and `calibration_diagnostics.v8.schema.json` beside it, for consumers that check
+the file without importing them, such as the collector. Its description lists the model rules a JSON Schema cannot express;
 see the [package README](packages/microcosm-diagnostics/README.md).
 
 Standard publication uploads the locally built `releases/<id>/` artifacts to

changelog.d/1341-diagnostics-schema-9-catalogs.changed.md

@@ -0,0 +1 @@
+Calibration diagnostics are published as schema 9: each provider, category, geography and dimension label is declared once in `hierarchy_catalogs`, and target rows refer to it by ID, so one geography or dimension value can no longer carry two labels in a published file (microcosm#1341). Producers still assemble schema 8; the writer (`catalog_calibration_diagnostics`) builds the catalogs and refuses rows that label one identifier two ways, with the failure code `label_conflict` and no file, unless a reviewed label choice in `microcosm.calibrate.hierarchy_labels` names the label. The UK choices give a `geography` dimension value its area's label (the payload's geography label, `UK_GEOGRAPHY_ID_TO_LABEL`, or the ONS local-authority names), which resolves the 88 areas the October dense release names two ways; the US choices name the label chosen in review for each of the three values two US tables word differently ("No adjusted gross income", "All returns, total", "Age 85 plus"), covering only the two spellings Chronicle registers as reviewed (chronicle#377), so a third spelling is still refused; the Belgian choice writes `measure` = `country_total` as "Country total", covering only the four spellings Belgium's Chronicle packages give it, without which a Belgian build would stop at its diagnostics step. Every disagreement a choice settles is logged as a warning. The release contract, the UK local scorer and the dense assembler accept schema 8 and 9, and a generated `calibration_diagnostics.v9.schema.json` ships beside the schema 8 file. The transport diagnostics step (`diagnostics.calibration@1`) and the UK full build's calibrated node (`uk.full.gates.calibrated`, through its `terminal_code` parameter) include the catalogs, label-choice, geography-constant and (UK) schema modules in their keys, so a resumed graph store recomputes the diagnostics when one of those modules changes. A later edit to the label choices, the catalogs or the schema module leaves the UK preflight and dense solve keys unchanged; an edit to the UK local-authority names (`local_authority_names.json`) or to `uk_hierarchy_label_choices` still re-solves, because keys before the solve cover them. This change itself gives the UK dense solve a new key: it edits `microcosm.calibrate.diagnostics`, which the key of `calibrate.adam@1`, and so of every UK calibration kernel, covers, and `uk_runtime/graph_terminal.py` and `uk_runtime/diagnostics.py`, which the UK preflight's key covers.

docs/agent-guide.md

@@ -455,14 +455,15 @@ module instead of copying it or reconstructing alternate views in consumers.
 
 ## Calibration diagnostics JSON Schema
 
-`packages/microcosm-diagnostics/src/microcosm/diagnostics/schemas/calibration_diagnostics.v8.schema.json`
-is generated from the schema-8 models in `microcosm.diagnostics.schema`. After
-changing one, regenerate it with
+`packages/microcosm-diagnostics/src/microcosm/diagnostics/schemas/calibration_diagnostics.v9.schema.json`
+(what builds publish) and `calibration_diagnostics.v8.schema.json` beside it
+(documents published before schema 9) are generated from the models in
+`microcosm.diagnostics.schema`. After changing one, regenerate both with
 `uv run --no-sync python tools/generate_calibration_diagnostics_json_schema.py`.
 A new model validator also needs an entry in
-`CALIBRATION_DIAGNOSTICS_V8_JSON_SCHEMA_GAPS` and a document in
-`test_json_schema.py` that the schema accepts and the models refuse; the
-diagnostics tests fail until both exist.
+`CALIBRATION_DIAGNOSTICS_V9_JSON_SCHEMA_GAPS` (or the schema-8 list) and a
+document in `test_json_schema_v9.py` (or `test_json_schema.py`) that the schema
+accepts and the models refuse; the diagnostics tests fail until both exist.
 
 ## Review this guide
 

docs/calibration-target-hierarchy.md

@@ -1,7 +1,8 @@
 # Calibration target hierarchy
 
-Microcosm diagnostics schema 8 publishes a fixed sequence of hierarchy tier
-types for every registry-backed calibration target:
+Microcosm calibration diagnostics (schema 8, and schema 9 which current
+builds publish) carry a fixed sequence of hierarchy tier types for every
+registry-backed calibration target:
 
 1. provider
 2. category
@@ -127,6 +128,100 @@ that labels the IRS SOI district rows translated to the 119th Congress plan.
 That release's materialize stage assembles its admin and population targets
 into one registry before the engine pass, so a conflict stops it at the start.
 
+## Diagnostics schema 9: labels declared once
+
+Schema 8 copies every provider, category, geography and dimension label onto
+every target row. The registry check above covers geography ids only, and
+Chronicle reports a value two packages word differently only as a warning, so
+a schema 8 file could still label one identifier two ways. Published examples:
+the US national release `populace-us-2024-0fb05b6-4b57d15a287c-20260930T150401Z`
+labels the AGI band `under_1` "No adjusted gross income" (106 targets) and "No
+adjusted gross income and deficit" (2), and the October 2026 UK dense release
+gives 88 areas two names, because Chronicle's local facts carry the area as a
+dimension named `geography` whose value is the lower-case GSS code and each
+publisher names it its own way (HMRC "Hartlepool UA", ONS "Hartlepool").
+
+Schema 9 (`CalibrationDiagnosticsV9`) declares each of those labels once, in a
+top-level `hierarchy_catalogs` object, and each row's `hierarchy` refers to
+them by id:
+
+```json
+{
+  "schema_version": 9,
+  "hierarchy_catalogs": {
+    "providers": [{"id": "obr", "label": "Office for Budget Responsibility"}],
+    "categories": [
+      {"id": "obr.efo_receipts", "label": "Economic and fiscal outlook receipts",
+       "provider_id": "obr"}
+    ],
+    "geographies": [
+      {"id": "K02000001", "label": "United Kingdom", "level": "country"}
+    ],
+    "dimensions": [
+      {"id": "obr.efo_line", "label": "Economic and fiscal outlook line",
+       "values": [{"id": "income_tax", "label": "Income tax"}]}
+    ]
+  },
+  "targets": [
+    {
+      "name": "obr.income_tax@2025",
+      "target_name": "obr.income_tax",
+      "hierarchy": {
+        "category_id": "obr.efo_receipts",
+        "geography_id": "K02000001",
+        "dimensions": [{"id": "obr.efo_line", "value_id": "income_tax"}],
+        "target": {"id": "obr.income_tax", "label": "Income tax receipts"}
+      }
+    }
+  ]
+}
+```
+
+The provider is the category's; providers and categories move into catalogs
+with geographies and dimensions, mirroring the authoring contract's catalogs,
+so a category cannot appear under two providers either. Each value is listed
+under its own dimension. A geography id has one label and one level. The
+target's own label stays on the row, and every other field is schema 8's.
+
+Producers still assemble schema 8 (`CALIBRATION_DIAGNOSTICS_ASSEMBLY_SCHEMA_VERSION`);
+the writer publishes schema 9 (`catalog_calibration_diagnostics`). It refuses,
+with the failure code `label_conflict` and no file, rows that give one id more
+than one label (or one geography more than one level) unless a reviewed label
+choice resolves it (`microcosm.calibrate.hierarchy_labels`):
+
+- UK: a `geography` dimension value takes its area's label: the payload's own
+  geography label for that code, else `UK_GEOGRAPHY_ID_TO_LABEL`, else the ONS
+  April 2023 local-authority names in `local_authority_names.json` (the UK
+  build's writer adds those). This resolves all 88 dense-release areas, two of
+  them constituencies that only the payload's geography names, and the
+  national releases' "Yorkshire And The Humber".
+- US: `under_1` is "No adjusted gross income", `all` is "All returns, total",
+  and `age` `85_plus` is "Age 85 plus", the labels chosen in review on
+  2026-10-21. A US feed re-pin brings the same labels from Chronicle
+  (chronicle#377 for `under_1`), so these choices matter only for feeds and
+  releases built before it. Each choice covers only the two spellings
+  Chronicle registers as reviewed variants; a third spelling is refused.
+- Belgium: `measure` `country_total` is "Country total", the label chosen in
+  review on 2026-10-22 and the one Chronicle's canonical register gives it
+  (chronicle#377). Each Belgian package (SPF Finances, ONSS, ONEM, NBB)
+  labels its national figure with its own measure name, so without this choice
+  the transport graph's diagnostics step raises and a Belgian build fails. The
+  choice covers only those four spellings; another is refused.
+
+Every disagreement a choice settles is logged as a warning naming the id, each
+spelling with its first target, and the label written, so a choice never hides
+a variant.
+
+A choice applies only where the rows disagree, so an id every row labels one
+way keeps its label. Any other disagreement stops the writer until the source
+is corrected or a reviewed choice is added. The same catalogs are written by
+the transport diagnostics kernel and the UK graph's final validation kernel,
+which raise instead of writing.
+
+Releases published before schema 9 keep their schema-8 files;
+`parse_calibration_diagnostics` reads both, and
+`expand_calibration_diagnostics` rebuilds schema 8 from schema 9.
+
 ## Propagation stages
 
 The expected data flow is:
@@ -143,9 +238,11 @@ The expected data flow is:
    readable for historical artifacts.
 7. Solver compilation carries the hierarchy unchanged from `TargetSpec` to
    `Target`.
-8. Diagnostics schema 8 serializes `targets[].hierarchy` without constructing
-   provider, category, geography, dimension, or target labels.
-9. Release validation requires every schema-8 tier and label.
+8. Diagnostics assembly (schema 8) serializes `targets[].hierarchy` without
+   constructing provider, category, geography, dimension, or target labels;
+   the writer publishes schema 9, declaring each label once in
+   `hierarchy_catalogs`.
+9. Release validation requires every tier and label, in schema 8 or 9.
 10. The calibration dashboard selects its parser from top-level
     `schema_version` and builds navigation from the ordered hierarchy.
 

docs/transport-graph-kernels.md

@@ -16,7 +16,7 @@ described here.
 | `targets.problem@1` | `transport/target_kernels.py` | compute | population slices; params `entities`, `weight_entity`; artifact `surface` | `problem` (`microcosm.calibrate.ordered-problem` v1) |
 | `calibrate.ordered_adam@1` | `calibrate/ordered_kernels.py` | REWEIGHT | one slice on the weight entity; artifact `problem`; params `epochs`, `learning_rate`, `mass`, optional `max_weight_ratio` with `weight_anchor="design"` | weights; `solution`, `result` |
 | `takeup.compare@1` | `transport/target_kernels.py` | compute | one declared source (hold-out facts); population slices; params as `targets.compile@1` plus `entities`, `weight_entity` | `comparison` (`microcosm.targets.comparison` v1) |
-| `diagnostics.calibration@1` | `transport/terminal_kernels.py` | compute | one slice on the weight entity; artifacts `problem`, `solution`, `result`, `surface`; params `weight_entity`, optional `build` | `diagnostics` (`microcosm.diagnostics.calibration` v8) |
+| `diagnostics.calibration@1` | `transport/terminal_kernels.py` | compute | one slice on the weight entity; artifacts `problem`, `solution`, `result`, `surface`; params `weight_entity`, optional `build` | `diagnostics` (`microcosm.diagnostics.calibration` v9) |
 | `gates.battery@1` | `transport/gate_kernels.py` | GATE | params `country`, `gates`, `gates_sha256`, `phase`, `release_candidate`, optional `synthetic_smoke`, `upstream`, and `entities` with `weight_entity`; any artifacts as evidence | `gate_report` (`microcosm.gates.phase-report` v1) |
 | `export.prepare@1` | `transport/terminal_kernels.py` | compute | population slices; the gate reports; params `entities`, `weight_entity`, `time_period`, optional `bindings` | `export_descriptor` |
 | `export.readback@1` | `transport/terminal_kernels.py` | GATE | one declared source (the written H5); artifact `export_descriptor` | `export_readback` |
@@ -106,9 +106,15 @@ that rejects; if it lowers them, the solver's clamp is the tighter bound.
 
 `diagnostics.calibration@1` rebuilds the completed calibration from the
 `problem`, `solution` and `result` artifacts without solving, and emits the
-schema-8 model. Each row describes its target as compiled (its own entity,
-measure and filter), with its hierarchy from the surface's registry. The
-kernel refuses a population whose installed weights are not the solution's,
+schema-9 document (the schema-8 model with its hierarchy labels in
+`hierarchy_catalogs`; a second label for one identifier without a chosen label
+fails the node). Its implementation hash covers the catalogs module
+(`microcosm.diagnostics.catalogs`), the label choices
+(`microcosm.calibrate.hierarchy_labels`) and the geography constants the UK
+choices read, so an edit to a chosen label gives the node a new key and a
+resumed store recomputes it. Each row describes its target as compiled (its
+own entity, measure and filter), with its hierarchy from the surface's
+registry. The kernel refuses a population whose installed weights are not the solution's,
 a result from another solve, and a surface other than the one the problem
 binds (`targets.problem@1` always binds it). So a diagnostics node belongs
 downstream of the one calibration it describes: a build with several

packages/microcosm-build/src/microcosm/build/transport/terminal_kernels.py

@@ -7,8 +7,10 @@
 - ``diagnostics.calibration@1`` rebuilds a completed calibration from the
   ordered ``problem``/``solution``/``result`` artifacts (no solve), checks
   that the population it reads carries exactly that solution's weights, and
-  emits the ``microcosm-diagnostics`` schema-8 model as canonical JSON. The
-  target hierarchy comes from the target surface's registry.
+  emits the ``microcosm-diagnostics`` schema 9 document as canonical JSON:
+  the schema 8 model with its hierarchy labels in ``hierarchy_catalogs``,
+  using the registry country's label choices. The target hierarchy comes
+  from the target surface's registry.
 - ``export.prepare@1`` describes the exact entity tables an export must
   write (columns, dtypes, row count and a content hash per table, plus the
   period) and refuses when any declared gate report does not permit the
@@ -44,10 +46,13 @@
 import microcosm.calibrate._target_loss_attribution as loss_attribution_module
 import microcosm.calibrate.artifacts as calibrate_artifacts_module
 import microcosm.calibrate.diagnostics as calibrate_diagnostics_module
+import microcosm.calibrate.geography_constants as geography_constants_module
+import microcosm.calibrate.hierarchy_labels as hierarchy_labels_module
 import microcosm.calibrate.matrix as matrix_module
 import microcosm.calibrate.registry as registry_module
 import microcosm.calibrate.solve as solve_module
 import microcosm.calibrate.target as target_module
+import microcosm.diagnostics.catalogs as diagnostics_catalogs_module
 import microcosm.diagnostics.schema as diagnostics_schema_module
 import microcosm.frame.adapters.axiom as axiom_adapter_module
 import microcosm.frame.bundle as frame_bundle_module
@@ -65,7 +70,11 @@
     decode_problem,
     decode_solution,
 )
-from microcosm.diagnostics import CalibrationDiagnosticsV8
+from microcosm.calibrate.hierarchy_labels import hierarchy_label_choices
+from microcosm.diagnostics import (
+    CalibrationDiagnosticsV8,
+    catalog_calibration_diagnostics,
+)
 from microcosm.frame import EntitySchema, Frame, engine_tables
 from microcosm.frame.adapters.axiom import AxiomEntityTableDataset
 from microcosm.graph import (
@@ -185,7 +194,7 @@ def _installed_weights(context: KernelContext, entity: str, ref: str) -> pd.Seri
 
 
 class DiagnosticsCalibrationKernel(KernelBase):
-    """``diagnostics.calibration@1``: one calibration's schema-8 diagnostics."""
+    """``diagnostics.calibration@1``: one calibration's schema 9 diagnostics."""
 
     ref = "diagnostics.calibration@1"
     capabilities = Capabilities(
@@ -198,6 +207,9 @@ class DiagnosticsCalibrationKernel(KernelBase):
     _optional = frozenset({"build"})
 
     def implementation_hash(self) -> str:
+        # The published document is built by the catalogs module with the
+        # registry country's label choices (which read the UK geography
+        # labels), so an edit to a chosen label re-keys this node.
         return source_hash(
             type(self),
             graph_inputs,
@@ -206,6 +218,9 @@ def implementation_hash(self) -> str:
             calibrate_artifacts_module,
             calibrate_diagnostics_module,
             diagnostics_schema_module,
+            diagnostics_catalogs_module,
+            hierarchy_labels_module,
+            geography_constants_module,
             loss_attribution_module,
             solve_module,
             matrix_module,
@@ -278,7 +293,13 @@ def run(self, context: KernelContext) -> KernelResult:
                 "surface_sha256": surface.sha256,
             },
         )
-        model = CalibrationDiagnosticsV8.model_validate(payload)
+        # Published as schema 9: the hierarchy labels go into catalogs, and a
+        # row set that labels one identifier two ways without a chosen label
+        # is refused (HierarchyLabelConflictError).
+        model = catalog_calibration_diagnostics(
+            CalibrationDiagnosticsV8.model_validate(payload),
+            label_choices=hierarchy_label_choices(surface.registry.country),
+        )
         diagnostics = canonical_json(
             model.model_dump(mode="python", exclude_defaults=True)
         )