From 97428cd9f4d3d3a668ee1474f6fa970b2de0ecf7 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 08:32:08 -0400 Subject: [PATCH 01/17] Record UK shared-graph contract lane baseline Source-only verification that WeightUpdate, the ordered-axis weight receipt, and the three KernelContext frame fields are all absent on 15ebde806, with the exact #901 consumer read that fixes their shape. Co-Authored-By: Claude Opus 5 --- PROGRESS-uk-shared-graph-contracts.md | 66 +++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 PROGRESS-uk-shared-graph-contracts.md diff --git a/PROGRESS-uk-shared-graph-contracts.md b/PROGRESS-uk-shared-graph-contracts.md new file mode 100644 index 000000000..bd18785be --- /dev/null +++ b/PROGRESS-uk-shared-graph-contracts.md @@ -0,0 +1,66 @@ +# UK-enabling shared graph contracts (amendments 25 and 26) + +Lane journal. Append-only within this lane; historicize rather than +overwrite once the branch merges (CLAUDE.md, "Root journals are history"). + +## State + +Bounded, source-only slice extracting the shared graph contract that +María's UK full-build graph (#901) consumes, as numbered amendments on +current `main`. No UK graph stage, calibration science or country +kernel is added here — those stay in #901. + +- Worktree: `_worktrees/microcosm-uk-shared-graph-contracts-20260913` +- Branch: `uk-shared-graph-contracts-20260913` +- Base: `origin/main` `15ebde806cd1a262363f7217fe535c7234ff757f` +- Reviewed UK head: #901 `051fb972b19d319d58277bd63306d0d0e0947ce2` + (draft, base `microcosm-us-launch-integration-20260909`, unchanged + since 2026-09-10T21:03:10Z; re-verified via `gh` on 2026-09-13) +- Source review followed: `uk-parallel-review.md` (2026-09-12), section + "Best independent implementation slice" + +**Runtime is UNTESTED in this lane.** Instructions forbid pytest, +production imports, engine, native sources and installation here. Only +stdlib `ast`/`ruff`/CI-inventory source checks were run. A finite +invented-only runtime plan is at the end of this file for root review +*before* execution. + +## Absence verified on base 15ebde806 + +| Contract | Present on main? | Evidence | +| --- | --- | --- | +| `WeightUpdate` (same-kind weight replacement) | **absent** | `grep -rni weightupdate .` over the worktree returns nothing; `decl.py:320` carries only `WeightTransition`, whose `__post_init__` requires `to_kind` strictly later in `WEIGHT_KINDS`, and `population.py:1944` rejects a non-forward move. A same-kind update is therefore unrepresentable. | +| `weight_update_receipt` / ordered-axis evidence | **absent** | no `weight_update` module under `packages/microcosm-graph/src/microcosm/graph/`. | +| `KernelContext.frame_metadata` | **absent** | `kernel.py:361-370` lists the complete field set; no metadata field. | +| `KernelContext.frame_mass_log` | **absent** (read side) | same field list. The *write* side already exists: `population._append_frame_mass_log` (`population.py:2117`) already ingests `receipt['frame_mass_log_append']`, so only the kernel's view of the incoming log is missing. | +| `KernelContext.frame_column_order` | **absent** | same field list. The executor projects `tables[entity]` in declaration order (`executor.py:600-628`), not the population version's own column order, so a consumer cannot reconstruct the frame layout. | + +Frame-side prerequisites already on main: `Frame.mass_log`, +`Frame.metadata`, `MassChangeRecord` and `_freeze_metadata` +(`packages/microcosm-frame/src/microcosm/frame/bundle.py`). + +Interface lock on base matches the files exactly: +`decl.py ed0a859adcae12510d5ba74d51c694617201f7b448b108a3f602410f5da44876`, +`kernel.py dbf57c137330f0f12744c557ee594586a1b308b6d6adaba1938e2b6efded21ca`. + +## Actual consumer read before specifying shape + +`packages/microcosm-build/src/microcosm/build/uk_runtime/graph_population.py` +at `051fb972` (SHA-256 `fc6f5b33127020f6e0529b39304715fe2028d47a6627b152b1e52e0d69f2efdc`): + +- `context_frame` (L58-80) reads `context.frame_column_order.get(entity, ...)`, + `context.weights`, `context.strata`, `getattr(context, "frame_mass_log", ())` + and `getattr(context, "frame_metadata", {})`. +- `UKSampleNormalizationKernel` (L386-412 region) declares + `WeightUpdate("household", weight_kind, "Normalize sampled source-family mass.")` + with `mass="declared"` and places `weight_update_receipt(ids)` under + `receipt["weight_update"]`. + +## Done + +- (nothing yet; baseline recorded) + +## Next + +- Amendment 25: `WeightUpdate` + ordered-axis receipt. +- Amendment 26: `KernelContext` frame metadata / mass log / column order. From ef2dc69c32bc02d206de2857272d73cf73541abf Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 08:40:32 -0400 Subject: [PATCH 02/17] Declare a same-kind weight update (graph amendment 25) WeightTransition only moves a weight kind forward, so a stage that recomputes weights it already holds -- a sampling normalization, a re-solve of an existing calibration -- had no declaration at all, and the only way to express it was to misdeclare a transition. WeightUpdate(entity, kind, reason, mass) is that declaration and is deliberately narrower: incumbent, declared and returned kinds must all agree; mass is conserve or declared, never free; and reason is required, non-empty and normative. Positional replacement values are not self-describing, so a kernel binds the ordered entity axis it computed them against with weight_update_receipt, and the executor recomputes that binding from the incumbent axis. A cache hit reconstructs the KernelResult and re-applies the REWEIGHT to the current base, so replay re-enters the same check rather than a parallel rule. to_kind is a property, not a field, so the two declarations have disjoint field sets and a WeightUpdate can never canonicalize -- or serialize -- to the same bytes as a WeightTransition. The transition payload is byte-for-byte unchanged. Node gains no field, so no existing node key moves. decl.py is re-locked; kernel.py is untouched. Raised by the source review of the UK full-build graph (#901, head 051fb972), whose uk.full.normalize node is the first consumer. Its UK graph stages and calibration science stay in that branch. Runtime UNTESTED in this lane: source-only ast/ruff and CI test inventory checks only. Co-Authored-By: Claude Opus 5 --- ...red-graph-contracts-weight-update.added.md | 1 + docs/graph-acceptance.md | 40 ++ docs/graph-interface.lock | 2 +- .../src/microcosm/graph/__init__.py | 7 + .../src/microcosm/graph/decl.py | 87 ++++- .../src/microcosm/graph/explain.py | 14 +- .../src/microcosm/graph/population.py | 69 +++- .../src/microcosm/graph/serialize.py | 51 ++- .../src/microcosm/graph/weight_update.py | 69 ++++ .../tests/test_graph_weight_update.py | 349 ++++++++++++++++++ 10 files changed, 672 insertions(+), 17 deletions(-) create mode 100644 changelog.d/uk-shared-graph-contracts-weight-update.added.md create mode 100644 packages/microcosm-graph/src/microcosm/graph/weight_update.py create mode 100644 packages/microcosm-graph/tests/test_graph_weight_update.py diff --git a/changelog.d/uk-shared-graph-contracts-weight-update.added.md b/changelog.d/uk-shared-graph-contracts-weight-update.added.md new file mode 100644 index 000000000..2b5340f9d --- /dev/null +++ b/changelog.d/uk-shared-graph-contracts-weight-update.added.md @@ -0,0 +1 @@ +Added `WeightUpdate`, a declared same-kind replacement of an entity's weight values, with `weight_update_receipt` binding the ordered entity axis the replacement values are positional against (graph amendment 25). diff --git a/docs/graph-acceptance.md b/docs/graph-acceptance.md index c78073571..403d70188 100644 --- a/docs/graph-acceptance.md +++ b/docs/graph-acceptance.md @@ -512,6 +512,46 @@ lock unchanged: cost before scaling. Extracted with the independently reviewed observer isolation repair on 2026-09-12. +25. **A weight update that keeps its kind is declarable.** + `WeightTransition` only ever moves a kind forward, so a stage that + recomputes weights it already holds — a sampling normalization, a + re-solve of an existing calibration — could not be declared at all, + and the only way to express it was to misdeclare a transition. + `WeightUpdate(entity, kind, reason, mass)` is that declaration and is + deliberately narrower than a transition: the incumbent kind, the + declared kind and the returned weights' kind must all be the same one; + `mass` is `conserve` or `declared` (`WEIGHT_UPDATE_MASS_POLICIES`), + because an update that neither changes kind nor bounds mass records + nothing a reader could check it against; and `reason` is required, + non-empty and normative. + + Positional replacement values are not self-describing: the same vector + is correct against one row order and silently wrong against another. + A kernel therefore binds its ordered entity axis with + `microcosm.graph.weight_update.weight_update_receipt` under + `receipt['weight_update']`, and the executor recomputes that binding + from the incumbent axis it is about to apply the values to. This is + checked on replay by construction rather than by a parallel rule: a + cache hit reconstructs the `KernelResult` with its restored weights + and receipt and re-applies the REWEIGHT to the current base, so it + re-enters the same function. A count mismatch, a missing binding and a + binding against a different axis are each a rejection. + + `WeightUpdate.to_kind` is a property, not a field, so the two + declarations have disjoint field sets (`{entity, to_kind, mass}` and + `{entity, kind, reason, mass}`) and a `WeightUpdate` can never + canonicalize, or serialize, to the same bytes as a + `WeightTransition`. Declaration JSON discriminates on those names, and + a transition's payload is byte-for-byte what it was before this + amendment, so every declaration written earlier restores unchanged. + Existing `to_kind` readers — the design-weight cap and the calibration + view — keep working through the property; the view drops the arrow + that would claim a kind moved. `Node` gains no field, so no existing + node key moves. `decl.py` is re-locked. Raised by the source review of + the UK full-build graph (#901, head `051fb972`), whose + `uk.full.normalize` node is the first consumer; its UK graph stages + and calibration science stay in that branch. + Adding a normative field with a default changes the canonical projection of every node that carries it, so node keys moved with amendments 11 and 13's sibling field `entrants`; no released artifact pins a graph key yet. diff --git a/docs/graph-interface.lock b/docs/graph-interface.lock index b9b01741c..8940b370c 100644 --- a/docs/graph-interface.lock +++ b/docs/graph-interface.lock @@ -1,2 +1,2 @@ -ed0a859adcae12510d5ba74d51c694617201f7b448b108a3f602410f5da44876 decl.py +e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933 decl.py dbf57c137330f0f12744c557ee594586a1b308b6d6adaba1938e2b6efded21ca kernel.py diff --git a/packages/microcosm-graph/src/microcosm/graph/__init__.py b/packages/microcosm-graph/src/microcosm/graph/__init__.py index fdc4d6be5..37227aed5 100644 --- a/packages/microcosm-graph/src/microcosm/graph/__init__.py +++ b/packages/microcosm-graph/src/microcosm/graph/__init__.py @@ -16,6 +16,7 @@ PARTITION_DTYPES, ROWS_ALL, WEIGHT_KINDS, + WEIGHT_UPDATE_MASS_POLICIES, ArtifactInput, ArtifactOutput, ArtifactType, @@ -30,6 +31,7 @@ SourceRef, StructuralDelta, WeightTransition, + WeightUpdate, compile_graph, ) from .errors import ( @@ -57,6 +59,7 @@ ) from .keys import platform_fingerprint from .randomness import keyed_uniform +from .weight_update import WEIGHT_UPDATE_AXIS_SCHEMA, weight_update_receipt __all__ = [ "platform_fingerprint", @@ -117,6 +120,9 @@ "StoreUnavailable", "StructuralDelta", "WeightTransition", + "WeightUpdate", + "WEIGHT_UPDATE_AXIS_SCHEMA", + "WEIGHT_UPDATE_MASS_POLICIES", "compile_graph", "describe", "explain_html", @@ -127,6 +133,7 @@ "load_source_bytes", "run_graph", "source_hash", + "weight_update_receipt", ] _FRAME_SERIES = "0.1" diff --git a/packages/microcosm-graph/src/microcosm/graph/decl.py b/packages/microcosm-graph/src/microcosm/graph/decl.py index 16b2ba0e2..f866be049 100644 --- a/packages/microcosm-graph/src/microcosm/graph/decl.py +++ b/packages/microcosm-graph/src/microcosm/graph/decl.py @@ -59,6 +59,7 @@ "PARTITION_DTYPES", "ROWS_ALL", "WEIGHT_KINDS", + "WEIGHT_UPDATE_MASS_POLICIES", "CompiledGraph", "Graph", "GraphError", @@ -70,6 +71,7 @@ "SourceRef", "StructuralDelta", "WeightTransition", + "WeightUpdate", "compile_graph", ] @@ -97,6 +99,10 @@ #: Mass policies a weight transition or structural node may declare. MASS_POLICIES = frozenset({"conserve", "free", "declared"}) +#: Mass policies a same-kind :class:`WeightUpdate` may declare. ``free`` is +#: deliberately absent (amendment 25). +WEIGHT_UPDATE_MASS_POLICIES = frozenset({"conserve", "declared"}) + #: The dtypes a mass-partition column may have. PARTITION_DTYPES = frozenset({"int32", "int64", "string"}) @@ -348,6 +354,71 @@ def __post_init__(self) -> None: ) +@dataclass(frozen=True) +class WeightUpdate: + """A declared numerical replacement of weights that keeps their kind. + + :class:`WeightTransition` only ever moves a weight *kind* forward, so a + stage that recomputes the numbers of weights it already holds — a + sampling normalization, a re-solve of an existing calibration — cannot + be declared at all. This is that declaration, and it is deliberately + narrower than a transition (amendment 25): + + - The kind does not move. The executor checks the incumbent kind, the + declared kind and the returned weights' kind are the same one. + - Mass is ``conserve`` or ``declared``; ``free`` is not offered, + because an update that may move mass arbitrarily and does not + change kind records nothing a reader could check it against. + - ``reason`` is required, non-empty, and normative: it enters the node + key, so a node that replaces weights for a different stated purpose + is a different node. + - The kernel must bind the ordered entity axis its replacement values + are positional against, through + :func:`~microcosm.graph.weight_update.weight_update_receipt`. The + executor recomputes that binding from the incumbent axis, on cold + execution and on replay. + + Design-weight ancestry is untouched: the executor carries the original + design anchors exactly as it does for any other node. + + Attributes: + entity: The entity whose explicit weights are replaced. + kind: The unchanged weight kind; one of :data:`WEIGHT_KINDS`. + reason: Why the numbers are replaced. Normative, non-empty. + mass: ``conserve`` or ``declared``. + """ + + entity: str + kind: str + reason: str + mass: str = "declared" + + def __post_init__(self) -> None: + _name("WeightUpdate.entity", self.entity) + _nonempty("WeightUpdate.reason", self.reason) + if self.kind not in WEIGHT_KINDS: + raise GraphError( + f"WeightUpdate.kind {self.kind!r} is not one of {WEIGHT_KINDS}." + ) + if self.mass not in WEIGHT_UPDATE_MASS_POLICIES: + raise GraphError( + f"WeightUpdate.mass {self.mass!r} is not one of " + f"{sorted(WEIGHT_UPDATE_MASS_POLICIES)}; an update that keeps " + "its kind does not get unconstrained free mass." + ) + + @property + def to_kind(self) -> str: + """The unchanged kind, for shared structural-weight accounting. + + A property, not a field, so it stays out of the normative + projection and a :class:`WeightUpdate` can never canonicalize to + the same bytes as a :class:`WeightTransition`. + """ + + return self.kind + + @dataclass(frozen=True) class Node: """One unit of computation and cell ownership. @@ -376,7 +447,9 @@ class Node: base: For a structural node other than ``CREATE``: the population version it transforms. sources: Names of :class:`SourceRef` entries this node reads. - weights: A declared weight-kind transition, if any. + weights: A declared :class:`WeightTransition` (the kind moves + forward) or :class:`WeightUpdate` (the numbers are replaced + and the kind does not move), if any. mass: Mass policy for structural nodes that change rows or weights. entrants: ``EXPAND`` nodes only: the kernel may add rows that copy no base row. Such a row has null lineage, the kernel supplies @@ -396,7 +469,7 @@ class Node: structural: StructuralDelta = StructuralDelta.NONE base: str | None = None sources: tuple[str, ...] = () - weights: WeightTransition | None = None + weights: WeightTransition | WeightUpdate | None = None mass: str = "conserve" description: str = "" citation: str = "" @@ -474,6 +547,13 @@ def __post_init__(self) -> None: f"Node {self.id!r}: entrants add mass, so an entrant-admitting " "node cannot declare mass='conserve'." ) + if self.weights is not None and not isinstance( + self.weights, WeightTransition | WeightUpdate + ): + raise GraphError( + f"Node {self.id!r}: weights must be a WeightTransition or a " + "WeightUpdate, not a look-alike." + ) if self.weights is not None and self.structural is not StructuralDelta.REWEIGHT: raise GraphError( f"Node {self.id!r}: a weight transition changes the population " @@ -482,7 +562,8 @@ def __post_init__(self) -> None: if self.structural is StructuralDelta.REWEIGHT: if self.weights is None: raise GraphError( - f"Node {self.id!r}: a REWEIGHT node declares its WeightTransition." + f"Node {self.id!r}: a REWEIGHT node declares its " + "WeightTransition or WeightUpdate." ) if self.mass != self.weights.mass: raise GraphError( diff --git a/packages/microcosm-graph/src/microcosm/graph/explain.py b/packages/microcosm-graph/src/microcosm/graph/explain.py index d17d8be94..5e1d51936 100644 --- a/packages/microcosm-graph/src/microcosm/graph/explain.py +++ b/packages/microcosm-graph/src/microcosm/graph/explain.py @@ -13,7 +13,7 @@ from typing import TYPE_CHECKING from .availability import execution_state -from .decl import GATE_OUTCOMES, CompiledGraph, StructuralDelta +from .decl import GATE_OUTCOMES, CompiledGraph, StructuralDelta, WeightUpdate from .manifest import NodeReceipt, RunManifest from .population import mass_record_receipt from .view import describe @@ -1196,12 +1196,22 @@ def _render_calibration(compiled: CompiledGraph, manifest: RunManifest) -> str: mass = _mass_payload(manifest, node, receipt) transition = node.weights assert transition is not None + # A same-kind update does not move the kind, so it does not get the + # arrow that says it did (amendment 25). + kind_label = ( + f"{_escape(transition.entity)} → {_escape(transition.to_kind)}" + if not isinstance(transition, WeightUpdate) + else ( + f"{_escape(transition.entity)} · {_escape(transition.kind)} " + f"updated ({_escape(transition.reason)})" + ) + ) cards.append( '
' '
' f"

{_escape(node.id)}

" f'

{_escape(node.kernel)} · ' - f"{_escape(transition.entity)} → {_escape(transition.to_kind)}

" + f"{kind_label}

" f'{_escape(transition.mass)} mass' "

Declared targets and results

" + target_table diff --git a/packages/microcosm-graph/src/microcosm/graph/population.py b/packages/microcosm-graph/src/microcosm/graph/population.py index bdeef2b50..a23266b2a 100644 --- a/packages/microcosm-graph/src/microcosm/graph/population.py +++ b/packages/microcosm-graph/src/microcosm/graph/population.py @@ -24,9 +24,11 @@ Owned, Ownership, StructuralDelta, + WeightUpdate, ) from .kernel import KernelResult from .store import _encode_object_scalar +from .weight_update import weight_update_receipt __all__ = [ "MassRecord", @@ -1140,7 +1142,11 @@ def patch( node, transitioning=node.weights.entity, ) - frame = _apply_weight_transition(population, frame, node, result) + frame = ( + _apply_weight_update(population, frame, node, result) + if isinstance(node.weights, WeightUpdate) + else _apply_weight_transition(population, frame, node, result) + ) elif result.weights is not None: raise PopulationError( f"Node {node.id!r} returned weights without declaring a transition." @@ -1971,6 +1977,67 @@ def _apply_weight_transition( return _replace_weights(frame, transition.entity, result.weights) +def _apply_weight_update( + population: Population, frame: Frame, node: Node, result: KernelResult +) -> Frame: + """Replace an entity's weight values without moving their kind. + + The kind is checked three ways — incumbent, declaration and returned + weights must all be the same one — and the kernel's ordered-axis + binding is recomputed against the incumbent axis these values are + about to be applied to. A cached hit re-enters this function with the + restored weights and receipt, so replay is checked by the same code + rather than a parallel rule (amendment 25). + """ + + update = node.weights + assert isinstance(update, WeightUpdate) + if result.weights is None: + raise PopulationError( + f"Node {node.id!r} declares a weight update but returned no weights." + ) + if update.entity not in population.frame.weighted_entities: + raise PopulationError( + f"Node {node.id!r} cannot update inherited weights for " + f"{update.entity!r}; explicit weights are required." + ) + old = population.frame.weights_for(update.entity) + declared_kind = WeightKind(update.kind) + if old.kind is not declared_kind: + raise PopulationError( + f"Node {node.id!r} declares a same-kind weight update to " + f"{update.kind!r}, but the incumbent weights are " + f"{old.kind.value!r}; a change of kind is a WeightTransition." + ) + if result.weights.kind is not declared_kind: + raise PopulationError( + f"Node {node.id!r} declared {update.kind!r} weights but the kernel " + f"returned {result.weights.kind.value!r}." + ) + + id_column = population.frame.schema.entity_id_column(update.entity) + axis = population.frame.table(update.entity)[id_column].tolist() + if len(result.weights.values) != len(axis): + raise PopulationError( + f"Node {node.id!r} returned {len(result.weights.values)} weights for " + f"an incumbent {update.entity!r} axis of {len(axis)} rows." + ) + declared_axis = result.receipt.get("weight_update") + if not isinstance(declared_axis, Mapping): + raise PopulationError( + f"Node {node.id!r} weight update requires receipt['weight_update']; " + "positional weights without their ordered axis are unverifiable." + ) + expected = weight_update_receipt(axis) + if dict(declared_axis) != expected: + raise PopulationError( + f"Node {node.id!r} bound its replacement weights to a different " + f"{update.entity!r} axis than the incumbent population carries." + ) + + return _replace_weights(frame, update.entity, result.weights) + + def _replace_weights(frame: Frame, entity: str, replacement: Weights) -> Frame: weights = { weighted: frame.weights_for(weighted) for weighted in frame.weighted_entities diff --git a/packages/microcosm-graph/src/microcosm/graph/serialize.py b/packages/microcosm-graph/src/microcosm/graph/serialize.py index e513c3b7a..b9b06a40b 100644 --- a/packages/microcosm-graph/src/microcosm/graph/serialize.py +++ b/packages/microcosm-graph/src/microcosm/graph/serialize.py @@ -19,6 +19,7 @@ SourceRef, StructuralDelta, WeightTransition, + WeightUpdate, ) __all__ = ["graph_from_json", "graph_to_json"] @@ -149,15 +150,7 @@ def _node_payload(node: Node) -> dict[str, object]: "structural": node.structural.value, "base": node.base, "sources": list(node.sources), - "weights": ( - None - if node.weights is None - else { - "entity": node.weights.entity, - "to_kind": node.weights.to_kind, - "mass": node.weights.mass, - } - ), + "weights": _weights_payload(node.weights), "mass": node.mass, **({"entrants": True} if node.entrants else {}), "description": node.description, @@ -288,10 +281,48 @@ def _owned_from_payload(value: object, label: str) -> Owned: ) -def _weights_from_payload(value: object, label: str) -> WeightTransition | None: +def _weights_payload( + weights: WeightTransition | WeightUpdate | None, +) -> dict[str, object] | None: + """Discriminate the two weight declarations by their own field names. + + ``WeightUpdate`` exposes ``to_kind`` as a property, so projecting it + the way a transition is projected would round-trip it back as a + transition and silently change what the node means. The transition + payload is byte-for-byte what it was before amendment 25, so every + declaration serialized before it restores unchanged. + """ + + if weights is None: + return None + if isinstance(weights, WeightUpdate): + return { + "entity": weights.entity, + "kind": weights.kind, + "reason": weights.reason, + "mass": weights.mass, + } + return { + "entity": weights.entity, + "to_kind": weights.to_kind, + "mass": weights.mass, + } + + +def _weights_from_payload( + value: object, label: str +) -> WeightTransition | WeightUpdate | None: if value is None: return None payload = _mapping(value, label) + if "kind" in payload: + _exact_fields(payload, {"entity", "kind", "reason", "mass"}, label) + return WeightUpdate( + entity=_string(payload["entity"], f"{label}.entity"), + kind=_string(payload["kind"], f"{label}.kind"), + reason=_string(payload["reason"], f"{label}.reason"), + mass=_string(payload["mass"], f"{label}.mass"), + ) _exact_fields(payload, {"entity", "to_kind", "mass"}, label) return WeightTransition( entity=_string(payload["entity"], f"{label}.entity"), diff --git a/packages/microcosm-graph/src/microcosm/graph/weight_update.py b/packages/microcosm-graph/src/microcosm/graph/weight_update.py new file mode 100644 index 000000000..69c1c5699 --- /dev/null +++ b/packages/microcosm-graph/src/microcosm/graph/weight_update.py @@ -0,0 +1,69 @@ +"""Ordered entity-axis evidence for a declared same-kind weight update. + +A :class:`~microcosm.graph.decl.WeightUpdate` kernel returns positional +weight values. Positions alone are not evidence: the same vector is correct +against one row order and silently wrong against another, and nothing in a +replayed receipt would show the difference. A kernel therefore binds the +ordered entity axis its values were computed against, and the executor +recomputes that binding from the incumbent axis it is about to apply them +to — on cold execution and on every replay of the cached receipt +(amendment 25). + +The binding is a digest, not the ids: an axis of millions of rows does not +belong in a manifest, and the executor only ever needs to answer whether +the axis it holds is the axis the kernel used. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from numbers import Integral + +from .canonical import canonical_json, sha256_domain + +__all__ = ["WEIGHT_UPDATE_AXIS_SCHEMA", "weight_update_receipt"] + +#: The tagged schema of the axis binding. The executor authors the same +#: token when it recomputes, so a receipt written under another schema is a +#: mismatch rather than an unchecked pass. +WEIGHT_UPDATE_AXIS_SCHEMA = "microcosm.graph.weight-update-axis.v1" + + +def weight_update_receipt(entity_ids: Sequence[int | str]) -> dict[str, object]: + """Bind positional replacement weights to a unique, ordered entity axis. + + Place the result under ``KernelResult.receipt['weight_update']``. + + Args: + entity_ids: The entity ids, in the order the returned weight values + are positional against. Integers or non-empty strings, unique. + ``bool`` is refused: it is an ``Integral`` in Python, and an + axis of ``True``/``False`` is a mistake, not an id space. + + Returns: + The tagged binding: its schema, the row count, and a domain- + separated SHA-256 over the canonical JSON of the ordered ids. + + Raises: + ValueError: An id is neither an integer nor a non-empty string, or + the ids repeat. + """ + + ids: list[int | str] = [] + for value in entity_ids: + if isinstance(value, Integral) and not isinstance(value, bool): + ids.append(int(value)) + elif isinstance(value, str) and value: + ids.append(value) + else: + raise ValueError( + "Weight update ids must be integers or non-empty strings; got " + f"{value!r}." + ) + if len(set(ids)) != len(ids): + raise ValueError("Weight update ids must be unique.") + return { + "schema": WEIGHT_UPDATE_AXIS_SCHEMA, + "count": len(ids), + "entity_ids_sha256": sha256_domain("weight-update-axis", canonical_json(ids)), + } diff --git a/packages/microcosm-graph/tests/test_graph_weight_update.py b/packages/microcosm-graph/tests/test_graph_weight_update.py new file mode 100644 index 000000000..16b33bf0c --- /dev/null +++ b/packages/microcosm-graph/tests/test_graph_weight_update.py @@ -0,0 +1,349 @@ +"""Amendment 25: a same-kind weight update is declarable and axis-bound. + +``WeightTransition`` only moves a kind forward, so a stage that recomputes +weights it already holds — a sampling normalization, a re-solve of an +existing calibration — had no declaration at all. These properties are +about the one that does: the kind cannot move, mass cannot be free, and +positional replacement values are refused unless the kernel binds the +ordered entity axis they were computed against. + +Everything here runs real shared graph operations over the invented toy +country. No country model, engine, or build artifact is involved. +""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import pytest + +from microcosm.frame import WeightKind, Weights +from microcosm.graph import ( + WEIGHT_UPDATE_AXIS_SCHEMA, + WEIGHT_UPDATE_MASS_POLICIES, + ContentStore, + Graph, + GraphError, + KernelResult, + Node, + NodeRejectedError, + Slice, + StructuralDelta, + WeightTransition, + WeightUpdate, + graph_from_json, + graph_to_json, + weight_update_receipt, +) +from microcosm.graph.canonical import normative + +if "_toy" not in sys.modules: + _SPEC = importlib.util.spec_from_file_location( + "_toy", Path(__file__).with_name("_toy.py") + ) + sys.modules["_toy"] = importlib.util.module_from_spec(_SPEC) + _SPEC.loader.exec_module(sys.modules["_toy"]) +toy = sys.modules["_toy"] + +UPDATE_REF = "reweight.same_kind@1" + + +class ScaleSameKind(toy.ToyKernel): + """A REWEIGHT that replaces weight values and keeps their kind. + + ``axis`` selects which ordered axis the kernel claims its positional + values belong to: the incumbent one, the incumbent one reversed, or + none at all. ``returns_kind`` lets a property separate "the incumbent + disagrees with the declaration" from "the kernel disagrees with it". + """ + + def compute(self, context): + entity = str(context.params["entity"]) + incumbent = context.weights[entity] + after = incumbent.values * float(context.params["factor"]) + returns = context.params.get("returns_kind") + kind = incumbent.kind if returns is None else WeightKind(str(returns)) + ids = context.tables[entity][toy.id_column(entity)].tolist() + receipt: dict[str, object] = { + "mass": toy._mass_record( + context, incumbent.values, after, str(context.params["policy"]) + ) + } + axis = str(context.params.get("axis", "incumbent")) + if axis == "incumbent": + receipt["weight_update"] = weight_update_receipt(ids) + elif axis == "reversed": + receipt["weight_update"] = weight_update_receipt(list(reversed(ids))) + elif axis == "short": + receipt["weight_update"] = weight_update_receipt(ids[:-1]) + elif axis != "absent": # pragma: no cover - guards the fixture itself + raise AssertionError(f"unknown axis fixture {axis!r}") + return KernelResult(weights=Weights(values=after, kind=kind), receipt=receipt) + + +def registry_with_update(): + """The toy registry plus the same-kind update kernel.""" + registry = toy.toy_registry() + registry.register(ScaleSameKind(UPDATE_REF, toy._REWEIGHT)) + return registry + + +def update_node( + *, + kind: str = "design", + reason: str = "reinstall normalized source mass", + mass: str = "declared", + factor: float = 2.0, + axis: str = "incumbent", + returns_kind: str | None = None, +) -> Node: + """One same-kind update of the toy household weights, on ``survey``.""" + return Node( + "update", + UPDATE_REF, + structural=StructuralDelta.REWEIGHT, + base="survey", + inputs=(Slice("person", ("age",)), Slice("household", ("household_size",))), + params={ + "entity": "household", + "factor": factor, + "policy": mass, + "axis": axis, + "returns_kind": returns_kind, + }, + weights=WeightUpdate("household", kind, reason, mass=mass), + mass=mass, + description="same-kind weight update", + ) + + +def update_graph(**kwargs) -> Graph: + return Graph("toy", (toy.SOURCE,), (toy.CREATE, update_node(**kwargs))) + + +def run_update(root: Path, **kwargs) -> object: + return toy.run_toy(update_graph(**kwargs), root, registry=registry_with_update()) + + +# ---------------------------------------------------------------------- +# The declaration +# ---------------------------------------------------------------------- + + +def test_weight_update_declares_a_non_empty_reason() -> None: + """The stated purpose is required, and it is normative.""" + with pytest.raises(GraphError, match="WeightUpdate.reason"): + WeightUpdate("household", "design", "") + one = WeightUpdate("household", "design", "normalize sampled mass") + other = WeightUpdate("household", "design", "re-solve the calibration") + assert normative(one) != normative(other) + + +def test_weight_update_refuses_free_mass() -> None: + """An update that neither moves kind nor bounds mass records nothing.""" + assert WEIGHT_UPDATE_MASS_POLICIES == frozenset({"conserve", "declared"}) + for mass in sorted(WEIGHT_UPDATE_MASS_POLICIES): + assert WeightUpdate("household", "design", "why", mass=mass).mass == mass + with pytest.raises(GraphError, match="free"): + WeightUpdate("household", "design", "why", mass="free") + + +def test_weight_update_refuses_an_unknown_kind() -> None: + with pytest.raises(GraphError, match="WeightUpdate.kind"): + WeightUpdate("household", "provisional", "why") + + +def test_weight_update_is_never_a_transition() -> None: + """Disjoint field sets, so the two can never canonicalize alike. + + ``to_kind`` is a property on the update, which is what keeps it out of + the normative projection. + """ + update = WeightUpdate("household", "calibrated", "re-solve", mass="declared") + transition = WeightTransition("household", "calibrated", mass="declared") + assert update.to_kind == "calibrated" + assert set(normative(update)) == {"entity", "kind", "reason", "mass"} + assert set(normative(transition)) == {"entity", "to_kind", "mass"} + assert normative(update) != normative(transition) + + +def test_reweight_node_accepts_either_declaration() -> None: + """A REWEIGHT node still needs one of them, and rejects a look-alike.""" + with pytest.raises(GraphError, match="REWEIGHT node declares"): + Node("n", UPDATE_REF, structural=StructuralDelta.REWEIGHT, base="survey") + with pytest.raises(GraphError, match="look-alike"): + Node( + "n", + UPDATE_REF, + structural=StructuralDelta.REWEIGHT, + base="survey", + weights={"entity": "household", "kind": "design", "reason": "no"}, + mass="declared", + ) + + +def test_node_mass_must_agree_with_the_update() -> None: + with pytest.raises(GraphError, match="disagrees"): + Node( + "n", + UPDATE_REF, + structural=StructuralDelta.REWEIGHT, + base="survey", + weights=WeightUpdate("household", "design", "why", mass="declared"), + mass="conserve", + ) + + +# ---------------------------------------------------------------------- +# JSON round trips +# ---------------------------------------------------------------------- + + +def test_weight_update_round_trips_as_itself() -> None: + """Not as a transition, which its ``to_kind`` property would allow.""" + graph = update_graph() + restored = graph_from_json(graph_to_json(graph)) + assert restored == graph + weights = restored.node("update").weights + assert isinstance(weights, WeightUpdate) + assert (weights.entity, weights.kind, weights.mass) == ( + "household", + "design", + "declared", + ) + assert weights.reason == "reinstall normalized source mass" + + +def test_transition_payload_is_unchanged_by_the_amendment() -> None: + """Every declaration serialized before amendment 25 restores unchanged.""" + graph = Graph("toy", (toy.SOURCE,), (toy.CREATE, toy.POOL)) + text = graph_to_json(graph) + assert '"weights":{"entity":"household","mass":"free","to_kind":"importance"}' in ( + text + ) + restored = graph_from_json(text) + assert restored == graph + assert isinstance(restored.node("pool").weights, WeightTransition) + + +def test_round_trip_refuses_a_mixed_weights_payload() -> None: + """``kind`` selects the update arm; its fields are then exact.""" + text = graph_to_json(update_graph()) + mixed = text.replace(',"reason":"reinstall normalized source mass"', "") + with pytest.raises(ValueError, match="weights"): + graph_from_json(mixed) + both = text.replace('"kind":"design"', '"kind":"design","to_kind":"design"') + with pytest.raises(ValueError, match="weights"): + graph_from_json(both) + + +def test_axis_receipt_is_a_tagged_digest() -> None: + """The binding answers one question and does not carry the axis.""" + receipt = weight_update_receipt([3, 1, 2]) + assert set(receipt) == {"schema", "count", "entity_ids_sha256"} + assert receipt["schema"] == WEIGHT_UPDATE_AXIS_SCHEMA + assert receipt["count"] == 3 + assert receipt != weight_update_receipt([1, 2, 3]) + assert receipt == weight_update_receipt([3, 1, 2]) + with pytest.raises(ValueError, match="unique"): + weight_update_receipt([1, 1]) + with pytest.raises(ValueError, match="integers or non-empty strings"): + weight_update_receipt([True]) + with pytest.raises(ValueError, match="integers or non-empty strings"): + weight_update_receipt([""]) + + +# ---------------------------------------------------------------------- +# The shared graph operation +# ---------------------------------------------------------------------- + + +def test_same_kind_update_replaces_values_and_keeps_the_kind(tmp_path: Path) -> None: + """The real property: new numbers, same kind, mass declared.""" + run = run_update(tmp_path / "run") + before = run.manifest.population("survey").weights_for("household") + after = run.manifest.population("update").weights_for("household") + assert after.kind is WeightKind.DESIGN is before.kind + assert list(after.values) == [value * 2.0 for value in before.values] + + +def test_conserving_update_is_accepted(tmp_path: Path) -> None: + run = run_update(tmp_path / "run", mass="conserve", factor=1.0) + weights = run.manifest.population("update").weights_for("household") + assert weights.kind is WeightKind.DESIGN + + +def test_update_refuses_a_kind_that_moved(tmp_path: Path) -> None: + """Declaring a different kind than the incumbent is a transition.""" + with pytest.raises(NodeRejectedError, match="a change of kind is a"): + run_update(tmp_path / "run", kind="importance", returns_kind="importance") + + +def test_update_refuses_weights_of_another_kind(tmp_path: Path) -> None: + """The kernel must also return the kind the node declared.""" + with pytest.raises(NodeRejectedError, match="the kernel returned"): + run_update(tmp_path / "run", returns_kind="importance") + + +def test_update_requires_its_ordered_axis(tmp_path: Path) -> None: + with pytest.raises(NodeRejectedError, match="unverifiable"): + run_update(tmp_path / "run", axis="absent") + + +def test_update_refuses_a_foreign_axis(tmp_path: Path) -> None: + """Same ids, other order: the values would land on the wrong rows.""" + with pytest.raises(NodeRejectedError, match="different .household. axis"): + run_update(tmp_path / "run", axis="reversed") + + +def test_update_refuses_a_short_axis(tmp_path: Path) -> None: + with pytest.raises(NodeRejectedError, match="different .household. axis"): + run_update(tmp_path / "run", axis="short") + + +# ---------------------------------------------------------------------- +# Replay +# ---------------------------------------------------------------------- + + +def test_cold_then_required_replay_revalidates_the_axis(tmp_path: Path) -> None: + """A hit re-applies the REWEIGHT, so the axis is checked again. + + The cached result is reconstructed with its stored weights and receipt + and passed back through the same application, which is why replay + needs no parallel rule. ``resume="require"`` proves the second run read + the store rather than recomputing. + """ + cold = run_update(tmp_path / "run") + assert cold.misses() == set(cold.compiled.order) + + registry = registry_with_update() + warm = toy.run_toy( + update_graph(), + tmp_path / "run", + sources=cold.sources, + registry=registry, + store=ContentStore(tmp_path / "run" / "store"), + resume="require", + ) + assert warm.misses() == set() + assert toy.total_calls(registry) == 0 + assert warm.keys() == cold.keys() + replayed = warm.manifest.population("update").weights_for("household") + original = cold.manifest.population("update").weights_for("household") + assert replayed.kind is original.kind + assert list(replayed.values) == list(original.values) + + +def test_reason_is_part_of_the_node_identity(tmp_path: Path) -> None: + """Two updates that state different purposes are different nodes.""" + first = run_update(tmp_path / "first") + second = toy.run_toy( + update_graph(reason="re-solve the calibration"), + tmp_path / "second", + registry=registry_with_update(), + ) + assert first.keys()["survey"] == second.keys()["survey"] + assert first.keys()["update"] != second.keys()["update"] From f33d47cdac3d34a9df491703a1307bbd33d94b72 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 08:47:48 -0400 Subject: [PATCH 03/17] Carry the version's metadata, mass log and column order (graph amendment 26) The executor projects each entity table in declaration order, so KernelContext.tables is not the population version's layout, and the version's Frame metadata and mass log were not reachable from a kernel at all. A kernel that has to hand its declared projection back to a legacy function as a Frame could not rebuild one without inventing the parts it could not see. KernelContext gains three read-only fields: - frame_metadata: the version's metadata. The executor passes Frame.metadata, which Frame has already deeply frozen; this class adds a read-only view and does not itself deep-freeze a mapping built some other way. This deliberately differs from the UK branch, which imports microcosm.frame.bundle._freeze_metadata into the frozen interface -- a frozen contract should not depend on another shard's private name. - frame_mass_log: the *incoming* Frame mass records. The write side already existed (receipt['frame_mass_log_append']); only the read side was missing. - frame_column_order: the version's own column order, restricted to the projected columns. An entry that is not exactly an ordering of that table's columns is refused, so an order can neither hide a column the node was given nor name one it was not -- a column name is itself information about the version. The three ride after artifacts and before tolerances, so amendment 17's statement that numerics rides at the end stays literally true, and amendment 19's that artifacts rides before the pair does too; the unit assertion of adjacency becomes the ordering amendment 19 actually claimed. Nothing here is normative: Node is untouched and no node key moves. kernel.py is re-locked; decl.py is untouched. The acceptance suite's B2 field set follows in its own commit, as amendment 19's did (a2b6dfb0b). Runtime UNTESTED in this lane: source-only ast/ruff and CI test inventory checks only. Co-Authored-By: Claude Opus 5 --- ...red-graph-contracts-frame-context.added.md | 1 + docs/graph-acceptance.md | 43 ++ docs/graph-interface.lock | 2 +- .../src/microcosm/graph/executor.py | 13 + .../src/microcosm/graph/kernel.py | 54 ++- .../tests/test_graph_frame_context.py | 435 ++++++++++++++++++ .../tests/test_graph_kernel_contract.py | 4 +- 7 files changed, 549 insertions(+), 3 deletions(-) create mode 100644 changelog.d/uk-shared-graph-contracts-frame-context.added.md create mode 100644 packages/microcosm-graph/tests/test_graph_frame_context.py diff --git a/changelog.d/uk-shared-graph-contracts-frame-context.added.md b/changelog.d/uk-shared-graph-contracts-frame-context.added.md new file mode 100644 index 000000000..64fe69ac1 --- /dev/null +++ b/changelog.d/uk-shared-graph-contracts-frame-context.added.md @@ -0,0 +1 @@ +Added `KernelContext.frame_metadata`, `frame_mass_log` and `frame_column_order`, so a kernel can reconstruct its population version's declared slices without seeing any column it did not declare (graph amendment 26). diff --git a/docs/graph-acceptance.md b/docs/graph-acceptance.md index 403d70188..96a4a4fa8 100644 --- a/docs/graph-acceptance.md +++ b/docs/graph-acceptance.md @@ -552,6 +552,49 @@ lock unchanged: `uk.full.normalize` node is the first consumer; its UK graph stages and calibration science stay in that branch. +26. **The context carries the version's metadata, mass log and column + order.** The executor projects each entity table in *declaration* + order, so `KernelContext.tables` is not the population version's + layout, and the version's `Frame` metadata and mass log were not + reachable from a kernel at all. A kernel that has to hand a declared + projection back to a legacy function as a `Frame` therefore could not + reconstruct one without inventing the parts it could not see. + `kernel.py` gains three read-only fields: + + - `frame_metadata` — the version's own metadata. The executor passes + `Frame.metadata`, which `Frame` has already deeply frozen; + `KernelContext` adds a read-only view over it and does **not** itself + deep-freeze a mapping built some other way. (This is a deliberate + difference from the UK branch, which imports + `microcosm.frame.bundle._freeze_metadata` into the frozen interface: + a frozen contract should not depend on another shard's private name.) + - `frame_mass_log` — the version's `Frame` mass records, in order, and + specifically the *incoming* ones. A node needing a stage's completed + records must run after that stage's structural boundary or read its + predecessor's evidence; incidental node order is not authority. The + write side already existed (`receipt['frame_mass_log_append']`); only + the read side was missing. + - `frame_column_order` — entity to the version's own column order, + restricted to the columns projected into `tables`. An entry that is + not exactly an ordering of that table's columns is refused, so an + order can neither hide a column the node was given nor name one it + was not: a column *name* is itself information about the version, and + B1's "nothing else is visible" covers names as well as values. + + The three ride after `artifacts` and before `tolerances`, so amendment + 17's statement that `numerics` rides at the end of the context stays + literally true and amendment 19's that `artifacts` rides before the + pair does too — the unit assertion of *adjacency* becomes the ordering + amendment 19 actually claimed. The acceptance suite's B2 field set + gains the three in its own commit, as amendment 19's did. Nothing here + is normative: `Node` is untouched, no canonical projection changes, and + no node key moves. The fields are rebuilt from a restored `Frame` on a + cache hit exactly as they are from a computed one, which is what + amendment 22's metadata-preserving Frame format makes possible. + `kernel.py` is re-locked. Raised by the same source review as amendment + 25; the UK full-build graph's `context_frame` helper (#901, head + `051fb972`) is the first consumer. + Adding a normative field with a default changes the canonical projection of every node that carries it, so node keys moved with amendments 11 and 13's sibling field `entrants`; no released artifact pins a graph key yet. diff --git a/docs/graph-interface.lock b/docs/graph-interface.lock index 8940b370c..f09fe7bb1 100644 --- a/docs/graph-interface.lock +++ b/docs/graph-interface.lock @@ -1,2 +1,2 @@ e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933 decl.py -dbf57c137330f0f12744c557ee594586a1b308b6d6adaba1938e2b6efded21ca kernel.py +51f45e899ba578a6b2324636a9879253266b067812f68b648eacfcd5a184d245 kernel.py diff --git a/packages/microcosm-graph/src/microcosm/graph/executor.py b/packages/microcosm-graph/src/microcosm/graph/executor.py index b61a678d4..c9687beb9 100644 --- a/packages/microcosm-graph/src/microcosm/graph/executor.py +++ b/packages/microcosm-graph/src/microcosm/graph/executor.py @@ -655,6 +655,16 @@ def _project_context( ) strata = frame.strata.loc[person_mask].copy() _freeze_series(strata) + # The projection above orders each table by declaration, not by the + # version's own layout, so a consumer rebuilding the version's tables + # needs that layout separately -- restricted to what it was given, so it + # never learns the name of a column it cannot read (amendment 26). + column_order: dict[str, tuple[str, ...]] = {} + for entity, table in tables.items(): + projected = set(table.columns) + column_order[entity] = tuple( + column for column in frame.table(entity).columns if column in projected + ) return KernelContext( node=node, tables=MappingProxyType(tables), @@ -664,6 +674,9 @@ def _project_context( rng=np.random.default_rng(seed(key)), sources=MappingProxyType({name: sources[name] for name in node.sources}), artifacts={} if artifacts is None else artifacts, + frame_metadata=frame.metadata, + frame_mass_log=frame.mass_log, + frame_column_order=MappingProxyType(column_order), tolerances=tolerances, numerics=numerics, ) diff --git a/packages/microcosm-graph/src/microcosm/graph/kernel.py b/packages/microcosm-graph/src/microcosm/graph/kernel.py index d6b16909d..528d11836 100644 --- a/packages/microcosm-graph/src/microcosm/graph/kernel.py +++ b/packages/microcosm-graph/src/microcosm/graph/kernel.py @@ -57,7 +57,7 @@ import numpy as np import pandas as pd -from microcosm.frame import Frame, Weights +from microcosm.frame import Frame, MassChangeRecord, Weights from .decl import ArtifactType, Node, Param, StructuralDelta @@ -348,6 +348,24 @@ class KernelContext: validates the versioned payload before using it, because a nominal type does not itself verify serialized data (amendment 19). + frame_metadata: The population version's own metadata. The + executor passes :attr:`microcosm.frame.Frame.metadata`, which + ``Frame`` has already deeply frozen; this class adds a + read-only view over that mapping and does not itself deep-freeze + a mapping built some other way (amendment 26). + frame_mass_log: The population version's ``Frame`` mass records, in + order. This is the *incoming* log: a node that needs a stage's + completed records must run after that stage's structural + boundary or read its predecessor's evidence, because incidental + node order is not authority (amendment 26). + frame_column_order: Entity to the population version's own column + order, restricted to the columns projected into ``tables``. + The executor projects ``tables`` in declaration order, so this + is the only way to reconstruct the version's layout. It never + names a column the node did not get: an entry that is not + exactly an ordering of that table's columns is refused, so an + undeclared column cannot be smuggled in as a name (amendment + 26). tolerances: ``(entity, column)`` of each declared input column to the :class:`Tolerance` its owning kernel declared, or ``None`` for a bitwise owner. A gate compares against these. @@ -366,6 +384,9 @@ class KernelContext: rng: np.random.Generator sources: Mapping[str, Path] = field(default_factory=dict) artifacts: Mapping[str, ArtifactValue] = field(default_factory=dict) + frame_metadata: Mapping[str, object] = field(default_factory=dict) + frame_mass_log: tuple[MassChangeRecord, ...] = () + frame_column_order: Mapping[str, tuple[str, ...]] = field(default_factory=dict) tolerances: Mapping[tuple[str, str], Tolerance | None] = field(default_factory=dict) numerics: Mapping[tuple[str, str], NumericScope] = field(default_factory=dict) @@ -382,6 +403,37 @@ def __post_init__(self) -> None: ) object.__setattr__(self, "artifacts", MappingProxyType(values)) + metadata = dict(self.frame_metadata) + if any(not isinstance(name, str) or not name for name in metadata): + raise TypeError( + "KernelContext.frame_metadata keys must be non-empty strings." + ) + object.__setattr__(self, "frame_metadata", MappingProxyType(metadata)) + + if not isinstance(self.frame_mass_log, tuple) or any( + not isinstance(record, MassChangeRecord) for record in self.frame_mass_log + ): + raise TypeError( + "KernelContext.frame_mass_log must be a tuple of MassChangeRecord." + ) + + # An order is an order *of the projected columns*, so it can neither + # name a column the node was not given nor hide one it was. + order = dict(self.frame_column_order) + for entity, columns in order.items(): + table = self.tables.get(entity) + if ( + table is None + or not isinstance(columns, tuple) + or len(set(columns)) != len(columns) + or set(columns) != set(table.columns) + ): + raise TypeError( + "KernelContext.frame_column_order must order exactly the " + f"projected columns of each entity; {entity!r} does not." + ) + object.__setattr__(self, "frame_column_order", MappingProxyType(order)) + @dataclass(frozen=True) class KernelResult: diff --git a/packages/microcosm-graph/tests/test_graph_frame_context.py b/packages/microcosm-graph/tests/test_graph_frame_context.py new file mode 100644 index 000000000..3f9634389 --- /dev/null +++ b/packages/microcosm-graph/tests/test_graph_frame_context.py @@ -0,0 +1,435 @@ +"""Amendment 26: the context carries the version's metadata, mass log, order. + +The executor projects each table in *declaration* order, so a kernel that +reconstructs its population version's layout cannot do it from +``KernelContext.tables`` alone, and it cannot see the version's metadata or +its incoming ``Frame`` mass log at all. These properties are about the three +fields that close that gap, and about the one thing they must never do: +name a column the node was not given. + +Everything here runs real shared graph operations over the invented toy +country. No country model, engine, or build artifact is involved. +""" + +from __future__ import annotations + +import dataclasses +import importlib.util +import sys +from pathlib import Path + +import pandas as pd +import pytest + +from microcosm.frame import Frame, MassChangeRecord +from microcosm.graph import ( + ContentStore, + Graph, + KernelContext, + KernelResult, + Node, + Owned, + Slice, + compile_graph, + run_graph, +) + +if "_toy" not in sys.modules: + _SPEC = importlib.util.spec_from_file_location( + "_toy", Path(__file__).with_name("_toy.py") + ) + sys.modules["_toy"] = importlib.util.module_from_spec(_SPEC) + _SPEC.loader.exec_module(sys.modules["_toy"]) +toy = sys.modules["_toy"] + +SOURCE_REF = "source.metadata@1" +PROBE_REF = "probe.frame_context@1" + +#: Metadata the CREATE kernel puts on the version, including a nested value +#: so the frozen projection is exercised rather than a flat string map. +VERSION_METADATA = {"time_period": "2024", "vintage": ("frs", "was")} + + +class MetadataSource(toy.ToyKernel): + """CREATE: the toy population, carrying declared version metadata.""" + + def compute(self, context: KernelContext) -> KernelResult: + frame = toy.read_toy_frame(context.sources["survey"]) + tables = {entity: frame.table(entity) for entity in frame.entities} + tables.update({link: frame.link(link) for link in frame.links}) + return KernelResult( + frame=Frame( + tables, + frame.schema, + { + entity: frame.weights_for(entity) + for entity in frame.weighted_entities + }, + frame.strata, + metadata=VERSION_METADATA, + ), + receipt={"persons": frame.n("person")}, + ) + + +class FrameContextProbe(toy.ToyKernel): + """Owns one column and records the frame view it was handed. + + With ``mass_reason`` set it also appends one ``Frame`` mass record + asserting the household total is unchanged, which is what the legacy + ``Frame`` contract calls an explicit conservation check. + """ + + def __init__(self, ref: str, capabilities, *, variant: str = "base") -> None: + super().__init__(ref, capabilities, variant=variant) + self.seen: list[dict[str, object]] = [] + + def compute(self, context: KernelContext) -> KernelResult: + self.seen.append( + { + "node": context.node.id, + "column_order": { + entity: tuple(columns) + for entity, columns in context.frame_column_order.items() + }, + "projected": { + entity: tuple(table.columns) + for entity, table in context.tables.items() + }, + "metadata": dict(context.frame_metadata), + "mass_log": tuple(context.frame_mass_log), + } + ) + ids = pd.Index(context.tables["person"]["person_id"], name="person_id") + result_columns = { + ("person", str(context.params["target"])): pd.Series( + 1.0, index=ids, dtype="float64" + ) + } + reason = context.params.get("mass_reason") + if reason is None: + return KernelResult(columns=result_columns) + total = float(context.weights["household"].values.sum()) + return KernelResult( + columns=result_columns, + receipt={ + "frame_mass_log_append": [ + { + "entity": "household", + "old_total": total, + "new_total": total, + "declared_factor": None, + "reason": str(reason), + } + ] + }, + ) + + +def build_registry() -> tuple[object, FrameContextProbe]: + """The toy registry with the metadata source and the probe registered.""" + registry = toy.toy_registry() + registry.register(MetadataSource(SOURCE_REF, toy._CREATE)) + probe = FrameContextProbe(PROBE_REF, toy._DETERMINISTIC) + registry.register(probe) + return registry, probe + + +CREATE = dataclasses.replace(toy.CREATE, kernel=SOURCE_REF) + + +def probe_node( + node_id: str, + *, + columns: tuple[str, ...], + target: str, + mass_reason: str | None = None, +) -> Node: + """A probe reading ``columns`` of the person entity, in that order. + + A probe that states a mass record also reads the household entity, + because the record is stated against the household total and a node + only receives weights for the entities it projects. + """ + inputs = (Slice("person", columns),) + if mass_reason is not None: + inputs = (*inputs, Slice("household", ("household_size",))) + return Node( + node_id, + PROBE_REF, + inputs=inputs, + outputs=(Owned("person", target, "float64"),), + params={"target": target, "mass_reason": mass_reason}, + population="survey", + ) + + +#: ``income`` before ``age`` is the reverse of the source table's own order, +#: so the projection and the version's layout genuinely disagree. +FIRST = probe_node( + "probe_first", + columns=("income", "age"), + target="probe_a", + mass_reason="households are unchanged by a derivation", +) +SECOND = probe_node("probe_second", columns=("probe_a",), target="probe_b") + + +def probe_graph(*extra: Node) -> Graph: + return Graph("toy", (toy.SOURCE,), (CREATE, FIRST, SECOND, *extra)) + + +def run_probe( + root: Path, + graph: Graph | None = None, + *, + registry=None, + probe=None, + store: ContentStore | None = None, + sources=None, + resume: str = "auto", + observer=None, +): + """Run ``graph`` and return ``(manifest, probe, sources, store)``.""" + if registry is None: + registry, probe = build_registry() + if sources is None: + sources = {"survey": toy.copy_source(root / "source")} + if store is None: + store = ContentStore(root / "store") + manifest = run_graph( + compile_graph(graph or probe_graph()), + sources=dict(sources), + store=store, + kernels=registry, + resume=resume, + _population_observer=observer, + ) + return manifest, probe, sources, store + + +def observation(probe: FrameContextProbe, node_id: str) -> dict[str, object]: + (found,) = [item for item in probe.seen if item["node"] == node_id] + return found + + +# ---------------------------------------------------------------------- +# The field contract +# ---------------------------------------------------------------------- + + +def test_the_three_fields_ride_before_the_amendment_13_17_pair() -> None: + """Amendment 17's "numerics rides at the end" stays literally true.""" + fields = [f.name for f in dataclasses.fields(KernelContext)] + assert fields[-2:] == ["tolerances", "numerics"] + assert fields[fields.index("artifacts") + 1 : fields.index("tolerances")] == [ + "frame_metadata", + "frame_mass_log", + "frame_column_order", + ] + + +def bare_context(**kwargs) -> KernelContext: + return KernelContext( + node=Node("n", PROBE_REF), + tables={"person": pd.DataFrame({"person_id": [1, 2], "age": [30, 40]})}, + weights={}, + strata=pd.Series(dtype="string"), + params={}, + rng=None, + **kwargs, + ) + + +def test_the_fields_default_to_an_empty_view() -> None: + context = bare_context() + assert dict(context.frame_metadata) == {} + assert context.frame_mass_log == () + assert dict(context.frame_column_order) == {} + + +def test_column_order_refuses_an_undeclared_column() -> None: + """The one thing an order must never do is name a column not given. + + A name is itself information about the version, so an order that + mentions an unprojected column is refused rather than trimmed. + """ + with pytest.raises(TypeError, match="exactly the projected columns"): + bare_context(frame_column_order={"person": ("person_id", "age", "income")}) + + +def test_column_order_refuses_a_partial_or_repeated_order() -> None: + with pytest.raises(TypeError, match="exactly the projected columns"): + bare_context(frame_column_order={"person": ("person_id",)}) + with pytest.raises(TypeError, match="exactly the projected columns"): + bare_context(frame_column_order={"person": ("person_id", "person_id", "age")}) + with pytest.raises(TypeError, match="exactly the projected columns"): + bare_context(frame_column_order={"person": ["person_id", "age"]}) + + +def test_column_order_refuses_an_unprojected_entity() -> None: + with pytest.raises(TypeError, match="exactly the projected columns"): + bare_context(frame_column_order={"household": ("household_id",)}) + + +def test_mass_log_refuses_anything_but_mass_records() -> None: + with pytest.raises(TypeError, match="tuple of MassChangeRecord"): + bare_context(frame_mass_log=({"entity": "household"},)) + with pytest.raises(TypeError, match="tuple of MassChangeRecord"): + bare_context( + frame_mass_log=[ + MassChangeRecord( + entity="household", + old_total=1.0, + new_total=1.0, + declared_factor=None, + reason="why", + ) + ] + ) + + +def test_metadata_is_a_read_only_view() -> None: + source = {"time_period": "2024"} + context = bare_context(frame_metadata=source) + assert dict(context.frame_metadata) == source + with pytest.raises(TypeError): + context.frame_metadata["time_period"] = "2025" # type: ignore[index] + source["time_period"] = "2025" + assert context.frame_metadata["time_period"] == "2024" + with pytest.raises(TypeError, match="non-empty strings"): + bare_context(frame_metadata={"": "no"}) + + +# ---------------------------------------------------------------------- +# What the executor actually supplies +# ---------------------------------------------------------------------- + + +def test_column_order_is_the_versions_own_order_not_the_projection( + tmp_path: Path, +) -> None: + """The real property: two different orders over the same column set.""" + _, probe, _, _ = run_probe(tmp_path / "run") + seen = observation(probe, "probe_first") + order = seen["column_order"]["person"] + projected = seen["projected"]["person"] + assert set(order) == set(projected) + assert order != projected + assert projected.index("income") < projected.index("age") + assert order.index("age") < order.index("income") + + +def test_column_order_names_no_column_the_node_did_not_read(tmp_path: Path) -> None: + """``receives_x`` exists on the version and is in neither view.""" + _, probe, _, _ = run_probe(tmp_path / "run") + for key in ("column_order", "projected"): + person = observation(probe, "probe_first")[key]["person"] + assert "receives_x" not in person + assert "is_adult" not in person + assert set(observation(probe, "probe_second")["column_order"]["person"]) == { + "person_id", + "person_household_id", + "person_release_id", + "probe_a", + } + + +def test_version_metadata_reaches_every_node_of_the_version(tmp_path: Path) -> None: + _, probe, _, _ = run_probe(tmp_path / "run") + for node_id in ("probe_first", "probe_second"): + metadata = observation(probe, node_id)["metadata"] + assert metadata["time_period"] == "2024" + assert tuple(metadata["vintage"]) == ("frs", "was") + + +def test_mass_log_is_the_incoming_log(tmp_path: Path) -> None: + """The appending node sees the log before its own record; its successor after.""" + _, probe, _, _ = run_probe(tmp_path / "run") + assert observation(probe, "probe_first")["mass_log"] == () + after = observation(probe, "probe_second")["mass_log"] + assert len(after) == 1 + assert after[0].entity == "household" + assert after[0].reason == "households are unchanged by a derivation" + assert after[0].old_total == after[0].new_total + + +# ---------------------------------------------------------------------- +# Replay +# ---------------------------------------------------------------------- + + +def test_required_replay_is_a_full_hit(tmp_path: Path) -> None: + """The three fields are additive: no key moves, everything replays.""" + cold_manifest, _, sources, store = run_probe(tmp_path / "run") + assert not any(receipt.hit for receipt in cold_manifest.nodes.values()) + + registry, probe = build_registry() + warm = run_graph( + compile_graph(probe_graph()), + sources=dict(sources), + store=ContentStore(tmp_path / "run" / "store"), + kernels=registry, + resume="require", + ) + assert all(receipt.hit for receipt in warm.nodes.values()) + assert probe.seen == [] + assert {n: r.key for n, r in warm.nodes.items()} == { + n: r.key for n, r in cold_manifest.nodes.items() + } + + +def test_a_new_node_over_restored_populations_sees_the_same_frame( + tmp_path: Path, +) -> None: + """The fields are rebuilt from a restored Frame, not only a computed one. + + The store round trip is where metadata and the ``Frame`` mass log could + quietly disappear, so the property is stated against a node that runs + cold on top of cache hits. + """ + _, cold_probe, sources, _ = run_probe(tmp_path / "run") + expected = observation(cold_probe, "probe_second") + + third = probe_node("probe_third", columns=("probe_a",), target="probe_c") + registry, probe = build_registry() + warm = run_graph( + compile_graph(probe_graph(third)), + sources=dict(sources), + store=ContentStore(tmp_path / "run" / "store"), + kernels=registry, + resume="auto", + ) + assert {n for n, r in warm.nodes.items() if not r.hit} == {"probe_third"} + seen = observation(probe, "probe_third") + assert seen["metadata"] == expected["metadata"] + assert seen["mass_log"] == expected["mass_log"] + assert seen["column_order"] == expected["column_order"] + + +def test_a_retained_mutating_observer_changes_nothing(tmp_path: Path) -> None: + """Amendment 24 still holds across the new fields. + + The observer keeps every snapshot and rewrites its tables and its + metadata view after the callback returns; what later nodes read through + the frame fields, and the run's identity, are unchanged. + """ + plain_manifest, plain_probe, sources, _ = run_probe(tmp_path / "plain") + retained: list[object] = [] + + def observe(node_id: str, population) -> None: + retained.append(population) + for entity in population.frame.entities: + table = population.frame.table(entity) + for column in table.columns: + table.loc[:, column] = table[column].iloc[0] + + observed_manifest, observed_probe, _, _ = run_probe( + tmp_path / "observed", sources=sources, observer=observe + ) + assert len(retained) == len(observed_manifest.nodes) + assert {n: r.key for n, r in observed_manifest.nodes.items()} == { + n: r.key for n, r in plain_manifest.nodes.items() + } + for node_id in ("probe_first", "probe_second"): + assert observation(observed_probe, node_id) == observation(plain_probe, node_id) diff --git a/packages/microcosm-graph/tests/test_graph_kernel_contract.py b/packages/microcosm-graph/tests/test_graph_kernel_contract.py index 885116345..fe524a25d 100644 --- a/packages/microcosm-graph/tests/test_graph_kernel_contract.py +++ b/packages/microcosm-graph/tests/test_graph_kernel_contract.py @@ -340,7 +340,9 @@ def test_context_artifacts_default_empty_and_are_immutable() -> None: """Amendment 19: ``artifacts`` rides before the amendment-13/17 pair.""" fields = [f.name for f in dataclasses.fields(KernelContext)] assert fields[-2:] == ["tolerances", "numerics"] - assert fields[fields.index("artifacts") + 1] == "tolerances" + # Amendment 19 claims artifacts rides *before* the pair, not adjacent to + # it; amendment 26's three frame fields ride between them. + assert fields.index("artifacts") < fields.index("tolerances") node = Node("draw", "fit.draw@1") bare = KernelContext( node=node, From 895aabf191f5ddd675ad6ebb9376546830eacfb4 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 08:47:57 -0400 Subject: [PATCH 04/17] B2: the kernel context carries the version's frame view (amendment 26) Co-Authored-By: Claude Opus 5 --- packages/microcosm-graph/tests/test_acceptance_b_ownership.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/packages/microcosm-graph/tests/test_acceptance_b_ownership.py b/packages/microcosm-graph/tests/test_acceptance_b_ownership.py index e4f065284..51b882ffe 100644 --- a/packages/microcosm-graph/tests/test_acceptance_b_ownership.py +++ b/packages/microcosm-graph/tests/test_acceptance_b_ownership.py @@ -117,6 +117,9 @@ def test_b2_executor_enforces_ownership(tmp_path: Path) -> None: "rng", "sources", "artifacts", # amendment 19: declared typed opaque artifact inputs + "frame_metadata", # amendment 26: the population version's metadata + "frame_mass_log", # amendment 26: the version's incoming Frame mass log + "frame_column_order", # amendment 26: the version's order, projected only "tolerances", # amendment 13: declared tolerances of the inputs' owners "numerics", # amendment 17: per-coordinate numeric class, bound, platform } From 6f4ba4ec989eba93786b5d88033631ad253fdc5c Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 08:50:06 -0400 Subject: [PATCH 05/17] Record the UK shared-graph contract lane receipts and runtime plan Source-only receipt for amendments 25 and 26: exact heads, the absence proof on 15ebde806, the #901 consumer that fixed the contract shape, file hashes, the contract decisions that differ from #901, and the finite invented-only runtime plan left for root to execute. Runtime UNTESTED in this lane. Co-Authored-By: Claude Opus 5 --- PROGRESS-uk-shared-graph-contracts.md | 109 +++++++++++++++++ .../901-uk-shared-graph-contracts-receipts.md | 115 ++++++++++++++++++ 2 files changed, 224 insertions(+) create mode 100644 experiments/901-uk-shared-graph-contracts-receipts.md diff --git a/PROGRESS-uk-shared-graph-contracts.md b/PROGRESS-uk-shared-graph-contracts.md index bd18785be..0bdde5d7c 100644 --- a/PROGRESS-uk-shared-graph-contracts.md +++ b/PROGRESS-uk-shared-graph-contracts.md @@ -64,3 +64,112 @@ at `051fb972` (SHA-256 `fc6f5b33127020f6e0529b39304715fe2028d47a6627b152b1e52e0d - Amendment 25: `WeightUpdate` + ordered-axis receipt. - Amendment 26: `KernelContext` frame metadata / mass log / column order. + +--- + +## Done (2026-09-13) + +| Commit | What | +| --- | --- | +| `97428cd9f` | Lane baseline: absence proof + the exact #901 consumer read | +| `ef2dc69c3` | Amendment 25: `WeightUpdate` + `weight_update_receipt` | +| `f33d47cda` | Amendment 26: `KernelContext` frame metadata / mass log / column order | +| `895aabf19` | Acceptance suite B2 field set, isolated (as amendment 19's was, `a2b6dfb0b`) | + +The two amendments are separable: 25 touches `decl.py` (re-locked) and +leaves `kernel.py` byte-identical; 26 touches `kernel.py` (re-locked) and +leaves `decl.py` byte-identical. Either can be dropped without the other. + +## Contract decisions + +1. **`reason` is normative.** It enters the node key, so two updates that + state different purposes are different nodes. Follows #901's own + declaration; the consequence is stated in the amendment. +2. **`to_kind` stays a property, not a field.** That is what keeps the two + declarations' field sets disjoint (`{entity, to_kind, mass}` vs + `{entity, kind, reason, mass}`), so neither canonical bytes nor + declaration JSON can confuse them — and existing `to_kind` readers (the + design-weight cap, the calibration view) keep working unchanged. +3. **No `free` mass on an update.** #901 declares this too. An update that + neither moves kind nor bounds mass records nothing checkable. +4. **Replay is structural, not a parallel rule.** `_load_cached_result` + already reconstructs the `KernelResult` and re-applies REWEIGHT to the + current base, so a cache hit re-enters `_apply_weight_update`. No + executor change was needed for the axis check. +5. **The frozen interface does not import another shard's private name.** + #901's `kernel.py` imports `microcosm.frame.bundle._freeze_metadata`. + This lane does not: `Frame` has already deeply frozen the metadata the + executor passes, and `KernelContext` adds a read-only view over it. The + docstring says exactly that and claims no deep freeze of its own. +6. **The three fields ride between `artifacts` and `tolerances`**, not at + the end as in #901, so amendment 17's "numerics rides at the end of the + context" stays literally true. Only amendment 19's unit assertion of + *adjacency* relaxes, to the ordering it actually claimed. +7. **A column order may not name an unprojected column.** Set equality + with the projected columns, not a subset: a column *name* is itself + information about the version. + +## Remaining risks + +- **Runtime is UNTESTED here.** No pytest, import, engine or install was + run, per the lane's instructions. Everything below the source level is + unverified; see the runtime plan. +- The acceptance-suite commit `895aabf19` is the one change this lane made + to a file the charter assigns to the suite lane. It is isolated to one + file and follows the precedent the amendment-19 doc text states + explicitly ("the acceptance suite's B2 field set gains it in its own + commit"). If root's owner disagrees, dropping that commit leaves B2 red + and the rest intact. +- Amendment 25 changes no node key; amendment 26 changes none either. + Neither re-pins a spec digest. If a spec/seed digest moves in CI, that + is main drift, not this lane (see `[[spec-engine-attested-modules]]`). +- `_context_digest` (B4's mutation check) was **not** extended to the + three new fields. They are immutable views over an immutable `Frame`, so + there is nothing for a kernel to mutate; stated here so the omission is + a decision rather than an oversight. +- No claim is made that any UK build, native lane, calibration or release + passes. This lane read source only. + +## Next (for root, before execution) + +Finite, invented-only runtime plan — nothing below touches a country +model, engine, native source, gated microdata or the network. + +``` +uv sync --all-packages --locked +uv run pytest packages/microcosm-graph/tests/test_graph_weight_update.py \ + packages/microcosm-graph/tests/test_graph_frame_context.py +uv run pytest packages/microcosm-graph/tests/test_graph_interface_lock.py \ + packages/microcosm-graph/tests/test_acceptance_b_ownership.py \ + packages/microcosm-graph/tests/test_graph_kernel_contract.py \ + packages/microcosm-graph/tests/test_graph_serialize.py \ + packages/microcosm-graph/tests/test_graph_decl.py \ + packages/microcosm-graph/tests/test_acceptance_d_weights.py \ + packages/microcosm-graph/tests/test_graph_population.py \ + packages/microcosm-graph/tests/test_graph_executor.py \ + packages/microcosm-graph/tests/test_graph_explain.py \ + packages/microcosm-graph/tests/test_acceptance_replays.py +uv run pytest packages/microcosm-graph packages/microcosm-frame +uv run ruff check . +uv run python tools/ci_test_groups.py --verify +``` + +Expected: 19 + 19 new tests pass; the ten regression files stay green; +`ruff check` and `--verify` already pass here. The four places a source-only +lane could be wrong, in the order worth checking: + +1. `test_graph_frame_context.py::test_a_retained_mutating_observer_changes_nothing` + mutates a detached snapshot with `table.loc[:, column] = table[column].iloc[0]`. + If pandas copy-on-write makes that a no-op on the snapshot, the test + passes vacuously rather than falsely; tighten it rather than trust it. +2. The toy person column order is asserted from `fixtures/toy_country/person.csv` + (`person_id, person_household_id, person_release_id, age, income, ...`). + If that fixture changes, `test_column_order_is_the_versions_own_order_not_the_projection` + is the test that notices. +3. `_append_frame_mass_log` requires the record to bracket the real + household totals; the probe states an unchanged total from + `context.weights["household"].values.sum()`. A mismatch would surface + as `PopulationError` from `test_mass_log_is_the_incoming_log`. +4. `test_round_trip_refuses_a_mixed_weights_payload` edits canonical JSON + by string surgery and depends on lexicographic key order + (`entity, kind, mass, reason`). diff --git a/experiments/901-uk-shared-graph-contracts-receipts.md b/experiments/901-uk-shared-graph-contracts-receipts.md new file mode 100644 index 000000000..58e428d4a --- /dev/null +++ b/experiments/901-uk-shared-graph-contracts-receipts.md @@ -0,0 +1,115 @@ +# Shared graph contracts extracted for the UK full-build graph (#901) + +Source-only lane receipt, 2026-09-13. Runtime **UNTESTED** — see +`PROGRESS-uk-shared-graph-contracts.md` for the decisions, risks and the +finite invented-only runtime plan this lane leaves for root. + +## Heads + +| Thing | Exact identity | +| --- | --- | +| Worktree | `_worktrees/microcosm-uk-shared-graph-contracts-20260913` | +| Branch | `uk-shared-graph-contracts-20260913` | +| Base (reviewed main) | `15ebde806cd1a262363f7217fe535c7234ff757f` | +| #901 head (reviewed, re-verified) | `051fb972b19d319d58277bd63306d0d0e0947ce2` | +| #901 live state | OPEN, draft, `CONFLICTING`, updated 2026-09-10T21:03:10Z | +| Source review followed | `uk-parallel-review.md`, 2026-09-12 | + +`origin/main` was re-fetched after the work and is still `15ebde806`. + +## Absence, before anything was written + +`grep -rni weightupdate` over the worktree at `15ebde806` returns nothing. +`decl.py:320` carries only `WeightTransition`, which requires `to_kind` +strictly later in `WEIGHT_KINDS`; `population.py:1944` rejects a +non-forward move. `kernel.py:361-370` is the complete `KernelContext` +field list and has no metadata, mass-log or column-order field. The +interface lock matched both files exactly. So both extensions were +genuinely absent, and no duplicate patch was produced. + +The write half of the mass-log contract already existed on main +(`population._append_frame_mass_log`, `receipt['frame_mass_log_append']`); +only the kernel's view of the incoming log was missing. That is why this +lane adds a read side and no second ledger. + +## Consumer that fixed the shape + +`packages/microcosm-build/src/microcosm/build/uk_runtime/graph_population.py` +at `051fb972`, SHA-256 +`fc6f5b33127020f6e0529b39304715fe2028d47a6627b152b1e52e0d69f2efdc`: + +- `context_frame` (L58-80): `context.frame_column_order.get(...)`, + `getattr(context, "frame_mass_log", ())`, + `getattr(context, "frame_metadata", {})`. +- `uk.full.normalize`: `WeightUpdate("household", weight_kind, "Normalize + sampled source-family mass.")`, `mass="declared"`, and + `receipt["weight_update"] = weight_update_receipt(ids)` where `ids` is + the household-id axis of the context frame. + +Both signatures land exactly as #901 calls them, so no UK edit is needed +to consume this. Nothing was copied from #901's executor, and +`graph.attachments._PopulationRetention` is not imported anywhere. + +## Amendments + +**25 — a same-kind weight update is declarable.** `decl.py` gains +`WeightUpdate(entity, kind, reason, mass)` and +`WEIGHT_UPDATE_MASS_POLICIES`; a new non-frozen +`microcosm/graph/weight_update.py` gains `weight_update_receipt`. The +incumbent, declared and returned kinds must agree; mass is `conserve` or +`declared`; `reason` is required, non-empty and normative. The kernel +binds its ordered entity axis and the executor recomputes that binding +from the incumbent axis, on cold execution and on replay — replay for +free, because `_load_cached_result` already re-applies REWEIGHT to the +current base through the same function. `to_kind` is a property, so the +two declarations' field sets are disjoint and declaration JSON round-trips +each as itself; the transition payload is byte-for-byte unchanged. + +**26 — the context carries the version's frame view.** `kernel.py` gains +`frame_metadata`, `frame_mass_log` and `frame_column_order`, riding after +`artifacts` and before `tolerances` so amendment 17's "numerics rides at +the end" stays literally true. A column order must be exactly an ordering +of the projected columns, so it can neither hide a column the node was +given nor name one it was not. The frozen interface does **not** import +`microcosm.frame.bundle._freeze_metadata` (as #901's does); the executor +passes `Frame.metadata`, already deeply frozen by `Frame`, and the +docstring claims only the read-only view it actually adds. + +Neither amendment adds a `Node` field, changes a canonical projection, or +moves a node key. `decl.py` is re-locked by 25, `kernel.py` by 26. + +## Identity + +| File | SHA-256 | +| --- | --- | +| `decl.py` | `e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933` | +| `kernel.py` | `51f45e899ba578a6b2324636a9879253266b067812f68b648eacfcd5a184d245` | +| `weight_update.py` | `0ccfe6fcd257ef62b1f771b12eecc8ac5d447f5aa7d0403102e0ae290de16720` | +| `population.py` | `33d1bb7bacea22870940288bf1907fb9eb24df7c245a216ff802e7fb41f5208f` | +| `serialize.py` | `e5bf83c1082154f148626b6a36676614c6ff6c3fe0721aed94a1501da3021b1f` | +| `executor.py` | `ec4473bb033c4b1a36180c1518a42c755a46a2265d10461df1dfa00065862364` | +| `explain.py` | `734a7b0e31692c31a99528cd83d9e74d3e508a317d913c1c169724a42d69d0de` | +| `graph/__init__.py` | `697c59a37989a36124e6d43c7b07dd3b0582d965f97303c1fb02c88b41db2d48` | +| `tests/test_graph_weight_update.py` | `771ae0becbc57a4dd6198b9df229ec8c8262a5597647f335b7d5ed9be83471ff` | +| `tests/test_graph_frame_context.py` | `d58250d2194c81002be7282cd53597a7b9cff0906bb879b4b3926b4ac5b91ada` | +| `docs/graph-interface.lock` | `b42811ff0411dc179aaf9ddcf827aa270db866e08112cef8d1839df99c5a1f02` | + +The lock was re-recorded as part of each numbered amendment, never +refreshed to make a test green: amendment 25 moved only the `decl.py` +line, amendment 26 only the `kernel.py` line. + +## What stays with María + +Every UK graph stage, kernel, calibration target, geography ladder, +scorecard and release gate in #901. This lane adds no country graph, no +UK node, and no second definition of anything #901 owns. + +## Checks actually run + +`ruff check .` (clean), `ruff format --check` on every file this lane +touched (clean), `python -I -B -S` stdlib `ast` parses of all 17 changed +files, and `tools/ci_test_groups.py --verify` (`verification=ok`; both new +test files land in `fast/rest` and the engine lane beside their 27 sibling +graph tests, neither `[defaulted]`). No pytest, no import of the +production package, no engine, no install, no network beyond `gh` +metadata and public blob reads. From 6104459f693db98ea835fb13044ef5ac1f866224 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 09:18:12 -0400 Subject: [PATCH 06/17] File the independent adjudication and open the fix round The reviewer returned its verdict through the tool result because its session had no write tool, so the text is filed here unedited rather than restated. Records the round's scope: F1, F3-F6 accepted, F2 not. Co-Authored-By: Claude Opus 5 --- FABLE-REVIEW.md | 48 +++++++++++++++++++++++++++ PROGRESS-uk-shared-graph-contracts.md | 31 +++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 FABLE-REVIEW.md diff --git a/FABLE-REVIEW.md b/FABLE-REVIEW.md new file mode 100644 index 000000000..b5a4456bd --- /dev/null +++ b/FABLE-REVIEW.md @@ -0,0 +1,48 @@ + + +**Verdict: REQUEST_CHANGES** for the source proposal. Two findings are actionable design defects in the new contracts; the rest are hardening and test-strength items. I could not save FABLE-REVIEW.md: this session is in plan mode with no write tool exposed, so the full review is below for root to file verbatim. + +## Reviewed identity + +| Item | Value | +| --- | --- | +| HEAD | `6f4ba4ec989eba93786b5d88033631ad253fdc5c` (resolved from `.git/refs/heads/uk-shared-graph-contracts-20260913`) | +| Base | `15ebde806cd1a262363f7217fe535c7234ff757f` (as given; not independently resolved) | +| Method | Read-only. No git diff was available without a shell, so I reviewed the current contents of every file in the receipts' identity table plus both new test files, the B2 edit, the kernel-contract unit test, and the amendment 25/26 text. No imports, pytest, engine, or data. | + +## Ranked findings + +**1. HIGH. `frame_mass_log` leaks non-declared same-version sibling output into kernel inputs with no key binding.** +`_project_context` passes `population.frame.mass_log` at `packages/microcosm-graph/src/microcosm/graph/executor.py:678`. For an ordinary node that population is the cumulative version state, updated after every ordinary node at `executor.py:2569-2570`, and `_append_frame_mass_log` runs for ordinary nodes too at `population.py:1157`. So node X sees records appended by any earlier ordinary node A in the same version, whether or not A is an ancestor. X's key binds only `frame_key(version)` plus declared input owners (`keys.py:187-209`), never A. Result: adding, removing, or re-parameterising A changes X's visible input while X's key is unchanged, so a cache hit replays output computed against a different log. This is a new charter-A hole; before amendment 26 kernels could not see the log at all. The doc sentence "incidental node order is not authority" describes the hazard but nothing enforces it. +Exact fix, minimal: in `run_graph` record `boundary_logs[node.id] = updated.frame.mass_log` when a structural node is admitted, and pass `frame_mass_log=boundary_logs[compiled.versions[node_id]]` for ordinary nodes; structural nodes keep the full incumbent log, which their key already binds through `members` (`keys.py:221-225`). Alternative that keeps sibling visibility: track which node appended each record and reject projection when a record's author is not in `_transitive_ancestors(compiled, node_id)`. Either way `test_mass_log_is_the_incoming_log` at `test_graph_frame_context.py:346-354` must change: the successor is an ordinary node reading the appender's column, and the fix makes it see `()` unless the ancestor rule is used. + +**2. HIGH. A `WeightUpdate` on kind `design` leaves design anchors stale and, after an EXPAND, mixed.** +Anchors are captured once at CREATE (`population.py:333-338`) and only carried afterwards (`population.py:1159`, `_carry_design_weights` at `2125-2181`). `_apply_weight_update` replaces the frame's design-kind values (`population.py:2038`) without re-anchoring. Consequences: the calibrated cap at `population.py:2323-2351` and `realized_max_weight_ratio` at `2354-2375` compare against pre-update design weights, so a normalisation by factor k makes every later cap ratio off by k; an EXPAND after the update anchors entrants from the updated values (`2166-2179`) while retained rows keep original anchors. The #901 consumer, `uk.full.normalize`, is exactly a design-weight normalisation. The decl docstring at `decl.py:381-382` states "ancestry is untouched" as if deliberate, but the mixed-anchor case is not a coherent contract. +Exact fix: in `patch` after line 1159, when `isinstance(node.weights, WeightUpdate)` and the kind is `design`, set `design_weights[entity] = frame.weights_for(entity).values`, and say so in amendment 25. If root prefers the current semantics, refuse `kind="design"` in `WeightUpdate.__post_init__` instead; leaving it silent is the one option I would not accept. + +**3. MEDIUM. `_context_digest` omission is defensible but the fields share live objects.** +Actual behaviour checked: `frame_metadata` is a `MappingProxyType` over a copy whose leaves are `Frame`-frozen tuples, frozensets and `_FrozenMapping` (`bundle.py:1346-1362`); `frame_column_order` values are `tuple[str]`; `frame_mass_log` is the population's own tuple of frozen `MassChangeRecord` objects, shared by reference (`executor.py:678`). Through the public API nothing is writable, so the omission at `executor.py:459-488` does not produce false passes. But `object.__setattr__` on a shared record silently rewrites the live version's log, and unlike `tables` the executor would not notice. This matches the existing treatment of `context.node`, so it is not blocking. Cheap hardening: digest `canonical_json(column_order)`, the mass-log record fields, and `store._encode_frame_metadata(frame_metadata)` inside `_context_digest`. + +**4. MEDIUM. Two new tests are weaker than their names.** +`test_cold_then_required_replay_revalidates_the_axis` (`test_graph_weight_update.py:311-337`) only proves a hit succeeds; it never shows the axis check executes on replay. `test_a_retained_mutating_observer_changes_nothing` (`test_graph_frame_context.py:410-435`) says it rewrites the metadata view but only rewrites tables; the three new fields are untouched. + +**5. LOW. The motivating "re-solve an existing calibration" case is unreachable with the shared kernel.** `calibrate.adam@1` returns no `receipt['weight_update']` and its error text at `packages/microcosm-calibrate/src/microcosm/calibrate/kernels.py:205-209` still demands a `WeightTransition`. Declaring it as a `WeightUpdate` rejects at `population.py:2025-2030`. Not this lane's file, but amendment 25's text should not claim the case is covered. + +**6. LOW. Cosmetic.** `decl.py:568-572` says "weight transition's" for an update mismatch. `test_graph_kernel_contract.py:339-345` was also edited, so the receipts' "isolated to one test file" claim applies to the acceptance suite only; that is acceptable under the charter. + +## Reproducing tests to invent + +- **T1 leak:** probe kernel writes `float(len(context.frame_mass_log))` into `person.x`, reading only `person.age`. Run `{survey, X}` cold: x = 0. Run `{survey, A, X}` into the same store where A appends a record: X hits and reports 0 while a cold run gives 1. Same key, different truth. +- **T2 anchors:** `WeightUpdate("household","design",…)` with factor 2, then a `calibrated` transition returning weights equal to the updated design weights with `max_weight_ratio=1.5`. Currently rejected as ratio 2.0; the receipt reports realized ratio 2.0 against a frame whose own design weights it equals. +- **T3 replay:** monkeypatch `microcosm.graph.population.weight_update_receipt` to return a foreign digest during the `resume="require"` run and assert `NodeRejectedError` matching "different .household. axis". +- **T4 tamper:** a kernel that calls `object.__setattr__(context.frame_mass_log[0], "reason", "x")`; assert the next node sees the original reason. Currently the live log changes and no rejection fires. + +## Residual runtime risks + +- Nothing here was executed. Pandas copy-on-write behaviour in the observer test and the fixture column order at `fixtures/toy_country/person.csv` line 1 (age before income) are the two places a source-only read can be wrong. +- In `resume="require"`, a mismatched axis receipt fails mid-run at apply time rather than in `_preflight_require`, since preflight validates record shape only. +- Column order is bound to the key by construction (owners are ancestors, order follows depth then id, rewrites keep position); I found no leak there. diff --git a/PROGRESS-uk-shared-graph-contracts.md b/PROGRESS-uk-shared-graph-contracts.md index 0bdde5d7c..2e0519236 100644 --- a/PROGRESS-uk-shared-graph-contracts.md +++ b/PROGRESS-uk-shared-graph-contracts.md @@ -173,3 +173,34 @@ lane could be wrong, in the order worth checking: 4. `test_round_trip_refuses_a_mixed_weights_payload` edits canonical JSON by string surgery and depends on lexicographic key order (`entity, kind, mass, reason`). + +--- + +# Fix round (2026-09-13, after the independent Fable adjudication) + +## State + +The lane's four commits (`97428cd9f`..`6f4ba4ec9`) stand. An independent +read-only adjudication of exactly `6f4ba4ec9` over base `15ebde806` +returned **REQUEST_CHANGES**; its verbatim text is now filed in this +packet as `FABLE-REVIEW.md` (the reviewer ran without a write tool and +returned the review through its tool result instead). + +Root owns adjudication. Findings F1, F3, F4, F5 and F6 are accepted and +implemented in this round. **F2 is not accepted as stated** — see the +point-by-point response below and in `FIX-RESULT.md`. + +`origin/main` re-fetched before editing: still +`15ebde806cd1a262363f7217fe535c7234ff757f`, nothing new to merge, branch +is 5 commits ahead and 0 behind. + +**Runtime remains UNTESTED in this round too.** No pytest, no production +import, no engine, no install, no network. Source, stdlib `ast`, `ruff` +and the CI group inventory only. + +## Next + +- F1: project ordinary nodes from their version boundary's mass log. +- F3/F4: detach the exposed frame objects; digest all three fields. +- F2: source-backed rejection plus the anchor-invariance tests. +- F5/F6: correct the docs' motivating claim and the disclosure. From c0e275543374ee18febf5f1811c5fee9cfc6d32a Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 09:23:29 -0400 Subject: [PATCH 07/17] F1: a node sees the mass log its own key binds (amendment 26) The projection handed every node `population.frame.mass_log`. For an ordinary node that is the version's *cumulative* log, updated after each member, while its key binds only the version's structural boundary and the owners of the columns it declared. Adding or re-parameterising an unrelated sibling therefore changed what a node was shown without moving its key, so a cache hit could replay output computed against a different log. `run_graph` now records each version's log as that version is admitted -- cold execution and a restored hit both reach the same line -- and projects ordinary nodes from that boundary. A structural node still receives the cumulative log, because `keys.py` binds its base *and* every ordinary member of that version through `members`. The properties distinguish the two cases: an unread same-version appender is invisible to an ordinary member and moves neither its key nor its stored bytes, cold-with-sibling equals cold-without and replays across, while the same appender is visible to the structural boundary whose key it moves. Co-Authored-By: Claude Opus 5 --- ...red-graph-contracts-frame-context.added.md | 2 +- docs/graph-acceptance.md | 28 +- docs/graph-interface.lock | 2 +- .../src/microcosm/graph/executor.py | 42 ++- .../src/microcosm/graph/kernel.py | 18 +- .../tests/test_graph_frame_context.py | 332 ++++++++++++++++-- 6 files changed, 381 insertions(+), 43 deletions(-) diff --git a/changelog.d/uk-shared-graph-contracts-frame-context.added.md b/changelog.d/uk-shared-graph-contracts-frame-context.added.md index 64fe69ac1..a66fead06 100644 --- a/changelog.d/uk-shared-graph-contracts-frame-context.added.md +++ b/changelog.d/uk-shared-graph-contracts-frame-context.added.md @@ -1 +1 @@ -Added `KernelContext.frame_metadata`, `frame_mass_log` and `frame_column_order`, so a kernel can reconstruct its population version's declared slices without seeing any column it did not declare (graph amendment 26). +Added `KernelContext.frame_metadata`, `frame_mass_log` and `frame_column_order`, so a kernel can reconstruct its population version's declared slices without seeing any column it did not declare. The mass log a node receives is the one its key binds: its version's structural boundary for an ordinary node, the base version's cumulative log for a structural one (graph amendment 26). diff --git a/docs/graph-acceptance.md b/docs/graph-acceptance.md index 96a4a4fa8..936dbda46 100644 --- a/docs/graph-acceptance.md +++ b/docs/graph-acceptance.md @@ -568,12 +568,28 @@ lock unchanged: difference from the UK branch, which imports `microcosm.frame.bundle._freeze_metadata` into the frozen interface: a frozen contract should not depend on another shard's private name.) - - `frame_mass_log` — the version's `Frame` mass records, in order, and - specifically the *incoming* ones. A node needing a stage's completed - records must run after that stage's structural boundary or read its - predecessor's evidence; incidental node order is not authority. The - write side already existed (`receipt['frame_mass_log_append']`); only - the read side was missing. + - `frame_mass_log` — the `Frame` mass records the node's *key* binds, + in order. An ordinary node's key binds its version's structural + boundary (`population_input`) and the owners of the columns it + declared (`input_artifacts`); it does not bind the other ordinary + members of its version. Its log is therefore the version's + **boundary** log, captured when the structural node was admitted, and + a record another member appends afterwards is not visible to it. The + alternative — the cumulative log the version carries at the moment the + node runs — would be a kernel input no key binds: adding or + re-parameterising an unrelated sibling would change what the node + sees while its key, and so its cache entry, stayed put, and a hit + would replay output computed against a different log. A structural + node is given its base version's cumulative log instead, because its + key does bind it: `keys.py` binds the base's frame identity *and* + every ordinary member of that version through `members`, which + `compile_graph` fills with `members.get(base, ())`. Boundaries are + captured where the version is admitted, so cold execution and a + restored cache hit record the same one. A node needing a stage's + completed records therefore runs after that stage's structural + boundary or reads its predecessor's evidence; incidental node order is + not authority. The write side already existed + (`receipt['frame_mass_log_append']`); only the read side was missing. - `frame_column_order` — entity to the version's own column order, restricted to the columns projected into `tables`. An entry that is not exactly an ordering of that table's columns is refused, so an diff --git a/docs/graph-interface.lock b/docs/graph-interface.lock index f09fe7bb1..cb01bf926 100644 --- a/docs/graph-interface.lock +++ b/docs/graph-interface.lock @@ -1,2 +1,2 @@ e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933 decl.py -51f45e899ba578a6b2324636a9879253266b067812f68b648eacfcd5a184d245 kernel.py +a49bb4461e7928118b05c13351a99b7585e5068a0868f6bc40634ebb057f29ca kernel.py diff --git a/packages/microcosm-graph/src/microcosm/graph/executor.py b/packages/microcosm-graph/src/microcosm/graph/executor.py index c9687beb9..aec8a3161 100644 --- a/packages/microcosm-graph/src/microcosm/graph/executor.py +++ b/packages/microcosm-graph/src/microcosm/graph/executor.py @@ -17,7 +17,7 @@ import numpy as np import pandas as pd -from microcosm.frame import Frame, WeightKind, Weights +from microcosm.frame import Frame, MassChangeRecord, WeightKind, Weights from . import keys as graph_keys from .artifact_edges import scope_payload, typed_contracts, value_from_descriptor @@ -561,7 +561,17 @@ def _project_context( tolerances: Mapping[tuple[str, str], Tolerance | None], numerics: Mapping[tuple[str, str], NumericScope], artifacts: Mapping[str, ArtifactValue] | None = None, + mass_log: tuple[MassChangeRecord, ...] = (), ) -> KernelContext: + """Project one node's frozen inputs out of the population it reads. + + ``mass_log`` is the ``Frame`` mass log this node's *key* binds, which is + not in general the cumulative log carried by ``population``: see the + boundary selection in :func:`run_graph`. It defaults to the empty log, + so a caller that projects a context outside the executor states no mass + history rather than inheriting one it never bound (amendment 26). + """ + if population is None: return KernelContext( node=node, @@ -675,7 +685,7 @@ def _project_context( sources=MappingProxyType({name: sources[name] for name in node.sources}), artifacts={} if artifacts is None else artifacts, frame_metadata=frame.metadata, - frame_mass_log=frame.mass_log, + frame_mass_log=mass_log, frame_column_order=MappingProxyType(column_order), tolerances=tolerances, numerics=numerics, @@ -2297,6 +2307,11 @@ def run_graph( _preflight_require(compiled, store, keys, implementations, kernels) populations: dict[str, Population] = {} + # Each structural version's ``Frame`` mass log as that version was + # admitted. ``populations`` cannot answer this: an ordinary member + # replaces its version's entry there, so by the time a later member runs + # the boundary state is gone. + boundary_mass_logs: dict[str, tuple[MassChangeRecord, ...]] = {} receipts: dict[str, NodeReceipt] = {} receipt_payloads: dict[str, Mapping[str, object]] = {} for node_id in compiled.order: @@ -2417,6 +2432,25 @@ def run_graph( ) for binding in node.artifact_inputs } + if incumbent is None: + incoming_mass_log: tuple[MassChangeRecord, ...] = () + elif node.structural is StructuralDelta.NONE: + # An ordinary node's key binds its version's structural + # boundary and the owners of the columns it declared + # (``population_input`` and ``input_artifacts`` in keys.py) -- + # never a sibling that merely ran earlier in the same version + # and appended a mass record. Projecting the cumulative log + # would hand the kernel an input its own key does not bind, so + # adding or re-parameterising that sibling would change what a + # node sees while its key, and therefore its cache entry, + # stayed put. It is projected from the boundary instead. + incoming_mass_log = boundary_mass_logs[compiled.versions[node_id]] + else: + # A structural node's key binds its base *and* every ordinary + # member of that version (``members`` in keys.py, built from + # ``compiled.predecessors``), so the base version's cumulative + # log is bound by the key that will name its cache entry. + incoming_mass_log = incumbent.frame.mass_log context = _project_context( node, incumbent, @@ -2425,6 +2459,7 @@ def run_graph( tolerances=input_tolerances, numerics=input_numerics, artifacts=artifact_values, + mass_log=incoming_mass_log, ) before = _context_digest(context) try: @@ -2570,6 +2605,9 @@ def run_graph( populations[compiled.versions[node_id]] = updated else: populations[node.id] = updated + # Captured at admission, so cold execution and a restored cache + # hit record the same boundary: both reach this line. + boundary_mass_logs[node.id] = updated.frame.mass_log if _population_observer is not None: _population_observer(node_id, _observer_snapshot(updated)) diff --git a/packages/microcosm-graph/src/microcosm/graph/kernel.py b/packages/microcosm-graph/src/microcosm/graph/kernel.py index 528d11836..f718c5f48 100644 --- a/packages/microcosm-graph/src/microcosm/graph/kernel.py +++ b/packages/microcosm-graph/src/microcosm/graph/kernel.py @@ -353,11 +353,19 @@ class KernelContext: ``Frame`` has already deeply frozen; this class adds a read-only view over that mapping and does not itself deep-freeze a mapping built some other way (amendment 26). - frame_mass_log: The population version's ``Frame`` mass records, in - order. This is the *incoming* log: a node that needs a stage's - completed records must run after that stage's structural - boundary or read its predecessor's evidence, because incidental - node order is not authority (amendment 26). + frame_mass_log: The ``Frame`` mass records this node's key binds, + in order. For an ordinary node that is its population version's + *boundary* log -- the log as that version was admitted -- so a + record another member of the same version appends afterwards is + not visible here: an ordinary node's key binds its version's + structural boundary and the owners of the columns it declared, + not its siblings, and a view the key does not bind could not + survive a cache hit. A structural node receives its base + version's cumulative log, which its key does bind, through that + base and that version's members. A node that needs a stage's + completed records therefore runs after that stage's structural + boundary or reads its predecessor's evidence; incidental node + order is not authority (amendment 26). frame_column_order: Entity to the population version's own column order, restricted to the columns projected into ``tables``. The executor projects ``tables`` in declaration order, so this diff --git a/packages/microcosm-graph/tests/test_graph_frame_context.py b/packages/microcosm-graph/tests/test_graph_frame_context.py index 3f9634389..0f0725e31 100644 --- a/packages/microcosm-graph/tests/test_graph_frame_context.py +++ b/packages/microcosm-graph/tests/test_graph_frame_context.py @@ -21,7 +21,7 @@ import pandas as pd import pytest -from microcosm.frame import Frame, MassChangeRecord +from microcosm.frame import Frame, MassChangeRecord, WeightKind, Weights from microcosm.graph import ( ContentStore, Graph, @@ -30,6 +30,8 @@ Node, Owned, Slice, + StructuralDelta, + WeightTransition, compile_graph, run_graph, ) @@ -44,6 +46,7 @@ SOURCE_REF = "source.metadata@1" PROBE_REF = "probe.frame_context@1" +REWEIGHT_PROBE_REF = "probe.frame_context_reweight@1" #: Metadata the CREATE kernel puts on the version, including a nested value #: so the frozen projection is exercised rather than a flat string map. @@ -72,6 +75,22 @@ def compute(self, context: KernelContext) -> KernelResult: ) +def observed(context: KernelContext) -> dict[str, object]: + """Everything a property asserts about one node's frame view.""" + return { + "node": context.node.id, + "column_order": { + entity: tuple(columns) + for entity, columns in context.frame_column_order.items() + }, + "projected": { + entity: tuple(table.columns) for entity, table in context.tables.items() + }, + "metadata": dict(context.frame_metadata), + "mass_log": tuple(context.frame_mass_log), + } + + class FrameContextProbe(toy.ToyKernel): """Owns one column and records the frame view it was handed. @@ -85,25 +104,14 @@ def __init__(self, ref: str, capabilities, *, variant: str = "base") -> None: self.seen: list[dict[str, object]] = [] def compute(self, context: KernelContext) -> KernelResult: - self.seen.append( - { - "node": context.node.id, - "column_order": { - entity: tuple(columns) - for entity, columns in context.frame_column_order.items() - }, - "projected": { - entity: tuple(table.columns) - for entity, table in context.tables.items() - }, - "metadata": dict(context.frame_metadata), - "mass_log": tuple(context.frame_mass_log), - } - ) + self.seen.append(observed(context)) ids = pd.Index(context.tables["person"]["person_id"], name="person_id") + # The output *is* the mass-log view, so the stored bytes of a node + # differ whenever what it was shown differs. A key that survives an + # unrelated sibling therefore has to have been shown the same log. result_columns = { ("person", str(context.params["target"])): pd.Series( - 1.0, index=ids, dtype="float64" + float(len(context.frame_mass_log)), index=ids, dtype="float64" ) } reason = context.params.get("mass_reason") @@ -126,12 +134,39 @@ def compute(self, context: KernelContext) -> KernelResult: ) +class ReweightProbe(toy.ToyKernel): + """REWEIGHT: records the frame view it was handed, then scales weights. + + A structural node, unlike an ordinary one, has a key that binds its + base *and* every ordinary member of that version, so it is the node + that may be shown the version's cumulative mass log. + """ + + def __init__(self, ref: str, capabilities, *, variant: str = "base") -> None: + super().__init__(ref, capabilities, variant=variant) + self.seen: list[dict[str, object]] = [] + + def compute(self, context: KernelContext) -> KernelResult: + self.seen.append(observed(context)) + before = context.weights["household"].values + after = before * 2.0 + return KernelResult( + weights=Weights(values=after, kind=WeightKind.IMPORTANCE), + receipt={"mass": toy._mass_record(context, before, after, "free")}, + ) + + def build_registry() -> tuple[object, FrameContextProbe]: - """The toy registry with the metadata source and the probe registered.""" + """The toy registry with the metadata source and both probes registered. + + The structural probe is reachable as ``registry.get(REWEIGHT_PROBE_REF)`` + for the properties that are about a boundary rather than a member. + """ registry = toy.toy_registry() registry.register(MetadataSource(SOURCE_REF, toy._CREATE)) probe = FrameContextProbe(PROBE_REF, toy._DETERMINISTIC) registry.register(probe) + registry.register(ReweightProbe(REWEIGHT_PROBE_REF, toy._REWEIGHT)) return registry, probe @@ -144,6 +179,7 @@ def probe_node( columns: tuple[str, ...], target: str, mass_reason: str | None = None, + population: str = "survey", ) -> Node: """A probe reading ``columns`` of the person entity, in that order. @@ -160,7 +196,21 @@ def probe_node( inputs=inputs, outputs=(Owned("person", target, "float64"),), params={"target": target, "mass_reason": mass_reason}, - population="survey", + population=population, + ) + + +def reweight_probe_node(node_id: str = "boundary", *, base: str = "survey") -> Node: + """A structural boundary that observes the log it is handed.""" + return Node( + node_id, + REWEIGHT_PROBE_REF, + structural=StructuralDelta.REWEIGHT, + base=base, + inputs=(Slice("person", ("age",)), Slice("household", ("household_size",))), + weights=WeightTransition("household", "importance", mass="free"), + mass="free", + description="design -> importance, observing the frame view", ) @@ -343,15 +393,241 @@ def test_version_metadata_reaches_every_node_of_the_version(tmp_path: Path) -> N assert tuple(metadata["vintage"]) == ("frs", "was") -def test_mass_log_is_the_incoming_log(tmp_path: Path) -> None: - """The appending node sees the log before its own record; its successor after.""" - _, probe, _, _ = run_probe(tmp_path / "run") - assert observation(probe, "probe_first")["mass_log"] == () - after = observation(probe, "probe_second")["mass_log"] - assert len(after) == 1 - assert after[0].entity == "household" - assert after[0].reason == "households are unchanged by a derivation" - assert after[0].old_total == after[0].new_total +# ---------------------------------------------------------------------- +# The mass log is the one the node's key binds +# ---------------------------------------------------------------------- + + +#: One ordinary member that appends a ``Frame`` mass record, and one that +#: reads a different column of the same version and declares nothing of the +#: appender's. ``compile_graph`` orders equal-depth nodes by id, so +#: ``appender`` runs first and its record is in the version's cumulative log +#: by the time ``unrelated`` is projected. +APPEND_REASON = "households are unchanged by a derivation" + + +def sibling_graph(*, appender: bool = True, reason: str = APPEND_REASON) -> Graph: + """``unrelated`` beside an optional, unread same-version mass appender.""" + unrelated = probe_node("unrelated", columns=("income",), target="unrelated_a") + if not appender: + return Graph("toy", (toy.SOURCE,), (CREATE, unrelated)) + return Graph( + "toy", + (toy.SOURCE,), + ( + CREATE, + probe_node( + "appender", + columns=("age",), + target="appender_a", + mass_reason=reason, + ), + unrelated, + ), + ) + + +def boundary_graph() -> Graph: + """An appender, a structural boundary over it, and a member of the new version.""" + return Graph( + "toy", + (toy.SOURCE,), + ( + CREATE, + probe_node( + "appender", + columns=("age",), + target="appender_a", + mass_reason=APPEND_REASON, + ), + reweight_probe_node(), + probe_node( + "after_boundary", + columns=("age",), + target="after_a", + population="boundary", + ), + ), + ) + + +def boundary_graph_with_reason(reason: str) -> Graph: + """``boundary_graph`` with the appender stating a different purpose.""" + return Graph( + "toy", + (toy.SOURCE,), + tuple( + probe_node( + "appender", + columns=("age",), + target="appender_a", + mass_reason=reason, + ) + if node.id == "appender" + else node + for node in boundary_graph().nodes + ), + ) + + +def unrelated_bytes(store: ContentStore, manifest) -> tuple[bytes, bytes]: + """The stored bytes of ``unrelated``'s output column.""" + key = manifest.nodes["unrelated"].artifacts[("person", "unrelated_a")] + return toy.artifact_bytes(store, key) + + +def only_record(view: object) -> MassChangeRecord: + """The single mass record in an observed view, asserted to be alone.""" + log = tuple(view) # type: ignore[call-overload] + assert len(log) == 1 + return log[0] + + +def test_an_ordinary_node_sees_its_versions_boundary_log(tmp_path: Path) -> None: + """Not the cumulative log, and specifically not a sibling's record. + + ``appender`` runs first and appends one record to the ``survey`` + version. ``unrelated`` declares no column of the appender's, so its key + binds ``survey``'s boundary and nothing else; showing it the record + would be showing it an input its key does not bind. + """ + _, probe, _, _ = run_probe(tmp_path / "run", graph=sibling_graph()) + assert observation(probe, "appender")["mass_log"] == () + assert observation(probe, "unrelated")["mass_log"] == () + + +def test_the_appenders_own_record_reaches_the_version_it_leaves( + tmp_path: Path, +) -> None: + """The record is applied — it is the *view* that is bounded, not the log.""" + manifest, _, _, _ = run_probe(tmp_path / "run", graph=sibling_graph()) + record = only_record(manifest.population("survey").mass_log) + assert record.entity == "household" + assert record.reason == APPEND_REASON + assert record.old_total == record.new_total + + +def test_a_keyed_structural_predecessor_does_see_the_cumulative_log( + tmp_path: Path, +) -> None: + """A structural node's key binds its base *and* that version's members. + + That is the difference the projection turns on: the boundary may be + shown what an ordinary member may not, because re-parameterising the + appender moves the boundary's key and so cannot be replayed onto it. + """ + registry, probe = build_registry() + manifest, _, sources, _ = run_probe( + tmp_path / "run", graph=boundary_graph(), registry=registry, probe=probe + ) + boundary = registry.get(REWEIGHT_PROBE_REF) + record = only_record(observation(boundary, "boundary")["mass_log"]) + assert record.reason == APPEND_REASON + + other_registry, _ = build_registry() + moved = run_graph( + compile_graph(boundary_graph_with_reason("a different stated reason")), + sources=dict(sources), + store=ContentStore(tmp_path / "moved" / "store"), + kernels=other_registry, + ) + assert moved.nodes["boundary"].key != manifest.nodes["boundary"].key + assert moved.nodes["after_boundary"].key != manifest.nodes["after_boundary"].key + + +def test_a_member_of_the_next_version_sees_the_carried_boundary_log( + tmp_path: Path, +) -> None: + """The boundary carries the record forward, so its own members do see it.""" + registry, probe = build_registry() + run_probe(tmp_path / "run", graph=boundary_graph(), registry=registry, probe=probe) + record = only_record(observation(probe, "after_boundary")["mass_log"]) + assert record.reason == APPEND_REASON + + +def test_a_restored_boundary_log_is_the_one_a_cold_member_sees( + tmp_path: Path, +) -> None: + """The boundary survives the store round trip, not only the computed frame.""" + registry, probe = build_registry() + _, _, sources, _ = run_probe( + tmp_path / "run", graph=boundary_graph(), registry=registry, probe=probe + ) + expected = observation(probe, "after_boundary") + + later = probe_node( + "later", columns=("age",), target="later_a", population="boundary" + ) + warm_registry, warm_probe = build_registry() + warm = run_graph( + compile_graph( + Graph("toy", (toy.SOURCE,), (*boundary_graph().nodes, later)), + ), + sources=dict(sources), + store=ContentStore(tmp_path / "run" / "store"), + kernels=warm_registry, + ) + assert {n for n, r in warm.nodes.items() if not r.hit} == {"later"} + assert observation(warm_probe, "later")["mass_log"] == expected["mass_log"] + + +def test_an_unrelated_appender_changes_neither_the_key_nor_the_view( + tmp_path: Path, +) -> None: + """Same key, same truth: cold with the sibling equals cold without it. + + This is the property the boundary rule exists for. ``unrelated``'s + output column *is* its mass-log view, so equal stored bytes mean it was + shown the same log in both graphs — and its key is the same, so the + store would serve either result for the other. + """ + with_manifest, with_probe, sources, with_store = run_probe( + tmp_path / "with", graph=sibling_graph() + ) + without_manifest, without_probe, _, without_store = run_probe( + tmp_path / "without", graph=sibling_graph(appender=False), sources=sources + ) + assert ( + with_manifest.nodes["unrelated"].key == without_manifest.nodes["unrelated"].key + ) + assert observation(with_probe, "unrelated")["mass_log"] == () + assert observation(without_probe, "unrelated")["mass_log"] == () + assert unrelated_bytes(with_store, with_manifest) == unrelated_bytes( + without_store, without_manifest + ) + + +def test_required_replay_agrees_when_the_sibling_is_added(tmp_path: Path) -> None: + """A store written with the appender serves the graph without it.""" + _, _, sources, _ = run_probe(tmp_path / "run", graph=sibling_graph()) + registry, probe = build_registry() + warm = run_graph( + compile_graph(sibling_graph(appender=False)), + sources=dict(sources), + store=ContentStore(tmp_path / "run" / "store"), + kernels=registry, + resume="require", + ) + assert all(receipt.hit for receipt in warm.nodes.values()) + assert probe.seen == [] + + +def test_required_replay_agrees_when_the_sibling_is_reparameterized( + tmp_path: Path, +) -> None: + """Re-stating the appender's reason moves its key, and no other node's.""" + first_manifest, _, sources, _ = run_probe(tmp_path / "run", graph=sibling_graph()) + registry, probe = build_registry() + second = run_graph( + compile_graph(sibling_graph(reason="a different stated reason")), + sources=dict(sources), + store=ContentStore(tmp_path / "run" / "store"), + kernels=registry, + ) + assert second.nodes["appender"].key != first_manifest.nodes["appender"].key + assert second.nodes["unrelated"].key == first_manifest.nodes["unrelated"].key + assert {n for n, r in second.nodes.items() if not r.hit} == {"appender"} + assert [item["node"] for item in probe.seen] == ["appender"] # ---------------------------------------------------------------------- From 6ec46d62c33ed89d8f9923326ea10b43070d5468 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 09:29:10 -0400 Subject: [PATCH 08/17] F3/F4: detach the frame view, and check it for mutation (amendment 26) `Frame` deeply freezes its metadata and its mass records, but a frozen dataclass still yields to `object.__setattr__`. Passing the version's own `_FrozenMapping` leaves and `MassChangeRecord`s by reference therefore made every kernel -- and anything retaining a context past its own mutation check -- a live handle on the population: one `object.__setattr__` on a shared record rewrote the version's log, and unlike a table nothing would have noticed. The projection now hands out a deep copy of the metadata and rebuilt mass records, through the same rule `_observer_snapshot` already followed (now one shared `_detached_record`). `_context_digest` binds all three fields: the metadata through the frame format's own store codec, each mass record field by field, and the projected column order. The codec is guarded, so a value `Frame` would never have admitted is reported as the mutation it is rather than raising while the comparison is computed. The properties cover both halves, and are not vacuous: the rewrite is asserted to have landed on the retained copy before the live version and the next node are asserted unchanged. The observer property now rewrites nested metadata and mass records too -- rewriting tables alone never touched these fields. Co-Authored-By: Claude Opus 5 --- ...red-graph-contracts-frame-context.added.md | 2 +- docs/graph-acceptance.md | 15 ++ docs/graph-interface.lock | 2 +- .../src/microcosm/graph/executor.py | 94 +++++-- .../src/microcosm/graph/kernel.py | 21 +- .../tests/test_graph_frame_context.py | 254 +++++++++++++++++- 6 files changed, 355 insertions(+), 33 deletions(-) diff --git a/changelog.d/uk-shared-graph-contracts-frame-context.added.md b/changelog.d/uk-shared-graph-contracts-frame-context.added.md index a66fead06..b70127ecc 100644 --- a/changelog.d/uk-shared-graph-contracts-frame-context.added.md +++ b/changelog.d/uk-shared-graph-contracts-frame-context.added.md @@ -1 +1 @@ -Added `KernelContext.frame_metadata`, `frame_mass_log` and `frame_column_order`, so a kernel can reconstruct its population version's declared slices without seeing any column it did not declare. The mass log a node receives is the one its key binds: its version's structural boundary for an ordinary node, the base version's cumulative log for a structural one (graph amendment 26). +Added `KernelContext.frame_metadata`, `frame_mass_log` and `frame_column_order`, so a kernel can reconstruct its population version's declared slices without seeing any column it did not declare. The mass log a node receives is the one its key binds: its version's structural boundary for an ordinary node, the base version's cumulative log for a structural one. All three are detached from the live population before a kernel sees them, and all three enter the executor's input-mutation check (graph amendment 26). diff --git a/docs/graph-acceptance.md b/docs/graph-acceptance.md index 936dbda46..d0aa9ce75 100644 --- a/docs/graph-acceptance.md +++ b/docs/graph-acceptance.md @@ -597,6 +597,21 @@ lock unchanged: was not: a column *name* is itself information about the version, and B1's "nothing else is visible" covers names as well as values. + All three are detached before a kernel sees them and are covered by + B4's before/after mutation check. Detachment is not redundant with + `Frame`'s freezing: a frozen dataclass still yields to + `object.__setattr__`, so passing the version's own `_FrozenMapping` + leaves and `MassChangeRecord`s by reference would make every kernel — and + anything that retains a context past its own mutation check — a live + handle on the population. The executor therefore hands out a deep copy + of the metadata and rebuilt mass records (the rule `_observer_snapshot` + already followed, now shared), and `_context_digest` binds the metadata + through the frame format's own store codec, the mass records field by + field, and the projected column order. Neither is a substitute for the + other: the digest catches a kernel whose output stops being a function + of its declared inputs, while detachment is what stops a retained view + from rewriting the live version after that check has passed. + The three ride after `artifacts` and before `tolerances`, so amendment 17's statement that `numerics` rides at the end of the context stays literally true and amendment 19's that `artifacts` rides before the diff --git a/docs/graph-interface.lock b/docs/graph-interface.lock index cb01bf926..578315abd 100644 --- a/docs/graph-interface.lock +++ b/docs/graph-interface.lock @@ -1,2 +1,2 @@ e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933 decl.py -a49bb4461e7928118b05c13351a99b7585e5068a0868f6bc40634ebb057f29ca kernel.py +2df6cc5b5b396adae578f885aedbd038d2c0e51f56b4a31b64c2f6b69e1b918d kernel.py diff --git a/packages/microcosm-graph/src/microcosm/graph/executor.py b/packages/microcosm-graph/src/microcosm/graph/executor.py index aec8a3161..8c15d6b57 100644 --- a/packages/microcosm-graph/src/microcosm/graph/executor.py +++ b/packages/microcosm-graph/src/microcosm/graph/executor.py @@ -78,6 +78,7 @@ StoreCorrupt, StoreMiss, StoreUnavailable, + _encode_frame_metadata, ) __all__ = ["NodeRejected", "NodeRejectedError", "run_graph"] @@ -351,6 +352,25 @@ def _freeze_frame(table: pd.DataFrame) -> pd.DataFrame: return frozen +def _detached_record(record): + """Rebuild one frozen record with independently copied field values. + + Reconstructed through the dataclass constructor rather than by a + reflective copy of its namespace, because source identities may seal + these classes. A frozen dataclass is not immune to ``object.__setattr__``, + so a record handed out by reference is a live handle on whatever holds + it; every hand-out goes through here. + """ + + return replace( + record, + **{ + field.name: deepcopy(getattr(record, field.name)) + for field in fields(record) + }, + ) + + def _observer_snapshot(population: Population) -> Population: """Detach every observation from the executable population and its cache. @@ -369,22 +389,12 @@ def _observer_snapshot(population: Population) -> Population: tables = {name: frame.table(name) for name in frame.entities} tables.update({name: frame.link(name) for name in frame.links}) tables, strata = pickle.loads(pickle.dumps((tables, frame.strata), protocol=5)) - - def copied_record(record): - return replace( - record, - **{ - field.name: deepcopy(getattr(record, field.name)) - for field in fields(record) - }, - ) - snapshot = Frame( tables, replace( frame.schema, group_entities=deepcopy(frame.schema.group_entities), - links=tuple(copied_record(link) for link in frame.schema.links), + links=tuple(_detached_record(link) for link in frame.schema.links), ), { entity: Weights( @@ -393,7 +403,9 @@ def copied_record(record): for entity in frame.weighted_entities }, strata, - mass_log=tuple(copied_record(record) for record in frame.mass_log), + mass_log=tuple(_detached_record(record) for record in frame.mass_log), + # ``Frame.__init__`` re-freezes metadata, which rebuilds every nested + # mapping, so the snapshot shares no metadata object with its parent. metadata=frame.metadata, ) return Population( @@ -401,7 +413,9 @@ def copied_record(record): population.version, dict(population.owners), dict(population.weight_kind), - mass_ledger=tuple(copied_record(record) for record in population.mass_ledger), + mass_ledger=tuple( + _detached_record(record) for record in population.mass_ledger + ), design_weights=population.design_weights, ) @@ -485,9 +499,54 @@ def _context_digest(context: KernelContext) -> bytes: ) digest.update(len(value.payload).to_bytes(8, "little")) digest.update(value.payload) + # The amendment-26 frame fields are kernel inputs like any other, so B4's + # before/after comparison covers them too. They are handed out detached + # (`_project_context`), so a kernel that rewrites one cannot reach the + # live version -- but it can still make its own node's output a function + # of something other than what it was given, and that is what this + # catches. + digest.update(b"frame-metadata\0") + digest.update(_frame_metadata_payload(context.frame_metadata)) + digest.update(b"frame-column-order\0") + for entity, columns in context.frame_column_order.items(): + _update_scalar(digest, entity) + for column in columns: + _update_scalar(digest, column) + digest.update(b"frame-mass-log\0") + for record in context.frame_mass_log: + for value in ( + getattr(record, "entity", None), + getattr(record, "old_total", None), + getattr(record, "new_total", None), + getattr(record, "declared_factor", None), + getattr(record, "reason", None), + ): + _update_scalar(digest, value) return digest.digest() +#: What a metadata tree digests to when the store codec cannot encode it. +#: Reached only after a kernel has replaced a value with something ``Frame`` +#: would never have admitted, which is itself the mutation being reported. +_UNCODABLE_FRAME_METADATA = b"" + + +def _frame_metadata_payload(metadata: Mapping[str, object]) -> bytes: + """Digest bytes for a frame-metadata view, without trusting its contents. + + ``store._encode_frame_metadata`` is the codec the frame format already + uses, so an unchanged view digests exactly as it persists. It is a codec + for *valid* metadata, though, and this runs a second time after a kernel + has held the object: it has to report a mutation rather than raise while + computing the comparison that would report it. + """ + + try: + return canonical_json(_encode_frame_metadata(metadata)) + except (TypeError, ValueError, RecursionError): + return _UNCODABLE_FRAME_METADATA + + def _structural_columns(frame: Frame, entity: str) -> list[str]: columns = [frame.schema.entity_id_column(entity)] if entity == frame.schema.person_entity: @@ -684,8 +743,13 @@ def _project_context( rng=np.random.default_rng(seed(key)), sources=MappingProxyType({name: sources[name] for name in node.sources}), artifacts={} if artifacts is None else artifacts, - frame_metadata=frame.metadata, - frame_mass_log=mass_log, + # Detached before the kernel sees them: a frozen dataclass still + # yields to ``object.__setattr__``, so passing the version's own + # ``_FrozenMapping`` leaves and mass records by reference would make + # every kernel -- and anything that retains a context past its own + # mutation check -- a live handle on the population (amendment 26). + frame_metadata=deepcopy(frame.metadata), + frame_mass_log=tuple(_detached_record(record) for record in mass_log), frame_column_order=MappingProxyType(column_order), tolerances=tolerances, numerics=numerics, diff --git a/packages/microcosm-graph/src/microcosm/graph/kernel.py b/packages/microcosm-graph/src/microcosm/graph/kernel.py index f718c5f48..a402e5b37 100644 --- a/packages/microcosm-graph/src/microcosm/graph/kernel.py +++ b/packages/microcosm-graph/src/microcosm/graph/kernel.py @@ -349,10 +349,12 @@ class KernelContext: nominal type does not itself verify serialized data (amendment 19). frame_metadata: The population version's own metadata. The - executor passes :attr:`microcosm.frame.Frame.metadata`, which - ``Frame`` has already deeply frozen; this class adds a - read-only view over that mapping and does not itself deep-freeze - a mapping built some other way (amendment 26). + executor passes a deep copy of + :attr:`microcosm.frame.Frame.metadata`, so the view is detached + from the live version as well as deeply frozen by ``Frame``; + this class adds a read-only view over that mapping and does not + itself deep-freeze or copy a mapping built some other way + (amendment 26). frame_mass_log: The ``Frame`` mass records this node's key binds, in order. For an ordinary node that is its population version's *boundary* log -- the log as that version was admitted -- so a @@ -365,15 +367,20 @@ class KernelContext: base and that version's members. A node that needs a stage's completed records therefore runs after that stage's structural boundary or reads its predecessor's evidence; incidental node - order is not authority (amendment 26). + order is not authority. The executor hands out rebuilt records, + not the version's own: a frozen dataclass still yields to + ``object.__setattr__``, so a record passed by reference would be + a live handle on the population (amendment 26). frame_column_order: Entity to the population version's own column order, restricted to the columns projected into ``tables``. The executor projects ``tables`` in declaration order, so this is the only way to reconstruct the version's layout. It never names a column the node did not get: an entry that is not exactly an ordering of that table's columns is refused, so an - undeclared column cannot be smuggled in as a name (amendment - 26). + undeclared column cannot be smuggled in as a name. All three + frame fields are ordinary inputs: the executor's before/after + comparison covers them, so rewriting one is refused exactly as + rewriting a table is (amendment 26). tolerances: ``(entity, column)`` of each declared input column to the :class:`Tolerance` its owning kernel declared, or ``None`` for a bitwise owner. A gate compares against these. diff --git a/packages/microcosm-graph/tests/test_graph_frame_context.py b/packages/microcosm-graph/tests/test_graph_frame_context.py index 0f0725e31..f93fec76a 100644 --- a/packages/microcosm-graph/tests/test_graph_frame_context.py +++ b/packages/microcosm-graph/tests/test_graph_frame_context.py @@ -17,6 +17,7 @@ import importlib.util import sys from pathlib import Path +from types import MappingProxyType import pandas as pd import pytest @@ -28,6 +29,7 @@ KernelContext, KernelResult, Node, + NodeRejectedError, Owned, Slice, StructuralDelta, @@ -47,10 +49,21 @@ SOURCE_REF = "source.metadata@1" PROBE_REF = "probe.frame_context@1" REWEIGHT_PROBE_REF = "probe.frame_context_reweight@1" +ISOLATION_REF = "probe.frame_context_isolation@1" -#: Metadata the CREATE kernel puts on the version, including a nested value -#: so the frozen projection is exercised rather than a flat string map. -VERSION_METADATA = {"time_period": "2024", "vintage": ("frs", "was")} +#: What a tampering probe or observer writes, so an assertion can tell +#: "the rewrite never happened" from "the rewrite never escaped". +TAMPERED = "tampered" + +#: Metadata the CREATE kernel puts on the version, including a nested +#: mapping and a nested sequence, so the frozen projection is exercised +#: rather than a flat string map -- and so a property can rewrite a value +#: one level below the mapping the context hands out. +VERSION_METADATA = { + "time_period": "2024", + "vintage": ("frs", "was"), + "provenance": {"source": "frs", "year": 2024}, +} class MetadataSource(toy.ToyKernel): @@ -156,6 +169,76 @@ def compute(self, context: KernelContext) -> KernelResult: ) +def rewrite(context: KernelContext, field: str) -> None: + """Rewrite one of the three frame fields of ``context``, in place. + + This is the adversary the amendment defends against, so it uses the + capability a kernel actually has rather than a public setter: a frozen + dataclass yields to ``object.__setattr__``, and the nested metadata + leaves are frozen dataclasses. The nested field is found through + ``dataclasses.fields`` rather than named, so the property survives a + rename inside ``microcosm.frame``. + """ + + if field == "metadata": + nested = context.frame_metadata["provenance"] + object.__setattr__( + nested, dataclasses.fields(nested)[0].name, (("source", TAMPERED),) + ) + elif field == "mass_record": + object.__setattr__(context.frame_mass_log[0], "reason", TAMPERED) + elif field == "column_order": + object.__setattr__( + context, + "frame_column_order", + MappingProxyType( + { + entity: tuple(reversed(columns)) + for entity, columns in context.frame_column_order.items() + } + ), + ) + else: # pragma: no cover - guards the fixture itself + raise AssertionError(f"unknown frame field {field!r}") + + +class IsolationProbe(toy.ToyKernel): + """Retains the frame view it is handed, and rewrites views on request. + + ``retain`` keeps this node's own context. ``tamper`` rewrites every + context retained so far, which happens *after* those nodes' mutation + checks have already passed -- at that point only detachment keeps the + live version intact. ``tamper_self`` rewrites one field of this node's + own context instead, which the before/after comparison has to refuse. + """ + + def __init__(self, ref: str, capabilities, *, variant: str = "base") -> None: + super().__init__(ref, capabilities, variant=variant) + self.retained: list[KernelContext] = [] + self.seen: list[dict[str, object]] = [] + + def compute(self, context: KernelContext) -> KernelResult: + if context.params.get("tamper"): + for held in self.retained: + rewrite(held, "metadata") + rewrite(held, "mass_record") + if context.params.get("retain"): + self.retained.append(context) + self.seen.append(observed(context)) + ids = pd.Index(context.tables["person"]["person_id"], name="person_id") + result = KernelResult( + columns={ + ("person", str(context.params["target"])): pd.Series( + 1.0, index=ids, dtype="float64" + ) + } + ) + tamper_self = context.params.get("tamper_self") + if tamper_self is not None: + rewrite(context, str(tamper_self)) + return result + + def build_registry() -> tuple[object, FrameContextProbe]: """The toy registry with the metadata source and both probes registered. @@ -167,6 +250,7 @@ def build_registry() -> tuple[object, FrameContextProbe]: probe = FrameContextProbe(PROBE_REF, toy._DETERMINISTIC) registry.register(probe) registry.register(ReweightProbe(REWEIGHT_PROBE_REF, toy._REWEIGHT)) + registry.register(IsolationProbe(ISOLATION_REF, toy._DETERMINISTIC)) return registry, probe @@ -630,6 +714,133 @@ def test_required_replay_agrees_when_the_sibling_is_reparameterized( assert [item["node"] for item in probe.seen] == ["appender"] +# ---------------------------------------------------------------------- +# Isolation: the fields are detached, and rewriting one is refused +# ---------------------------------------------------------------------- + + +def isolation_node( + node_id: str, + *, + target: str, + retain: bool = False, + tamper: bool = False, + tamper_self: str | None = None, +) -> Node: + """One ordinary member of the ``boundary`` version, with a tamper role.""" + return Node( + node_id, + ISOLATION_REF, + inputs=(Slice("person", ("age",)),), + outputs=(Owned("person", target, "float64"),), + params={ + "target": target, + "retain": retain, + "tamper": tamper, + "tamper_self": tamper_self, + }, + population="boundary", + ) + + +def isolation_graph(*nodes: Node) -> Graph: + """``boundary_graph``'s appender and boundary, then the given members. + + The members sit in the ``boundary`` version because that is the version + whose boundary log is non-empty: a property about rewriting a mass + record needs a node that was actually handed one. + """ + return Graph( + "toy", + (toy.SOURCE,), + ( + CREATE, + probe_node( + "appender", + columns=("age",), + target="appender_a", + mass_reason=APPEND_REASON, + ), + reweight_probe_node(), + *nodes, + ), + ) + + +def run_isolation(root: Path, *nodes: Node): + """Run ``isolation_graph`` and return ``(manifest, isolation probe)``.""" + registry, probe = build_registry() + manifest, _, _, _ = run_probe( + root, graph=isolation_graph(*nodes), registry=registry, probe=probe + ) + return manifest, registry.get(ISOLATION_REF) + + +def test_a_retained_context_cannot_rewrite_the_live_version(tmp_path: Path) -> None: + """Detachment, stated where the mutation check cannot reach. + + ``iso_b_tamper`` rewrites the nested metadata and the mass record of a + context ``iso_a_retain`` was handed, after that node finished and its + own before/after comparison passed. The rewrite lands -- the assertions + on the retained view prove the property is not vacuous -- and reaches + neither the node that runs next nor the version itself. + """ + manifest, isolation = run_isolation( + tmp_path / "run", + isolation_node("iso_a_retain", target="iso_a", retain=True), + isolation_node("iso_b_tamper", target="iso_b", tamper=True), + isolation_node("iso_c_reader", target="iso_c"), + ) + (held,) = isolation.retained + assert held.frame_mass_log[0].reason == TAMPERED + assert held.frame_metadata["provenance"]["source"] == TAMPERED + + reader = observation(isolation, "iso_c_reader") + assert only_record(reader["mass_log"]).reason == APPEND_REASON + assert reader["metadata"]["provenance"]["source"] == "frs" + for version in ("survey", "boundary"): + assert manifest.population(version).metadata["provenance"]["source"] == "frs" + assert only_record(manifest.population("boundary").mass_log).reason == APPEND_REASON + + +def test_rewriting_nested_metadata_is_refused(tmp_path: Path) -> None: + """A leaf one level below the mapping the context hands out.""" + with pytest.raises(NodeRejectedError, match="mutated its input context"): + run_isolation( + tmp_path / "run", + isolation_node("iso_tamper", target="iso_t", tamper_self="metadata"), + ) + + +def test_rewriting_a_mass_record_is_refused(tmp_path: Path) -> None: + """The record is the node's input, not a description of one.""" + with pytest.raises(NodeRejectedError, match="mutated its input context"): + run_isolation( + tmp_path / "run", + isolation_node("iso_tamper", target="iso_t", tamper_self="mass_record"), + ) + + +def test_rewriting_the_projected_column_order_is_refused(tmp_path: Path) -> None: + """Reversing an order is a different claim about the version's layout.""" + with pytest.raises(NodeRejectedError, match="mutated its input context"): + run_isolation( + tmp_path / "run", + isolation_node("iso_tamper", target="iso_t", tamper_self="column_order"), + ) + + +def test_an_untouched_frame_view_still_passes_the_mutation_check( + tmp_path: Path, +) -> None: + """The digest additions do not make an ordinary node look mutated.""" + manifest, isolation = run_isolation( + tmp_path / "run", isolation_node("iso_quiet", target="iso_q") + ) + assert [item["node"] for item in isolation.seen] == ["iso_quiet"] + assert manifest.nodes["iso_quiet"].hit is False + + # ---------------------------------------------------------------------- # Replay # ---------------------------------------------------------------------- @@ -686,11 +897,16 @@ def test_a_new_node_over_restored_populations_sees_the_same_frame( def test_a_retained_mutating_observer_changes_nothing(tmp_path: Path) -> None: """Amendment 24 still holds across the new fields. - The observer keeps every snapshot and rewrites its tables and its - metadata view after the callback returns; what later nodes read through - the frame fields, and the run's identity, are unchanged. + The observer keeps every snapshot and rewrites its tables, its nested + metadata *and* its mass records after the callback returns. Table + mutation alone would not touch the amendment-26 fields at all, so it is + the metadata and mass-record rewrites that make this property about + them; the run is over ``boundary_graph`` because that is the graph + whose snapshots carry a mass record to rewrite. """ - plain_manifest, plain_probe, sources, _ = run_probe(tmp_path / "plain") + plain_manifest, plain_probe, sources, _ = run_probe( + tmp_path / "plain", graph=boundary_graph() + ) retained: list[object] = [] def observe(node_id: str, population) -> None: @@ -699,13 +915,33 @@ def observe(node_id: str, population) -> None: table = population.frame.table(entity) for column in table.columns: table.loc[:, column] = table[column].iloc[0] + for record in population.frame.mass_log: + object.__setattr__(record, "reason", TAMPERED) + nested = population.frame.metadata["provenance"] + object.__setattr__( + nested, dataclasses.fields(nested)[0].name, (("source", TAMPERED),) + ) observed_manifest, observed_probe, _, _ = run_probe( - tmp_path / "observed", sources=sources, observer=observe + tmp_path / "observed", + graph=boundary_graph(), + sources=sources, + observer=observe, ) assert len(retained) == len(observed_manifest.nodes) assert {n: r.key for n, r in observed_manifest.nodes.items()} == { n: r.key for n, r in plain_manifest.nodes.items() } - for node_id in ("probe_first", "probe_second"): + for node_id in ("appender", "after_boundary"): assert observation(observed_probe, node_id) == observation(plain_probe, node_id) + + # The rewrites landed on the snapshots, and on nothing else. + rewritten = [record for snapshot in retained for record in snapshot.frame.mass_log] + assert rewritten and all(record.reason == TAMPERED for record in rewritten) + assert all( + snapshot.frame.metadata["provenance"]["source"] == TAMPERED + for snapshot in retained + ) + version = observed_manifest.population("boundary") + assert version.metadata["provenance"]["source"] == "frs" + assert only_record(version.mass_log).reason == APPEND_REASON From 09827aec81155bf889632789cd2dba123215e14c Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 09:54:10 -0400 Subject: [PATCH 09/17] Record the resumed fix round and what it re-verified Co-Authored-By: Claude Opus 5 --- PROGRESS-uk-shared-graph-contracts.md | 50 +++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/PROGRESS-uk-shared-graph-contracts.md b/PROGRESS-uk-shared-graph-contracts.md index 2e0519236..b6b8810aa 100644 --- a/PROGRESS-uk-shared-graph-contracts.md +++ b/PROGRESS-uk-shared-graph-contracts.md @@ -204,3 +204,53 @@ and the CI group inventory only. - F3/F4: detach the exposed frame objects; digest all three fields. - F2: source-backed rejection plus the anchor-invariance tests. - F5/F6: correct the docs' motivating claim and the disclosure. + +## Fix round, resumed (2026-09-13) + +The first fix-round process died on an external API DNS error after two +commits. Nothing was reset; the checkout resumed clean at `6ec46d62c`. +`origin/main` re-fetched on resume: still `15ebde806`, branch 8 ahead / 0 +behind. Restrictions unchanged — no pytest, import, engine, native source, +install, publication or disallowed log; source, stdlib `ast`, `ruff` and +the CI group inventory only. + +### Verified on resume, not assumed from the commit messages + +- `executor.py` boundary selection: `boundary_mass_logs[node.id]` is + written at exactly one place — beside `populations[node.id] = updated` + in the structural arm of the admission step (`executor.py:2669-2674`), + which both a cold run and a restored hit reach — and read at exactly one + place, the `StructuralDelta.NONE` arm that projects a context + (`executor.py:2508`). Its key set is therefore a subset of + `populations`', so the ordinary lookup cannot miss a version whose + incumbent lookup succeeded. +- Key binding re-read at source: `keys.py:206-209` binds an ordinary + node's `population_input` to `frame_key(version_key)` only, and + `keys.py:214-225` binds a structural node's `base` *and* `members`. The + projection matches that split exactly. +- `store.py:1157-1174`/`1334-1368` round-trip `Frame.mass_log`, so a + restored boundary carries the same records a computed one does. +- Detachment: `Frame.__init__` calls `_freeze_metadata` + (`bundle.py:112`), and `_freeze_metadata_value` rebuilds every nested + mapping, tuple and frozenset (`bundle.py:1382-1405`), so the + `_observer_snapshot` metadata hand-off shares no mutable-by-`setattr` + object with its parent. `_project_context` deep-copies the metadata and + rebuilds every record through `_detached_record`. +- `_context_digest` additions cannot raise while computing the comparison + that reports a mutation: `canonical_json` raises only `TypeError` / + `ValueError` (`canonical.py:_json_value`), `_encode_frame_metadata` + only `TypeError` (`store.py:1060-1082`), and both are caught alongside + `RecursionError`. +- `_project_context`'s only other caller, + `packages/microcosm-build/tests/test_uk_uc_capital_coherence.py:223`, + passes no `mass_log` and so takes the documented empty default. +- `ruff check .` clean; `ruff format --check` clean on every touched file; + stdlib `ast` parses clean. + +### Next + +- F4b: the required-replay axis property is still the weak one the review + named. Strengthen it. +- F2: source-backed rejection plus the anchor-invariance properties. +- F5/F6: the unsupported `calibrate.adam` motivating claim, the + transition-only wording, and an accurate receipts disclosure. From f5aa65d408b6c65be37e4b0e1a31dc85aa729c79 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 10:01:30 -0400 Subject: [PATCH 10/17] F2/F4/F5/F6: state what an update does to design ancestry, and prove it The review's F2 asked for a design-kind `WeightUpdate` to re-anchor `Population.design_weights`, on the grounds that a later calibration cap is otherwise "off by k" and that an `EXPAND` afterwards leaves anchors "mixed". Read against the source, the first is the declared semantics and the second is not what the code does. An anchor is the design weight a row entered the population carrying. `Population.design_weights` is captured once, at CREATE (`Population.from_frame`), and afterwards only carried by stable entity id (`_carry_design_weights`), which `patch` hands to `Population.from_frame` explicitly so the re-derive-from-the-frame default is never taken. `_carry_design_weights` maps an EXPAND's *copied* rows back to their source row's original anchor; only a row with no lineage at all reads the current frame, because it has no earlier weight to be anchored on. So the values move and no anchor does -- for retained, cloned and entrant rows alike -- which the new properties assert by running the same EXPAND over an updated and an un-updated population and comparing the anchors. `max_weight_ratio` with `weight_anchor='design'` is therefore still relative to the original design weights, which is what its refusal has always said ("original design weight"). Re-anchoring would let an unrelated normalization silently widen every cap declared upstream of it by that normalization's factor. The review's own T2 case is asserted as the intended outcome: calibrated weights equal to design weights doubled by an update are refused at a cap of 1.5 and admitted exactly at 2.0, reporting a realized ratio of 2.0. The `WeightUpdate` docstring and amendment 25 now say all of this instead of "ancestry is untouched". F4: the required-replay property only showed that a hit succeeds. The stored record's axis binding is now rewritten to the same ids in reverse, and to a short axis, and each is refused on a run that must hit the record with no kernel called -- so the check runs on the replay path and rejects there. A third property re-files the record unchanged and still replays, so the two refusals are about the binding rather than about re-filing. The weak test is renamed to the claim it does support. F5: amendment 25 claimed a re-solve of an existing calibration as a case it covers. `calibrate.adam@1` emits no `receipt['weight_update']`, so that declaration would be refused as unverifiable; it is now marked a future consumer adaptation, in the amendment and in the two docstrings that repeated it. F6: the node mass-policy mismatch message said "weight transition's" on a path both declarations reach. `decl.py` is re-locked for the amendment-25 text. Co-Authored-By: Claude Opus 5 --- ...red-graph-contracts-weight-update.added.md | 2 +- docs/graph-acceptance.md | 34 ++- docs/graph-interface.lock | 2 +- .../src/microcosm/graph/decl.py | 29 ++- .../tests/test_graph_population.py | 238 ++++++++++++++++++ .../tests/test_graph_weight_update.py | 126 +++++++++- 6 files changed, 410 insertions(+), 21 deletions(-) diff --git a/changelog.d/uk-shared-graph-contracts-weight-update.added.md b/changelog.d/uk-shared-graph-contracts-weight-update.added.md index 2b5340f9d..76d148bf4 100644 --- a/changelog.d/uk-shared-graph-contracts-weight-update.added.md +++ b/changelog.d/uk-shared-graph-contracts-weight-update.added.md @@ -1 +1 @@ -Added `WeightUpdate`, a declared same-kind replacement of an entity's weight values, with `weight_update_receipt` binding the ordered entity axis the replacement values are positional against (graph amendment 25). +Added `WeightUpdate`, a declared same-kind replacement of an entity's weight values, with `weight_update_receipt` binding the ordered entity axis the replacement values are positional against, checked against the incumbent axis on cold execution and on every replay. An update replaces weight values only: design anchors, and so any `max_weight_ratio` declared against them, are unchanged (graph amendment 25). diff --git a/docs/graph-acceptance.md b/docs/graph-acceptance.md index d0aa9ce75..ef930562d 100644 --- a/docs/graph-acceptance.md +++ b/docs/graph-acceptance.md @@ -514,9 +514,9 @@ lock unchanged: 25. **A weight update that keeps its kind is declarable.** `WeightTransition` only ever moves a kind forward, so a stage that - recomputes weights it already holds — a sampling normalization, a - re-solve of an existing calibration — could not be declared at all, - and the only way to express it was to misdeclare a transition. + recomputes weights it already holds — a sampling normalization is the + case this was extracted for — could not be declared at all, and the + only way to express it was to misdeclare a transition. `WeightUpdate(entity, kind, reason, mass)` is that declaration and is deliberately narrower than a transition: the incumbent kind, the declared kind and the returned weights' kind must all be the same one; @@ -537,6 +537,34 @@ lock unchanged: re-enters the same function. A count mismatch, a missing binding and a binding against a different axis are each a rejection. + An update replaces weight *values* and does not re-anchor design + ancestry. `Population.design_weights` is captured once, at `CREATE` + (`Population.from_frame`), and afterwards only carried by stable entity + id (`_carry_design_weights`), which `patch` passes on explicitly so the + re-derive-from-the-frame default is never taken. A design-kind update + therefore leaves every existing row's anchor where it was; a later + `EXPAND`'s copied rows still inherit the anchor of the row they copy + rather than that row's current value; and `max_weight_ratio` with + `weight_anchor='design'` keeps the denominator it was written against — + the error text has always said "original design weight". A row admitted + with no ancestor is anchored on whatever design weight the `EXPAND` + installs for it, because it has no earlier weight to be anchored on; + that is the anchor definition applied to a row with no ancestry, not a + mixture. Re-anchoring on a same-kind update would instead let an + unrelated normalization silently widen every cap declared upstream of + it by that normalization's factor, which is a non-local change to an + already-declared contract. A stage that wants a cap against normalized + weights states the ratio it means. + + The shared calibration kernel does **not** consume this yet: + `calibrate.adam@1` emits no `receipt['weight_update']` + (`packages/microcosm-calibrate/src/microcosm/calibrate/kernels.py`), so + declaring a re-solve of an existing calibration as a `WeightUpdate` + would be refused by the axis check as unverifiable, and its own guard + still asks for a `WeightTransition` by name. Re-solving an existing + calibration through the shared kernel is a **future consumer + adaptation**, not a case this amendment already covers. + `WeightUpdate.to_kind` is a property, not a field, so the two declarations have disjoint field sets (`{entity, to_kind, mass}` and `{entity, kind, reason, mass}`) and a `WeightUpdate` can never diff --git a/docs/graph-interface.lock b/docs/graph-interface.lock index 578315abd..85920c480 100644 --- a/docs/graph-interface.lock +++ b/docs/graph-interface.lock @@ -1,2 +1,2 @@ -e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933 decl.py +11c2abd77f50389c6cd0b51e26cb3ebd5cbd0770ba96e9de1b53c71e9725eaa4 decl.py 2df6cc5b5b396adae578f885aedbd038d2c0e51f56b4a31b64c2f6b69e1b918d kernel.py diff --git a/packages/microcosm-graph/src/microcosm/graph/decl.py b/packages/microcosm-graph/src/microcosm/graph/decl.py index f866be049..a443f7f9b 100644 --- a/packages/microcosm-graph/src/microcosm/graph/decl.py +++ b/packages/microcosm-graph/src/microcosm/graph/decl.py @@ -360,8 +360,8 @@ class WeightUpdate: :class:`WeightTransition` only ever moves a weight *kind* forward, so a stage that recomputes the numbers of weights it already holds — a - sampling normalization, a re-solve of an existing calibration — cannot - be declared at all. This is that declaration, and it is deliberately + sampling normalization is the case this was extracted for — cannot be + declared at all. This is that declaration, and it is deliberately narrower than a transition (amendment 25): - The kind does not move. The executor checks the incumbent kind, the @@ -378,8 +378,23 @@ class WeightUpdate: executor recomputes that binding from the incumbent axis, on cold execution and on replay. - Design-weight ancestry is untouched: the executor carries the original - design anchors exactly as it does for any other node. + An update replaces weight *values*; it does not re-anchor design + ancestry, and it is not a way to redefine a calibration cap. A row's + design anchor is the design weight it entered the population carrying: + ``Population.design_weights`` is captured once, at ``CREATE`` + (``population.Population.from_frame``), and afterwards only carried by + stable entity id (``population._carry_design_weights``), which ``patch`` + passes on explicitly so the re-derive-from-the-frame default is never + taken. So a design-kind update leaves every existing row's anchor + exactly where it was, a later ``EXPAND``'s copied rows still inherit + the anchor of the row they copy rather than that row's current value, + and ``max_weight_ratio`` with ``weight_anchor='design'`` keeps the + denominator it was written against. A row admitted with no ancestor is + anchored on the design weight the ``EXPAND`` installs for it, whatever + that is — it has no earlier weight to be anchored on. Re-anchoring here + instead would let an unrelated normalization silently widen every cap + declared upstream of it by that normalization's factor; a stage that + wants a cap against normalized weights states the ratio it means. Attributes: entity: The entity whose explicit weights are replaced. @@ -566,9 +581,11 @@ def __post_init__(self) -> None: "WeightTransition or WeightUpdate." ) if self.mass != self.weights.mass: + # Reached by a WeightTransition and a WeightUpdate alike, so + # the text names neither (amendment 25). raise GraphError( - f"Node {self.id!r}: mass policy {self.mass!r} disagrees with its " - f"weight transition's {self.weights.mass!r}." + f"Node {self.id!r}: mass policy {self.mass!r} disagrees with " + f"its declared weight change's {self.weights.mass!r}." ) declared_inputs = {(s.entity, c) for s in self.inputs for c in s.columns} for s in self.inputs: diff --git a/packages/microcosm-graph/tests/test_graph_population.py b/packages/microcosm-graph/tests/test_graph_population.py index 3735edda9..52c7fdfc1 100644 --- a/packages/microcosm-graph/tests/test_graph_population.py +++ b/packages/microcosm-graph/tests/test_graph_population.py @@ -18,6 +18,7 @@ Slice, StructuralDelta, WeightTransition, + WeightUpdate, ) from microcosm.graph.kernel import KernelResult from microcosm.graph.population import ( @@ -36,6 +37,7 @@ weight_cap_receipt, ) from microcosm.graph.store import ContentStore, _encode_object_scalar +from microcosm.graph.weight_update import weight_update_receipt def _frame() -> Frame: @@ -1359,6 +1361,242 @@ def test_calibration_cap_stays_anchored_to_original_design_after_filter() -> Non ) +# ---------------------------------------------------------------------- +# Amendment 25: what a same-kind design update does to design ancestry +# ---------------------------------------------------------------------- + + +#: Households 10, 20 and 30 enter the toy population carrying these design +#: weights, so these are the anchors every cap below is stated against. +ORIGINAL_DESIGN = np.array([1.0, 2.0, 3.0]) + + +def _design_update_node(node_id: str = "normalize") -> Node: + """A REWEIGHT that replaces design weights without moving their kind.""" + return Node( + node_id, + "test@1", + structural=StructuralDelta.REWEIGHT, + base="source", + weights=WeightUpdate( + "household", + "design", + "normalize sampled source-family mass", + mass="declared", + ), + mass="declared", + ) + + +def _design_update_result(population: Population, *, factor: float) -> KernelResult: + """Scale every design weight by ``factor``, declaring the mass it moves. + + The declared totals are derived from the incumbent frame rather than + written down, because a uniform scaling multiplies every stratum's mass + by the same factor; a hand-copied number here would only be a second + chance to be wrong. + """ + frame = population.frame + before = frame.stratum_mass() + id_column = frame.schema.entity_id_column("household") + ids = frame.table("household")[id_column].tolist() + return KernelResult( + weights=Weights( + frame.weights_for("household").values * factor, WeightKind.DESIGN + ), + receipt={ + "weight_update": weight_update_receipt(ids), + "mass": { + "policy": "declared", + "before": float(before.sum()), + "after": float(before.sum()) * factor, + "stratum_before": {key: float(value) for key, value in before.items()}, + "stratum_after": { + key: float(value) * factor for key, value in before.items() + }, + }, + }, + ) + + +def _clone_and_entrant_expand_node(*, base: str) -> Node: + """An EXPAND that clones one household and admits one true entrant. + + The toy household table carries no data columns, so the entrant row + materializes nothing; ``expand_cells`` is empty exactly as it is for the + lineage EXPAND above. + """ + return Node( + "grow", + "test@1", + structural=StructuralDelta.EXPAND, + base=base, + params={ + "expand_cells": (), + "expand_weight_entity": "household", + "expand_weight_kind": "design", + }, + mass="free", + entrants=True, + ) + + +#: What the EXPAND declares for the household that has no ancestor. It is a +#: number the kernel states, not one derived from the incumbent weights, so +#: it is the same in every run below. +ENTRANT_DESIGN_WEIGHT = 7.0 + + +def _clone_and_entrant_expand_result(population: Population) -> KernelResult: + """Clone household 10 as 40 and admit 50 from nothing. + + The returned design weights are the incumbent ones with the clone's copy + of its source appended, then the entrant's declared weight. + """ + incumbent = population.frame.weights_for("household").values + return KernelResult( + expand={ + "person": pd.Series( + [], + index=pd.Index([], dtype="int64", name="person_id"), + dtype="int64", + ), + "household": pd.Series( + pd.array([10, pd.NA], dtype="Int64"), + index=pd.Index([40, 50], dtype="int64", name="household_id"), + ), + }, + weights=Weights( + np.array([*incumbent, incumbent[0], ENTRANT_DESIGN_WEIGHT]), + WeightKind.DESIGN, + ), + ) + + +def test_a_same_kind_design_update_leaves_the_original_anchors_invariant() -> None: + """New numbers, same kind, same ancestry. + + ``_apply_weight_update`` replaces the frame's design *values*; + ``_carry_design_weights`` carries ``Population.design_weights`` forward + by stable entity id, and ``patch`` passes the carried anchors to + ``Population.from_frame`` explicitly, so the default that would re-derive + anchors from the frame is never taken. An update is therefore not a + re-anchoring, and a cap declared against ``weight_anchor='design'`` + keeps the denominator it was written against. + """ + population = _population() + np.testing.assert_array_equal( + population.design_weights["household"], ORIGINAL_DESIGN + ) + + updated = patch( + population, _design_update_node(), _design_update_result(population, factor=2.0) + ) + + weights = updated.frame.weights_for("household") + assert weights.kind is WeightKind.DESIGN + np.testing.assert_array_equal(weights.values, ORIGINAL_DESIGN * 2.0) + np.testing.assert_array_equal(updated.design_weights["household"], ORIGINAL_DESIGN) + assert not updated.design_weights["household"].flags.writeable + + +def test_a_design_update_moves_no_anchor_for_retained_clone_or_entrant_rows() -> None: + """The same EXPAND over an updated and an un-updated population. + + Retained rows keep their own original anchor, a clone inherits the + anchor of the row it copies -- not that row's *current* design weight -- + and a row with no ancestor is anchored on the design weight the EXPAND + installs for it, which the kernel declares rather than derives. All + three agree across the two runs, so the update moved no anchor at all; + the frames' design weights are a factor apart, so the property is not + vacuous. + """ + plain = _population() + updated = patch( + plain, _design_update_node(), _design_update_result(plain, factor=2.0) + ) + np.testing.assert_array_equal( + updated.frame.weights_for("household").values, ORIGINAL_DESIGN * 2.0 + ) + + over_updated = patch( + updated, + _clone_and_entrant_expand_node(base="normalize"), + _clone_and_entrant_expand_result(updated), + ) + over_plain = patch( + plain, + _clone_and_entrant_expand_node(base="source"), + _clone_and_entrant_expand_result(plain), + ) + + expected = np.array([1.0, 2.0, 3.0, 1.0, ENTRANT_DESIGN_WEIGHT]) + np.testing.assert_array_equal(over_updated.design_weights["household"], expected) + np.testing.assert_array_equal(over_plain.design_weights["household"], expected) + np.testing.assert_array_equal( + over_updated.frame.table("household")["household_id"], + np.array([10, 20, 30, 40, 50]), + ) + # Not vacuous: the two versions' design *values* differ by the factor for + # every row that existed before the update, and the clone copies the + # updated value while inheriting the original anchor. + np.testing.assert_array_equal( + over_updated.frame.weights_for("household").values, + np.array([2.0, 4.0, 6.0, 2.0, ENTRANT_DESIGN_WEIGHT]), + ) + np.testing.assert_array_equal( + over_plain.frame.weights_for("household").values, + np.array([1.0, 2.0, 3.0, 1.0, ENTRANT_DESIGN_WEIGHT]), + ) + + +def test_a_design_update_does_not_move_the_calibration_cap_denominator() -> None: + """``max_weight_ratio`` stays relative to the original design weights. + + Calibrated weights equal to the *updated* design weights are twice the + original anchors, so a cap of 1.5 refuses them and a cap of 2.0 admits + them exactly at the limit. Re-anchoring on the update would instead make + the same numbers a ratio of 1.0, silently widening every cap declared + upstream of an unrelated normalization by that normalization's factor. + """ + population = _population() + updated = patch( + population, _design_update_node(), _design_update_result(population, factor=2.0) + ) + normalized = updated.frame.weights_for("household").values + + def calibrated_node(cap: float) -> Node: + return Node( + "calibrated", + "test@1", + structural=StructuralDelta.REWEIGHT, + base="normalize", + params={"max_weight_ratio": cap, "weight_anchor": "design"}, + weights=WeightTransition("household", "calibrated", mass="free"), + mass="free", + ) + + with pytest.raises(PopulationError, match="calibrated.*original design"): + patch( + updated, + calibrated_node(1.5), + KernelResult(weights=Weights(normalized, WeightKind.CALIBRATED)), + ) + + at_cap = calibrated_node(2.0) + accepted = patch( + updated, + at_cap, + KernelResult(weights=Weights(normalized, WeightKind.CALIBRATED)), + ) + np.testing.assert_array_equal(accepted.design_weights["household"], ORIGINAL_DESIGN) + assert weight_cap_receipt(accepted, at_cap) == { + "weight_anchor": "design", + "max_weight_ratio": 2.0, + "realized_max_weight_ratio": 2.0, + } + + def test_design_cap_fails_closed_when_source_has_no_design_lineage() -> None: frame = _frame() importance_frame = Frame( diff --git a/packages/microcosm-graph/tests/test_graph_weight_update.py b/packages/microcosm-graph/tests/test_graph_weight_update.py index 16b33bf0c..7a7d8cc9e 100644 --- a/packages/microcosm-graph/tests/test_graph_weight_update.py +++ b/packages/microcosm-graph/tests/test_graph_weight_update.py @@ -1,11 +1,15 @@ """Amendment 25: a same-kind weight update is declarable and axis-bound. ``WeightTransition`` only moves a kind forward, so a stage that recomputes -weights it already holds — a sampling normalization, a re-solve of an -existing calibration — had no declaration at all. These properties are -about the one that does: the kind cannot move, mass cannot be free, and -positional replacement values are refused unless the kernel binds the -ordered entity axis they were computed against. +weights it already holds — a sampling normalization is the case this was +extracted for — had no declaration at all. These properties are about the +one that does: the kind cannot move, mass cannot be free, and positional +replacement values are refused unless the kernel binds the ordered entity +axis they were computed against. + +What an update does *not* do is re-anchor design ancestry; those +properties live beside the other design-anchor ones in +``test_graph_population.py``. Everything here runs real shared graph operations over the invented toy country. No country model, engine, or build artifact is involved. @@ -38,6 +42,7 @@ weight_update_receipt, ) from microcosm.graph.canonical import normative +from microcosm.graph.executor import _cache_record_key if "_toy" not in sys.modules: _SPEC = importlib.util.spec_from_file_location( @@ -308,13 +313,15 @@ def test_update_refuses_a_short_axis(tmp_path: Path) -> None: # ---------------------------------------------------------------------- -def test_cold_then_required_replay_revalidates_the_axis(tmp_path: Path) -> None: - """A hit re-applies the REWEIGHT, so the axis is checked again. +def test_required_replay_reapplies_the_stored_update(tmp_path: Path) -> None: + """A hit re-applies the REWEIGHT rather than restoring its frame. The cached result is reconstructed with its stored weights and receipt - and passed back through the same application, which is why replay - needs no parallel rule. ``resume="require"`` proves the second run read - the store rather than recomputing. + and passed back through the same application, which is why replay needs + no parallel rule. ``resume="require"`` proves the second run read the + store rather than recomputing. That the axis check *rejects* on that + path is the next three properties; this one only establishes that the + path is taken and agrees. """ cold = run_update(tmp_path / "run") assert cold.misses() == set(cold.compiled.order) @@ -337,6 +344,105 @@ def test_cold_then_required_replay_revalidates_the_axis(tmp_path: Path) -> None: assert list(replayed.values) == list(original.values) +def household_axis(run: object) -> list[int]: + """The incumbent ``survey`` household axis the update is applied to.""" + household = run.manifest.population("survey").entity("household") + return household[toy.id_column("household")].tolist() + + +def rebind_cached_axis(run: object, ids: list[int]) -> dict[str, object]: + """Rewrite the stored update record's axis binding to name ``ids``. + + The cache record is filed under a key derived from the node key, not + from its own content, and its receipt body carries no digest of its + own (`_require_record_shape` validates the envelope and the execution + evidence), so this is a faithful stand-in for a record produced against + a different axis rather than a store forgery the loader would catch. + """ + key = run.keys()["update"] + record_key = _cache_record_key(key) + record = run.store.load_json(record_key) + receipt = dict(record["receipt"]) + receipt["weight_update"] = weight_update_receipt(ids) + record["receipt"] = receipt + run.store.put_json(record_key, record, node_key=key, verify_existing=False) + return record + + +def test_required_replay_refuses_a_cached_binding_against_another_axis( + tmp_path: Path, +) -> None: + """The stored binding is re-checked, not trusted, on a hit. + + The cold run's record is rewritten to bind the same household ids in + the reverse order — the one case a count check cannot catch — and the + replay is then required to hit that record. It is refused, and no + kernel runs, so the refusal came from re-applying the cached result + rather than from recomputing it. This is also where the check fires: + ``resume="require"``'s preflight validates record *shape*, so a foreign + axis surfaces at apply time, mid-run. + """ + cold = run_update(tmp_path / "run") + ids = household_axis(cold) + assert len(ids) > 1 + rebind_cached_axis(cold, list(reversed(ids))) + + registry = registry_with_update() + with pytest.raises(NodeRejectedError, match="different .household. axis"): + toy.run_toy( + update_graph(), + tmp_path / "run", + sources=cold.sources, + registry=registry, + store=ContentStore(tmp_path / "run" / "store"), + resume="require", + ) + assert toy.total_calls(registry) == 0 + + +def test_required_replay_refuses_a_cached_binding_of_the_wrong_length( + tmp_path: Path, +) -> None: + """A shortened axis is refused on replay for the same reason.""" + cold = run_update(tmp_path / "run") + rebind_cached_axis(cold, household_axis(cold)[:-1]) + + registry = registry_with_update() + with pytest.raises(NodeRejectedError, match="different .household. axis"): + toy.run_toy( + update_graph(), + tmp_path / "run", + sources=cold.sources, + registry=registry, + store=ContentStore(tmp_path / "run" / "store"), + resume="require", + ) + assert toy.total_calls(registry) == 0 + + +def test_an_untouched_cached_binding_still_replays(tmp_path: Path) -> None: + """The rewrite above is what fails, not the rewriting. + + The record is written back through the same call with the binding it + already had, so the two properties above cannot be passing because a + re-filed record is unreadable. + """ + cold = run_update(tmp_path / "run") + rebind_cached_axis(cold, household_axis(cold)) + + registry = registry_with_update() + warm = toy.run_toy( + update_graph(), + tmp_path / "run", + sources=cold.sources, + registry=registry, + store=ContentStore(tmp_path / "run" / "store"), + resume="require", + ) + assert warm.misses() == set() + assert toy.total_calls(registry) == 0 + + def test_reason_is_part_of_the_node_identity(tmp_path: Path) -> None: """Two updates that state different purposes are different nodes.""" first = run_update(tmp_path / "first") From ecc98107ec191a353eeca1f9ce60cd5177095b4b Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 10:02:17 -0400 Subject: [PATCH 11/17] Disclose the fix round's contract deltas and refresh the receipts identity Co-Authored-By: Claude Opus 5 --- .../901-uk-shared-graph-contracts-receipts.md | 88 +++++++++++++++++-- 1 file changed, 82 insertions(+), 6 deletions(-) diff --git a/experiments/901-uk-shared-graph-contracts-receipts.md b/experiments/901-uk-shared-graph-contracts-receipts.md index 58e428d4a..3831c997b 100644 --- a/experiments/901-uk-shared-graph-contracts-receipts.md +++ b/experiments/901-uk-shared-graph-contracts-receipts.md @@ -80,19 +80,40 @@ moves a node key. `decl.py` is re-locked by 25, `kernel.py` by 26. ## Identity +Files as of the fix round (`f5aa65d40`). The four earlier rows this table +carried for `decl.py`, `executor.py` and the two new test files were the +pre-fix bytes and are superseded here. + | File | SHA-256 | | --- | --- | -| `decl.py` | `e78c7359b48813c57eff6f697359682753b74f86f565f4e6974421e5a4048933` | -| `kernel.py` | `51f45e899ba578a6b2324636a9879253266b067812f68b648eacfcd5a184d245` | +| `decl.py` | `11c2abd77f50389c6cd0b51e26cb3ebd5cbd0770ba96e9de1b53c71e9725eaa4` | +| `kernel.py` | `2df6cc5b5b396adae578f885aedbd038d2c0e51f56b4a31b64c2f6b69e1b918d` | | `weight_update.py` | `0ccfe6fcd257ef62b1f771b12eecc8ac5d447f5aa7d0403102e0ae290de16720` | | `population.py` | `33d1bb7bacea22870940288bf1907fb9eb24df7c245a216ff802e7fb41f5208f` | | `serialize.py` | `e5bf83c1082154f148626b6a36676614c6ff6c3fe0721aed94a1501da3021b1f` | -| `executor.py` | `ec4473bb033c4b1a36180c1518a42c755a46a2265d10461df1dfa00065862364` | +| `executor.py` | `5ab918e495fe4f8dd32c16155fe8c7a911e60e171cdbc8edb790626ce2d58c19` | | `explain.py` | `734a7b0e31692c31a99528cd83d9e74d3e508a317d913c1c169724a42d69d0de` | | `graph/__init__.py` | `697c59a37989a36124e6d43c7b07dd3b0582d965f97303c1fb02c88b41db2d48` | -| `tests/test_graph_weight_update.py` | `771ae0becbc57a4dd6198b9df229ec8c8262a5597647f335b7d5ed9be83471ff` | -| `tests/test_graph_frame_context.py` | `d58250d2194c81002be7282cd53597a7b9cff0906bb879b4b3926b4ac5b91ada` | -| `docs/graph-interface.lock` | `b42811ff0411dc179aaf9ddcf827aa270db866e08112cef8d1839df99c5a1f02` | +| `tests/test_graph_weight_update.py` | `6ff1296b1d769454b9664ca95bf084b91cf8bd05ada99ed58d54c8d4768751c7` | +| `tests/test_graph_frame_context.py` | `bcee6caccb7ffce47f04785557d246d2e8d5211d760ce5a344f3023cb1d3bf0c` | +| `tests/test_graph_population.py` | `b1aeabac04dbe6af8aeaf3b1691c7306a1442a142279043868500fdfe6476eec` | +| `tests/test_graph_kernel_contract.py` | `b13105cd302702eadcb30278545e794c16f3d819f571693810fb8f838473b126` | +| `tests/test_acceptance_b_ownership.py` | `a78ad49a341d08fe2a76e98e94d6dfe2fb60140ce5589101603b93f933f5bb34` | +| `docs/graph-interface.lock` | `b857403be2206156b844958cbd6abcb25ef951d05c0cc11e22554169a9343d2e` | + +### Every test file this lane touched + +An earlier draft of this receipt said the lane's test change was "isolated +to one file". That is true only of the **acceptance suite**, which the +charter assigns to the suite lane. In full: + +| File | Suite? | Change | +| --- | --- | --- | +| `test_graph_weight_update.py` | no | new (amendment 25) | +| `test_graph_frame_context.py` | no | new (amendment 26) | +| `test_graph_population.py` | no | three design-anchor properties added beside the existing ones (fix round, F2) | +| `test_graph_kernel_contract.py` | no | one assertion of *adjacency* relaxed to the ordering amendment 19 actually claims | +| `test_acceptance_b_ownership.py` | **yes** | B2's `KernelContext` field set, in its own commit (`895aabf19`), as amendment 19's was | The lock was re-recorded as part of each numbered amendment, never refreshed to make a test green: amendment 25 moved only the `decl.py` @@ -113,3 +134,58 @@ test files land in `fast/rest` and the engine lane beside their 27 sibling graph tests, neither `[defaulted]`). No pytest, no import of the production package, no engine, no install, no network beyond `gh` metadata and public blob reads. + +--- + +## Fix round (2026-09-13), after the independent adjudication + +An independent read-only review of `6f4ba4ec9` over `15ebde806` returned +REQUEST_CHANGES; its verbatim text is `FABLE-REVIEW.md` in this packet. +Root adjudicated. What changed in the contracts above: + +**Amendment 26, mass log (F1).** The projection handed every node +`population.frame.mass_log`, which for an ordinary node is its version's +*cumulative* log. An ordinary node's key binds only its version's +structural boundary and the owners of the columns it declared, so a +sibling that ran earlier in the same version and appended a record could +change what the node was shown without moving its key. `run_graph` now +records each version's log as that version is admitted — one line, which +cold execution and a restored hit both reach — and projects ordinary nodes +from that boundary. A structural node still receives the cumulative log, +which its key binds through `base` and `members`. + +**Amendment 26, isolation (F3/F4).** `Frame` deeply freezes its metadata +and its mass records, but a frozen dataclass still yields to +`object.__setattr__`, so passing those objects by reference made every +kernel a live handle on the population. The projection now hands out a +deep copy of the metadata and rebuilt records, through the rule +`_observer_snapshot` already followed (now the shared `_detached_record`), +and `_context_digest` binds all three fields — the metadata through the +frame format's own store codec, each mass record field by field, and the +projected column order. + +**Amendment 25, design ancestry (F2 — not accepted as stated).** The +review asked a design-kind update to re-anchor `Population.design_weights`. +Root refused: an anchor is the design weight a row entered carrying, it is +captured once at CREATE and afterwards only carried by stable entity id, +and `_carry_design_weights` maps an EXPAND's copied rows back to their +source's *original* anchor rather than its current value. Re-anchoring +would silently redefine the denominator of every `max_weight_ratio` +declared upstream of an unrelated normalization. The amendment and the +`WeightUpdate` docstring now state the anchor rule instead of "ancestry is +untouched", and `test_graph_population.py` asserts it. + +**Amendment 25, motivating claim (F5).** The amendment claimed a re-solve +of an existing calibration as a covered case. `calibrate.adam@1` emits no +`receipt['weight_update']`, so that declaration would be refused by the +axis check; the claim is now marked a future consumer adaptation. + +### Checks actually run in the fix round + +`ruff check .` (clean), `ruff format --check` on every touched file +(clean), stdlib `ast` parses of every touched Python file, and +`tools/ci_test_groups.py --verify` (`verification=ok`; the three touched +graph test files appear under `[fast]` and `[engine]`, none under +`[defaulted]`). Still no pytest, no import of the production package, no +engine, no install, no native source and no publication. `origin/main` +re-fetched on resume: `15ebde806`, nothing to merge. From c1a7920f72ff1ee5a9ea4d0430c9be9b9454e0df Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 10:04:23 -0400 Subject: [PATCH 12/17] Copy the cloned household's members, as a copied group requires `_remapped_expand_memberships` aligns clone ordinals: a copied group needs the same number of copies of every incumbent member, so cloning household 10 without copying persons 1 and 2 would have been refused rather than proving anything about anchors. The clone now carries its members, which also lets the property assert the remapped memberships; the entrant household still joins with none, which is allowed. Co-Authored-By: Claude Opus 5 --- .../tests/test_graph_population.py | 24 ++++++++++++------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/packages/microcosm-graph/tests/test_graph_population.py b/packages/microcosm-graph/tests/test_graph_population.py index 52c7fdfc1..7a79766c9 100644 --- a/packages/microcosm-graph/tests/test_graph_population.py +++ b/packages/microcosm-graph/tests/test_graph_population.py @@ -1422,9 +1422,9 @@ def _design_update_result(population: Population, *, factor: float) -> KernelRes def _clone_and_entrant_expand_node(*, base: str) -> Node: """An EXPAND that clones one household and admits one true entrant. - The toy household table carries no data columns, so the entrant row - materializes nothing; ``expand_cells`` is empty exactly as it is for the - lineage EXPAND above. + Copied rows carry their source's storage, and the toy household table + has no column but its id, so nothing here has to be materialized; + ``expand_cells`` is empty exactly as it is for the lineage EXPAND above. """ return Node( "grow", @@ -1448,17 +1448,20 @@ def _clone_and_entrant_expand_node(*, base: str) -> Node: def _clone_and_entrant_expand_result(population: Population) -> KernelResult: - """Clone household 10 as 40 and admit 50 from nothing. + """Clone household 10 as 40, and admit household 50 from nothing. - The returned design weights are the incumbent ones with the clone's copy - of its source appended, then the entrant's declared weight. + Household 10's members are copied with it, because a copied group + requires the same number of copies of every incumbent member + (``_remapped_expand_memberships``); the entrant household joins with + none. The returned design weights are the incumbent ones, then the + clone's copy of its source's weight, then the entrant's declared one. """ incumbent = population.frame.weights_for("household").values return KernelResult( expand={ "person": pd.Series( - [], - index=pd.Index([], dtype="int64", name="person_id"), + [1, 2], + index=pd.Index([5, 6], dtype="int64", name="person_id"), dtype="int64", ), "household": pd.Series( @@ -1537,6 +1540,11 @@ def test_a_design_update_moves_no_anchor_for_retained_clone_or_entrant_rows() -> over_updated.frame.table("household")["household_id"], np.array([10, 20, 30, 40, 50]), ) + person = over_updated.frame.table("person") + np.testing.assert_array_equal(person["person_id"], np.array([1, 2, 3, 4, 5, 6])) + np.testing.assert_array_equal( + person["person_household_id"], np.array([10, 10, 20, 30, 40, 40]) + ) # Not vacuous: the two versions' design *values* differ by the factor for # every row that existed before the update, and the clone copies the # updated value while inheriting the original anchor. From 03b5cb8e2b0376a9badf1c9e4cc14cad60dd214d Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 10:05:32 -0400 Subject: [PATCH 13/17] Record the fix round's outcome, the F2 source reading and the runtime plan Co-Authored-By: Claude Opus 5 --- PROGRESS-uk-shared-graph-contracts.md | 111 ++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) diff --git a/PROGRESS-uk-shared-graph-contracts.md b/PROGRESS-uk-shared-graph-contracts.md index b6b8810aa..436b5c399 100644 --- a/PROGRESS-uk-shared-graph-contracts.md +++ b/PROGRESS-uk-shared-graph-contracts.md @@ -120,6 +120,10 @@ leaves `decl.py` byte-identical. Either can be dropped without the other. explicitly ("the acceptance suite's B2 field set gains it in its own commit"). If root's owner disagrees, dropping that commit leaves B2 red and the rest intact. + *(2026-09-13, fix round: "isolated to one file" is true of the* + *acceptance suite only. `test_graph_kernel_contract.py` — not a suite* + *file — was also edited, and the fix round adds three properties to* + *`test_graph_population.py`. The full list is in the receipts.)* - Amendment 25 changes no node key; amendment 26 changes none either. Neither re-pins a spec digest. If a spec/seed digest moves in CI, that is main drift, not this lane (see `[[spec-engine-attested-modules]]`). @@ -127,6 +131,10 @@ leaves `decl.py` byte-identical. Either can be dropped without the other. three new fields. They are immutable views over an immutable `Frame`, so there is nothing for a kernel to mutate; stated here so the omission is a decision rather than an oversight. + *(2026-09-13, fix round: this reasoning was wrong and the decision is* + *reversed. A frozen dataclass still yields to `object.__setattr__`, so* + *the fields are now handed out detached and all three are digested —* + *commit `6ec46d62c`. Superseded; kept for the record.)* - No claim is made that any UK build, native lane, calibration or release passes. This lane read source only. @@ -254,3 +262,106 @@ the CI group inventory only. - F2: source-backed rejection plus the anchor-invariance properties. - F5/F6: the unsupported `calibrate.adam` motivating claim, the transition-only wording, and an accurate receipts disclosure. + +## Done (fix round) + +| Commit | What | +| --- | --- | +| `6104459f6` | File `FABLE-REVIEW.md` and open the fix round | +| `c0e275543` | F1: a node sees the mass log its own key binds | +| `6ec46d62c` | F3/F4: detach the frame view, and digest it for mutation | +| `09827aec8` | Record the resume and what it re-verified | +| `f5aa65d40` | F2/F4/F5/F6: design ancestry stated and proven; replay axis properties; the `calibrate.adam` claim withdrawn; `decl.py` re-locked | +| `c1a7920f7` | The clone fixture copies its household's members, as a copied group requires | +| `ecc98107e` | Receipts: fix-round deltas and refreshed identity | + +`decl.py` is now `11c2abd77f50389c6cd0b51e26cb3ebd5cbd0770ba96e9de1b53c71e9725eaa4` +and `kernel.py` `2df6cc5b5b396adae578f885aedbd038d2c0e51f56b4a31b64c2f6b69e1b918d`; +`docs/graph-interface.lock` matches both. `kernel.py` moved only in +amendment 26's commits, `decl.py` only in amendment 25's. + +## F2: why the re-anchoring was refused + +Read at source, not argued from the declaration text: + +1. `Population.design_weights` is set from the frame **once**, at CREATE + (`population.py:333-338` via `_create_population`, `executor.py:1461`). +2. Every later version gets its anchors from `_carry_design_weights` + (`population.py:2125-2181`), which aligns the incumbent anchors to the + new version by stable entity id. +3. `patch` passes those carried anchors to `Population.from_frame` + **explicitly** (`population.py:1159-1180`), so the `design_weights is + None` default that would re-derive them from the frame + (`population.py:333-338`) is never reached after CREATE. +4. `_apply_weight_update` (`population.py:2032-2040`) replaces the frame's + weight values. It touches `design_weights` not at all, and it runs + *before* step 2 in `patch`. +5. So a design-kind update moves no existing row's anchor. The EXPAND arm + of `_carry_design_weights` maps copied rows back to their **source + row's original anchor** (`population.py:2145-2157`, and the cached + twin at `941-981`), not to the source's current value, so a clone after + an update inherits the pre-update anchor too. Only a row with no + lineage reads `frame.weights_for(entity)` — and it has no earlier + weight to be anchored on. +6. The cap therefore stays what its own refusal has always called it: + `current > design * cap` against the **original** design weights + (`population.py:2333-2350`), with `realized_max_weight_ratio` reported + on the same denominator (`population.py:2354-2375`). + +The review's T2 case is real behaviour and is the intended outcome, now +asserted as such: calibrated weights equal to design weights an update +doubled are refused at `max_weight_ratio=1.5` and admitted exactly at +`2.0`, realized ratio `2.0`. Re-anchoring would make the same numbers a +ratio of `1.0` and would widen every cap declared upstream of an unrelated +normalization by that normalization's factor — a non-local change to an +already-declared contract, which is what the assignment forbade. + +No mixed-anchor defect beyond that definition was found, so nothing was +stopped. The one consequence worth naming, and now named in the +amendment: a row admitted *after* an update is anchored on whatever design +weight the EXPAND installs for it. If a kernel derives an entrant's design +weight from the updated incumbent values, that derived number is the +entrant's anchor — because it is the weight the row entered carrying. + +## Next (for root, before execution) — fix round + +Superseding the earlier plan's first block; the rest of that plan stands. + +``` +uv sync --all-packages --locked +uv run pytest packages/microcosm-graph/tests/test_graph_weight_update.py \ + packages/microcosm-graph/tests/test_graph_frame_context.py \ + packages/microcosm-graph/tests/test_graph_population.py +uv run pytest packages/microcosm-graph/tests/test_graph_interface_lock.py \ + packages/microcosm-graph/tests/test_acceptance_b_ownership.py \ + packages/microcosm-graph/tests/test_graph_kernel_contract.py \ + packages/microcosm-graph/tests/test_graph_executor.py \ + packages/microcosm-graph/tests/test_acceptance_replays.py \ + packages/microcosm-graph/tests/test_acceptance_d_weights.py +uv run pytest packages/microcosm-graph packages/microcosm-frame +uv run ruff check . +uv run python tools/ci_test_groups.py --verify +``` + +Where a source-only round could still be wrong, in the order worth +checking: + +1. `test_graph_population.py::test_a_design_update_moves_no_anchor_for_retained_clone_or_entrant_rows` + builds an EXPAND by hand. The clone-ordinal rule in + `_remapped_expand_memberships` already forced one correction here + (`c1a7920f7`); if another of `_patch_expand`'s guards fires, it will be + a `PopulationError` naming the guard, not a wrong anchor. +2. The three cached-axis properties in `test_graph_weight_update.py` + re-file a store record through `ContentStore.put_json(..., + verify_existing=False)`. That is the documented replace path, but it is + the one place these tests touch store internals; the third property + (re-file unchanged, still replays) exists to separate "the binding was + refused" from "the re-filed record was unreadable". +3. `_context_digest`'s metadata arm calls `store._encode_frame_metadata` + twice per node. It is guarded against a kernel-planted value the codec + cannot encode, but the guard's `except` list is reasoned from the + codec's source, not observed. +4. The four risks the first plan listed are unchanged, except that the + observer property named there now also rewrites nested metadata and + mass records, so a pandas copy-on-write no-op could no longer make the + whole property vacuous. From a7e61b98a04a5afe51c4e134947cfae1cbed20e6 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 10:08:31 -0400 Subject: [PATCH 14/17] Write the fix-round result report Co-Authored-By: Claude Opus 5 --- FIX-RESULT.md | 352 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 352 insertions(+) create mode 100644 FIX-RESULT.md diff --git a/FIX-RESULT.md b/FIX-RESULT.md new file mode 100644 index 000000000..f7a0af8c8 --- /dev/null +++ b/FIX-RESULT.md @@ -0,0 +1,352 @@ +# Fix round result — UK-enabling shared graph contracts (amendments 25, 26) + +Source-only. **Runtime is UNTESTED.** No pytest, no import of the +production package, no engine, no country model, no native source, no +install, no gated data, no network beyond `git fetch`, no publication. The +checks that were run are stdlib `ast` parses, `ruff check` / `ruff format +--check`, `tools/ci_test_groups.py --verify`, and reading source. Every +behavioural claim below is a claim about what the source says, not about +an observed run; the bounded runtime plan root needs is at the end. + +## Identity + +| Thing | Value | +| --- | --- | +| Worktree | `_worktrees/microcosm-uk-shared-graph-contracts-20260913` | +| Branch | `uk-shared-graph-contracts-20260913` | +| Base | `origin/main` `15ebde806cd1a262363f7217fe535c7234ff757f` (re-fetched on resume; unchanged, 0 behind) | +| HEAD | `03b5cb8e2` | +| Reviewed head the adjudication ran on | `6f4ba4ec989eba93786b5d88033631ad253fdc5c` | + +### Commits, in order + +| Commit | Kind | What | +| --- | --- | --- | +| `97428cd9f` | journal | Lane baseline: absence proof, the #901 consumer read | +| `ef2dc69c3` | source+tests+docs | Amendment 25: `WeightUpdate`, `weight_update_receipt` | +| `f33d47cda` | source+tests+docs | Amendment 26: the three `KernelContext` frame fields | +| `895aabf19` | acceptance suite | B2's field set, isolated (as amendment 19's was) | +| `6f4ba4ec9` | journal+receipts | Lane receipts and the first runtime plan | +| `6104459f6` | journal | File `FABLE-REVIEW.md`; open the fix round | +| `c0e275543` | source+tests+docs | **F1** — a node sees the mass log its own key binds | +| `6ec46d62c` | source+tests+docs | **F3/F4** — detach the frame view; digest all three fields | +| `09827aec8` | journal | Record the resume and what it re-verified | +| `f5aa65d40` | source+tests+docs | **F2/F4/F5/F6** — design ancestry stated and proven; cached-axis replay properties; the `calibrate.adam` claim withdrawn; `decl.py` re-locked | +| `c1a7920f7` | tests | The clone fixture copies its household's members, as a copied group requires | +| `ecc98107e` | receipts | Fix-round contract deltas; refreshed identity; full test-file disclosure | +| `03b5cb8e2` | journal | Fix-round outcome, the F2 source reading, the runtime plan | + +The two commits the previous process had already landed (`c0e275543`, +`6ec46d62c`) were preserved and re-verified rather than trusted; what that +verification consisted of is in `PROGRESS-uk-shared-graph-contracts.md` +under "Verified on resume, not assumed from the commit messages". + +### File hashes at `03b5cb8e2` + +| File | SHA-256 | +| --- | --- | +| `graph/decl.py` | `11c2abd77f50389c6cd0b51e26cb3ebd5cbd0770ba96e9de1b53c71e9725eaa4` | +| `graph/kernel.py` | `2df6cc5b5b396adae578f885aedbd038d2c0e51f56b4a31b64c2f6b69e1b918d` | +| `graph/executor.py` | `5ab918e495fe4f8dd32c16155fe8c7a911e60e171cdbc8edb790626ce2d58c19` | +| `graph/population.py` | `33d1bb7bacea22870940288bf1907fb9eb24df7c245a216ff802e7fb41f5208f` | +| `graph/weight_update.py` | `0ccfe6fcd257ef62b1f771b12eecc8ac5d447f5aa7d0403102e0ae290de16720` | +| `graph/serialize.py` | `e5bf83c1082154f148626b6a36676614c6ff6c3fe0721aed94a1501da3021b1f` | +| `graph/explain.py` | `734a7b0e31692c31a99528cd83d9e74d3e508a317d913c1c169724a42d69d0de` | +| `graph/__init__.py` | `697c59a37989a36124e6d43c7b07dd3b0582d965f97303c1fb02c88b41db2d48` | +| `docs/graph-interface.lock` | `b857403be2206156b844958cbd6abcb25ef951d05c0cc11e22554169a9343d2e` | + +The lock file records `decl.py` and `kernel.py` at exactly the hashes +above; `test_graph_interface_lock.py` is what enforces that. The lock moved +twice in this lane and both times for a numbered amendment: +`kernel.py` in amendment 26's commits (`f33d47cda`, then the F1 and F3/F4 +docstring edits), `decl.py` in amendment 25's (`ef2dc69c3`, then +`f5aa65d40`'s semantics correction). It was never refreshed to make a test +green. + +### Test files touched, with their roles + +| File | Acceptance suite? | Change | +| --- | --- | --- | +| `test_graph_weight_update.py` | no | new (amendment 25); fix round adds three cached-axis replay properties and renames the weak one | +| `test_graph_frame_context.py` | no | new (amendment 26); fix round adds the boundary-log and isolation properties | +| `test_graph_population.py` | no | fix round adds three design-anchor properties beside the existing ones | +| `test_graph_kernel_contract.py` | no | one assertion of field *adjacency* relaxed to the ordering amendment 19 actually claims | +| `test_acceptance_b_ownership.py` | **yes** | B2's `KernelContext` field set, in its own commit `895aabf19` | + +The lane's earlier "isolated to one test file" claim was true of the +acceptance suite only. That is now stated accurately in the receipts and +historicized in the journal. + +--- + +## Point-by-point response to the adjudication + +### F1 — `frame_mass_log` leaked a same-version sibling's record. **Accepted; fixed.** (`c0e275543`) + +The finding was correct. `_project_context` passed +`population.frame.mass_log`, which for an ordinary node is the version's +cumulative log (`executor.py:2669` rewrites the version entry after every +ordinary member, and `_append_frame_mass_log` runs for ordinary nodes). +An ordinary node's key binds `frame_key(version_key)` and the owners of +the columns it declared (`keys.py:206-209`, `keys.py:181-201`) — never a +sibling — so a cache hit could replay output computed against a different +log. + +The fix is the reviewer's own minimal option. `run_graph` records +`boundary_mass_logs[node.id] = updated.frame.mass_log` where a structural +version is admitted (`executor.py:2674`), beside +`populations[node.id] = updated`, which both cold execution and a restored +hit reach; an ordinary node is projected from +`boundary_mass_logs[compiled.versions[node_id]]` (`executor.py:2511`), a +structural node from `incumbent.frame.mass_log`, whose key binds the base +*and* every ordinary member through `members` (`keys.py:214-225`). The two +maps are written at the same point, so the boundary lookup cannot miss a +version whose incumbent lookup (`executor.py:2413`, which runs first) +succeeded. + +`test_mass_log_is_the_incoming_log` was replaced, as the review said it +had to be. The properties now distinguish the two cases: an unread +same-version appender is invisible to an ordinary member and moves neither +its key nor its stored bytes; cold-with-sibling equals cold-without and +each replays into the other's store under `resume="require"`; the same +appender *is* visible to the structural boundary, whose key it moves. The +ancestor-tracking alternative was not taken: it would make a node's input a +function of graph topology the key does not bind either. + +### F2 — a design-kind `WeightUpdate` should re-anchor. **Not accepted.** Source-backed response. + +The premise that anchors become "stale and mixed" is not what the source +does, and the proposed fix would redefine a declared contract. + +1. **Anchors are captured once, at CREATE.** + `_create_population` calls `Population.from_frame(frame, node.id)` + (`executor.py:1461`), whose `design_weights is None` default reads the + frame's design weights (`population.py:333-338`). That default is + reached exactly once per graph. +2. **Afterwards they are only carried.** Every later version gets + `design_weights=_carry_design_weights(...)` passed **explicitly** + (`population.py:1159`, `population.py:1180`), so the re-derive default + never runs again. +3. **An update does not touch them.** `_apply_weight_update` + (`population.py:2032-2040`) returns a frame with replaced weight values + and nothing else, and it runs *before* the carry in `patch`. +4. **Clones are not "mixed".** The review's EXPAND case says entrants + anchor from updated values "while retained rows keep original anchors". + The code separates three cases, not two: + `_carry_design_weights` maps an EXPAND's **copied** rows back to their + source row's *original* anchor (`population.py:2145-2157`; the cached + twin at `941-981` does the same), retained rows keep their own, and only + a row with **no lineage at all** reads `frame.weights_for(entity)` — + because it has no earlier weight to be anchored on. The assignment's + framing is exactly the code's. +5. **The cap denominator is the original by declaration, not by accident.** + `_assert_design_weight_cap` compares `current > design * cap` against + the carried anchors (`population.py:2333-2350`) and its refusal has + always read "above N * original design weight"; + `realized_max_weight_ratio` uses the same denominator + (`population.py:2354-2375`), under a parameter the node must spell + `weight_anchor='design'`. +6. **Re-anchoring would be the silent change.** It would let a + normalization node inserted anywhere upstream widen every already- + declared `max_weight_ratio` by its own factor — a non-local + redefinition of a contract other nodes wrote against. + +The review's T2 case is real behaviour, and it is now asserted as the +intended outcome rather than left implicit: with design weights doubled by +an update, calibrated weights equal to them are refused at +`max_weight_ratio=1.5` and admitted exactly at `2.0`, reporting a realized +ratio of `2.0`. A stage that wants a cap against normalized weights states +the ratio it means. + +**No mixed-anchor defect beyond that definition was established**, so +nothing was stopped with a counterexample. One consequence is worth naming +and is now named in the amendment and the docstring: a row admitted +*after* an update is anchored on whatever design weight the `EXPAND` +installs for it. If a kernel derives an entrant's design weight from the +updated incumbent values, that derived number becomes the entrant's +anchor — because it is the weight the row entered carrying. That is the +anchor rule applied to a row with no ancestry, not a mixture of two rules. + +The review's alternative ("refuse `kind='design'` in +`WeightUpdate.__post_init__`") was also declined: the lane's recorded read +of #901 shows `uk.full.normalize` declaring +`WeightUpdate("household", weight_kind, ...)` with `weight_kind` a +variable, so refusing the design arm would refuse a consumer this +amendment exists for, to avoid a defect that is not there. + +What changed instead: `WeightUpdate`'s docstring no longer says "ancestry +is untouched" as a bare assertion but states the anchor rule and its three +cases; amendment 25 gains the same paragraph; and +`test_graph_population.py` gains three properties — +`test_a_same_kind_design_update_leaves_the_original_anchors_invariant`, +`test_a_design_update_moves_no_anchor_for_retained_clone_or_entrant_rows` +(the same EXPAND run over an updated and an un-updated population, with +the anchors asserted equal and the frames' design values asserted a factor +apart so it is not vacuous), and +`test_a_design_update_does_not_move_the_calibration_cap_denominator` (T2). + +### F3 — the three fields shared live objects. **Accepted; fixed.** (`6ec46d62c`) + +The review rated this non-blocking; root took it as blocking, because +"nothing writable through the public API" is not the property that matters +once a kernel holds the object: a frozen dataclass still yields to +`object.__setattr__`, and a shared `MassChangeRecord` is a live handle on +the version's log that, unlike a table, nothing would notice. + +`_project_context` now hands out `deepcopy(frame.metadata)` and records +rebuilt through `_detached_record`, the rule `_observer_snapshot` already +followed and now shares. `_context_digest` binds all three fields: the +metadata through `store._encode_frame_metadata` (the frame format's own +codec, so an unchanged view digests as it persists), each mass record +field by field, and the projected column order. The codec call is guarded — +a value `Frame` would never have admitted is reported as the mutation it +is rather than raising while the comparison that would report it is being +computed. + +Detachment and the digest are not redundant: the digest catches a kernel +whose output stops being a function of its declared inputs; detachment is +what stops a *retained* view from rewriting the live version after that +node's check has already passed. + +### F4 — two tests were weaker than their names. **Accepted; fixed.** (`6ec46d62c`, `f5aa65d40`) + +- The observer property now rewrites nested metadata and mass records as + well as tables, over the graph whose snapshots actually carry a mass + record, and asserts the rewrite landed on the snapshot before asserting + the live version and the next node are unchanged. +- `test_cold_then_required_replay_revalidates_the_axis` is renamed + `test_required_replay_reapplies_the_stored_update`, which is what it + proves. Three new properties carry the claim it did not: + `test_required_replay_refuses_a_cached_binding_against_another_axis` + rewrites the stored record's binding to the same household ids in + reverse order — the one case a count check cannot catch — and requires + the replay to hit that record; it is refused with `different 'household' + axis` and no kernel runs, so the refusal came from re-applying the + cached result. A second does the same with a short axis. A third re-files + the record *unchanged* and still replays, so the two refusals are about + the binding rather than about a record having been re-filed. The tests + also pin where the check fires: `resume="require"`'s preflight validates + record shape, so a foreign axis surfaces at apply time, mid-run — which + is the residual the review itself flagged. + +### F5 — the `calibrate.adam` motivating claim. **Accepted; withdrawn.** (`f5aa65d40`) + +Confirmed at source: `packages/microcosm-calibrate/src/` contains no +`weight_update` string at all, so `calibrate.adam@1` emits no +`receipt['weight_update']`, and `_apply_weight_update` would refuse the +declaration as unverifiable (`population.py:2025-2031`). Its own guard +also still asks for a `WeightTransition` by name. Amendment 25, the +`WeightUpdate` docstring and the test module docstring no longer offer +"a re-solve of an existing calibration" as a covered case; the amendment +now states plainly that re-solving through the shared kernel is a **future +consumer adaptation**. + +### F6 — cosmetic and disclosure. **Accepted.** (`f5aa65d40`, `ecc98107e`) + +The node mass-policy mismatch message said "weight transition's" on a path +both declarations reach; it now names neither. The receipts disclose every +test file touched and which one is the acceptance suite's, and the +journal's "isolated to one file" line is historicized in place rather than +edited away. + +--- + +## Bounded runtime plan for root + +Finite and invented-only: the toy country fixture under +`packages/microcosm-graph/tests/fixtures/toy_country/` and `tmp_path`. No +country model, engine, native source, gated microdata, credential or +network. Nothing below publishes, promotes or writes outside `tmp_path` +and the uv environment. + +``` +uv sync --all-packages --locked + +# 1. The new and changed properties. +uv run pytest packages/microcosm-graph/tests/test_graph_weight_update.py \ + packages/microcosm-graph/tests/test_graph_frame_context.py \ + packages/microcosm-graph/tests/test_graph_population.py + +# 2. Everything that reads the two frozen files, the executor, or replay. +uv run pytest packages/microcosm-graph/tests/test_graph_interface_lock.py \ + packages/microcosm-graph/tests/test_acceptance_b_ownership.py \ + packages/microcosm-graph/tests/test_graph_kernel_contract.py \ + packages/microcosm-graph/tests/test_graph_executor.py \ + packages/microcosm-graph/tests/test_graph_explain.py \ + packages/microcosm-graph/tests/test_graph_serialize.py \ + packages/microcosm-graph/tests/test_graph_decl.py \ + packages/microcosm-graph/tests/test_acceptance_replays.py \ + packages/microcosm-graph/tests/test_acceptance_d_weights.py + +# 3. Both shards whole. +uv run pytest packages/microcosm-graph packages/microcosm-frame + +# 4. Lint and the CI partition (both already pass here). +uv run ruff check . +uv run python tools/ci_test_groups.py --verify +``` + +### Helper dependencies each step needs + +| Step | Needs | Why | +| --- | --- | --- | +| 1 | `packages/microcosm-graph/tests/_toy.py`, `fixtures/toy_country/*.csv`, `schema.json` | every graph-level property runs the toy country; `test_graph_frame_context.py` also asserts the **person column order** from `person.csv`'s header, so a fixture column reorder is a genuine (and intended) failure there | +| 1 | `pandas`, `numpy` from the locked env | the anchor properties call `patch()` directly and build pandas lineage objects; no store, no executor | +| 1 | a writable `tmp_path` | the frame-context and weight-update properties build real `ContentStore`s under it | +| 2 | `docs/graph-interface.lock` | `test_graph_interface_lock.py` reads it from the repo, not the wheel | +| 3 | nothing further | | +| 4 | `tools/ci_test_groups.py` | partition authority; `--verify` must stay `verification=ok` | + +Expected: `test_graph_weight_update.py`'s 22 and +`test_graph_frame_context.py`'s 26 properties pass, `test_graph_population.py` +passes all 60 (57 of them pre-existing, 3 added this round), the +regression files stay green, `ruff check` and `--verify` stay clean, and no +node key moves anywhere (neither amendment adds a `Node` field or changes a canonical +projection, so no spec or seed digest should move; if one does, that is +main drift, not this lane). + +### Where a source-only round could still be wrong, worth checking in this order + +1. **`test_a_design_update_moves_no_anchor_for_retained_clone_or_entrant_rows` + builds an EXPAND by hand.** `_patch_expand`'s guards are many; one + already forced a correction in this round (`c1a7920f7` — a copied group + requires the same number of copies of every incumbent member, + `_remapped_expand_memberships`). A second guard firing would surface as + a `PopulationError` naming that guard, not as a wrong anchor. +2. **The three cached-axis properties re-file a store record** through + `ContentStore.put_json(..., verify_existing=False)`. That is the + documented replace path (`store._put` → + `_replace_write_only_collision`), and the record key is derived from the + node key rather than from content, so a rewritten receipt is not a + key/content mismatch. The third property (re-file unchanged, still + replays) is the control that separates "the binding was refused" from + "the re-filed record was unreadable". +3. **`_context_digest`'s metadata arm** calls `store._encode_frame_metadata` + twice per node. Its guard (`TypeError`, `ValueError`, `RecursionError`) + is reasoned from the codec's and `canonical_json`'s source, not + observed. It also costs one encode per node per side; metadata is small, + but that cost is real and unmeasured. +4. **`deepcopy(frame.metadata)` per node projection** is likewise + unmeasured. `Frame` metadata is stage-level, not row-level, so this is + expected to be negligible; it has not been timed. +5. **Pandas copy-on-write in the observer property.** The earlier plan + flagged that a table-only rewrite could make that property vacuous; it + now also rewrites nested metadata and mass records through + `object.__setattr__`, which copy-on-write does not affect, so the + property can no longer be vacuous in the way flagged — but the table arm + of it can still be. +6. **The mass-record probe** in `test_graph_frame_context.py` states an + unchanged household total from `context.weights["household"].values.sum()`; + if `_append_frame_mass_log`'s bracketing disagrees, it surfaces as a + `PopulationError` from the boundary-log properties. + +## What this round does not claim + +No UK build, calibration, native lane, release or publication was run, +prepared or authorized. No country model was imported. `#901` is +untouched: this lane adds no UK graph stage, kernel, target or gate, and +the two amendments remain separable in source — 25 touches `decl.py`, +`serialize.py`, `explain.py`, `population.py`, `graph/__init__.py` and the +new `weight_update.py`, leaving `kernel.py` byte-identical; 26 touches +`kernel.py` and `executor.py`, leaving `decl.py` byte-identical. Either can +be dropped without the other. From c5c1fd567717dab66c925ceff2b0844df776b1cf Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 11:04:22 -0400 Subject: [PATCH 15/17] Preserve graph context framing and original design ratios --- .../src/microcosm/graph/executor.py | 4 + .../src/microcosm/graph/explain.py | 38 ++++-- .../tests/test_graph_explain.py | 116 ++++++++++++++++++ .../tests/test_graph_frame_context.py | 74 +++++++++++ .../tests/test_graph_population.py | 54 ++++++-- 5 files changed, 265 insertions(+), 21 deletions(-) diff --git a/packages/microcosm-graph/src/microcosm/graph/executor.py b/packages/microcosm-graph/src/microcosm/graph/executor.py index 8c15d6b57..33116b689 100644 --- a/packages/microcosm-graph/src/microcosm/graph/executor.py +++ b/packages/microcosm-graph/src/microcosm/graph/executor.py @@ -508,8 +508,12 @@ def _context_digest(context: KernelContext) -> bytes: digest.update(b"frame-metadata\0") digest.update(_frame_metadata_payload(context.frame_metadata)) digest.update(b"frame-column-order\0") + # Frame both levels: flattening entity names and columns lets a changed + # mapping reinterpret an entity name as a column without changing bytes. + digest.update(len(context.frame_column_order).to_bytes(8, "little")) for entity, columns in context.frame_column_order.items(): _update_scalar(digest, entity) + digest.update(len(columns).to_bytes(8, "little")) for column in columns: _update_scalar(digest, column) digest.update(b"frame-mass-log\0") diff --git a/packages/microcosm-graph/src/microcosm/graph/explain.py b/packages/microcosm-graph/src/microcosm/graph/explain.py index 5e1d51936..2a3b87d76 100644 --- a/packages/microcosm-graph/src/microcosm/graph/explain.py +++ b/packages/microcosm-graph/src/microcosm/graph/explain.py @@ -884,9 +884,11 @@ def _frame_ratios(frame, anchor, entity: str) -> list[float]: except (KeyError, ValueError): return [] design_by_id = dict(zip(design_ids, design_weights, strict=True)) + if any(entity_id not in design_by_id for entity_id in entity_ids): + return [] ratios = [] for entity_id, current in zip(entity_ids, values, strict=True): - design = float(design_by_id.get(entity_id, 0.0)) + design = float(design_by_id[entity_id]) current_value = float(current) if design > 0: ratios.append(current_value / design) @@ -902,6 +904,14 @@ def _population_ratios( manifest: RunManifest, node: Node, ) -> tuple[list[float], list[float]]: + """Infer ratios only when the manifest retains the original anchor. + + Manifest populations are Frames, not Populations with carried design + anchors. CREATE design weights suffice across updates and filters, but + an EXPAND needs copy/entrant ancestry not carried by those Frames. In + that case leave the fallback unavailable; explicit receipt samples + remain usable. A later design-kind update never becomes a new anchor. + """ if node.weights is None: return [], [] before = None if node.base is None else manifest.populations.get(node.base) @@ -912,17 +922,24 @@ def _population_ratios( anchor_node = node.base anchor = None while anchor_node is not None: - candidate = manifest.populations.get(anchor_node) - if candidate is not None and entity in candidate.weighted_entities: - weights = candidate.weights_for(entity) - if str(_value(weights.kind)) == "design": - anchor = candidate - break declaration = compiled.graph.node(anchor_node) + if declaration.structural is StructuralDelta.EXPAND: + return [], [] + if declaration.structural is StructuralDelta.CREATE: + candidate = manifest.populations.get(anchor_node) + if candidate is not None and entity in candidate.weighted_entities: + weights = candidate.weights_for(entity) + if str(_value(weights.kind)) == "design": + anchor = candidate + break anchor_node = declaration.base if anchor is None: return [], [] - return _frame_ratios(before, anchor, entity), _frame_ratios(after, anchor, entity) + before_ratios = _frame_ratios(before, anchor, entity) + after_ratios = _frame_ratios(after, anchor, entity) + if not before_ratios or not after_ratios: + return [], [] + return before_ratios, after_ratios def _numeric_sequence(value: object) -> list[float]: @@ -965,7 +982,10 @@ def _histogram_svg(before: Sequence[float], after: Sequence[float]) -> str: finite = [*finite_before, *finite_after] nonfinite = len(before) + len(after) - len(finite) if not finite: - return '

Weight-ratio samples are not present in this manifest.

' + return ( + '

Weight-ratio samples are not recorded or original ' + "design ancestry is unavailable in this manifest.

" + ) low, high = min(finite), max(finite) bin_count = min(12, max(4, int(math.sqrt(len(finite))))) if math.isclose(low, high): diff --git a/packages/microcosm-graph/tests/test_graph_explain.py b/packages/microcosm-graph/tests/test_graph_explain.py index b8bac925f..661a46282 100644 --- a/packages/microcosm-graph/tests/test_graph_explain.py +++ b/packages/microcosm-graph/tests/test_graph_explain.py @@ -13,7 +13,9 @@ import pytest import microcosm.graph as graph_api +from microcosm.frame import WeightKind, Weights from microcosm.graph import describe, explain_html, graph_to_json +from microcosm.graph.explain import _population_ratios ROOT = Path(__file__).parents[3] @@ -240,6 +242,120 @@ def test_calibration_view_uses_targets_ratios_and_mass(explanation) -> None: assert "urban" in rendered +class _NormalizeDesign(toy.ToyKernel): + """A real same-kind update for the original-anchor explanation check.""" + + def compute(self, context): + before = context.weights["household"].values + after = before * 2.0 + return graph_api.KernelResult( + weights=Weights(after, WeightKind.DESIGN), + receipt={ + "mass": toy._mass_record(context, before, after, "declared"), + "weight_update": graph_api.weight_update_receipt( + context.tables["household"]["household_id"].tolist() + ), + }, + ) + + +@pytest.mark.parametrize("filtered", [False, True], ids=("same-axis", "filtered")) +def test_ratio_fallback_keeps_create_anchors_after_design_update_and_replay( + tmp_path: Path, filtered: bool +) -> None: + """The chart and the actual cap receipt both use the original denominator.""" + normalize = graph_api.Node( + "normalize", + "explain.normalize@1", + structural=graph_api.StructuralDelta.REWEIGHT, + base="survey", + inputs=( + graph_api.Slice("person", ("age",)), + graph_api.Slice("household", ("household_size",)), + ), + weights=graph_api.WeightUpdate( + "household", "design", "toy normalization", mass="declared" + ), + mass="declared", + ) + nodes = [toy.CREATE, normalize] + base = normalize.id + if filtered: + selection = toy.select_node("adults", base=base) + nodes.append(selection) + base = selection.id + calibrated = toy.reweight_node( + "calibrated", base=base, to_kind="calibrated", factor=1.0 + ) + calibrated = replace( + calibrated, + params={ + **dict(calibrated.params), + "max_weight_ratio": 2.0, + "weight_anchor": "design", + }, + ) + graph = graph_api.Graph("toy", (toy.SOURCE,), (*nodes, calibrated)) + + def registry(): + value = toy.toy_registry() + value.register(_NormalizeDesign(normalize.kernel, toy._REWEIGHT)) + return value + + cold = toy.run_toy(graph, tmp_path / "cold", registry=registry()) + replay = toy.run_toy( + graph, + tmp_path / "replay", + sources=cold.sources, + store=cold.store, + registry=registry(), + resume="require", + ) + assert all(receipt.hit for receipt in replay.manifest.nodes.values()) + assert toy.total_calls(replay.registry) == 0 + for run in (cold, replay): + before, after = _population_ratios(run.compiled, run.manifest, calibrated) + count = run.manifest.populations[calibrated.id].n("household") + assert before == [2.0] * count + assert after == [2.0] * count + receipt = run.manifest.nodes[calibrated.id].receipt + assert receipt["realized_max_weight_ratio"] == 2.0 + assert "weight_ratios" not in receipt + assert "Before (n=" in explain_html(run.compiled, run.manifest) + + +def test_ratio_fallback_declines_expand_ancestry_but_preserves_receipt_samples( + tmp_path: Path, +) -> None: + """Manifest Frames cannot reconstruct copied/entrant original anchors.""" + expand, claim = toy.entrant_expand_node() + calibrated = toy.reweight_node( + "calibrated", base=expand.id, to_kind="calibrated", factor=1.0 + ) + graph = graph_api.Graph( + "toy", (toy.SOURCE,), (toy.CREATE, expand, claim, calibrated) + ) + run = toy.run_toy(graph, tmp_path / "run") + + assert _population_ratios(run.compiled, run.manifest, calibrated) == ([], []) + rendered = explain_html(run.compiled, run.manifest) + assert "original design ancestry is unavailable" in rendered + original = run.manifest.nodes[calibrated.id] + updated = replace( + original, + receipt={ + **dict(original.receipt), + "weight_ratios": {"before": (1.0, 2.0), "after": (2.0, 3.0)}, + }, + ) + supplied = replace( + run.manifest, nodes={**dict(run.manifest.nodes), calibrated.id: updated} + ) + rendered = explain_html(run.compiled, supplied) + assert "Before (n=2)" in rendered and "After (n=2)" in rendered + assert "original design ancestry is unavailable" not in rendered + + def test_calibration_view_renders_partition_mass_with_deltas(tmp_path: Path) -> None: run = toy.run_toy(toy.full_graph(), tmp_path / "run") original = run.manifest.nodes["calibrated"] diff --git a/packages/microcosm-graph/tests/test_graph_frame_context.py b/packages/microcosm-graph/tests/test_graph_frame_context.py index f93fec76a..cdefd5bba 100644 --- a/packages/microcosm-graph/tests/test_graph_frame_context.py +++ b/packages/microcosm-graph/tests/test_graph_frame_context.py @@ -73,6 +73,11 @@ def compute(self, context: KernelContext) -> KernelResult: frame = toy.read_toy_frame(context.sources["survey"]) tables = {entity: frame.table(entity) for entity in frame.entities} tables.update({link: frame.link(link) for link in frame.links}) + mass_log = () + reason = context.params.get("source_mass_reason") + if reason is not None: + total = float(frame.weights_for("household").values.sum()) + mass_log = (MassChangeRecord("household", total, total, None, reason),) return KernelResult( frame=Frame( tables, @@ -82,6 +87,7 @@ def compute(self, context: KernelContext) -> KernelResult: for entity in frame.weighted_entities }, frame.strata, + mass_log=mass_log, metadata=VERSION_METADATA, ), receipt={"persons": frame.n("person")}, @@ -198,6 +204,28 @@ def rewrite(context: KernelContext, field: str) -> None: } ), ) + elif field in ("column_order_collapse", "column_order_regroup"): + (first_entity, first_columns), (second_entity, second_columns) = ( + context.frame_column_order.items() + ) + if field == "column_order_collapse": + changed = {first_entity: (*first_columns, second_entity, *second_columns)} + else: + # Keep two entries and the same flattened string sequence, but + # reinterpret the first entity's last column as the second entity. + changed = { + first_entity: first_columns[:-1], + first_columns[-1]: (second_entity, *second_columns), + } + assert changed != dict(context.frame_column_order) + assert [ + value for entity, columns in changed.items() for value in (entity, *columns) + ] == [ + value + for entity, columns in context.frame_column_order.items() + for value in (entity, *columns) + ] + object.__setattr__(context, "frame_column_order", MappingProxyType(changed)) else: # pragma: no cover - guards the fixture itself raise AssertionError(f"unknown frame field {field!r}") @@ -830,6 +858,28 @@ def test_rewriting_the_projected_column_order_is_refused(tmp_path: Path) -> None ) +def test_collapsing_entity_column_groups_is_refused(tmp_path: Path) -> None: + node = isolation_node( + "iso_collapse", target="iso_t", tamper_self="column_order_collapse" + ) + node = dataclasses.replace( + node, inputs=(*node.inputs, Slice("household", ("household_id",))) + ) + with pytest.raises(NodeRejectedError, match="mutated its input context"): + run_isolation(tmp_path / "run", node) + + +def test_moving_a_column_to_an_entity_boundary_is_refused(tmp_path: Path) -> None: + node = isolation_node( + "iso_regroup", target="iso_t", tamper_self="column_order_regroup" + ) + node = dataclasses.replace( + node, inputs=(*node.inputs, Slice("household", ("household_id",))) + ) + with pytest.raises(NodeRejectedError, match="mutated its input context"): + run_isolation(tmp_path / "run", node) + + def test_an_untouched_frame_view_still_passes_the_mutation_check( tmp_path: Path, ) -> None: @@ -866,6 +916,30 @@ def test_required_replay_is_a_full_hit(tmp_path: Path) -> None: } +def test_a_cached_create_preserves_its_nonempty_boundary_log(tmp_path: Path) -> None: + reason = "source weights reconciled before graph admission" + source = dataclasses.replace(CREATE, params={"source_mass_reason": reason}) + first = probe_node("first", columns=("age",), target="first_count") + graph = Graph("toy", (toy.SOURCE,), (source, first)) + cold, probe, sources, store = run_probe(tmp_path / "run", graph=graph) + expected = observation(probe, "first")["mass_log"] + assert len(expected) == 1 and expected[0].reason == reason + + registry, next_probe = build_registry() + second = probe_node("second", columns=("age",), target="second_count") + warm = run_graph( + compile_graph(Graph("toy", (toy.SOURCE,), (source, second))), + sources=dict(sources), + store=store, + kernels=registry, + ) + assert warm.nodes[source.id].hit is True + assert warm.nodes["second"].hit is False + assert warm.nodes[source.id].key == cold.nodes[source.id].key + assert observation(next_probe, "second")["mass_log"] == expected + assert warm.population(source.id).mass_log == expected + + def test_a_new_node_over_restored_populations_sees_the_same_frame( tmp_path: Path, ) -> None: diff --git a/packages/microcosm-graph/tests/test_graph_population.py b/packages/microcosm-graph/tests/test_graph_population.py index 7a79766c9..d6d6505c6 100644 --- a/packages/microcosm-graph/tests/test_graph_population.py +++ b/packages/microcosm-graph/tests/test_graph_population.py @@ -1422,9 +1422,9 @@ def _design_update_result(population: Population, *, factor: float) -> KernelRes def _clone_and_entrant_expand_node(*, base: str) -> Node: """An EXPAND that clones one household and admits one true entrant. - Copied rows carry their source's storage, and the toy household table - has no column but its id, so nothing here has to be materialized; - ``expand_cells`` is empty exactly as it is for the lineage EXPAND above. + Both members of the copied household are cloned without reassignment. + A true entrant person has all carried cells and its stratum declared, + and joins the new entrant household. """ return Node( "grow", @@ -1432,7 +1432,13 @@ def _clone_and_entrant_expand_node(*, base: str) -> Node: structural=StructuralDelta.EXPAND, base=base, params={ - "expand_cells": (), + "expand_cells": ( + ("person", "person_household_id", "int64"), + ("person", "keep", "bool"), + ("person", "owned", "boolean"), + ("person", "nullable", "boolean"), + ("person", "amount", "float64"), + ), "expand_weight_entity": "household", "expand_weight_kind": "design", }, @@ -1452,27 +1458,50 @@ def _clone_and_entrant_expand_result(population: Population) -> KernelResult: Household 10's members are copied with it, because a copied group requires the same number of copies of every incumbent member - (``_remapped_expand_memberships``); the entrant household joins with - none. The returned design weights are the incumbent ones, then the - clone's copy of its source's weight, then the entrant's declared one. + (``_remapped_expand_memberships``); person 7 enters without ancestry + and belongs to household 50. The returned design weights are the + incumbent ones, the clone's source weight, then the entrant's weight. """ incumbent = population.frame.weights_for("household").values + person = population.frame.table("person") + ids = pd.Index([1, 2, 3, 4, 5, 6, 7], name="person_id") + entrant = { + "person_household_id": 50, + "keep": True, + "owned": False, + "nullable": pd.NA, + "amount": 4.0, + } + columns = { + ("person", column): pd.Series( + [*person[column], *person[column].iloc[:2], value], + index=ids, + dtype=person[column].dtype, + ) + for column, value in entrant.items() + } + columns[("person", "person_household_id")] = pd.Series( + [10, 10, 20, 30, 40, 40, 50], index=ids, dtype="int64" + ) return KernelResult( expand={ "person": pd.Series( - [1, 2], - index=pd.Index([5, 6], dtype="int64", name="person_id"), - dtype="int64", + pd.array([1, 2, pd.NA], dtype="Int64"), + index=pd.Index([5, 6, 7], dtype="int64", name="person_id"), ), "household": pd.Series( pd.array([10, pd.NA], dtype="Int64"), index=pd.Index([40, 50], dtype="int64", name="household_id"), ), }, + columns=columns, weights=Weights( np.array([*incumbent, incumbent[0], ENTRANT_DESIGN_WEIGHT]), WeightKind.DESIGN, ), + strata=pd.Series( + ["entrant"], index=pd.Index([7], name="person_id"), dtype=object + ), ) @@ -1541,10 +1570,11 @@ def test_a_design_update_moves_no_anchor_for_retained_clone_or_entrant_rows() -> np.array([10, 20, 30, 40, 50]), ) person = over_updated.frame.table("person") - np.testing.assert_array_equal(person["person_id"], np.array([1, 2, 3, 4, 5, 6])) + np.testing.assert_array_equal(person["person_id"], np.array([1, 2, 3, 4, 5, 6, 7])) np.testing.assert_array_equal( - person["person_household_id"], np.array([10, 10, 20, 30, 40, 40]) + person["person_household_id"], np.array([10, 10, 20, 30, 40, 40, 50]) ) + assert over_updated.frame.strata.iloc[-1] == "entrant" # Not vacuous: the two versions' design *values* differ by the factor for # every row that existed before the update, and the clone copies the # updated value while inheriting the original anchor. From 5742c17cef27f02db40997b39d7ac7e8b4f8492e Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 11:12:20 -0400 Subject: [PATCH 16/17] Project a declared household field in context mutation tests --- packages/microcosm-graph/tests/test_graph_frame_context.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/microcosm-graph/tests/test_graph_frame_context.py b/packages/microcosm-graph/tests/test_graph_frame_context.py index cdefd5bba..21deecea2 100644 --- a/packages/microcosm-graph/tests/test_graph_frame_context.py +++ b/packages/microcosm-graph/tests/test_graph_frame_context.py @@ -863,7 +863,7 @@ def test_collapsing_entity_column_groups_is_refused(tmp_path: Path) -> None: "iso_collapse", target="iso_t", tamper_self="column_order_collapse" ) node = dataclasses.replace( - node, inputs=(*node.inputs, Slice("household", ("household_id",))) + node, inputs=(*node.inputs, Slice("household", ("household_size",))) ) with pytest.raises(NodeRejectedError, match="mutated its input context"): run_isolation(tmp_path / "run", node) @@ -874,7 +874,7 @@ def test_moving_a_column_to_an_entity_boundary_is_refused(tmp_path: Path) -> Non "iso_regroup", target="iso_t", tamper_self="column_order_regroup" ) node = dataclasses.replace( - node, inputs=(*node.inputs, Slice("household", ("household_id",))) + node, inputs=(*node.inputs, Slice("household", ("household_size",))) ) with pytest.raises(NodeRejectedError, match="mutated its input context"): run_isolation(tmp_path / "run", node) From 072c2e88b2a66782a351044a662d7e8ada2e9f61 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sun, 13 Sep 2026 12:06:42 -0400 Subject: [PATCH 17/17] Record verified shared graph contract acceptance --- FABLE-REVIEW.md | 5 ++ FIX-RESULT.md | 4 + PROGRESS-uk-shared-graph-contracts.md | 3 + .../901-uk-shared-graph-contracts-receipts.md | 3 + ...ared-graph-contract-acceptance-20260913.md | 73 +++++++++++++++++++ 5 files changed, 88 insertions(+) create mode 100644 experiments/uk-shared-graph-contract-acceptance-20260913.md diff --git a/FABLE-REVIEW.md b/FABLE-REVIEW.md index b5a4456bd..bebe6cef2 100644 --- a/FABLE-REVIEW.md +++ b/FABLE-REVIEW.md @@ -1,3 +1,8 @@ +> Historical review of the initial proposal, preserved verbatim. The findings +> were subsequently resolved or adjudicated. See the +> [13 September acceptance record](experiments/uk-shared-graph-contract-acceptance-20260913.md) +> for the tested revision and remaining scope. +