Repository navigation
Add the transport skeleton composer, rules registry and local build driver (NZ graph G6) - #1161
Conversation
Prepare isolated source codecs and shared population, geography, target, calibration, gate and terminal kernels. Route all pinned rules bindings through one simulate.rules_by_ref kernel, validate adapter roots and relations, and bind injected Python adapters to explicit cache configuration identities. Add engine-free property and differential coverage for registry dispatch. Full composer and CLI work remains blocked on the absent AS bridge, gap and bands kernels; no stand-in policy calculations are introduced. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
- compose.py: compose_transport_graph(spec, config) builds the skeleton a country's transport_graph.json spec resource declares. A node parameter that is an object must be an explicit resource selection (value, json or the resource's SHA-256) or the prepared engine-reference map, so a node binds only the resources it selects; no whole-spec fingerprint enters any node. Rules nodes name a binding and get its authenticated engine_ref and period, all through the one simulate.rules_by_ref@1 router. Kernels outside the skeleton vocabulary enter only through TransportGraphConfig.extensions, the typed extension point for the entitlement package; composition refuses an extension that changes any skeleton node's predecessors. validate_transport_activation refuses unresolved evidence with every item named (the benefit-unit age limit, placeholder references, the receipt contract, engine pins, the S4 rent factor, a missing declaration or selected resource). - cli.py: prepare (registry; CREATE's column inventory, read by running transport.create@1 once on the declared params), compile, then run ancestor-closed checkpoints on <out>/.graph-store: geography, calibration, export preparation (then the skeleton H5), each terminal endpoint, the whole graph. Writes graph.json, manifest.json, checkpoint manifests, artifacts, the H5 and descriptor, calibration diagnostics, explorer.html and build-status.json (with the outputs still owed) under --out only. --resume require refuses a cold store before creating anything; an --out that overlaps an input is refused. No upload or staging path. - registry.py: build the engines' schema and unit nesting from the benefit-unit rule (the schema CREATE builds) instead of naming a country schema, check injected and Axiom adapters against both, and drop country words from messages, so the transport source rules pass. - tools/build_transport.py: the six-line shim. - Tests on an engine-free toy package (synthetic donor, facts for both roles, toy RuleSpec tree, three pure-Python engines in one run_graph): purity; a Hypothesis resource-locality property (one resource edit re-keys exactly its selectors and their descendants); A1, A2, A4, A5; A3 for a calibration reference; C2 for a gates.json threshold and a hold-out reference; hold-out ancestry; B1; extension isolation; E3; CLI outputs, warm resume, H5 rewrite and corruption refusal; differentials (router vs stock simulate.rules@1, prepared vs independent CREATE inventory, registry vs CREATE schema, declared vs executed keys); the packaged spec's named refusals; no policy literal or remote import. Invariants: the graph is a pure function of the spec resources and the prepared config; editing one resource re-keys exactly the nodes that select it and their descendants; the extension point never re-keys a skeleton node; no calibration ancestor reads the hold-out facts or a family-engine column; every rules node runs through the one router; every output lies under --out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both prose properties drew from alphabets that include a space, so Hypothesis could draw a whitespace-only string such as " ", which graph/decl.py's _description refuses. The G5a A4 test failed this way in 2 of 6 local runs on main (the same fix is microcosm#1160); the composed graph's purity/prose test has the same latent case. Map both strategies through str.strip: the result is empty or contains a non-whitespace character, which is what decl.py accepts. tests: composed prose property 4/4 runs passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Write timestamp-free HDF5 exports without changing shared export kernels, validate calibration ancestry and activation selections, and preserve literal resource data. Cache CREATE inventories, refuse unsafe CLI paths and ids, isolate extension specs, and expose binding-driven rules nodes with additional sources and checkpoints. Narrow pin-provenance claims and expand regressions. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
The round-2 review of #1161 showed that a pre-existing `<out>/.graph-store/objects` link sends ContentStore writes outside --out: ContentStore creates `objects`, `tmp` and shard directories with mkdir(exist_ok=True), which follows an existing link, and the driver only checked the store root itself. `_contained_store_root(out)` now checks the root with `_destination` and, when a store tree already exists, walks it without following links and refuses any entry that is a link (root, objects, tmp, shards, objects). Both store entry points use it: main(), before the spec is loaded, and execute_transport_graph(), before the ContentStore is opened. The agent guide and changelog now say what is guarded (links present when a build starts) and what is not (links planted while it runs). Tests: a link at objects, tmp or objects/aa is refused by _contained_store_root, by execute_transport_graph and by main(), and nothing is written to the link target; a plain store tree is accepted. With the old root-only check restored in memory (a pytest plugin, no tracked file changed), the 4 execution and main() cases fail (DID NOT RAISE; main() reached spec loading). test_transport_composed_graph.py 123 passed; test_transport_graph.py 15 passed; ruff check and format pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged by the NZ hub (session local_e407a527) at fe47f88 with --merge. Gates: |
G6 of the Microcosm NZ graph plan (Max's d962): the transport skeleton composer, its rules registry, the local build driver and the shim. Under the hub's G6/G7 split amendment, the AS chain is out of scope here. The bridges,
nz.as, scenarios, variants, gaps, bands and the WFF tripwire come in G7 through the extension point this PR adds. The PR builds on #1159 (G5a, merged).What it adds (
packages/microcosm-build/src/microcosm/build/transport/)registry.py(extends G5a's):build_transport_registry(...)checks the rules bindings: it hashes the actual RuleSpec module and tree bytes, and checks schema, roots, entity names (each engine reports its own entity), output variables and graph dtypes. The engine commit, wheel SHA-256 and exported-tree RuleSpec commit are caller declarations that are format-checked only; they are not verified against the installed engine. It binds every rules binding into onesimulate.rules_by_ref@1instance, and registers the codecs plus the G5a population, geography, G5b target/ordered-calibration/terminal and G5c gate kernels. Each engine's schema and nesting come from the benefit-unit rule, the same expression CREATE uses, so no country schema is named inbuild/transport/. Injected pure-Python adapters declare theircache_identity()and get refs that identify Python execution.compose.py:compose_transport_graph(spec, config) -> Graph.transport_graph.json(alegacy_jsonspec resource) declares the skeleton.SKELETON_KERNELSadmits only kernels that exist; anything else must come throughTransportGraphConfig.extensions, a typed protocol. Composition refuses an extension that would change a skeleton node's compiled key. Extensions may declare additional sources (collisions refused) and ordered pre-export checkpoints; a public helper builds a rules node from a binding's prepared reference and period. Every extension factory gets its own deep copy of the spec.nz.calibratemay declare hold-out facts, select hold-out references, carry the prepared reference map, or embed the family/AS engine reference..,..or NUL are refused at compose time.validate_transport_activationrefuses before CREATE or any engine is built. It checks every selected reference resource for non-active rows, every null scenario knob (no country knob name in the transport code) and every selected resource, and names each gap.cli.py+tools/build_transport.py(6-line shim).transport.create@1once, so CREATE's logic isn't copied. The inventory is cached under a key of the kernel implementation, resolved parameters and source bytes, so warm reruns skip it; a cold build still runs CREATE twice (inventory, then in the graph).<out>/.graph-store: geography, calibration,export.prepare(then writesskeleton.h5), each terminal, then the whole graph.graph.json,manifest.json, the per-checkpoint manifests, the evidence index, the export descriptor,calibration-diagnostics.json,explorer.html, and abuild-status.jsonthat listsnot_yet_producedoutputs from the spec'spending_outputs..graph-store, lies under--out: before opening the store it refuses any link in an existing store tree (root,objects,tmp, shards), because the store creates those directories by following links. It does not guard against links planted while a build runs.--resume requirerefuses a cold store before any preparation. An--outthat overlaps an input or--spec-diris refused. Argument and build errors print a refusal, never a traceback.--outdirectories give byte-identicalskeleton.h5and identical keys for every node.nz.asFILTER based onnz.calibrate, not innz.open. A REWEIGHT node's key binds every member of its base version, so encoding innz.openwould make calibration depend on family entitlement inputs and break hold-out ancestry. This follows plan §2.2/§2.3.Invariants (property tests on an engine-free toy NZ package with three pure-Python engines in one
run_graph; toy run ≈4 s)graph_to_json.--outdirectories give equal keys for every node and byte-identical artifacts, includingskeleton.h5; a reloadedrequirerun has zero misses.descendants(nz.targets.compile).gates.jsonthreshold or hold-out reference edit is descendant-exact and disjoint from the calibration closure.nz.calibratedeclaresnz_holdout_facts, selects hold-out references, carries the prepared reference map, embeds the family engine reference, or touches family-engine columns; enforced at compose time and tested for each rule. The packaged spec's reference names are also checked.simulate.rules@1per node and column; the prepared CREATE inventory equals G5a's composition; the registry schema equals CREATE's frame schema.build/transport/*.py, plus the existing source-scan rules intest_transport_graph.py(no country word, no float literal).Real-spec activation (refuses, by design)
The packaged NZ spec refuses before reading any input, naming every gap:
receipt_contract.json(the closure's receipt requirements are prose);rent_stock_factor;transport_graph.jsonitself.That file is left for D1/G8, because several of its inputs are still undecided.
Tests (one file at a time, by the hub)
At fe47f88 (store containment fix):
test_transport_composed_graph.py123 (115 plus 8 containment cases),test_transport_graph.py15. With the old root-only store check restored in memory, the 4 execution andmain()containment cases fail.At e7953f3 (review fixes):
test_transport_composed_graph.py115,test_transport_registry.py31,test_transport_graph.py15. At 25c47a5 (after merging main with #1160 and #1162):test_transport_composed_graph.py115,test_transport_population_kernels.py27,test_telemetry_emitter.py33. All passed. The fix lane's baselines show the new tests failing on 5bd688c (determinism 1 failed; refusals 20 failed; composer 29 failed; CLI 4 failed; entity echo 2 failed); evidence in the hub's job folder.Also in this PR
Both prose property tests now map their strategies through
str.strip, because a whitespace-only draw is an invalidNode.description. The G5a A4 test failed this way in 2 of 6 local runs on main. The same one-line fix is microcosm#1160; the change is identical, so whichever lands second merges cleanly.Review
_paramindirection, narrowed claims, and driver/test fixes; also flagged main's telemetry date failure (fixed by Stamp migrated-spool fixture rows inside the retention window #1162) and a plan wording error (fixed in the hub's plan and decision d1043)..graph-store/objectslink sent store writes outside--out; every other r1 fix held in its probes (8/8 composition probes). The review stopped early on its disk floor, so test runs and mutation checks were not completed.Part of Microcosm NZ v0 (d962), package G6 of the NZ hub's graph plan.
Verified, committed and opened by the NZ hub.
🤖 Generated with Claude Code