Skip to content

Export country-agnostic Microcosm graphs for Orrery, with the UK as the first consumer #1079

Description

@juaristi22

Microcosm needs a maintained route from any microcosm.graph declaration or compiled graph to a snapshot that Orrery can open. Build this as country-agnostic tooling. The UK will be its first consumer and the first complete integration example; the export API, schema and renderer-facing contract must work for other countries without changes to the shared exporter.

This is the first of three Microcosm workstreams: shared graph export, execution-evidence overlays, and variable lineage tracked in #865. Once all three are supported, use their outputs to assess remaining Orrery work and agree which repository should own it.

Deliverable

A documented, country-agnostic command/API that exports Microcosm declarations to Orrery's graph-explorer/v1 format, plus UK driver integration that saves the snapshot. The snapshot should expose readable steps, actual dependencies, source boundaries and population versions. Country adapters supply their own descriptions and grouping through a shared metadata contract. The shared exporter must not import UK build modules, branch on country names or depend on UK node-ID conventions.

Work

  • Reuse the adapter and tests in Export Microcosm schema metadata for Orrery #888. Supply its missing microcosm.graph.schema.v1 producer or adapt it to the current CompiledGraph; derive compiler facts and writer resolution from shared graph code.
  • Keep generic export in microcosm.graph; keep UK composition, labels and driver wiring in the UK adapter/build code. Country is metadata, not a selector for export algorithms.
  • Preserve operations, declared reads/writes, sources, typed artifact connections, population versions, structural changes and weight transitions. Keep stable domain IDs separate from display labels and content revisions.
  • Supply readable descriptions and presentation groups for sources, enrichment, geography, targets, calibration, checks and export. Groups must preserve real dependency edges. Descriptive edits must not invalidate numerical node keys.
  • Expose composite stages' existing operation descriptions while retaining their actual execution/cache boundary. For example, a coupled fit/draw stage must not appear as several independently executed nodes.
  • Support saved-graph export without executing kernels or reading licensed microdata. Add a documented export path to UK planning/build commands, retaining existing executable graph.json and using a distinct viewer filename such as graph.orrery.json.
  • State each export's scope: raw-spine/full dense build, checkpoint-based dense build or national build. Represent a checkpoint as an explicit boundary. Link upstream snapshots only through recorded checkpoint provenance; field names alone cannot establish continuity.
  • Make the earlier spine construction discoverable from a checkpoint-based view. Save the producing graph declaration/snapshot and its manifest identity with the checkpoint provenance, and export the exact artifact link into downstream builds. A viewer can then expand the earlier spine run without claiming those stages executed in the current national run. If upstream metadata is unavailable, expose that boundary explicitly rather than reconstructing historical lineage from today's declarations.
  • Serialize the final graph after terminal nodes are appended. In the national path, save the continued graph containing uk.full.national.readback alongside its final manifest. Refresh any inventory advertised as complete from that same final graph.

Acceptance criteria

  • The same public exporter handles a country-neutral synthetic graph and UK fixtures. A guard confirms that the shared exporter has no UK imports, country branches or UK node-name assumptions; another country needs only conforming graph/metadata inputs and driver wiring.
  • Synthetic UK spine, dense and national declarations export through the maintained public interface; dense coverage includes the optional household-size path.
  • Exported operation dependencies and typed artifact connections agree with the compiler, and all references resolve.
  • The saved national graph includes the final readback node; graph snapshots identify their exact scope and configuration.
  • A synthetic checkpoint-to-national example exports a verifiable link to the producing spine snapshot. The two runs remain distinguishable, and missing upstream metadata has an explicit representation.
  • Tests cover stable IDs, descriptive-edit invariance, deterministic output, large integer transport and explicit failure at export limits. Measure representative UK snapshot size before changing existing bounds; never silently truncate it.
  • A documented, pinned Orrery version parses the snapshot and opens a representative synthetic UK graph, with readable labels and usable groups. Record any viewer gaps for the later cross-repository assessment.
  • Document the commands, output files, scope boundaries and a one-sentence explanation of a kernel.

Relationship to #888

#888 implements the schema-to-viewer conversion and is useful foundation work. Merging it alone does not satisfy this issue: current main lacks the expected schema producer, UK driver wiring and the complete UK demonstration. Continue or merge that PR as a component, then complete this issue's remaining work. Supersede it only if implementation establishes that its contract needs replacement; preserve reusable code, tests and attribution.

Execution status and calibration results belong to #1080. Precise per-output variable lineage belongs to #865. Hosted build discovery and new Orrery features will be assessed after these Microcosm outputs exist. This export work does not require changing the national builder's checkpoint-based execution contract.

References: UK full-build graph, current serialization, Orrery document contract.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions