0.2.0 scope: import of external datasets

[JArmandoAnaya]

Draft — planning seed, not an implementation ticket. Captures partial information as of 2026-08-06; a dedicated design session will produce the real spec. Do not pick up for implementation.

The settled frame (Armando, 2026-08-06)

An import lands into a project. Point VisionSet at an existing labeled
dataset — COCO, YOLO, … — and materialize it as a project:

• Assets through the existing content-hash ingest.
• A schema derived from the dataset's own classes.
• Annotations as the initial trunk.

That is the shape. Everything below is what HEAD already provides for it, and
what is genuinely new.

The reuse thesis

Conceptually an ingest variant, not a new subsystem. The three mechanics an
import needs are shipped and load-bearing:

• Dedup. Asset identity is the SHA-256 of the content
  (Asset.content_hash); the same bytes ingested twice are the same asset, held
  by uq_asset_project_content_hash. An import re-running is free rather than
  duplicative, and an imported image that also arrives from a directory ingest
  is one asset.
• Schema versioning. SchemaService is the one door to an
  AnnotationSchema; versions are 1..N, never updated or deleted, and "active"
  is derived (highest version), not stored. A derived schema is just version 1.
• Trunk mechanics. Annotations hang off asset_id and nothing else;
  DatasetService promotes completed batches into the trunk as a union against
  current membership, idempotent, logged.

The genuinely new work is two things:

1. Per-format readers — the import-side twin of the export plugins. The port
   already exists: kernel/ports/importer.py declares a @runtime_checkable Importer Protocol (format_name, read(src: Path) -> Iterable[Annotation])
   discovered through the same visionset.formats entry-point group, and
   formats/registry.py filters exporters by port explicitly "because the group
   carries importers too". Nothing implements it. provenance="import" — one of
   the three values in Provenance = Literal["human", "model", "import"] — has
   no producer today.
2. The derive-schema step — turning a dataset's class list into a
   LabelClass tuple. Every derived class needs a geometry, and
   SchemaService.create_version refuses any outside IMPLEMENTED_GEOMETRIES
   (bbox, polygon, polyline, classification_tag at HEAD), so this step is
   also where a format carrying something VisionSet cannot represent first
   becomes a refusal.

Open questions (to name, not to answer here)

• The fidelity contract on import. The export side has a settled answer — a
  format declares lossy, supported_geometries and degraded_geometries, and
  a lossy export is refused until the caller consents. Mirror it: what happens
  to fields VisionSet cannot represent on the way in? Refuse the import,
  import-with-consent and report, or import and drop silently (which the export
  side explicitly rejects)? Is the declaration per format, like lossy, or
  computed per source dataset, since an import can actually look at the data —
  an advantage export never has.
• Provenance marking. provenance="import" is the obvious mark and is
  already in the domain. Annotation.job_id is the finer question: it says
  which round of work produced this label, and None means genuinely unknown.
  Imported annotations belong to no round — so is job_id: null the honest
  answer (the same shape the migration used for labels it could not attribute),
  or does an import synthesize a round so the lineage has a name? Related: a
  batch cut over already-labeled assets opens seeded (BatchService.approve
  starts an already-labeled asset at annotated), which is exactly the state an
  imported project's first correction batch would be in — that behavior comes for
  free and should be checked against, not designed around.
• One-shot or re-syncable. One-shot materialization is the working
  assumption. Re-sync — watching an external dataset and reconciling changes —
  means tracking external identity, resolving conflicts against local edits, and
  a notion of upstream that the model does not have. That belongs to a future
  commercial tier (separate project, not started), and the milestone's product
  boundary (OSS stays a simple local server, MLflow-style) is the reason to say
  so out loud rather than leave it open by omission.

References

• cf. #81 — its Importer bullet at HEAD: "Importer port made real (COCO/YOLO
  in) — the third provenance value, import, currently has no producer; imports
  are also the cheapest path to pre-annotations from external models."
• cf. #365 — the formats scope issue in this milestone, for the export-side
  lossiness contract this one mirrors.