Batch pre-labeling — unattended model annotation as a background job

[JArmandoAnaya]

This feature spans the kernel, the job system and the review UI. It is the second of the two
auto-labeling modes, following the ordering decision recorded on the ModelProvider port issue
(#418): interactive first, batch second, and both ship.

Status: placeholder — design deliberately deferred

This issue exists to hold the responsibilities that recorded decisions have already assigned to
the batch mode, so that they live on an issue rather than only in comments scattered across other
threads. The full design session for the feature is intentionally deferred until the interactive
click-to-suggest mode (#424) is in real use, because this design has to absorb what interactive
usage measures: the on-domain suggestion acceptance rate, real confidence distributions — the
data point contributed by the work surfacing provenance and confidence in the review loop
(#417) — and the NMS and threshold behaviour from the measurement spike recorded on #418.

Responsibilities already settled (do not re-litigate here)

1. Entry state is review_pending. The decision recorded on the interactive auto-labeling
   issue (Interactive auto-labeling — click-to-suggest in the editor canvas #424) sets the entry state by who exercised the judgment. Unattended prediction is a
   silent write, so batch pseudo-labels enter at review_pending. The constraint preserved from
   the AI-assist substrate issue (exercise the AI-assist substrate — ModelProvider pre-annotation, review loop in the UI, importers #81) — "predictions are written into review_pending, never
   silently annotated" — applies here as written.
2. This issue owns the kernel-side entry path. The annotation write gate admits writes at
   {unannotated, annotated}, so batch entry to review_pending requires its own sanctioned
   kernel path. Designing that path, without weakening the existing gate, is this issue's central
   kernel task.
3. Runs through a user-configured model connection (Inference section — user-configured model connections #421), via the ModelProvider port
   (ModelProvider: declare the kernel port for autolabeling — local and cloud adapters both ordinary #418), as a background job on the existing job infrastructure. Nothing is bundled, and
   nothing is downloaded at install time.
4. Raw model output gets cross-box NMS before entering, which is a finding from the
   measurement spike recorded on ModelProvider: declare the kernel port for autolabeling — local and cloud adapters both ordinary #418.
5. Provenance is provenance: model plus model_ref plus confidence — the same domain
   spelling that Model-produced annotations: provenance and confidence surfaced in the review loop #417 and Interactive auto-labeling — click-to-suggest in the editor canvas #424 use.

Open for the future design session (not now)

The teacher model default and prompting strategy; per-class thresholds; review-queue UX at
volume, meaning bulk accept and reject and confidence-sorted ordering, which connects directly to
#417; job scoping across batch, project and filter; and the interaction with the interactive tool
as the correction instrument.

Non-goals

• Anything that accelerates this issue's own schedule: nothing blocks on it while the interactive
  mode goes first.

[JArmandoAnaya]

Design rationale for responsibility 2, recorded here so that it survives to the design session.

The review_pending entry path must be atomic in the kernel rather than composed from existing sanctioned moves. The naive composition — write annotations at annotated, then transition the asset to review_pending — is legal move by move, since review_pending is reachable only from annotated and writes are legal at {unannotated, annotated}. That legality is exactly what makes it dangerous. The composition passes through annotated, so a job that crashes in that window leaves an asset carrying unreviewed model labels at annotated, which is precisely the state the constraint preserved from #81 forbids. An implementer who saw the composition but not the crash window would "simplify" the design into a violation.

The sanctioned path is therefore a single kernel operation — write-with-provenance and progress in one transaction — guarded like the other narrow port methods, rather than a sequence the job runner orchestrates. The batch job may crash before an asset is touched, or after it is fully entered at review_pending, but never in between. This applies the set_asset_progress precedent to a compound write: a narrow port method, an optimistic guard on the contended datum, and no version columns for their own sake.

The port this runs against is ModelProvider (#418), and the entry-state rule it honours comes from the interactive mode's decision (#424).

[JArmandoAnaya]

Joining this design session from the work that surfaces provenance and confidence in the review loop (#417): confidence triage — ordering or narrowing the review flow by model confidence.

It moved here because it is a wire question and a batch-review question at the same time, and #417 could only have answered it by designing a wire that this session will design anyway. That issue keeps criteria 1 and 2 — per-annotation provenance and confidence rendering, both frontend-only — and ends on those.

Three sub-decisions travel with it:

• The per-asset aggregate. Confidence is per annotation, while the review flow orders assets. A minimum, a mean, and a count below a threshold are three different queues.
• Order or narrow, and any threshold that choice implies, taking the observed score distribution as input rather than an assumed one.
• Server-side segment counts. GalleryScreen takes its counts from the batch's ProgressCounts rather than from loaded pages, so a confidence narrow that honours that rule needs its counts answered server-side too.

For the session's reference, here is the current wire: BatchAssetOut carries no annotation count and no confidence aggregate, and the asset-list endpoints take limit and offset only.

[JArmandoAnaya]

An operational note for the review surfaces, joining the three sub-decisions inherited from the provenance-and-confidence work (#417).

A confidence score means something different per capability, so the two cannot be pooled.

• Point-prompted segmentation scores mask quality against a human-placed prompt. Observed range 68–98% (2026-08-10).
• Text detection scores prompt affinity. Observed range 37–78%, as recorded on Model-produced annotations: provenance and confidence surfaced in the review loop #417.

Two consequences follow for anything built here.

1. Triage aggregates, thresholds and queues are per-capability. A single "below 60%" filter over a mixed batch selects almost every detection and almost no segmentation, which is a fact about the two scales rather than about the labels.
2. Any review surface that shows the number names what it measures. A bare percentage beside a shape is not readable without knowing which of the two scales it sits on.