feat(inference): connections declare what their model can be asked for

[JArmandoAnaya]

A connection row said where a model runs and whether its weights are here, and
nothing anywhere said what it can be asked to do. So the editor picked by the
only property it could see — find(row => row.setup_state === "ready") — and with
a text-prompt detector as the only ready connection, every S click round-tripped
and came back with a truthful UnsupportedPrompt refusal. A tool offered where it
could never work, one refusal at a time.

Three commits: the kernel/wire half, the editor half, the docs.

Naming

Proposed	Against
capabilities, a list on ConnectionOut	allowed_actions' grammar — a list of a closed enum, empty rather than null. "Empty" was the decision's own word for the unknown case, and a list is what makes that spelling exist.
ModelCapability.POINT_SUGGEST / TEXT_DETECT	the decision's proposal, kept. They name the prompt, not the answer: a region comes back either way, so naming the output would collapse the two and lose the only distinction that decides whether a request can be made.
SuggestBlocker gains "not-capable"	"not-ready"'s shape, and it reads beside it.
visionset.inference.families	the module the decision asked for: "one home for the family→capability mapping, next to the family sets it derives from".

The mapping is derived, not written

CAPABILITY_BY_FAMILY is built from SEGMENTER_FAMILIES and DETECTOR_FAMILIES
rather than listed beside them. A hand-written map lets a family be added to a set,
get an adapter, ship, and declare nothing — the model runs and no client offers it,
which is invisible from both ends. Derived, an adapter and its declaration are one
edit, and test_the_mapping_covers_exactly_what_this_build_can_run says so.

The vocabulary is the kernel's and the mapping is not, deliberately: what a tool can
ask for is a domain word, while which model_type values this build serves is a
fact about an optional runtime the kernel has no view of.

Step 0, before designing: where the resolver runs

The halt condition was "the resolver cannot run where the design assumes". It can.
family_of reads the model's own config out of the workspace cache with
local_files_only, so at download completion the file is already there and no
network is touched. fetch_weights — the one sequence the job, the CLI and a future
UI all run — now ends with that read and records what it found. Nothing resolves at
connection creation, which keeps the form's promise intact.

It moved from providers.py into families.py verbatim. weights.py needs it and
providers.py imports weights.cache_root, so calling it in place would have closed
an import cycle. One resolver, one spelling, one module — which is also where the
capability map wanted to live.

Three states, not two. model_family is NULL for nobody has looked, "" for
looked and the config declared nothing, and a family otherwise. "" is the answer
family_of already gave an unreadable config; keeping it distinct is what stops a
look that found nothing from being repeated on every read for the life of the
workspace.

The backfill is on the read path, and that is a layering fact

A migration is the natural home and cannot be one: the answer lives in a model cache
under the workspace, migrations run inside the kernel, and the kernel may not import
visionset.inference (contract 1) nor address that cache. So with_families fills a
row in on the first read of it — bounded to local + ready + never-asked, one
small JSON read, no network, and never again once the row has any answer. Rows
created from here on arrive with a family from the download and never enter the loop.

A build without the optional runtime records nothing rather than recording that it
found nothing: a build that cannot look has not looked, and writing "" there would
let a machine that later installs the runtime go on believing an answer nobody
produced.

Precedence

checking → no-connections → not-ready → not-capable → usable.

not-ready outranks not-capable and the ordering is load-bearing: an undownloaded
connection has no capability yet, because nothing has read its config. Ranking
capability first would tell somebody their SAM connection answers the wrong question
when the truth is that its weights have not arrived. #473's parked-class precedence
is untouched and still outranks every blocker — a class that can hold no shape is
not a fact about connections.

Two things the dispatch assumed, checked

• The per-project preference mechanism exists. frontend/ui-core/src/data/prefs.ts
  (readPref/writePref, namespaced visionset.prefs., with the storage-throws guard
  and an in-memory fallback). The key is suggest.connection.<projectId>. Nothing new
  was invented.
• There is no footer line naming the active connection at HEAD. The dispatch says
  it "stays"; a grep over the annotator finds nothing that displays a connection's name
  or model. I added one, because the single-candidate case is otherwise the only path
  where the panel never says what is answering — it is the same line the picker
  replaces once there is a choice.

Bounds

No batch-mode surface: text_detect is declared and has no tool (cf. #425). No HTTP
adapter work — an http connection declares nothing, and how a remote endpoint states
what it can do is that slice's inherited question (cf. #418). The suggest route's
contract is unchanged and it still refuses a wrong-family request; the client filter is
convenience, the kernel is the law.

providers.py changed by having the family sets and family_of moved out of it, with
no behavioural edit — I read "no changes to providers" as no change to how a provider
is resolved, since the same decision explicitly asks for the mapping to sit beside
those sets. MCP has no inference surface at HEAD and gains none.

Mutation verification

Each applied to a committed tree, reverted by its exact diff, anchor asserted before
and after.

Mutation	Result
the capability filter removed (candidates = ready)	5 red — including the DINO-only reproduction and "sends no request at all over one"
fetch_weights stops passing model_family	2 red in tests/inference/test_weights.py. The server capability test survives, and correctly: the read-path backfill covers for the missing write. The download-time write is guarded by the inference tests, not the wire ones.
wire.connection publishes the key under another name	2 red — the CLI/REST key-parity gate
ConnectionOut.capabilities hardcoded to []	2 red — the wire declaration and the backfill-over-HTTP test
CAPABILITY_BY_FAMILY hand-written, missing sam2_video	2 red — the derivation gate and the per-family case
not-capable ranked above not-ready	4 red — both the unit and the flow assertions on the ordering

Found, not fixed

• A ready local connection stays ready when its model_id is edited, though the
  new model's weights are not on the machine. Pre-existing. This change makes the row
  forget its family on such an edit — a stale capability is worse than none — but the
  setup state is out of scope here.
• test_configuring_a_connection_reaches_no_model_runtime asserts a process-global
  sys.modules fact, so it is order-dependent across the whole suite. It stays true,
  and three fixtures that fake a download now also fake the config read to keep it that
  way, each saying why. Worth narrowing to the claim its own docstring makes.

Rebased onto 5f2c61c

Four PRs landed mid-flight (#478, #479, #483, #484). One conflict, in
AnnotationPage.tsx: #484 moved every floating editor message — the suggest panel
included — into one top-right EditorNotices column. Resolved by keeping that
structure and passing the three new props to the SuggestPanel inside it; the
panel chooses the sentence and #484 owns the geometry, which is the split its own
docstring states. The whole tree was re-linted and the entire gate re-run below on
the rebased tip, with uv sync and pnpm install for the two lockfile bumps.

CI caught one the local gate could not (attempt 1 → fixed)

python came back with 15 failures and they were a real defect, not a test
problem. CI runs without the local-inference extra; my worktree had it
synced. So the config read that now follows a download raised
LocalInferenceUnavailable — and took the download with it. The connection landed
back at not_set_up beside a full cache, offering as its remedy the transfer that
had just succeeded.

A download that worked must not be undone by a question about it. None is
recorded instead — not knowing is recoverable, and the next read of the
connection asks again — and with_families already followed exactly that rule for
a build that cannot look, so both now go through one function.

Reproduced and re-verified the way CI has it: uv sync with no extras
(transformers and huggingface_hub genuinely absent), then the whole suite in
one process — 3081 passed, 9 skipped, exit 0. That single-process run is also
what exercises the sys.modules import-purity guard across directories.

Test plan

Full local gate, staged (harness ceiling), every stage exit 0. The pytest stages
below ran with the optional runtime present; the whole-suite run without it is
the one above, and it is the one that matches CI.

Stage	Result
pytest tests/architecture	21 passed
pytest tests/cli	331 passed, 2 skipped
pytest tests/examples	41 passed
pytest tests/formats	151 passed, 2 skipped
pytest tests/inference	171 passed, 3 skipped
pytest tests/jobs	30 passed
pytest tests/kernel	1415 passed, 4 skipped
pytest tests/mcp	246 passed
pytest tests/packaging	3 passed, 1 skipped
pytest tests/server	640 passed, 2 skipped
pytest tests/test_versioning.py	3 passed
ruff check . / ruff format --check .	passed / 361 files formatted
mypy src/visionset	153 files, no issues
lint-imports	4 kept, 0 broken
pnpm -r build / pnpm -r lint	passed
pnpm test	annotator 914, ui-core 875
scripts/check.sh generated	passed
scripts/check.sh browser	both suites passed, 104s

openapi.json and frontend/ui-core/src/generated/ regenerated and committed. Load
average 1.9 on 20 cores — the full gate, not the saturation fallback.

New coverage: the family read (config over model id, unreadable config, no runtime),
the capability derivation and its three empty cases, the download-time write and the
re-download that repairs a missing one, the backfill and its bounds, the wire
declaration per kind and state, the backfill over HTTP with its read count, the
capability filter and the blocker's precedence, the picker's presence and absence, and
the per-project memory including that one project's choice does not reach another.

The reproduction is a test: a DINO-only workspace shows the blocker and sends no
request — asserted as the absence of the call, not as the shape of a refusal.

Closes #480