feat(download): pull oci:// models via llmman serve

[ericcurtin]

What

Adds an oci:// scheme so a model published as a CNCF ModelPack artifact can be used anywhere a HuggingFace repo id works today.

Model distribution is increasingly moving to OCI registries -- the same registries, credentials, mirroring and air-gap tooling a deployment already uses for container images. For an isolated or bandwidth-constrained cluster that is often easier to run than reaching the Hub, and it lets one local registry mirror serve every node.

How

download_shard gains an early branch. An artifact is pulled whole, so there is no per-file list to walk and no allow_patterns counterpart -- the shard-aware pattern filtering simply does not apply. The daemon's aggregate progress is mapped onto the same RepoDownloadProgress the HuggingFace path emits, so the cluster UI and on_progress contract are unchanged.

The blocking pull runs via asyncio.to_thread so the event loop is not stalled, and skip_download=True resolves through llmman resolve --no-pull without pulling anything.

Acquisition is delegated to a running llmman serve rather than hand-rolled: llmman already implements the ModelPack media types, registry auth, resumable blob download and a content-addressed store.

New src/exo/download/llmman.py is the daemon client, stdlib-only (urllib), no new dependency:

• GET /api/version probes reachability and identity -- a server answering without a version field is reported as "not an llmman daemon", worth distinguishing from nothing listening.
• POST /api/pull streams NDJSON. An error arrives in-band at HTTP 200, and a stream that ends without success is also a failure -- both are errors, not a completed pull.
• llmman resolve --no-pull reports where the bytes landed; --no-pull keeps the daemon the only thing that touches the network.
• LLMMAN_HOST is honoured with llmman's own parsing, including rewriting a wildcard bind (0.0.0.0, [::]) to loopback.

A pull needs both the daemon reachable and the binary on PATH (or EXO_LLMMAN_BIN); each missing piece has its own actionable error, and neither is required unless an oci:// model id is used.

Explicit scheme, no sniffing. A bare registry/name:tag is indistinguishable from a HuggingFace repo id (mlx-community/Llama-3-8B-Instruct-4bit); guessing would silently hijack existing model ids. Every other id reaches exactly the path it did before.

Testing

New src/exo/download/tests/test_llmman.py, running against a real HTTP server on a loopback port rather than mocks, so the NDJSON contract is genuinely exercised.

$ pytest src/exo/download/tests/test_llmman.py
22 passed

All 22 executed here. Coverage: /api/version accepted, a non-llmman server rejected, nothing-listening reported actionably; pull success with forwarded byte progress and the exact request body asserted; in-band error at HTTP 200; a stream ending without success; non-OK status; a non-JSON diagnostic tolerated; scheme detection incl. case-insensitivity; that a HF repo id, a bare registry ref and a local path are not claimed; strip_scheme round-trips; empty reference rejected; every LLMMAN_HOST form incl. wildcard-to-loopback.

• ruff format and ruff check clean on both new files. download_utils.py reports 14 ruff findings both before and after this change (verified against a stashed clean tree), so none are introduced here.

Not verified here, flagged rather than implied: download_shard_from_oci itself is covered by inspection rather than execution -- exercising it needs the full exo stack (mlx/torch), unavailable in this environment. No end-to-end cluster run against a live llmman serve.

Disclosure: written with AI assistance.