Skip to content

Weight download progress: observable on the wire, recoverable from any screen visit #492

Description

@JArmandoAnaya

Downloading a model's weights from the Inference screen shows no usable progress
and no state prose, and what little it shows is coupled to the screen staying
open. Leaving the screen and coming back — or reloading — loses the view of a
transfer that is still running.

cf. #434, #454, #470, #471.

Observed

  1. Create a local connection and press Download weights.
  2. The row reads Downloading… with a 1 of 1 microline beside it — the
    handler reports once, at the end, so the number is a placeholder rather than
    progress. There is no bar.
  3. Navigate away and back, or reload. The row reads Not set up with a
    Download weights button, exactly as if nothing were happening. The
    transfer is still running in the worker; nothing on screen says so.

What has to be true

  1. The download is a server-side job and nothing the browser does affects
    it.
    Leaving the screen, navigating elsewhere or closing the browser never
    cancels or pauses it.
  2. The screen is a viewport onto the job, not the owner of it. Arriving —
    first visit, return visit, fresh tab, reload — shows the current state and
    progress, because everything rendered derives from the wire and nothing from
    client-held process state.
  3. Progress is legible: a determinate bar plus prose for every state the job
    can be in.

Verified before designing anything

1. Job ↔ request coupling: none, and nothing to fix

JobRunner is a dispatcher thread plus a ProcessPoolExecutor(spawn) owned by
the FastAPI application's lifespan (src/visionset/jobs/runner.py). A request
enqueues a row and calls runner.wake(); the response is a 202 and the
request is over. Nothing in the claim/dispatch/settle path holds a client
connection, and the only two ways a run stops early are
JobQueue.request_cancel — an explicit act, unreachable from the download
screen — and JobRunner.stop() at server shutdown.

So requirement 1 already holds. Nothing in this issue changes it, and the
contract test below is what stops it quietly stopping to hold.

2. Current wire vocabulary during a download: nothing

ConnectionSetupState is two-valued as shipped after #470not_set_up and
ready — and there is no third member. The wire says nothing at all about a
download in flight: ConnectionOut carries setup_state, allowed_actions,
capabilities and the configuration, and a running download changes none of
them. The 202 from POST /inference/connections/{id}/download hands back a
BackgroundJobOut, and the job id lives in React state on the row
(useWeightsRun's useState<string | null>) — which is precisely the
client-held process state requirement 2 rules out, and precisely why a reload
loses the transfer.

A downloading member is deliberately not added. ConnectionSetupState's
own docstring settles it: the state flip is the last statement of
fetch_weights, so "a run that fails partway leaves the row exactly where it
was" and "there is no third state meaning 'half fetched' and no window in which
a caller could read one". A downloading value would reintroduce exactly that
window, and a worker killed mid-transfer would strand a connection in it
forever. The liveness of a download is the job's state, which settles itself
— including sweep_orphans at startup, which settles a job whose worker died.
So the download travels on the wire as the job it is, hung off the connection it
is about.

3. Total size: yes, and it prices the same file set

measure() (inference/weights.py) reads model_info(revision=…, files_metadata=True) and sums every sibling; download() calls
snapshot_download(repo_id=…, revision=…) with no allow_patterns or
ignore_patterns, so it fetches every file in that revision. The two describe
the same set by construction, which the measure docstring already states.
Curated entries pin a revision (#470) and a connection is required to carry one,
so the pair the job fetches is the pair that can be priced. DownloadSizes is a
process-wide LRU keyed on model_id@revision, so the worker's lookup is one
metadata call and usually a cache hit behind the form that already asked.

A sizing failure must not fail the download: the two reach the network
independently and a transfer that can run should run. So a total that cannot be
read leaves bytes_total null — which the wire and every bar in this product
already tolerate — and the bar is indeterminate for that run.

4. Progress source: bytes on disk, not a library callback

huggingface_hub (locked at 1.26) exposes no byte-level callback for a
snapshot. snapshot_download's only injection point is tqdm_class, which is
handed to the thread_map over files: it counts files completed, not bytes.
Per-file byte bars come from http_get, which builds its own hf_tqdm and
takes no bar from the caller. Mapping file counts onto bytes is not an
alternative — a repository is typically one multi-gigabyte .safetensors beside
ten tiny JSON files, so "10 of 11 files" is 1% of the transfer and a bar drawn
from it would sit at 91% for the whole download.

So the implementation samples bytes on disk: the cached repository's
blobs/ directory, .incomplete parts included, which is where a transfer in
flight actually accumulates. The repository's path is asked of
scan_cache_dir rather than assembled here — the rule cached_file already
follows, because the cache layout belongs to that library and a hand-built path
is a mirror that breaks on the release that reorganises it.

No version bump is needed for any of this.

Shape

Backend

  • The download handler reports bytes: bytes_total from measure() before
    the first byte, bytes_done sampled at a bounded interval while
    snapshot_download runs. Never per-chunk row writes — the existing
    SqliteProgressReporter throttle (0.5 s) is the write bound, and the sampler
    adds its own so the filesystem is not walked faster than a bar can move.
  • Progress is monotonic and clamped: a transfer that retries and re-reads
    must not move a bar backwards, and a sample must not exceed the total.
  • Storage is the job row's existing processed/total, which are exactly "an
    absolute count of the units this handler works in" — bytes, here, as the
    integrity check's are files. The names bytes_done/bytes_total are given at
    the one boundary that knows the job type, so no client has to know that
    processed means bytes for this type and files for that one.
    (Rejected: two new columns on every job row, plus a migration, a second
    reporter path and two nulls carried by every other job type — a second
    encoding of the same number.)
  • The connection's wire model gains the download: job id, state, and both byte
    counts, resolved from the queue by job type and connection id. This is what
    makes recovery automatic
    — a screen that reads the connection list has the
    download without ever having held a job id.
  • allowed_actions stays authoritative for what may be asked; the invariants
    hold unchanged — never-half-ready (the flip is still the last statement),
    idempotency (a re-request finds the same cache and settles), and failure
    carrying prose on the job rather than a raw exception.
  • Contract tests: the download job completes with no observer polling at any
    point
    ; a poll mid-download reports 0 < bytes_done < bytes_total and the
    shipped state vocabulary; allowed_actions rows for a connection with a live
    download.

Frontend

  • Everything rendered derives from the wire. The connection list polls on a
    conditional interval, active only while the wire reports a live download, so
    recovery on return or reload is automatic by construction — asserted by a
    browser test that starts a download, remounts the screen, and finds the bar
    and the prose.
  • A determinate bar built from the existing Progress primitive, with prose per
    state and human-readable sizes (312 MB of 1.4 GB) beside it. Compact
    progress on the row itself, so returning to the screen answers "how is it
    going" at a glance.
  • A failure renders as a destructive alert carrying the wire's own prose, with
    the retry driven by allowed_actions — never a bare disabled control
    (DESIGN.md principle 9).

Structure

Two PRs, sequential. openapi.json and the generated TS client are a shared
surface, so the backend and the wire land first and alone; the screen follows on
top of a regenerated client.

Metadata

Metadata

Assignees

No one assigned

    Labels

    apivisionset.server — REST API and OpenAPI contractbackendvisionset Python distribution — kernel, server, cli, mcp, formatsenhancementNew feature or requestfrontendannotator / ui-core / app packagesuxUser experience / interaction design

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions