tools: check the WebUI catalog in the loader sync check

[CryptVenture]

Replaces #376, split as requested: this PR is the checker only. It changes no catalog entry, no spec and no UI file — the catalog corrections it makes visible follow as their own PRs.

The gap

check_loader_catalog_sync.py compared registered loaders, model_specs and model_manager_v2.py, then printed "in sync" without ever opening webui/configs/models_catalog.json — the file that decides which models the WebUI shows. A catalog entry could name a download_id that is not a package id, or a path no package installs into, and nothing in CI would see it.

The pass this adds

• the entry's family exists in model_specs
• download_id is an exact packages[].id in that entry's own family
• path matches the resolved package's target_directory or an installed file
• every spec family with installable GGUF packages has a catalog entry
• every distinct installable GGUF target directory is reachable — this is what catches shipped-but-unreachable checkpoints
• each spec's default and ui.recommended_package can actually be selected
• task and mode are checked against the vocabularies parsed out of session.cpp, not a hardcoded list
• duplicate entry ids
• model_params.json groups resolve to a family and to options that family declares

Exclusions are derived, not listed: a package counts as installable only when its effective download.kind is huggingface_snapshot, so the families carrying kind: unsupported for licence reasons drop out on their own. The summary line now separates manager_packages from server_packages, which exposes that the Python tool knows packages the native manager filters out.

It reports the current state, it does not demand a rewrite

New findings are advisories by default. Against main as it stands:

$ python3 tools/check_loader_catalog_sync.py
active_loaders=67 commented_loaders=0 specs=65 packages=189 manager_packages=189
  server_packages=185 catalog_entries=85 task_kinds=14
warning: ... catalog entry 'seed-vc' download_id 'seed_vc' is not a packages[].id in
  model_specs/seed_vc.json (advisory; --strict-catalog makes this fail)
... 85 more
ok: runtime loaders, model_specs and model_manager_v2 are in sync;
    86 WebUI catalog advisories reported (--strict-catalog fails on them)
$ echo $?
0

So the check can land now, on the current catalog, without changing a byte of data. --strict-catalog promotes the advisories to failures and exits 1 — that is the flag to turn on in CI once the catalog corrections have landed.

Validation

python3 tools/check_loader_catalog_sync.py                  # exit 0, 86 advisories
python3 tools/check_loader_catalog_sync.py --strict-catalog # exit 1, 86 findings
python3 tools/check_loader_catalog_sync.py --self-test      # Ran 10 tests, OK

The self-test suite grew with the pass: package install paths, download_id resolution, path matching, unknown task/family rejection, unreachable directory and default reporting, unsupported downloads counted as not installable, and model_params group resolution.

Scope

One file, tools/check_loader_catalog_sync.py. No build, runtime or model behaviour touched.

[0xShug0]

@CryptVenture Thank you for all the PRs. It’s getting a bit overwhelming to review them all at once. I do manual review and testing for PRs, so my review bandwidth is limited. I also need to balance that time across new models, new features, bug fixes, and other ongoing work in the project.

We recently added a policy limiting each user to 3 concurrent PRs, including draft PRs. I thought GitHub’s pull request limits would enforce this, but apparently draft PRs are not covered.

I already have three of your PRs open. Please keep the three PRs you think I should review first and close the others for now. You can reopen or resubmit them as the current ones are merged or closed.

[CryptVenture]

Understood, and apologies for the flood — that was my doing, not a gap in the policy.

I have closed 25 and kept three. All three are in app/server/, so they read as one subsystem rather than three contexts:

• server: take the state lock around model unload #402 — server: gate model unload and take the state lock around it. /v1/tasks/unload_models and /v1/tasks/unload_all_models were reachable without --ui-management, and the model lookup ran outside models_mutex_.
• server: add /v1/audio/transcriptions/details for transcript detail fields #401 — server: return ASR detail fields from /v1/audio/transcriptions and /v1/tasks/run. Word timings and segments were being dropped on both routes; the WebUI transcript view has nothing to render.
• server: forward language only where the model's contract accepts it #400 — server: forward language only where the model's contract accepts it. It is currently sent to models that reject it, and the request fails with a 500.

Each is a single commit touching one or two files, with the before/after evidence in the description, and CI is green on all three.

The rest are closed rather than abandoned — I will bring them back a few at a time as these merge or close, in an order that avoids stacking conflicts. The four older audio PRs (#359, #360, #365, #367) had also gone stale against main; I will rebase those before resubmitting.

Thanks for the review time, and for saying so plainly — the 3-PR cadence works better for me too.

[CryptVenture]

Closing for now to stay inside the 3-concurrent-PR policy (see the discussion on #422). Nothing is wrong with the change and CI is green on it; I will reopen it as review slots free.

[0xShug0]

Thank you!