Skip to content

feat: update model lists and defaults without a Scala change #64

Description

@JNK234

Problem

Keeping up with provider model churn currently requires a Scala change and a release. Two provider families broke within about five months (see #62), and one broke with no published notice at all — so this will keep happening.

Some of the machinery already exists, but not enough of it:

What works today. ModelRegistryLoader reads models-override.yaml (or .yml) from the NetLogo model directory and completely replaces the bundled section for each provider it names (ModelRegistryLoader.scala:68-71, mergeConfigs at :101-108). A user can correct a stale model list without touching the extension.

What does not. Two gaps make that insufficient:

  1. The override is per-model-directory. Every user, in every model folder, needs their own copy. There is no user-level or global override location, so a fix cannot be shared or applied once.
  2. Defaults are compiled in. defaultModel lives in ProviderRegistrations.scala (:51 anthropic, :79 gemini, :107 ollama). No YAML file can change them. So llm:set-provider "gemini" with no explicit model still selects a dead model even when the registry has been corrected — which is exactly the failure in bug: three provider default models are missing from the bundled model registry #62.

Possible directions

  • A user-level override path — e.g. ~/.netlogo-llm/models.yaml, consulted after the bundled registry and before the model-directory override. One edit fixes every model on the machine.
  • Move defaults into models.yaml. A default: key per provider section, with the descriptor value used only as a last-resort fallback. Then a data-only edit fixes both the list and the default.
  • Query the provider's live model list. Most providers expose one (GET /v1/models, Gemini's models?key=). Could back llm:list-models, or refresh a cache on demand via a primitive. Caveat: the list endpoint is not always truthful — Gemini's returns gemini-2.5-flash for keys that then get a 404 on generateContent, so a live list alone would not have prevented bug: three provider default models are missing from the bundled model registry #62.
  • Ship a fetch script in the repo that regenerates models.yaml from provider APIs, so updating the bundle is a data commit rather than a code change.

Open questions

  • Is a global override the right layer, or does per-model-directory scoping matter for reproducibility (a shared .nlogox pinning its own model list)?
  • If defaults move to YAML, what happens when a user override omits default: — fall back to the descriptor, or the first model listed?
  • Should a live query ever run automatically, or only on an explicit primitive? Automatic network calls at load time would be surprising in a classroom with no connectivity.
  • bug: model override registry leaks across model directories #29 (registry state leaks across model directories) touches the same loader and should probably be resolved alongside this.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions