You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Keeping up with provider model churn currently requires a Scala change and a release. Two provider families broke within about five months (see #62), and one broke with no published notice at all — so this will keep happening.
Some of the machinery already exists, but not enough of it:
What works today.ModelRegistryLoader reads models-override.yaml (or .yml) from the NetLogo model directory and completely replaces the bundled section for each provider it names (ModelRegistryLoader.scala:68-71, mergeConfigs at :101-108). A user can correct a stale model list without touching the extension.
What does not. Two gaps make that insufficient:
The override is per-model-directory. Every user, in every model folder, needs their own copy. There is no user-level or global override location, so a fix cannot be shared or applied once.
Defaults are compiled in.defaultModel lives in ProviderRegistrations.scala (:51 anthropic, :79 gemini, :107 ollama). No YAML file can change them. So llm:set-provider "gemini" with no explicit model still selects a dead model even when the registry has been corrected — which is exactly the failure in bug: three provider default models are missing from the bundled model registry #62.
Possible directions
A user-level override path — e.g. ~/.netlogo-llm/models.yaml, consulted after the bundled registry and before the model-directory override. One edit fixes every model on the machine.
Move defaults into models.yaml. A default: key per provider section, with the descriptor value used only as a last-resort fallback. Then a data-only edit fixes both the list and the default.
Query the provider's live model list. Most providers expose one (GET /v1/models, Gemini's models?key=). Could back llm:list-models, or refresh a cache on demand via a primitive. Caveat: the list endpoint is not always truthful — Gemini's returns gemini-2.5-flash for keys that then get a 404 on generateContent, so a live list alone would not have prevented bug: three provider default models are missing from the bundled model registry #62.
Ship a fetch script in the repo that regenerates models.yaml from provider APIs, so updating the bundle is a data commit rather than a code change.
Open questions
Is a global override the right layer, or does per-model-directory scoping matter for reproducibility (a shared .nlogox pinning its own model list)?
If defaults move to YAML, what happens when a user override omits default: — fall back to the descriptor, or the first model listed?
Should a live query ever run automatically, or only on an explicit primitive? Automatic network calls at load time would be surprising in a classroom with no connectivity.
Problem
Keeping up with provider model churn currently requires a Scala change and a release. Two provider families broke within about five months (see #62), and one broke with no published notice at all — so this will keep happening.
Some of the machinery already exists, but not enough of it:
What works today.
ModelRegistryLoaderreadsmodels-override.yaml(or.yml) from the NetLogo model directory and completely replaces the bundled section for each provider it names (ModelRegistryLoader.scala:68-71,mergeConfigsat:101-108). A user can correct a stale model list without touching the extension.What does not. Two gaps make that insufficient:
defaultModellives inProviderRegistrations.scala(:51anthropic,:79gemini,:107ollama). No YAML file can change them. Sollm:set-provider "gemini"with no explicit model still selects a dead model even when the registry has been corrected — which is exactly the failure in bug: three provider default models are missing from the bundled model registry #62.Possible directions
~/.netlogo-llm/models.yaml, consulted after the bundled registry and before the model-directory override. One edit fixes every model on the machine.models.yaml. Adefault:key per provider section, with the descriptor value used only as a last-resort fallback. Then a data-only edit fixes both the list and the default.GET /v1/models, Gemini'smodels?key=). Could backllm:list-models, or refresh a cache on demand via a primitive. Caveat: the list endpoint is not always truthful — Gemini's returnsgemini-2.5-flashfor keys that then get a 404 ongenerateContent, so a live list alone would not have prevented bug: three provider default models are missing from the bundled model registry #62.models.yamlfrom provider APIs, so updating the bundle is a data commit rather than a code change.Open questions
.nlogoxpinning its own model list)?default:— fall back to the descriptor, or the first model listed?Related