model-gateway: derive pin-alias names from the deployed GGUFs

ordo/gpu.py

@@ -68,9 +68,9 @@
 CAPABILITY when it does. Not all preemptible services are alike:
 
   - A burst render has no degraded path — reclaiming it means the work stops. ``strategy: stop``.
-  - The resident LLM does: it MIGRATES rather than dies. ``llamacpp-cpu`` runs the same Qwen3.6
-    A3B on CPU at the same 131072 window, and the model-gateway (LiteLLM) already carries
-    ``fallbacks: [{local-chat: [qwen3.6-35b-a3b-cpu]}]`` with a 30s cooldown that routes back to
+  - The resident LLM does: it MIGRATES rather than dies. ``llamacpp-cpu`` runs its own CPU
+    chat model at the same context window, and the model-gateway (LiteLLM) already carries
+    ``fallbacks: [{local-chat: [<cpu pin alias>]}]`` with a 30s cooldown that routes back to
     the GPU model as soon as it is healthy again. So llama.cpp yielding the 5090 is a
     GPU→CPU migration: availability is preserved, only throughput degrades. That is declared as
     ``strategy: failover`` with the degraded target and the component that reroutes named as

services/model-gateway/README.md

@@ -9,11 +9,16 @@ entrypoint that templates the config placeholders at startup.
 
 | id | mode | backend | notes |
 |----|------|---------|-------|
-| `local-chat` | chat | `llamacpp` (GPU) | Canonical auto-routing alias; fails over to `qwen3.6-35b-a3b-cpu` on GPU eviction. |
-| `qwen3.6-27b` | chat | `llamacpp` (GPU) | Explicit pin of the GPU deployment — no failover. |
-| `qwen3.6-35b-a3b-cpu` | chat | `llamacpp-cpu` | Always-on CPU MoE fallback (opt-in `cpu-fallback` profile). |
+| `local-chat` | chat | `llamacpp` (GPU) | Canonical auto-routing alias; fails over to the CPU pin on GPU eviction. |
+| *\<gpu model\>* | chat | `llamacpp` (GPU) | Explicit pin of the GPU deployment — no failover. Name derived from `LLAMACPP_MODEL` (basename, lowercased; e.g. `qwen3.8-27b-q6_k`). |
+| *\<cpu model\>*`-cpu` | chat | `llamacpp-cpu` | Always-on CPU fallback (opt-in `cpu-fallback` profile). Name derived from `LLAMACPP_CPU_MODEL`. |
 | `local-embed` | embedding | `llamacpp-embed` | nomic-embed-text-v1.5, 768-dim, ctx 8192. |
 
+The two pin-alias names are **derived at startup from the deployed GGUF filenames** — a
+model swap renames them automatically, so nothing version-named is hardcoded in the config.
+Only `local-chat` and `local-embed` are stable ids; anything that must survive a model swap
+should use those.
+
 Every entry carries a fully populated `model_info` block (mode, context window, output cap,
 capability flags, description, backing GGUF) — that block IS the gateway's model
 documentation for clients. Deployment-variable values are placeholder-templated by the
@@ -29,10 +34,14 @@ metadata tracks the running deployment instead of drifting:
 | `__GPU_WEIGHTS__` | `LLAMACPP_MODEL` (model.gguf) |
 | `__CPU_WEIGHTS__` | `LLAMACPP_CPU_MODEL` (Qwen3.6-35B-A3B-UD-Q4_K_M.gguf) |
 | `__EMBED_WEIGHTS__` | `LLAMACPP_EMBED_MODEL` (nomic-embed-text-v1.5.Q4_K_M.gguf) |
+| `__GPU_IMAGE__` | `LLAMACPP_IMAGE` (llama.cpp) |
+| `__GPU_MODEL_NAME__` | derived: `LLAMACPP_MODEL` basename, lowercased, `.gguf` stripped |
+| `__CPU_MODEL_NAME__` | derived: `LLAMACPP_CPU_MODEL` basename, lowercased, + `-cpu` |
+| `__GPU_SUPPORTS_VISION__` | derived: `true` iff `LLAMACPP_MMPROJ` is non-empty |
 
-The `supports_*` capability flags (tools, vision, reasoning) describe the deployed Qwen3.6
-family and are static in the config — revisit them if the active model swaps to a different
-model family.
+`supports_vision` is derived from whether the GPU server actually loads an mmproj. The
+remaining `supports_*` flags (tools, reasoning) describe the llama-server invocation
+(`--jinja`, `--reasoning-format`) rather than a model family, and are static in the config.
 
 The Ordo stack references it as a **project buildable image** (`ordo/model-gateway:latest`) — pinned by
 its build context, not pulled from a registry — so `ordo preflight` reports a missing one as

services/model-gateway/entrypoint.sh

@@ -16,14 +16,28 @@ CPU_CTX_SIZE="${LLAMACPP_CPU_CTX:-131072}"
 GPU_WEIGHTS="${LLAMACPP_MODEL:-model.gguf}"
 CPU_WEIGHTS="${LLAMACPP_CPU_MODEL:-Qwen3.6-35B-A3B-UD-Q4_K_M.gguf}"
 EMBED_WEIGHTS="${LLAMACPP_EMBED_MODEL:-nomic-embed-text-v1.5.Q4_K_M.gguf}"
+GPU_IMAGE="${LLAMACPP_IMAGE:-llama.cpp}"
+
+# The pickable pin-alias NAMES derive from the deployed weights (basename, lowercased,
+# .gguf stripped) — a model swap renames them automatically, so the template never
+# hardcodes a model generation. `local-chat`/`local-embed` stay stable by contract.
+GPU_MODEL_NAME="$(basename "${GPU_WEIGHTS}" .gguf | tr '[:upper:]' '[:lower:]')"
+CPU_MODEL_NAME="$(basename "${CPU_WEIGHTS}" .gguf | tr '[:upper:]' '[:lower:]')-cpu"
+
+# Vision support is a fact about the deployment (is an mmproj loaded?), not the template.
+if [ -n "${LLAMACPP_MMPROJ:-}" ]; then GPU_SUPPORTS_VISION=true; else GPU_SUPPORTS_VISION=false; fi
 
 sed -e "s|__MASTER_KEY__|${MASTER_KEY}|g" \
     -e "s|__CTX_SIZE__|${CTX_SIZE}|g" \
     -e "s|__N_PREDICT__|${N_PREDICT}|g" \
     -e "s|__CPU_CTX_SIZE__|${CPU_CTX_SIZE}|g" \
     -e "s|__GPU_WEIGHTS__|${GPU_WEIGHTS}|g" \
     -e "s|__CPU_WEIGHTS__|${CPU_WEIGHTS}|g" \
-    -e "s|__EMBED_WEIGHTS__|${EMBED_WEIGHTS}|g" /app/config.template.yaml > /tmp/config.yaml
+    -e "s|__EMBED_WEIGHTS__|${EMBED_WEIGHTS}|g" \
+    -e "s|__GPU_IMAGE__|${GPU_IMAGE}|g" \
+    -e "s|__GPU_MODEL_NAME__|${GPU_MODEL_NAME}|g" \
+    -e "s|__CPU_MODEL_NAME__|${CPU_MODEL_NAME}|g" \
+    -e "s|__GPU_SUPPORTS_VISION__|${GPU_SUPPORTS_VISION}|g" /app/config.template.yaml > /tmp/config.yaml
 
 # LiteLLM's proxy callback importer (get_instance_fn in
 # litellm/proxy/types_utils/utils.py) resolves "module.attr" relative to the

services/model-gateway/litellm_config.yaml

@@ -5,11 +5,12 @@ model_list:
   #
   # model_info blocks are the gateway's model documentation: everything /model/info and
   # /v1/models consumers see. All deployment-variable values (__CTX_SIZE__, __N_PREDICT__,
-  # __CPU_CTX_SIZE__, __*_WEIGHTS__) are entrypoint-substituted from the SAME .env the
-  # backend llama-server containers read, so this metadata cannot drift from what's running.
-  # The supports_* flags describe the deployed Qwen3.6 family (vision mmproj loaded on the
-  # GPU server, --jinja tool templates, --reasoning-format deepseek) — revisit them if the
-  # active model swaps to a different family.
+  # __CPU_CTX_SIZE__, __*_WEIGHTS__, __GPU_IMAGE__, __*_MODEL_NAME__,
+  # __GPU_SUPPORTS_VISION__) are entrypoint-substituted from the SAME .env the backend
+  # llama-server containers read, so this metadata — including the pickable pin-alias
+  # NAMES and the vision flag — cannot drift from what's running. The remaining static
+  # supports_* flags describe the llama-server invocation itself (--jinja tool templates,
+  # --reasoning-format), not any particular model family.
   - model_name: "local-chat"
     litellm_params:
       model: "openai/local-chat"
@@ -28,21 +29,23 @@ model_list:
       supports_function_calling: true
       supports_tool_choice: true
       supports_response_schema: true
-      supports_vision: true
+      supports_vision: __GPU_SUPPORTS_VISION__
       supports_reasoning: true
       description: >-
         Stable auto-routing alias — the id every Ordo consumer should use. Backed by the
         active GPU chat model on the RTX 5090 (see weights_file for the exact GGUF); fails
-        over to qwen3.6-35b-a3b-cpu when the scheduler leases the GPU to a media render, and
-        recovers automatically once the GPU model is healthy again.
+        over to the CPU deployment (__CPU_MODEL_NAME__) when the scheduler leases the GPU
+        to a media render, and recovers automatically once the GPU model is healthy again.
       weights_file: __GPU_WEIGHTS__
-      served_by: llamacpp (patched qwen36-swa llama.cpp build, RTX 5090)
+      served_by: llamacpp (__GPU_IMAGE__, RTX 5090)
 
   # Explicit, pickable REAL models so callers/UI see WHICH model they're using, not just the
-  # opaque `local-chat`. qwen3.6-27b = the GPU chat model; qwen3.6-35b-a3b-cpu = the always-on
-  # CPU fallback (also local-chat's failover target). The CPU one is only reachable when the
+  # opaque `local-chat`. Their NAMES are derived from the deployed GGUFs by the entrypoint
+  # (never hardcode a model generation here — it goes stale on the next swap):
+  # __GPU_MODEL_NAME__ = the GPU chat model; __CPU_MODEL_NAME__ = the always-on CPU
+  # fallback (also local-chat's failover target). The CPU one is only reachable when the
   # opt-in `llamacpp-cpu` service is running; otherwise it just sits unavailable (harmless).
-  - model_name: "qwen3.6-27b"
+  - model_name: "__GPU_MODEL_NAME__"
     litellm_params:
       model: "openai/local-chat"
       api_base: "http://llamacpp:8080/v1"
@@ -59,16 +62,17 @@ model_list:
       supports_function_calling: true
       supports_tool_choice: true
       supports_response_schema: true
-      supports_vision: true
+      supports_vision: __GPU_SUPPORTS_VISION__
       supports_reasoning: true
       description: >-
         Explicit pin of the GPU chat deployment (same backend as local-chat but with NO CPU
-        failover routing — errors when the GPU model is evicted for a render). Qwen3.6-27B
-        finetune, Q6_K, hybrid attention/SSM, vision via the shared Qwen3.6-27B mmproj.
+        failover routing — errors when the GPU model is evicted for a render). See
+        weights_file for the exact GGUF; supports_vision reflects whether an mmproj is
+        loaded on the GPU server.
       weights_file: __GPU_WEIGHTS__
-      served_by: llamacpp (patched qwen36-swa llama.cpp build, RTX 5090)
+      served_by: llamacpp (__GPU_IMAGE__, RTX 5090)
 
-  - model_name: "qwen3.6-35b-a3b-cpu"
+  - model_name: "__CPU_MODEL_NAME__"
     litellm_params:
       model: "openai/local-chat"
       api_base: "http://llamacpp-cpu:8080/v1"
@@ -88,9 +92,9 @@ model_list:
       supports_vision: false
       supports_reasoning: true
       description: >-
-        Always-on CPU fallback and local-chat's failover target: Qwen3.6-35B-A3B MoE
-        (~3B active params) running CPU-only on stock llama.cpp. Context matches the GPU
-        window so failover never truncates a conversation. Only reachable while the opt-in
+        Always-on CPU fallback and local-chat's failover target (see weights_file for the
+        exact GGUF), running CPU-only on stock llama.cpp. Context matches the GPU window
+        so failover never truncates a conversation. Only reachable while the opt-in
         cpu-fallback profile (llamacpp-cpu) is running. Slow but survives GPU eviction.
       weights_file: __CPU_WEIGHTS__
       served_by: llamacpp-cpu (stock llama.cpp, CPU-only)
@@ -126,7 +130,7 @@ general_settings:
 # the always-on CPU A3B so chat/agents keep working instead of erroring — then recovers to the
 # GPU model as soon as it's healthy again (short cooldown). No effect until llamacpp-cpu runs.
 router_settings:
-  fallbacks: [{"local-chat": ["qwen3.6-35b-a3b-cpu"]}]
+  fallbacks: [{"local-chat": ["__CPU_MODEL_NAME__"]}]
   cooldown_time: 30
   num_retries: 1