+- The testing agent has two models on OpenRouter, separate from the app's provider mode, and a fake-mode stack still needs both. The actor (`agent.act` and explorations) is `inclusionai/ling-3.0-flash-vl` ($0.021 and $0.062 per million input and output tokens on Novita, with DeepInfra as fallback). Recorded flows replay without it, so it runs only on a cache miss and in explorations. A small model is enough to click through these flows: with the cache off it passed the actor-heavy specs for about a fifth of GPT-6 Luna's price, while Gemma 3 12B, cheaper still, failed most of them. OpenRouter withdrew the free Qwen tier the suite used before, which now fails with `This model is unavailable for free`. The judge (`agent.assert`, `waitFor`, `extract`) is `meta/muse-spark-1.3-contributor` ($0.10 and $0.20 per million input and output tokens), the same tier as the app's live pin. It answers every judgment on every run. A full run of both viewports makes about 250 judgments and costs about ten cents. It takes about 45 minutes on one machine with three workers, which is why CI splits it into four jobs. The judge cannot act, because Meta's endpoint accepts only `tool_choice: "auto"`. `E2E_AGENT_MODEL` and `E2E_JUDGE_MODEL` override the two. The judge's tier may keep prompts, so use made-up debate content only. A transient OpenRouter error such as `[Meta] The requested model was not found` is infrastructure: rerun with `--last-failed`. The agent's context explains that fake AI answers are canned free-transport text.
0 commit comments