ModelRouter

A weekend side project. I wanted an auto model mode for a coding agent CLI, the kind where you stop picking Opus vs something cheaper on every message. I looked at how commercial agents seem to do it, tried to copy the shape of the idea, and this is what I ended up with.

It is not production code. It is not affiliated with anyone. It is a rough imitation with my own names, my own file layout, and a Python eval harness so I can poke at routing without paying for classifier calls.

What's in the box

Piece	What it does
src/route-core/	TypeScript routing core (signals → scorer → cost-aware selector)
eval/	Python port of the selector + prompt builder, runs offline
docs/	Notes I wrote while figuring this out

No build step. No npm install. Clone and read, or run the eval:

cd eval
python3 harness.py run

End-to-end flow

You pick auto. Before the expensive model runs, a small model scores a few fixed tiers. A selector picks the tier. The result gets cached for the session.

Everything below runs on the client. No routing microservice. Just prompts, JSON scores, and a selector function.

sequenceDiagram
    participant User
    participant CLI
    participant Signals as buildSignalsFromHistory
    participant Scorer as ScoreClassifier
    participant Selector as costAwareSelector
    participant Cache as session cache
    participant Agent as agent turn

    User->>CLI: pick "auto" + send message
    CLI->>Signals: conversation history
    Signals-->>CLI: ClassifierSignals
    CLI->>Scorer: signals + model cards
    Scorer-->>CLI: JSON scores per tier
    CLI->>Selector: scores + tier costs
    Selector-->>CLI: primary tier + fallbacks
    CLI->>Cache: store resolved tier
    CLI->>Agent: run turn on chosen tier
    Note over Agent: on failure → SAFE_FALLBACK_MODEL

Loading

More detail in docs/architecture.md.

How the selector decides

VIABILITY_THRESHOLD is 0.7. The rule is simple on purpose.

flowchart TD
    A[Classifier scores for each tier] --> B{Any tier ≥ 0.7?}
    B -->|yes| C[Pool viable tiers]
    C --> D[Pick lowest input cost]
    D --> E{Tie on cost?}
    E -->|yes| F[Higher score wins]
    E -->|no| G[Primary tier chosen]
    B -->|no| H[Pick highest score]
    H --> G
    I{Images in session?} --> J[Zero out tiers without image support]
    J --> A
    G --> K[Remaining tiers become fallbacks]

Loading

Image turns zero out tiers that cannot handle attachments before selection runs.

Data shape

This is the object graph that moves through the pipeline.

flowchart LR
    subgraph input
        H[conversation history]
    end

    subgraph signals
        S[ClassifierSignals]
    end

    subgraph classify
        X["<session> XML"]
        P[scorer prompts]
        J["JSON { scores, reasoning }"]
    end

    subgraph select
        O[ModelOption list]
    end

    H --> S
    S --> X
    S --> P
    P --> J
    J --> O

Loading

Key fields on ClassifierSignals: current message, recent turns, tool outcomes, image flag, failed-tool flag.

Repo map

flowchart TB
    subgraph root [ModelRouter]
        RC[src/route-core]
        INT[src/integration]
        EV[eval]
        DOC[docs]
    end

    RC --> signals[signals.ts]
    RC --> context[context.ts]
    RC --> cards[model-cards.ts]
    RC --> scorer[classifiers/score-classifier.ts]
    RC --> sel[selector.ts]
    RC --> router[task-router.ts]

    INT --> cache[session-cache.ts]

    EV --> harness[harness.py]
    EV --> core[router_core.py]
    EV --> fix[fixtures/*.jsonl]

Loading

Where to start reading

Question	File
How are tiers scored?	src/route-core/model-cards.ts
How is the winner picked?	src/route-core/selector.ts
How does history become XML?	src/route-core/signals.ts, context.ts
How would a CLI wire it?	src/route-core/task-router.ts

Tiers I hardcoded

Model	Input $/1M	Output $/1M	Catch
claude-opus-4-8	$15	$75	safe fallback
glm-5.2	$1.40	$4.40	long-context agentic, no images
kimi-k2.7-code	$0.95	$4
minimax-m3	$0.30	$1.2	no images

Prices are illustrative. Swap them in src/route-core/candidates.ts.

flowchart LR
    subgraph tiers [candidate tiers]
        O[claude-opus-4-8<br/>premium]
        G[glm-5.2<br/>core]
        K[kimi-k2.7-code<br/>mid]
        M[minimax-m3<br/>budget]
    end

    O -.->|fallback| O
    K -->|cheaper when viable| M
    G -->|steps in when kimi below bar| K

Loading

Eval harness

cd eval
python3 harness.py run
python3 harness.py replay-session fixtures/example_refactor.jsonl

flowchart LR
    subgraph tracks [offline eval tracks]
        T1[distilled selector<br/>gold scores from cards]
        T2[boundary cases<br/>synthetic 0.7 edge cases]
        T3[fixture replay<br/>prompt builder only]
    end

    T1 --> R[eval/reports/*.json]
    T2 --> R
    T3 --> R

Loading

Reports go to eval/reports/ (gitignored). See eval/README.md.

Honest limitations

• I have not run this against real agent benchmarks.
• The scorer path is stubbed in eval (no live LLM calls).
• Model cards are hand-written and probably wrong in places.
• Provider routing (OpenAI vs Gemini vs BYOK) is out of scope here.

If you want something battle-tested, use a product. If you want to see one person's attempt at the routing layer, you are in the right place.

Contributing

PRs welcome for docs, fixtures, and selector experiments. Please no personal session dumps.

License

MIT. See LICENSE.