feat: model-based routing — wildcard model-to-provider mapping in proxy#277
feat: model-based routing — wildcard model-to-provider mapping in proxy#277zhangyang-crazy-one wants to merge 61 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c3fe23b460
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) | ||
| .await | ||
| { | ||
| Ok(Some((_route_id, provider))) => (vec![provider], Some("model_route".to_string())), |
There was a problem hiding this comment.
When a model route matches, this replaces the provider list with only the route target, but the later success path still builds SuccessSyncInfo and calls ProxyServerState::sync_successful_provider_selection, which persists set_current_provider and may update the live backup whenever the route target differs from current_provider_id_at_start. In a setup where *sonnet* routes to provider B while the user’s current provider is A, the first successful routed request permanently switches the app to B, so subsequent non-routed traffic and live config no longer use the user-selected provider.
Useful? React with 👍 / 👎.
| let mut regex_str = String::from("(?i)"); | ||
| for (i, segment) in segments.iter().enumerate() { | ||
| if i > 0 { | ||
| regex_str.push_str(".*"); | ||
| } | ||
| regex_str.push_str(®ex::escape(segment)); | ||
| } | ||
|
|
||
| Regex::new(®ex_str) |
There was a problem hiding this comment.
Wildcard patterns are compiled without ^...$, so regex matching can succeed on any substring rather than the whole model id. For example, a user adding the documented prefix route claude-* would also match xclaude-opus, and sonnet* would match claude-sonnet even though no leading wildcard was supplied; this makes route precedence unpredictable when model ids merely contain another route’s prefix.
Useful? React with 👍 / 👎.
| pattern, | ||
| provider_id, | ||
| priority, | ||
| enabled: true, |
There was a problem hiding this comment.
The TUI edit flow always sends enabled: true to update_model_route, so editing a disabled model route’s pattern/provider/priority silently re-enables it. This differs from the CLI update path, which preserves existing.enabled, and it can cause a route the user intentionally disabled to start matching requests again after a harmless edit.
Useful? React with 👍 / 👎.
zhangyang-crazy-one
changed the title
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4e55a75d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| json!({ | ||
| "id": id, | ||
| "object": "model", | ||
| "created": 1700000000, | ||
| "owned_by": "cc-switch" | ||
| }) |
There was a problem hiding this comment.
When Claude Code is pointed at this proxy via ANTHROPIC_BASE_URL, its /model flow uses Anthropic's GET /v1/models schema; the Anthropic Models API documents each item with fields like type, display_name, and created_at (plus pagination fields). This handler only emits OpenAI-style entries (object, created, owned_by), so Anthropic clients that deserialize or display ModelInfo can reject the response or show no usable models even though routes exist. Consider emitting the Anthropic fields as well, or a protocol superset.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 05861bcea0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let request_model = body | ||
| .get("model") | ||
| .and_then(|value| value.as_str()) | ||
| .unwrap_or("unknown") |
There was a problem hiding this comment.
For standard Gemini proxy calls, the model is in the path (for example /v1beta/models/gemini-2.5-pro:generateContent) rather than in the JSON body, so this always sets request_model to unknown for those requests. Since model-route matching now uses this value before falling back to ProviderRouter, any --app gemini model route created through the new CLI/TUI will never match normal Gemini traffic.
Useful? React with 👍 / 👎.
| self.overlay = Overlay::TextInput(TextInputState { | ||
| title: texts::tui_model_route_add_priority_title().to_string(), | ||
| prompt: texts::tui_model_route_add_priority_prompt().to_string(), | ||
| input: TextInput::new("0".to_string()), |
There was a problem hiding this comment.
When editing an existing route in the TUI, the priority prompt is always prefilled with 0, so accepting the default after changing only the pattern/provider silently rewrites any non-zero priority to highest priority. That changes routing order for existing rules even though the user did not intend to edit priority; the edit flow should prefill or preserve the route's current priority.
Useful? React with 👍 / 👎.
| if version == 12 { | ||
| log::warn!("数据库版本 {version} 高于 SCHEMA_VERSION={SCHEMA_VERSION}，跳过迁移（兼容模式）"); | ||
| conn.execute("RELEASE schema_migration;", []).ok(); | ||
| return Ok(()); |
There was a problem hiding this comment.
This compatibility branch accepts user_version=12 and returns before running the column repairs that this patch relies on for model_routes. If a v12 database already has model_routes but lacks hit_count/last_hit_at (the code comments call out that case), create_tables will not alter the existing table and later list_model_routes SELECTs those columns, causing proxy/TUI/CLI model-route operations to fail with no such column.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a45cc8b5b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if version == 12 { | ||
| log::warn!("数据库版本 {version} 高于 SCHEMA_VERSION={SCHEMA_VERSION}，进入兼容模式并补齐列"); | ||
| Self::create_tables_on_conn(conn)?; |
There was a problem hiding this comment.
When opening a database with user_version 12, this compatibility branch returns after create_tables_on_conn, but CREATE TABLE IF NOT EXISTS does not alter an existing upstream model_routes table. The new DAO selects hit_count and last_hit_at, so a v12 database whose table lacks those local tracking columns will make model-route list/TUI/proxy lookups fail with no such column instead of running in compatibility mode. Run the same add_column_if_missing repairs here before returning.
Useful? React with 👍 / 👎.
| self.overlay = Overlay::ModelRouteProviderPicker { | ||
| pattern: raw, | ||
|
|
||
| selected: 0, |
There was a problem hiding this comment.
When editing an existing rule, the provider picker is always initialized at index 0. If the user only changes or accepts the pattern and presses Enter through the provider step, handle_model_route_provider_picker_key uses that selected row as the provider, so the route is silently moved to the first provider instead of preserving its current target. Initialize selected from the route's current provider_id before opening the picker.
Useful? React with 👍 / 👎.
… ModelRoute type - Add ModelRoute struct with camelCase serde, unit test for serialization - Register model_route module and public export in lib.rs - Add schema_migration_v10_adds_model_routes_table test (RED - no migration yet)
…es table - Bump SCHEMA_VERSION from 10 to 11 - Add CREATE TABLE model_routes to create_tables_on_conn (table 17) - Add migrate_v10_to_v11 function with identical schema to upstream - Add version 10 match arm to apply_schema_migrations_on_conn - All existing tests pass (2596 passed, 0 failed)
- Add list_model_routes (ordered by priority ASC, created_at ASC) - Add get_model_route (by id, returns Option) - Add create_model_route (with FK validation for provider_id) - Add update_model_route (with FK validation on provider_id change) - Add delete_model_route (checks changes count) - Add toggle_model_route (flips enabled, uses NOT enabled) - All 6 DAO unit tests pass within crate tests - Full lib test suite: 2602 passed, 0 failed
- Add model_route_dao_crud_roundtrip: tests create, get, FK validation, update, toggle, delete, list ordering by priority, and app_type filtering - Add model_route_cascade_delete_on_provider_removal: verifies ON DELETE CASCADE works when provider is deleted - All tests pass: 2604 lib tests, all integration test targets green - cargo fmt --check passes, no new clippy warnings
- Add proxy/model_router.rs with ModelRouter struct - Wildcard * to regex conversion with meta-character escaping - Priority-based route selection (lowest number wins) - Case-insensitive matching, enabled-only routing - Defensive empty model and missing provider handling - 16 unit tests covering exact, wildcard, priority, disabled, case-insensitive, regex meta-char, empty model, and missing provider
- Add model_router field to ProxyServerState struct and constructor - Add model_router and route_source fields to HandlerContext - HandlerContext::load() now calls model_router.match_route() first - Model route match bypasses failover queue with single provider - Unmatched/error cases fall back to existing ProviderRouter logic - Update all test_state() helpers: server.rs, handler_context.rs, handlers.rs, response_handler/tests.rs
- Add model_route_match_bypasses_failover_queue integration test - Add no_model_route_falls_back_to_provider_router integration test - Apply cargo fmt across all modified files - Remove unused Mutex import in model_router.rs tests - All 2622 tests pass with zero regressions
- Add ModelRouteCommand enum (List, Add, Remove, Toggle, Update) - Add ModelRoute variant to ProxyCommand with #[command(subcommand)] - Wire ModelRoute dispatch in execute() with get_state() - Add stub handle_model_route() function
- 13 tests covering list, add, remove, toggle, update operations - Tests for non-existent provider rejection - Tests for codex app type isolation - Seed helper for test providers
- Add print_model_routes() helper with comfy-table output - Implement handle_model_route() for List, Add, Remove, Toggle, Update - Add calls Phase 1 DAO methods directly - All 13 TDD tests pass
…, and state fields - Add SettingsModelRoutes variant to Route enum - Define ModelRouteRow and ModelRouteSnapshot types with provider name resolution - Add model_routes field to UiData with data loading from DB - Add SettingsItem::ModelRoutes to settings menu - Add model_routes_idx field to App struct with clamp_selections - Add i18n text for model routes title (EN/CN) - Add stub key handler and placeholder render dispatch - Create ui/model_routes.rs with table rendering
zhangyang-crazy-one
force-pushed
the
feat/model-based-routing-pr
branch
from
39675ac to
20feb74
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 20feb74877
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| regex_str.push_str(®ex::escape(segment)); | ||
| } | ||
|
|
||
| Regex::new(®ex_str) |
There was a problem hiding this comment.
For wildcard patterns that do not end in *, the generated regex is only start-anchored, so a rule such as *-4-5 also matches claude-haiku-4-55 because is_match can stop at the -4-5 prefix. Since the first matching route selects a single provider, suffix-style rules can route newer/different model IDs to the wrong provider; append $ when the pattern itself does not end with *.
Useful? React with 👍 / 👎.
| self.overlay = Overlay::ModelRouteProviderPicker { | ||
| pattern: raw, | ||
|
|
||
| selected: 0, |
There was a problem hiding this comment.
When editing an existing route, the provider picker always starts at index 0, so pressing Enter to keep the provider silently changes the route to the first provider whenever the current provider is not first in data.providers.rows. Seed selected from the existing route's provider_id so editing pattern/priority does not rewrite the provider unintentionally.
Useful? React with 👍 / 👎.
…Day#10/SaladDay#11 Dashboard multi-provider wave color: - observe_proxy_provider_activity no longer silently returns on the first tick, so provider samples align with main input/output samples from the start; resync provider samples after a proxy restart resets the main counter, fixing the wave degrading to a single accent color Codex review (PR SaladDay#277): - proxy: anchor non-trailing-* model route patterns at the end ($) so a suffix rule like "*-4-5" no longer matches "claude-haiku-4-55"; use "*sonnet*" to match a substring anywhere - tui: preselect the current provider when editing a model route so Enter does not silently move the route to the first provider
The multi-provider color stack was distributed across the full [0, stack_height) range by per-column token share, but the dot-matrix wave only renders the bottom `filled` rows (height scaled by window-max output). Minor providers' colors landed on blank rows and were invisible, so the legend color (e.g. DeepSeek blue) did not match the wave (all dominant purple). - compute_column_color_stacks takes column_filled_rows and fills only the rendered [stack_height-filled, stack_height) range: dominant at the bottom, minor on the top character row - split upper/lower color stacks so output and input shapes align independently - expose recent_samples/scale_samples (pub(super)) so the color logic reuses the same wave-scaling baseline as proxy_wave_lines - add color_stacks_only_fill_rendered_rows regression test
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c542b4a12a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ModelRouteCommand::Remove { id } => { | ||
| state.db.delete_model_route(&id)?; |
There was a problem hiding this comment.
When a route id from another app is supplied, this deletes it solely by id even though the command is executed under the currently selected --app; the toggle and update paths use the same unscoped id-only DAO calls. Since list/add are app-scoped, a user can accidentally remove or modify a Codex route while running --app claude; verify the existing route's app_type matches app or use app-scoped DAO mutations before changing it.
Useful? React with 👍 / 👎.
| let standard_models = match pattern_lower.as_str() { | ||
| "*haiku*" | "haiku" => vec!["claude-haiku-4-5-20251001"], | ||
| "*sonnet*" | "sonnet" => vec!["claude-sonnet-4-6"], | ||
| "*opus*" | "opus" => vec!["claude-opus-4-8"], | ||
| _ => Vec::new(), | ||
| }; |
There was a problem hiding this comment.
For an exact model route such as gpt-5 or deepseek-v4-pro, this match falls through to _ and adds nothing to /v1/models, so clients that populate model pickers from the proxy can miss configured route-only models or see an empty list when providers do not define env model mappings. Exact patterns without * should be emitted as model ids in addition to the role shortcuts.
Useful? React with 👍 / 👎.
Codex review SaladDay#5 (P3): exact model route patterns (no wildcard, e.g. gpt-5, deepseek-v4-pro) were not emitted in GET /v1/models, so clients populating model pickers missed route-only models. Non-wildcard patterns are now added as model ids alongside the role shortcuts. Codex review SaladDay#6 (P2): route Remove/Toggle/Update used id-only DAO calls under the selected --app, so a route id from another app could be deleted/toggled/rewritten (Update even forced app_type to the current app). Add require_route_for_app to verify the route belongs to the current app before any mutation.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e9132db64
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// clients can consume the response. | ||
| pub async fn handle_models(State(state): State<ProxyServerState>) -> impl IntoResponse { | ||
| let db = state.db; | ||
| let app_type = "claude"; |
There was a problem hiding this comment.
When an OpenAI/Codex client calls the newly registered GET /v1/models endpoint, this hard-coded app scope means the response is built from Claude providers and Claude model routes only; Codex/Gemini model routes created with --app codex or --app gemini will never appear in model discovery even though request routing uses those app-specific routes. I checked server.rs, and this handler is the only /v1/models route, so the app context needs to be passed/inferred instead of fixed to Claude.
Useful? React with 👍 / 👎.
| #[arg(long)] | ||
| provider_id: Option<String>, |
There was a problem hiding this comment.
The new command docs/requirements describe model-route update <id> ... [--provider], but with #[arg(long)] on the provider_id field Clap exposes the flag as the case-converted field name (--provider-id; Clap derive reference: https://docs.rs/clap/latest/clap/_derive/index.html#arg-attributes). Users following the documented --provider form will get an unknown-argument error when trying to move a route to a different provider, so add long = "provider" or an alias.
Useful? React with 👍 / 👎.
Merge SaladDay/cc-switch-cli main to resolve conflicts and pick up upstream v5.8.3/5.8.4 changes (deeplink command, Codex session history, proxy cache_control fix, live config merge, opencode modalities, ClaudeAPI preset). Resolved conflicts: - handler_context.rs: keep is_model_routed sync guard (SaladDay#1) + ModelRouter - server.rs: keep failover tests (not yet moved upstream) - data.rs: use derived Default (upstream) + keep model_routes field - content_config.rs: merge ProxySwitch (ours) + upstream ListenAddress/ ListenPort stop-hint handling - ui/config.rs: ProxySwitch key label is toggle, not edit
The model-route update command documented `--provider`, but Clap derived `--provider-id` from the field name, so users following the docs hit an unknown-argument error (Codex review SaladDay#8).
|
Re: codex review #7 ( #8 — fixed in #7 — by design, not a defect. The |
|
Hi @zhangyang-crazy-one, thanks a lot for this PR — it's thorough work, covering all four phases from the database through to the TUI, with the tests and Codex review followed up closely. You clearly put real care into it 👍 I noticed Phase 1 touches the database schema (the new How about we chat through the design first (either here in the comments or in a new issue)? Pinning it down before pushing the code further will keep things on a steadier footing. The routing engine, CLI, and TUI parts all look like they're heading in a good direction — it's really just the schema layer I'd like to firm up first. Thanks again 🙏 |
|
Thanks for the thorough review @SaladDay! Completely agree the schema layer should be pinned down before merge. On On the tracking fields ( Let me know which direction you prefer and I'll update the PR accordingly. The routing / CLI / TUI parts can wait — schema first, then rebase. Thanks again! |
The /v1/models discovery endpoint hardcoded app_type to "claude", so Codex/Gemini clients calling it never saw their own app's model routes or provider env models. Iterate AppType::all() and merge provider env models plus enabled routes from every app, so the endpoint is useful regardless of which client consumes it. Addresses Codex review P2 on PR SaladDay#277.
…gs schema Some databases reach user_version=v12 with tables stuck on older schemas (create_tables used a legacy DDL then set_user_version directly), so the v10->v11 migration loop never ran. This caused run_usage_maintenance to fail repeatedly with 'no such column: request_model/pricing_model'. - Add repair_usage_daily_rollups_columns: idempotently rebuilds the rollups table to the v11 schema (request_model/pricing_model columns + 6-column PK) and migrates historical rows, run in the repair phase of apply_schema_migrations. - Add pricing_model to repair_proxy_request_logs_columns: the rollup aggregation SQL reads l.pricing_model from proxy_request_logs, but this column was missing from the repair list (and from legacy tables). - Add test repair_usage_daily_rollups_rebuilds_legacy_table_when_already_at_current_schema.
|
@codex review this PR |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 13f88f0413
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| .and_then(|value| value.as_str()) | ||
| .map(|s| s.to_string()) | ||
| .or_else(|| extract_gemini_model_from_path(path)) | ||
| .unwrap_or_else(|| "unknown".to_string()); |
There was a problem hiding this comment.
When a request has no model field/path model, this sentinel is still passed into ModelRouter; a catch-all route such as * matches "unknown". That changes model-less endpoints like Codex /v1/responses/compact (the existing passthrough test sends only input) from using the current provider to being forced through the catch-all route, even though there is no model to route on. Keep the model as optional for routing, or skip model-route lookup when extraction fails.
Useful? React with 👍 / 👎.
| if takeover_enabled { | ||
| ProxyService::new(self.db.clone()) | ||
| .refresh_failover_live_snapshot_for_provider(app_type.as_str(), provider) | ||
| .update_live_backup_from_provider(app_type.as_str(), provider) |
There was a problem hiding this comment.
When automatic failover succeeds while proxy takeover is active, this now overwrites proxy_live_backup with the failover provider. That backup is later restored verbatim by restore_live_config_for_app when takeover is disabled or recovered, so the user's original live config captured at takeover start is lost; the hot-switch path still avoids this by writing a failover live snapshot instead. Refresh/write the failover snapshot here rather than mutating the live backup.
Useful? React with 👍 / 👎.
…ackup P2-1: Change request_model fallback from "unknown" to "" so model-less requests (e.g. Codex /v1/responses/compact) are not accidentally matched by catch-all model routes. match_route_internal already guards on model.is_empty(). P2-2: Replace update_live_backup_from_provider with refresh_failover_live_snapshot_for_provider in sync_successful_provider_selection. When auto-failover succeeds while takeover is active, the original takeover live backup must not be overwritten with the failover provider's config, or restore_live_config_for_app will restore the wrong config when takeover ends. refresh_failover_live_snapshot_for_provider writes a failover snapshot instead (matching the hot-switch path).
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
@codex review this PR |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6349a2c298
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| None => Action::None, | ||
| }, | ||
| } |
There was a problem hiding this comment.
Running cargo fmt --check under src-tauri/ fails on this match arm because rustfmt expects a trailing comma after the KeyCode::Enter arm body. The repository guidance notes CI runs cargo fmt --check for src-tauri/**, so this formatting miss will block the PR even though the code otherwise parses.
Useful? React with 👍 / 👎.
Add trailing comma to match arm to satisfy cargo fmt --check (CI gate). Codex P2.
Strengthen the auto-failover takeover test to verify that the original proxy_live_backup is preserved verbatim (still the user's provider) and the failover provider's config is captured in a dedicated failover snapshot. Locks in the P2-2 fix (refresh_failover_live_snapshot_for_provider).
…uting-pr # Conflicts: # src-tauri/src/database/tests.rs
2 participants
Summary
实现基于模型名称的路由功能：通过通配符模式（如
*sonnet*、claude-*）将不同的模型请求路由到指定的 provider。核心功能
数据库 (Phase 1)
model_routes表：存储模型模式到 provider 的映射，支持优先级排序路由引擎 (Phase 2)
ModelRouter：通配符*转正则匹配，大小写不敏感，按优先级选择CLI 命令 (Phase 3)
cc-switch proxy model-route list|add|remove|toggle|updateTUI 管理界面 (Phase 4)
Bug 修复（Codex review）
is_model_routed标记）^锚定，防止前缀越界匹配（如claude-*匹配xclaude-opus）CI 状态
架构对齐