Skip to content

🤖 feat: route skills to model classes (Settings-managed large/medium/small) - #3849

Open
asm wants to merge 27 commits into
coder:mainfrom
asm:skill-model-classes
Open

🤖 feat: route skills to model classes (Settings-managed large/medium/small)#3849
asm wants to merge 27 commits into
coder:mainfrom
asm:skill-model-classes

Conversation

@asm

@asm asm commented Aug 14, 2026

Copy link
Copy Markdown

Summary

Skills can now be routed to user-defined model size classes so mechanical skills (wrap-up chores, formatting passes, routine repo tasks) don't consume frontier-model tokens. Classes map a name to a model[+thinking] value (one-shot syntax) and are edited in Settings → Models → Model Classes; skills bind to a class via the spec-standard frontmatter metadata: model-class: small or a local skillModelClasses config table. The class model applies to that invocation only — the workspace model is untouched. One-shot overrides also compose with skill invocations now (/haiku+0 /deep-review), and an explicit one-shot always beats class routing.

Background

Models churn constantly, so per-skill bindings shouldn't name concrete models — they name a class (large/medium/small), and only the class map names models. Updating one class re-routes every bound skill.

  • 🤖 feat: add /<model> one-shot model override syntax #2142 introduced one-shot model overrides; this reuses that exact plumbing (per-send model) and extends it to compose with skill slash invocations.
  • Agent definition ai.model is not consulted when spawning sub-agent tasks #3038 describes the kindred gap for agent definitions (ai.model parsed but not consulted); this PR takes the same position for skills — a declared model preference should be honored — while keeping it strictly opt-in.
  • Portability: the binding uses the Agent Skills spec's metadata map, which other harnesses ignore. Frontmatter bindings to a class the user never defined are deliberately inert, so skills shipping metadata: model-class can never break users who haven't opted in. The config table exists for routing skills the user doesn't own — and because the table is the user's own explicit intent, a dangling table entry (naming a class that was deleted) fails loudly instead of silently unrouting.

Implementation

  • Config: modelClasses and skillModelClasses records (schema, load normalization, saveConfig whitelist, config.updateModelClasses route). Maps are stored verbatim — entries this build can't parse are preserved, not dropped, so edits from an older/newer build never destroy classes they don't understand. Validity is judged lazily at send time by the resolver.
  • Shared resolver (src/common/utils/ai/skillModelClasses.ts): binding resolution as a discriminated union (unbound / unknown-class / invalid-value / resolved), plus isModelServableWithProvidersConfig (modelAvailability.ts) wrapping the routing layer's isModelAvailable with the same exported provider/gateway predicates useRouting consumes — so a model reachable only via a configured gateway (e.g. OpenRouter) correctly counts as available, route-priority membership is honored, and the editor warning cannot drift from the send-time gate.
  • Send path (AgentSession.sendMessage): the override is resolved before the pricing gate, PDF-support preflight, and any history mutation, so those gates evaluate the model that will actually stream and a broken binding errors before persisting side effects. Routing is gated by a dedicated skipSkillModelRouting send option (set by explicit one-shot composition and compaction retries) rather than overloading skipAiSettingsPersistence. Bound-but-broken mappings (dangling table entry, invalid value, no configured route for the model) fail the send with an actionable error naming the fix and the one-shot bypass; unbound skills take a null fast-path and infrastructure failures (unreadable skill/config, providers state unavailable) fail open.
  • Compaction interplay: the auto-compaction threshold is computed against the routed model's context window, while the compaction request itself and its follow-up resume options carry the pre-routing model/thinking (the compaction model must fit the uncompacted history, and the user's model choice must survive the round-trip). Mid-stream forced compaction during a routed turn threads the same pre-routing options through the stream context. Routed sends only auto-compact when the history is within ROUTED_SEND_COMPACTION_HEADROOM_PERCENT (10 points) of the routed model's window — headroom for the pending turn, while still far above the workspace threshold so a small-context class model can't trigger surprise compaction of a history the workspace model handles fine.
  • Settings UI: a "Model Classes" section under Settings → Models with fixed canonical slots (large/medium/small — a shared vocabulary keeps skill frontmatter portable across machines), model + thinking selects per class, custom hand-edited classes preserved on save and listed read-only (unparseable raw values shown in a tooltip), and an inline "no configured route can serve this model" warning using the same predicate as the send-time check. Edits are disabled until config and routing state finish loading, so an early click can't clobber persisted classes; thinking suffixes carry across model swaps only when the target model's policy supports them.
  • Composer: parseCommandWithSkillInvocation composes a leading one-shot with a skill invocation by re-running parseCommand on the one-shot's message — registered commands and nested one-shots stay out of skill resolution, mirroring direct-invocation semantics exactly. Composed sends record the full command prefix (model /skill) in message metadata so transcript badges render what was actually typed. Numeric one-shot thinking is model-relative, so a thinking-only composed send (/+0 /skill) also passes the raw index (oneShotThinkingIndex) for the backend to re-resolve against the routed model's ladder — +0 means the class model's lowest level, not the workspace model's. Compact-and-retry rebuilds re-derive the one-shot's model and thinking from the original text (with skipAiSettingsPersistence, so a re-dispatch never persists one-shot values as new workspace defaults), and prepareCompactionMessage keeps carried one-shot fields from being clobbered by ambient stored options.
  • Attribution: when routing applies, the persisted user-message metadata is re-stamped with the routed model (requestedModel), so the pending-turn label and history consumers see the model that actually streams.

Review-round hardening

Sixteen Codex review rounds tightened the edges (all threads resolved):

  • Send-path ordering: routing resolves before the pricing gate, PDF preflight, and any history mutation; rejected manual/queued sends persist a visible error (never for edits, which return bare and restore the draft); queued PDF rejections surface instead of vanishing.
  • Compaction interplay: routed sends compact within a headroom of the routed window (pre-send AND mid-stream); the compaction request runs on whichever of the user/routed model has the larger usable window; the routed policy survives same-session retries, compact-and-retry rebuilds (model, thinking, prefix, skipAiSettingsPersistence), and process relaunch (durable compactionBaseOptions in retrySendOptions, honored even in child task workspaces).
  • Availability truth: the shared servability predicate and ProviderModelFactory.resolveModelRoute both apply model-aware OpenAI credential rules (Codex-OAuth-only serves the OAuth set; API keys attempt anything; custom openai-compatible providers shadowing the openai id are exempt).
  • Telemetry attribution: the accepted-send payload reports routedModel + post-floor routedThinkingLevel; persisted metadata re-stamps requestedModel. Queued-send event attribution is documented as a follow-up (needs backend-side event capture).
  • Editor integrity: class edits persist before publishing (no split-brain with a fast follow-up send), rows lock while their write is in flight, custom classes survive verbatim, and CI snapshots the wrapping layout at a pinned phone viewport.

Validation

  • ~70 tests across the feature: resolver statuses (frontmatter-inert vs table-loud, blank table entries, the opt-in guard), availability predicate (route-priority membership, disabled providers), config round-trip through the saveConfig whitelist (including preservation of unknown classes), end-to-end AgentSession routing and error paths via the session harness (gate ordering, skipSkillModelRouting exemption, thinking-only bindings, compaction follow-up model), composition parser cases, and editor UI behavior (clear preserves custom classes; load gating; warning states).
  • Full bun test src failure set is identical to main's on the same machine (pre-existing env-sensitive tests only).
  • Verified live in Storybook (ModelsSection stories now seed classes, including one pointing at an unconfigured provider to exercise the warning; row layout wraps at mobile widths) and in a packaged build used for daily work.

Risks

The sensitive area is the insertion in AgentSession.sendMessage. Scope is tightly bounded: only sends carrying agent-skill metadata without skipSkillModelRouting are considered, and workspaces with no modelClasses/table binding hit an early return before any skill read — no behavior change for anyone who hasn't opted in. Compaction interplay (threshold on the routed model, compaction request and mid-stream forced compaction on the user's model, follow-up resume options) is covered by tests. One known asymmetry, documented at the helper: the shared servability predicate mirrors the routing layer's gateway/priority gates but not per-request policy checks, so an editor warning can under-report in exotic policy setups — the send-time error remains authoritative.


🤖 Generated with Claude Code

@asm
asm marked this pull request as draft August 14, 2026 00:08
@asm
asm marked this pull request as ready for review August 14, 2026 03:20
@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60f19ad5e5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/browser/features/ChatInput/index.tsx
Comment thread src/browser/hooks/useCompactAndRetry.ts Outdated
Comment thread src/node/services/agentSession.ts
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

All three findings addressed in b1b0bf8:

  • Numeric thinking vs routed model: the frontend now passes the raw index (oneShotThinkingIndex send option) alongside the workspace-resolved level, and AgentSession re-resolves it against the routed class model when routing applies — /+0 /skill means the routed model's lowest allowed level. Covered by a routing test where the pre-resolved level ("medium") and the routed ladder ("off") differ.
  • One-shot thinking across compact-and-retry: the rebuilt follow-up carries the parsed thinking (named as-is; numeric resolved against the explicit model, or kept as a raw index for routed re-resolution) plus skipAiSettingsPersistence, and prepareCompactionMessage no longer lets ambient stored options clobber carried one-shot fields. This also fixes a latent issue: without the persistence flag, the re-dispatch would have persisted the one-shot model as the new workspace default.
  • requestedModel: when routing applies, the persisted user-message metadata is re-stamped with the routed model, so the incoming user event and history consumers attribute the send correctly. One deliberate limit: the frontend's fire-and-forget messageSent telemetry event still reports the requested model — threading the routed model through the send result would widen Result<void> across ~15 return sites, which felt too invasive here; happy to do it as a follow-up if maintainers prefer.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1b0bf8591

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Round-2 finding addressed in 6c4903d: routed sends now compact within ROUTED_SEND_COMPACTION_HEADROOM_PERCENT (10 points) of the routed model's window instead of requiring a full 100% — headroom for the pending message, attachments, and skill snapshot that the recorded usage doesn't include, while still staying far above the workspace threshold so a cheap skill invocation can't force an unrequested compaction of a history that fits.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c4903deb0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts
Comment thread src/node/services/agentSession.ts Outdated
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Both round-3 findings addressed in 297b210:

  • Mid-stream routed policy: checkMidStream now accepts a force-threshold override, and routed turns (identified by the stream context's compaction base options) pass the same routed-send headroom bar — a usage update during a routed turn no longer forces compaction at the workspace threshold+buffer against the smaller routed window. Monitor test covers the override at 75% (no trigger) and 92% (trigger).
  • Compaction model fit: on-send and mid-stream compaction now run with whichever of the user's / routed model has the larger usable context window (getEffectiveContextLimit comparison) — normally still the user's model, but a class routing UP past the user's window no longer summarizes on a model that can't read the history. The deferred follow-up keeps pre-routing options and re-routes at dispatch either way.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 297b210330

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/common/utils/ai/modelAvailability.ts Outdated
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Round-4 finding addressed in 50b68ee: added canDirectOpenAIServeModel (colocated with the existing Codex OAuth routing mirrors) reflecting the factory's credential selection — OAuth-required models need stored tokens even with an API key, and OAuth-only configs serve only the allowed model set — and the shared servability predicate now consults it for direct-OpenAI routes. An OAuth-ineligible class model no longer passes the preflight on a Codex-OAuth-only config; a later gateway in routePriority can win, or the user gets the actionable class error. Tests cover OAuth-only vs API-key vs OAuth-required combinations.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50b68ee8fc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/common/utils/ai/modelAvailability.ts
Comment thread src/node/services/agentSession.ts
Comment thread src/node/services/agentSession.ts
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

All three round-5 findings addressed in 6452b8a:

  • Factory route selection: resolveModelRoute (both call sites) now passes the canonical model into isProviderAvailableForRouting, which rejects direct OpenAI when tokens-only credentials can't serve the model — a usable gateway later in routePriority wins, matching the shared predicate. I also realigned canDirectOpenAIServeModel with the factory's actual fallback semantics (an API key attempts any model, including OAuth-preferred ones; tokens-only serves only the allowed set) and updated the tests accordingly.
  • PDF preflight: the client-side check now defers to the backend's routed-model gate whenever a routable skill invocation is present (skillInvocation && !modelOverride) — the backend validates against the class model and rejects with a persisted, visible error, so a PDF-capable class model bound to a skill can actually receive PDFs.
  • Send telemetry: sendMessage now returns SendMessageAccepted { routedModel } through AgentSession → WorkspaceService → router → wire schema, and ChatInput attributes messageSent telemetry to result.data?.routedModel ?? effectiveModel. Queued sends report no routed model (dispatch happens later) and fall back to the requested model, as documented on the schema. Routing tests assert the payload for both the routed and skip-flag cases.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6452b8a491

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/browser/hooks/useModelClasses.ts Outdated
Comment thread src/browser/features/Settings/Sections/ModelClassesEditor.tsx
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Both round-6 findings addressed in 47afc04:

  • Manual memoization removed: the subscription's fetch now lives inside the effect and the write-failure revert reaches it through a ref — no useCallback, no exhaustive-deps suppressions. (Note: useModelFallbacks, which this hook was modeled on, uses the same pre-existing useCallback pattern upstream; left untouched here as out of scope.)
  • Pinned phone snapshot: ModelsConfiguredPhone pins a Pixel phone matrix variant mirrored with globals.viewport, so CI snapshots the Model Classes rows at the width their wrapping layout exists for. Verified live at 375px: label/select wrap, inline no-route warning on the unconfigured row, no right-edge overflow.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47afc04612

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/browser/hooks/useModelClasses.ts Outdated
Comment thread src/browser/features/ChatInput/index.tsx Outdated
Comment thread src/browser/features/ChatInput/index.tsx Outdated
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Round-7 findings in 44facc0:

  • Split-brain class edits (fixed): useModelClasses now serializes writes and publishes state only on the write's ack — the editor can no longer advertise a mapping the backend doesn't have. Rapid edits build on the newest pending intent (no lost updates), and failures still revert via refetch.
  • Routed thinking telemetry (fixed): the accepted-send payload gains routedThinkingLevel (class suffix or re-resolved numeric one-shot), and messageSent attributes to it. Covered by a routing test asserting both payload fields.
  • Queued-send attribution (descoped as follow-up, with rationale): routing resolves at dispatch, not at queue-accept, so correct attribution for queued sends requires emitting the event backend-side at dispatch — and message_sent carries frontend-only provenance (frontendPlatform, runtime context) that a backend emitter would misreport. The durable record is already attributed correctly at dispatch via the persisted requestedModel stamp; only the fire-and-forget analytics event stays approximate for the queued minority. Happy to build dispatch-time backend capture as a follow-up if maintainers want it — it needs its own provenance design rather than a bolt-on here.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 44facc03ea

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts Outdated
Comment thread src/node/services/agentSession.ts
Comment thread src/browser/features/ChatInput/index.tsx Outdated
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

All three round-8 findings addressed in 79fb8e0:

  • Queued PDF rejections preserved: both PDF gate branches (unsupported input, size limit) now persist and surface the rejection through preserveRejectedManualSend — same contract as the pricing and model-class gates — so a queued skill send whose PDF the routed model rejects leaves a visible transcript error instead of silently discarding the user's text and attachment.
  • Post-policy routed thinking: the per-model floor resolution + clamping now live in one shared method (resolveThinkingFloorForModel / enforceThinkingFloorsForModel) used by both the stream request build and the accepted-send payload — routedThinkingLevel reports the clamped level the stream actually runs at.
  • Named one-shot fallback: messageSent falls back to the send's actual sendOptions.thinkingLevel (which carries a composed one-shot's thinking) rather than the ambient workspace setting.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79fb8e040b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts Outdated
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Round-9 finding addressed in 6284377: routedThinkingLevel now reports the effective level for every routed send — whatever optionsForStream carries (class suffix, re-resolved numeric one-shot, or a named/ambient level riding through), clamped by the shared per-model floor enforcement. A /+off /skill routed onto a floor-medium model reports medium. Test covers the ride-through case.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62843778d0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts
Comment thread src/browser/hooks/useCompactAndRetry.ts Outdated
@asm
asm force-pushed the skill-model-classes branch from 6284377 to 3d6ffbd Compare August 14, 2026 17:57
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Both round-10 findings addressed, and the branch is rebased onto latest main (the #3844 conflict in agentSession.ts resolved by adopting the new gateway-preserving lookupMinThinkingLevelOverride inside the shared floor helper):

  • Routed compaction context across retries: the auto-retry resume state now carries compactionBaseOptions, resumeStream threads it through to streamWithHistory, and the post-compaction context-exceeded retry reads it from the captured stream context — same-session restarts keep both the routed force threshold and the larger-window compaction model selection.
  • Leading-whitespace one-shots: the compact-retry reparse guard now trims before checking, matching parseCommand's own tolerance.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d6ffbd18d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/browser/hooks/useCompactAndRetry.ts Outdated
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Round-11 finding addressed: the compact-and-retry metadata rebuild now carries source.commandPrefix into buildAgentSkillMetadata, so recovered composed invocations keep their command badge.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9eb115404

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/common/utils/ai/modelAvailability.ts
@asm

asm commented Aug 14, 2026

Copy link
Copy Markdown
Author

@codex review

Round-12 finding addressed: canDirectOpenAIServeModel now recognizes a custom openai-compatible provider shadowing the openai id (via the existing isCustomOpenAICompatibleProviderConfig detector) and exempts it from built-in OpenAI credential rules — custom endpoints authenticate on their own terms, so availability falls back to the ordinary isConfigured gate. Test covers a keyless shadowing provider serving an OAuth-ineligible model.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ecc3f4b18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts
Comment thread src/node/services/agentSession.ts Outdated
Comment thread src/browser/features/Settings/Sections/ModelClassesEditor.tsx
…l paths)

- useModelClasses: a client swap now marks the hook unloaded and
  invalidates in-flight fetches, so a reconnect window can't accept a
  full-map write built from another process's stale map.
- docs: point model-class config at canonical ~/.xum/config.json and
  describe the real compaction base selection (larger-window model,
  Compact-agent settings still win); regenerated built-in skill content.
- rebase reconciliation: widen the new task/workspace seam's sendMessage
  to the SendMessageAccepted payload; return a real TurnStreamHandle from
  the routing test's streamMessage stub (turn-engine refactor).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm
asm force-pushed the skill-model-classes branch from f5dd3e3 to 9840eb0 Compare August 29, 2026 16:36
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — rebased onto latest main (turn-engine refactor and the task/workspace seam) and addressed all three findings from the last round; inline replies on each thread. Head is 9840eb0.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-29T18:35:13.811119Z 6a11d23 Manual request
🔒 Security Review Completed 2026-08-29T18:40:45.227413Z 6a11d23 Manual request

Security findings

Advisory findings (3)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 9840eb0d58

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9840eb0d58

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/browser/hooks/useModelClasses.ts
Comment thread src/common/utils/providers/codexOauthRouting.ts
Comment thread src/node/services/agentSession.ts Outdated
…, validate routed-retry context

Codex round 2 on the rebased branch:

- useModelClasses ties write acknowledgements (and failure refetches) to
  the client generation that issued them, so an old client's late ack can
  no longer publish over — or invalidate the in-flight fetch of — the
  replacement client after a reconnect.
- Codex OAuth speaks only the Responses endpoint: with openai pinned to
  wireFormat chatCompletions and no real API key, both the shared
  canDirectOpenAIServeModel preflight and the factory's route-selection
  gate now refuse the direct route (createModel rejects it with
  api_key_not_found), letting a configured gateway win instead.
- Persisted compactionBaseOptions from chat.jsonl is shape-validated
  (object with a non-empty model, nested field stripped) before it can
  mark a row routed or ride into the resume request; malformed values fall
  back to today's non-routed behavior. Regression test covers a corrupted
  boolean in a child task workspace.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — addressed all three round-2 findings (generation-guarded write acks, chatCompletions OAuth route gating in both predicates, shape-validated persisted routed-retry context); inline replies on each thread. Head is 4e83559.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 4e835591c2

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e835591c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/browser/hooks/useModelClasses.ts
Comment thread src/node/services/agentSession.ts
Comment thread src/node/services/agentSession.ts
…efore the pricing preflight

Codex round 3:

- useModelClasses resets the pending intent map, the write chain, and
  pending-row counts on a client swap (completions from the old client
  skip their bookkeeping via the generation guard), so a new edit can
  neither compose from dead intent nor queue behind a request that may
  never settle.
- The persisted routed-retry sanitizer applies the startup-model bar
  (normalizeSelectedModel + isValidModelFormat) instead of accepting any
  non-empty string, and forwards the normalized id.
- WorkspaceService.sendMessage defers its pricing preflight for skill
  sends: class routing resolves inside AgentSession, whose dispatch-time
  gate re-asserts pricing against the model that actually streams — the
  ambient-model preflight would reject a skill bound to a priced class on
  an unpriced workspace model. Deferral can't corrupt stored settings
  (composer skill sends re-persist the already-selected model; one-shots
  skip persistence).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — addressed all three round-3 findings (client-swap reset of pending write state, startup-model-bar validation for persisted routed context, service pricing preflight deferred to the routing-aware dispatch gate for skill sends); inline replies on each thread. Head is 836dd19.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 836dd19681

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 836dd19681

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/workspaceService.ts
Comment thread src/node/services/agentSession.ts Outdated
… on gate rejections

Codex round 4:

- The deferred pricing preflight for skill sends also deferred nothing for
  persistence: an unbound skill (or skipSkillModelRouting caller) could get
  its unpriced ambient model persisted before AgentSession rejected it.
  Skill sends now persist AI settings from the session's onAccepted callback
  — fired only after every dispatch-time gate passed, and carried through
  queued dispatch by MessageQueue. Test asserts the ordering.
- The class-routing and PDF rejection branches now thread
  internal.enqueuedAtMs into the preserved row and goal safety, matching the
  pricing branch, so a skill queued before a goal activation cannot wrongly
  pause the fresh goal (nor be misclassified after restart).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — addressed both round-4 findings (acceptance-deferred AI-settings persistence for skill sends, queue-timestamp threading on the routing/PDF rejection branches); inline replies on each thread. Head is e16c845.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e16c845436

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/workspaceService.ts
Comment thread src/node/services/agentSession.ts Outdated
Comment thread src/browser/features/Settings/Sections/ModelClassesEditor.tsx Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Security Review

Here are some automated security review suggestions for this pull request.

Reviewed commit: e16c845436

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread src/common/utils/ai/skillModelClasses.ts
…ontext, safe persistence

Codex round 5:

- Security: a project-scope skill's model-class frontmatter only applies in
  trusted projects (Project Trust is the existing consent boundary for
  repo-controlled configuration) — an attacker-controlled repository could
  otherwise silently reroute the transcript to any provider the user bound
  to a class. Global/built-in skills and the user's own skillModelClasses
  table are unaffected. Tested both directions; documented.
- Persisted compactionBaseOptions is now schema-parsed with
  SendMessageOptionsSchema (unknown keys stripped, malformed siblings like
  providerOptions: false reject the whole context) instead of validating
  only the model, so corrupted values can't reach provider request
  construction via buildAutoCompactionRequest.
- The acceptance-deferred AI-settings persist is wrapped: a persistence
  exception must not propagate through onAccepted and turn an accepted send
  (durable user row) into a partial failure.
- ModelClassesEditor candidates dedupe via the metadata-aware
  normalizeFallbackModelKey, so a cross-typed Coder selection
  (coder:openai/x on an anthropic instance) keeps its explicit gateway
  route instead of persisting a direct-provider id.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — addressed all four round-5 findings, including the security one: project-skill frontmatter routing is now gated on Project Trust (inline replies on each thread). Head is 121cb4b.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 121cb4b98e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 121cb4b98e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentSession.ts Outdated
Comment thread src/browser/features/Settings/Sections/ModelClassesEditor.tsx Outdated
Comment thread src/browser/features/Settings/Sections/ModelsSection.stories.tsx Outdated
…es, 390px story viewport

Codex round 6:

- The persisted routed-context sanitizer now filters to the exact key set
  pickStartupRetrySendOptions persists (exported as
  STARTUP_RETRY_DURABLE_SEND_OPTION_KEYS) BEFORE schema-parsing, and narrows
  muxMetadata to workspace-turn correlation like the durable pick does — a
  schema-valid but non-durable field (editMessageId) can no longer ride a
  corrupted row into buildAutoCompactionRequest and send the restored
  compaction request down the edit/truncation path.
- Model-class candidates dedupe by the metadata-aware key but keep each
  first-seen RAW selection as the value, so explicit gateway selections
  (openrouter:openai/x, default-typed coder:anthropic/x) persist their
  gateway identity instead of collapsing to direct-provider ids.
- New pixelPhone (390px) Storybook viewport mirrors the Pixel matrix's
  named phone width; the pinned ModelsConfiguredPhone story uses it so the
  local view renders at the exact width CI snapshots.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — addressed all three round-6 findings (durable-key whitelist before the schema parse, gateway-preserving candidate values, 390px pixelPhone story viewport); inline replies on each thread. Head is a7a2e82.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Security Review

Here are some automated security review suggestions for this pull request.

Reviewed commit: a7a2e8281d

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread src/node/services/agentSession.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a7a2e8281d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +67 to +69
const key = normalizeFallbackModelKey(model, providersConfig);
if (!candidatesByKey.has(key)) {
candidatesByKey.set(key, model);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep direct and gateway routes as separate candidates

When both a direct model and its explicit gateway form are available, normalizeFallbackModelKey() maps them to the same key, so this keeps only whichever appears first. For example, Settings supplies custom models before built-ins, so openrouter:openai/gpt-5 can remove openai:gpt-5 from every class picker; the user can no longer select the direct route even though the two values dispatch differently. Deduplicate exact selection identities rather than fallback-chain keys.

Useful? React with 👍 / 👎.

Comment on lines +3274 to +3278
const persisted = await this.preserveRejectedManualSend(
message,
options,
routingError,
internal?.enqueuedAtMs

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the typed command when routing rejects a queued skill

When a queued skill is rejected here because its class binding is invalid or unavailable, message is the model-facing text such as Using skill done: ...; the original /done ... command exists only in options.muxMetadata. preserveRejectedManualSend() creates a normal row without that metadata, so the only recovered transcript row loses the user's typed command and skill badge and instead exposes the rewritten provider prompt. Preserve the raw command display metadata on this rejection path.

Useful? React with 👍 / 👎.

Codex round 7 (security): a project skill shadows a global/built-in name at
collision time, so a name-keyed skillModelClasses entry — consent the user
gave for the skill they knew — could route an untrusted repo's shadow to the
bound provider. Scope now resolves from the package (never from the
client-supplied invocation metadata) before ANY binding applies: in an
untrusted project a project-scope skill gets no class routing at all,
frontmatter or table. The trusted table-binding fast path keeps skipping the
SKILL.md read. Flipped the untrusted-table test to assert shadow protection
and documented the rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asm

asm commented Aug 29, 2026

Copy link
Copy Markdown
Author

@codex review — addressed the round-7 security finding: authoritative package scope resolves before any binding applies, so untrusted project shadows get no routing via frontmatter or the table (inline reply on the thread). Head is 6a11d23.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 6a11d23f43

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Security Review

Here are some automated security review suggestions for this pull request.

Reviewed commit: 6a11d23f43

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

}
const projectTrusted = (() => {
try {
return isWorkspaceProjectTrusted(this.config, metadataResult.data);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Security: Require trust for scratch project-skill routing

Condition: a scratch workdir contains an attacker-controlled repository, a class maps to another configured provider, and the user invokes the repository's skill. This check calls isWorkspaceProjectTrusted, which returns true for every scratch workspace, so project frontmatter bypasses the untrusted-project guard. Scratch folders are writable and terminal-enabled, and discovery scans their .xum/skills; the hidden model-class then replaces the stream model and sends active history to that provider. Fresh evidence beyond the resolved project-trust finding is this unconditional scratch exception. Require explicit trust for project skills found in scratch workdirs.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant