From 3abe55c89a8eeb7407431c79fd6094f9039f83ca Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 14:42:01 +0000 Subject: [PATCH 001/178] fix: clarify Knowledge Legacy is local-record, not per-domain backup exports --- GOALS.md | 2 +- PRD.md | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/GOALS.md b/GOALS.md index 368d75186e..0c826c90e8 100644 --- a/GOALS.md +++ b/GOALS.md @@ -42,7 +42,7 @@ AI agents should be capable of operating fully autonomously across all connected ### 9. Knowledge Legacy -Preserve personal knowledge, identity, decision-making patterns, creative output, and life story beyond a single lifetime. The autobiography system, genome data, behavioral profiles, captured memories, written work, and built worlds form a durable record — not just of what you built, but of who you are and how you think. +Preserve personal knowledge, identity, decision-making patterns, creative output, and life story beyond a single lifetime. The autobiography system, genome data, behavioral profiles, captured memories, written work, and built worlds form a durable record — not just of what you built, but of who you are and how you think. PortOS itself is the backup of record (local data + automatic snapshots); download/export buttons exist only for sharing or handoff to other tools, never as a parallel backup mechanism. ### 10. Anywhere Access on Private Network diff --git a/PRD.md b/PRD.md index 1ab0933245..abd6c39867 100644 --- a/PRD.md +++ b/PRD.md @@ -22,7 +22,7 @@ Reused verbatim (condensed to objective statements) from [GOALS.md](./GOALS.md)' 6. **Developer Productivity Toolkit** — shell, git, browser control, and process tooling available from any device. 7. **Self-Improving Intelligence** — the system tunes its own routing/metrics from observed outcomes rather than staying static. 8. **Full Digital Autonomy** — agents can act across connected platforms (voice, Telegram, messaging, social) around the clock. -9. **Knowledge Legacy** — personal knowledge, identity, and creative output are preserved as a durable, exportable record. +9. **Knowledge Legacy** — personal knowledge, identity, and creative output are preserved as a durable, local record. PortOS itself is the backup of record (data lives on the user's hardware and is covered by automatic snapshots — see `docs/BACKUP.md`), so per-domain backup-style exports are out of scope; download/export buttons exist only for sharing or handoff to other tools (Sharing buckets, Legacy Bundle). 10. **Anywhere Access on Private Network** — every feature is reachable from any device on the user's Tailnet, with no public exposure. 11. **Health & Longevity** — health data (MeatSpace) is tracked and made actionable via mortality/longevity-aware goal scoring. 12. **Personal Productivity & Life Management** — calendar, goals, and communications are unified with the same tooling that manages digital projects. @@ -208,6 +208,7 @@ Reused verbatim (condensed to objective statements) from [GOALS.md](./GOALS.md)' | NR-7 | The system MUST NOT treat leakage of a free, non-monetary third-party API key (e.g. CivitAI) to an unintended host as a security finding requiring host-allowlisting or key-stripping. | Won't-fix precedent (#2200): worst case is quota abuse against a free service, borne by that service — no monetary loss or meaningful security consequence. Does not extend to paid/quota-billed providers or money-bearing/destructive-action keys, which retain full hardening requirements. | | NR-8 | The system MUST NOT send an AI-drafted outbound message (email, social post) without explicit user review-and-approve, regardless of how confident the draft is. | Full digital autonomy (Goal 8) extends to task execution, not to irreversible outward-facing communication acting under the user's identity without a human gate. | | NR-9 | The system MUST NOT instruct the user to run a shell/terminal command in order to complete a workflow PortOS can perform itself — including installing or removing a runtime, and searching for, downloading, or deleting a model. Blocked-state copy points at the in-app control, not at a command line. | PortOS is the control surface for the machine; sending the user to a terminal for one step of an otherwise-managed lifecycle is a dead end that breaks remote/mobile use (Anywhere Access) and leaves the app's own state stale. **Carve-out:** genuinely privileged one-time host setup PortOS deliberately refuses to perform (`pm2 startup`, `sudo` fan-control helpers, `gcloud auth login`) may be named as an operator step — the refusal must be deliberate and documented, not a gap in the UI. | +| NR-10 | The system MUST NOT add per-domain backup-style export endpoints (e.g. a generic "export my Brain/memories/thoughts to file" download) as a durability or backup story. | PortOS is a locally hosted app that is itself the user's backup of record — data lives on the user's hardware and is covered by automatic snapshots (`docs/BACKUP.md`). Download/export buttons exist only for sharing or handoff to other tools (Sharing buckets, Legacy Bundle, format-specific creative deliverables), never as a parallel backup mechanism. | --- @@ -222,6 +223,7 @@ Reused verbatim (condensed to objective statements) from [GOALS.md](./GOALS.md)' - **Federated peer-to-peer sharing beyond bucket-based Sharing** — direct P2P distribution between instances is a secondary goal, not yet built. - **Federated media-provider routing for image/video generation** — the queued-job delegation contract (FR-52/FR-53) is implemented and live for audio/music generation today; extending the same provider/consumer contract to image and video generation is tracked separately (issue #4348), not yet built. - **User-directed assignment of a CoS task to a specific federated peer instance** — task coordination across peers is currently opportunistic only (first peer to see a synced task claims it via the existing lease mechanism); an explicit "run this task on instance X" control is a decided, ready-to-work follow-up (issue #4520), not yet implemented. +- **Per-domain backup-style exports** — PortOS is locally hosted and is itself the backup of record (automatic snapshots); exports exist only for sharing or handoff to other tools, per NR-10. --- From 0fee6e5e6d69e13a53f96d812ca3d1ff768175a9 Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 14:44:45 +0000 Subject: [PATCH 002/178] =?UTF-8?q?fix:=20address=20review=20=E2=80=94=20n?= =?UTF-8?q?ame=20all=20permitted=20export=20classes=20consistently?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- GOALS.md | 2 +- PRD.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/GOALS.md b/GOALS.md index 0c826c90e8..5c1fb98fba 100644 --- a/GOALS.md +++ b/GOALS.md @@ -42,7 +42,7 @@ AI agents should be capable of operating fully autonomously across all connected ### 9. Knowledge Legacy -Preserve personal knowledge, identity, decision-making patterns, creative output, and life story beyond a single lifetime. The autobiography system, genome data, behavioral profiles, captured memories, written work, and built worlds form a durable record — not just of what you built, but of who you are and how you think. PortOS itself is the backup of record (local data + automatic snapshots); download/export buttons exist only for sharing or handoff to other tools, never as a parallel backup mechanism. +Preserve personal knowledge, identity, decision-making patterns, creative output, and life story beyond a single lifetime. The autobiography system, genome data, behavioral profiles, captured memories, written work, and built worlds form a durable record — not just of what you built, but of who you are and how you think. PortOS itself is the backup of record (local data + automatic snapshots); download/export buttons exist only for sharing or handoff to other tools (e.g. Sharing buckets, Legacy Bundle), never as a parallel backup mechanism. ### 10. Anywhere Access on Private Network diff --git a/PRD.md b/PRD.md index abd6c39867..9d0bc0976e 100644 --- a/PRD.md +++ b/PRD.md @@ -22,7 +22,7 @@ Reused verbatim (condensed to objective statements) from [GOALS.md](./GOALS.md)' 6. **Developer Productivity Toolkit** — shell, git, browser control, and process tooling available from any device. 7. **Self-Improving Intelligence** — the system tunes its own routing/metrics from observed outcomes rather than staying static. 8. **Full Digital Autonomy** — agents can act across connected platforms (voice, Telegram, messaging, social) around the clock. -9. **Knowledge Legacy** — personal knowledge, identity, and creative output are preserved as a durable, local record. PortOS itself is the backup of record (data lives on the user's hardware and is covered by automatic snapshots — see `docs/BACKUP.md`), so per-domain backup-style exports are out of scope; download/export buttons exist only for sharing or handoff to other tools (Sharing buckets, Legacy Bundle). +9. **Knowledge Legacy** — personal knowledge, identity, and creative output are preserved as a durable, local record. PortOS itself is the backup of record (data lives on the user's hardware and is covered by automatic snapshots — see `docs/BACKUP.md`), so per-domain backup-style exports are out of scope; download/export buttons exist only for sharing or handoff to other tools (Sharing buckets, Legacy Bundle, format-specific creative deliverables). 10. **Anywhere Access on Private Network** — every feature is reachable from any device on the user's Tailnet, with no public exposure. 11. **Health & Longevity** — health data (MeatSpace) is tracked and made actionable via mortality/longevity-aware goal scoring. 12. **Personal Productivity & Life Management** — calendar, goals, and communications are unified with the same tooling that manages digital projects. From feff28cd4225caf1523ba8bfe773f055540dee89 Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:09:47 +0000 Subject: [PATCH 003/178] give the OpenCode reviewer a model dropdown and hide reviewers this machine can't run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The reviewer picker's Model cell was a bare text input for OpenCode, and its Add row offered every reviewer in the roster regardless of whether the machine could run one. - OpenCode's options now come from the seeded OpenCode Zen CLI/TUI records, whose namespaced `opencode/*` ids the Harnesses page refreshes from `opencode models`. The local-runtime wrappers stay excluded: their ids resolve only under the config a PortOS-spawned provider injects, and the reviewer runs a bare `opencode` against the user's own config. - Each reviewer's options now UNION every provider record fronting its binary instead of reading one. Sourcing from a single record is what hid `claude-sonnet-5` behind the CLI record's retired `claude-sonnet-4-6` while `claude-code-tui` already listed the current tier. - The Add row folds a reviewer whose CLI is missing, or whose provider records are all switched off, behind a `+N unavailable` toggle. Hidden rather than dropped: both checks are local-machine-only and the reviewer list is federation-wide config, so the toggle reveals them badged with which signal fired, and an already-selected reviewer always renders its row. Migration 337 repairs the underlying data: `claude` has no `models` subcommand, so nothing in the app can refresh a stale Claude record. It inserts `claude-sonnet-5` after a retired `claude-sonnet-4-6` even on a curated list, which migration 153's exact-match policy skipped. Additive on purpose — the retired id and every tier pointer survive, since 4-6 still resolves and the defect is the new tier being absent, not the old one being present. --- client/src/components/cos/ReviewerPicker.jsx | 111 +++++++--- .../components/cos/ReviewerPicker.test.jsx | 62 +++++- client/src/hooks/useReviewerModelOptions.js | 197 +++++++++++++----- .../hooks/useReviewerModelOptions.test.jsx | 110 +++++++++- .../337-claude-sonnet-5-additive.js | 82 ++++++++ .../337-claude-sonnet-5-additive.test.js | 125 +++++++++++ 6 files changed, 598 insertions(+), 89 deletions(-) create mode 100644 scripts/migrations/337-claude-sonnet-5-additive.js create mode 100644 scripts/migrations/337-claude-sonnet-5-additive.test.js diff --git a/client/src/components/cos/ReviewerPicker.jsx b/client/src/components/cos/ReviewerPicker.jsx index b3672f02b7..118f10a6d1 100644 --- a/client/src/components/cos/ReviewerPicker.jsx +++ b/client/src/components/cos/ReviewerPicker.jsx @@ -65,10 +65,10 @@ const CUSTOM_MODEL_OPTION = '[custom]'; * * `modelOptions` is the resolved model-picker data, shaped like * `useReviewerModelOptions()`'s return: `{ optionsByReviewer, defaultModels, - * freeText, unavailable, loaded }`. Callers keep owning their own - * `api.getLocalLlmStatus` / `api.getProviders` fetches (that's what the hook is - * for) — passing nothing degrades every Model cell to a free-text input, which is - * still fully usable, rather than hiding the column. + * freeText, unavailable, providerDisabled, loaded }`. Callers keep owning their + * own `api.getLocalLlmStatus` / `api.getProviders` fetches (that's what the hook + * is for) — passing nothing degrades every Model cell to a free-text input, which + * is still fully usable, rather than hiding the column. * * `showRunFlags={false}` hides the stop-mode select and the "reviewer applies * fixes" checkbox for surfaces that can't honor them — the `/do:next` claim @@ -79,11 +79,16 @@ const CUSTOM_MODEL_OPTION = '[custom]'; * `installed` is a per-reviewer-slug install probe from the Code Review * Defaults endpoint (`GET /api/code-review/defaults`'s `installed` field, * #3606) — `{ claude: true, antigravity: false, ... }`. Only an explicit - * `false` renders a "not installed" badge; `undefined` (not a CLI reviewer, - * or the caller didn't fetch it) renders nothing. Warn-only: a reviewer stays - * selectable and selected even when flagged not-installed, since the CLI - * check is local-machine-only and a federated peer (or a later install) may - * satisfy it. + * `false` counts as missing; `undefined` (not a CLI reviewer, or the caller + * didn't fetch it) says nothing. + * + * Together with `modelOptions.providerDisabled`, that decides which reviewers + * the **Add** row offers up front: one whose CLI is missing here, or whose + * provider records are all switched off, is folded behind a `+N unavailable` + * toggle. Warn-only either way — the toggle reveals them with a badge and they + * stay selectable, and an ALREADY-SELECTED reviewer always renders its row + * (badged), since both checks are local-machine-only and the reviewer list is + * federation-wide config a peer may satisfy. */ export default function ReviewerPicker({ reviewers = [], @@ -108,6 +113,9 @@ export default function ReviewerPicker({ // pin maps use. Purely presentational — nothing is stored until an id is typed, // so this never has to round-trip through `onChange`. const [customModelTokens, setCustomModelTokens] = useState(() => new Set()); + // Whether the Add row also lists the reviewers this machine can't run (see + // `hiddenAddable`). Presentational only — nothing about it is stored. + const [showUnavailable, setShowUnavailable] = useState(false); const isCustomModel = (token) => customModelTokens.has(token.toLowerCase()); const setCustomModel = (token, on) => setCustomModelTokens((prev) => { const next = new Set(prev); @@ -122,7 +130,7 @@ export default function ReviewerPicker({ // the active provider's own reviewer (falling back to copilot when that // provider maps to none) — see `codeReviewDefaultsFromProvider`. const selected = Array.isArray(reviewers) ? [...new Set(reviewers.map(normalizeReviewerValue))] : []; - const available = REVIEWER_OPTIONS.filter(o => !selected.includes(o.value)); + const addable = REVIEWER_OPTIONS.filter(o => !selected.includes(o.value)); const hasNonCopilot = selected.some(r => r !== 'copilot'); const selectedUsernames = normalizeReviewUsernames(usernames); const atMaxUsernames = selectedUsernames.length >= MAX_REVIEW_USERNAMES; @@ -165,20 +173,53 @@ export default function ReviewerPicker({ // normally does", so clearing the select DELETES the key rather than writing `''`. const effortsMap = asMap(reviewerEfforts); const efforts = keyedLookup(effortsMap); - // Only an explicit `false` counts — `undefined` covers both "not a CLI - // reviewer" (copilot/lmstudio/ollama/@username) and "caller didn't fetch - // `installed`", neither of which should render a warning badge. - const notInstalled = (token) => installed?.[token] === false; - const renderInstalledBadge = (token) => notInstalled(token) && ( - - not installed - - ); + // Why this reviewer can't run here, or null when nothing says it can't. + // + // Two independent signals, both warn-only and both reported only when the + // caller actually fetched them — a reviewer stays selectable and selected + // either way, since the checks are local-machine-only and a federated peer + // (or a later install / a flip in Settings) may satisfy them: + // + // - `installed[token] === false` — the CLI binary isn't on PATH. Only an + // explicit `false` counts; `undefined` covers both "not a CLI reviewer" + // (copilot/@username) and "caller didn't fetch `installed`". + // - `providerDisabled[token]` — every provider record fronting that binary is + // switched off on this install, so the user has said they don't use it. A + // `/api/providers` that failed or hasn't landed reports nothing (see the + // hook), so this never fires on a slow page. + const unavailability = (token) => { + if (installed?.[token] === false) { + return { + label: 'not installed', + title: `${reviewerLabel(token)}'s CLI binary wasn't found on this machine. It still runs (federation-wide config), but the review loop here will report it unsatisfied until it's installed.` + }; + } + if (modelOptions?.providerDisabled?.[token]) { + return { + label: 'disabled', + title: `${reviewerLabel(token)}'s provider records are all switched off in Settings → AI Providers, so this machine isn't set up to use it. Adding it still works — the review loop spawns its CLI directly, and a federated peer may have it enabled.` + }; + } + return null; + }; + const renderUnavailableBadge = (token) => { + const reason = unavailability(token); + return reason && ( + + {reason.label} + + ); + }; + // The Add row lists what this machine can actually run, so a reviewer whose + // CLI is missing or whose providers are all switched off is folded behind a + // count instead of padding the row with things the review loop would report + // unsatisfied. HIDDEN, not dropped: the checks are local-machine-only and the + // reviewer list is federation-wide config, so the toggle reveals them (badged) + // rather than making a peer's reviewer unconfigurable from here. + const hiddenAddable = addable.filter(opt => unavailability(opt.value)); + const addOptions = showUnavailable + ? addable + : addable.filter(opt => !hiddenAddable.includes(opt)); const emit = (next) => onChange?.({ reviewers: selected, @@ -586,7 +627,7 @@ export default function ReviewerPicker({ {reviewerLabel(value)} - {renderInstalledBadge(value)} + {renderUnavailableBadge(value)} Model
{renderModelCell(value)}
@@ -630,10 +671,10 @@ export default function ReviewerPicker({ )} - {available.length > 0 && ( + {addable.length > 0 && (
Add: - {available.map(opt => ( + {addOptions.map(opt => ( ))} + {hiddenAddable.length > 0 && ( + + )}
)} diff --git a/client/src/components/cos/ReviewerPicker.test.jsx b/client/src/components/cos/ReviewerPicker.test.jsx index c40ab85c2b..427144c5be 100644 --- a/client/src/components/cos/ReviewerPicker.test.jsx +++ b/client/src/components/cos/ReviewerPicker.test.jsx @@ -32,10 +32,66 @@ describe('ReviewerPicker', () => { expect(screen.queryByText('not installed')).not.toBeInTheDocument(); }); - it('flags an unselected reviewer in the Add row too', () => { + it('flags an unselected reviewer once the Add row reveals it', async () => { + const user = userEvent.setup(); render( {}} />); - const addButton = screen.getByRole('button', { name: /Antigravity/ }); - expect(addButton).toHaveTextContent('not installed'); + await user.click(screen.getByRole('button', { name: /1 unavailable/ })); + expect(screen.getByRole('button', { name: /Antigravity/ })).toHaveTextContent('not installed'); + }); + }); + + // The Add row lists what this machine can actually run. Hidden, not dropped: + // both signals are local-machine-only and the reviewer list is + // federation-wide config, so a peer's reviewer stays configurable from here. + describe('unavailable reviewers in the Add row', () => { + const modelOptions = { providerDisabled: { kimi: true, cursor: true } }; + + it('hides a missing CLI and an all-off provider behind one count', () => { + render( + {}} + /> + ); + expect(screen.getByRole('button', { name: /3 unavailable/ })).toBeInTheDocument(); + for (const hidden of [/Antigravity/, /Kimi/, /Cursor Agent/]) { + expect(screen.queryByRole('button', { name: hidden })).not.toBeInTheDocument(); + } + // An available reviewer is still offered up front. + expect(screen.getByRole('button', { name: /Codex/ })).toBeInTheDocument(); + }); + + it('reveals them, badged with which signal fired, and adds them normally', async () => { + const onChange = vi.fn(); + const user = userEvent.setup(); + render( + + ); + await user.click(screen.getByRole('button', { name: /3 unavailable/ })); + expect(screen.getByRole('button', { name: /Kimi/ })).toHaveTextContent('disabled'); + expect(screen.getByRole('button', { name: /Antigravity/ })).toHaveTextContent('not installed'); + await user.click(screen.getByRole('button', { name: /Kimi/ })); + expect(onChange).toHaveBeenCalledWith(expect.objectContaining({ reviewers: ['copilot', 'kimi'] })); + }); + + it('keeps an already-selected unavailable reviewer visible, badged', () => { + render( + {}} /> + ); + expect(screen.getByText('Kimi').parentElement).toHaveTextContent('disabled'); + }); + + it('offers the whole roster when neither signal was fetched', () => { + render( {}} />); + expect(screen.queryByRole('button', { name: /unavailable/ })).not.toBeInTheDocument(); + expect(screen.getByRole('button', { name: /Kimi/ })).toBeInTheDocument(); }); }); diff --git a/client/src/hooks/useReviewerModelOptions.js b/client/src/hooks/useReviewerModelOptions.js index b1e8b2de92..e69792de4b 100644 --- a/client/src/hooks/useReviewerModelOptions.js +++ b/client/src/hooks/useReviewerModelOptions.js @@ -1,6 +1,6 @@ import { useEffect, useMemo, useState } from 'react'; import * as api from '../services/api'; -import { filterSelectableModels, selectableModelsForProvider, isAntigravityProvider, isGrokBuildCli, isKimiProvider, antigravityModelEffortLevels } from '../utils/providers'; +import { filterSelectableModels, selectableModelsForProvider, isAntigravityProvider, isCursorProvider, isGrokBuildCli, isKimiProvider, antigravityModelEffortLevels } from '../utils/providers'; import { MODEL_SELECTABLE_REVIEWERS } from '../components/cos/constants'; import { reviewerEffortLevels, normalizeReviewerSlug } from '../lib/reviewerPins'; import { LOCAL_LLM_BACKENDS } from '../lib/localLlmBackends'; @@ -11,6 +11,55 @@ import { LOCAL_LLM_BACKENDS } from '../lib/localLlmBackends'; // listing runs the `mtplx` wrapper, so it stays catalog-sourced and free-text. const PROBED_LOCAL_BACKENDS = LOCAL_LLM_BACKENDS.map((b) => b.id); +/** + * Every provider record that fronts a reviewer's binary, as predicates in + * PREFERENCE ORDER. Two reductions run over each list and they answer different + * questions: + * + * - the **option list** unions every matching record's catalog, because a + * reviewer runs one binary and any record fronting that binary lists ids that + * binary accepts. Sourcing from a single record made the picker hostage to + * that record's staleness — `claude-code` (CLI) listing `claude-sonnet-4-6` + * while `claude-code-tui` already listed `claude-sonnet-5` showed the reviewer + * the retired tier and hid the current one. + * - the **shown default** takes the FIRST match, so a reviewer spawned + * non-interactively reports the CLI record's default rather than the TUI's. + * + * A predicate rather than a bare id wherever the app already recognizes a + * provider by more than its shipped id (an `agy` configured by path), so this + * classifies the same records the rest of the UI does. + * + * What is deliberately NOT matched matters as much as what is: + * - **No Bedrock/Vertex record.** `claude-code-bedrock` lists + * `us.anthropic.*` ids that resolve only under that record's own environment. + * - **No `opencode-` preset.** Those enumerate ids that resolve + * only under the `OPENCODE_CONFIG_CONTENT` a PortOS-spawned provider injects, + * and the reviewer runs a bare `opencode` against the user's OWN config. The + * Zen CLI/TUI records are the exception and ARE matched: their ids are the + * namespaced `opencode/*` spellings that bare `opencode models` prints, and + * the Harnesses page's model refresh fills them from exactly that probe (see + * `server/services/harnesses.js#usesHarnessCatalog`), so they are the live + * catalog for the account the reviewer will bill. + * - **Not `opencode-zen` itself.** That is the HTTP-API record; its bare ids + * (`claude-opus-5`) are Zen's API model names, which `opencode -m` cannot + * resolve. + */ +const REVIEWER_PROVIDER_MATCHERS = Object.freeze({ + claude: [(p) => p.id === 'claude-code', (p) => p.id === 'claude-code-tui'], + codex: [(p) => p.id === 'codex', (p) => p.id === 'codex-tui'], + antigravity: [isAntigravityProvider], + // `grok` names one binary that ships as BOTH a `cli` and a `tui` provider, and + // the reviewer is spawned non-interactively, so the CLI's record wins the + // default — the broad predicate follows it for an install that only kept the TUI. + grok: [(p) => p.id === 'grok-cli', isGrokBuildCli], + cursor: [(p) => p.id === 'cursor-cli', isCursorProvider], + kimi: [(p) => p.id === 'kimi-cli', isKimiProvider], + opencode: [(p) => p.id === 'opencode-zen-cli', (p) => p.id === 'opencode-zen-tui'], + mtplx: [(p) => p.id === 'mtplx'], + lmstudio: [(p) => p.id === 'lmstudio'], + ollama: [(p) => p.id === 'ollama'], +}); + /** * Selectable model ids per model-taking reviewer, for `ReviewerPicker`'s Model * column. One hook so all four picker surfaces (Code Review Defaults, TaskAddForm, @@ -21,15 +70,16 @@ const PROBED_LOCAL_BACKENDS = LOCAL_LLM_BACKENDS.map((b) => b.id); * - `lmstudio` / `ollama` ids come from `/api/local-llm/status`, so they reflect * what's actually installed rather than a provider's stale stored `models`. * - Every other reviewer's tiers come from the provider catalog - * (`/api/providers`). That includes `mtplx`: its installed checkpoints live - * behind `/api/local-llm/mtplx/status`, which INVOKES the `mtplx` wrapper (a - * several-hundred-MB venv bootstrap on a cold version — see + * (`/api/providers`), unioned across the records listed in + * `REVIEWER_PROVIDER_MATCHERS`. That includes `mtplx`: its installed + * checkpoints live behind `/api/local-llm/mtplx/status`, which INVOKES the + * `mtplx` wrapper (a several-hundred-MB venv bootstrap on a cold version — see * `server/lib/mtplxRuntime.js`), and a picker mount must never pay that. The * shipped provider's catalog plus a free-text field is the honest trade. * - * The `claude` list spans BOTH usage modes: the `claude-code` provider tiers and - * the installed Ollama ids (an Ollama-backed `claude` CLI, where `--model` selects - * the local model). Deduped, order-preserving. + * The `claude` list spans BOTH usage modes: the `claude-code`/`claude-code-tui` + * provider tiers and the installed Ollama ids (an Ollama-backed `claude` CLI, + * where `--model` selects the local model). Deduped, order-preserving. * * `freeText` marks a reviewer whose picker must ALSO accept a typed id, not only * a pick: an Ollama-backed `claude` can run any locally-installed id, and a @@ -41,6 +91,14 @@ const PROBED_LOCAL_BACKENDS = LOCAL_LLM_BACKENDS.map((b) => b.id); * `unavailable` distinguishes "backend is down" from "backend has no models" so * the empty state can say the useful thing. Absent = not probed (every reviewer * outside PROBED_LOCAL_BACKENDS, `mtplx` included). + * + * `providerDisabled[reviewer]` is true when the reviewer HAS provider records on + * this install and every one of them is switched off — the signal + * `ReviewerPicker` uses to drop it from the Add row, so a machine that never + * enabled Kimi or Cursor isn't offered them. Only ever true from a landed fetch: + * a null/failed `/api/providers` matches no record, which reads as "nothing + * known", never as "switched off". + * * `loaded` flips once both fetches settle, so a consumer can tell "no options * yet" from "genuinely no options" (an empty list is a real answer, not a * pre-fetch placeholder). @@ -51,7 +109,7 @@ const PROBED_LOCAL_BACKENDS = LOCAL_LLM_BACKENDS.map((b) => b.id); * reviewer returns its static ladder. Lives here rather than in the picker so the * picker keeps doing no fetching of its own. * - * @returns {{ optionsByReviewer: Record, defaultModels: Record, freeText: Record, unavailable: Record, modelEffortLevels: (reviewer: string, model?: string|null) => readonly string[]|null, loaded: boolean, reviewers: string[] }} + * @returns {{ optionsByReviewer: Record, defaultModels: Record, freeText: Record, unavailable: Record, providerDisabled: Record, modelEffortLevels: (reviewer: string, model?: string|null) => readonly string[]|null, loaded: boolean, reviewers: string[] }} */ export default function useReviewerModelOptions() { const [localStatus, setLocalStatus] = useState(null); @@ -78,34 +136,49 @@ export default function useReviewerModelOptions() { const localIds = (backend) => (localStatus?.[backend]?.models || []) .map((m) => m.id || m.name) .filter(Boolean); - // `match` is a predicate rather than an id so a reviewer whose provider can be - // recognized by more than its shipped id (an `agy` configured by path) uses the - // same predicate the rest of the app does. Several matchers = preference order: - // `grok` names one binary that ships as BOTH a `cli` and a `tui` provider, and - // the reviewer is spawned non-interactively, so the CLI's catalog wins — the - // broad predicate is the fallback for an install that only kept the TUI. - const providerFor = (...matchers) => - matchers.reduce((found, match) => found || (providers || []).find(match), null); - const providerTiers = (...matchers) => { - const provider = providerFor(...matchers); - if (!provider) return []; + // Every record fronting each reviewer's binary, in matcher-preference order + // (not `providers` array order) so `[0]` is the record whose default the + // picker shows. De-duped by identity: two matchers commonly overlap + // (`grok-cli` is also an `isGrokBuildCli`). + // + // Resolved ONCE for the whole roster rather than per lookup — the option + // list, the shown default, the agy raw catalog and `providerDisabled` all + // ask the same question, and a per-call helper re-walked the provider array + // for every one of them. + const providersByReviewer = Object.fromEntries( + Object.entries(REVIEWER_PROVIDER_MATCHERS).map(([reviewer, matchers]) => { + const matched = []; + for (const match of matchers) { + for (const provider of providers || []) { + if (match(provider) && !matched.includes(provider)) matched.push(provider); + } + } + return [reviewer, matched]; + }) + ); + const providersFor = (reviewer) => providersByReviewer[reviewer] || []; + + // `selectableModelsForProvider` owns the per-provider normalization (today: + // agy's one-id-per-effort-tier catalog collapsed to base ids, so the row's + // separate Effort cell stays the effort control). Going through it rather + // than special-casing agy here keeps the rule in one place. + const selectableModels = (provider) => { // `models` may be empty on a CLI provider configured with only a // defaultModel — `[]` is truthy, so a bare `||` wouldn't fall through. const models = provider.models?.length ? provider.models : [provider.defaultModel]; - // `selectableModelsForProvider` owns the per-provider normalization (today: - // agy's one-id-per-effort-tier catalog collapsed to base ids, so the row's - // separate Effort cell stays the effort control). Going through it rather - // than special-casing agy here keeps the rule in one place. return filterSelectableModels(selectableModelsForProvider(provider, models)); }; + const providerTiers = (reviewer) => + Array.from(new Set(providersFor(reviewer).flatMap(selectableModels))); + // Show the configured provider default in the picker even when the user has // not saved a per-reviewer override. A concrete default is useful context; // configured-default sentinels intentionally resolve to null because the CLI // owns the choice and there is no model id PortOS can honestly display. - const providerDefault = (...matchers) => { - const provider = providerFor(...matchers); + const providerDefault = (reviewer) => { + const provider = providersFor(reviewer)[0]; if (!provider?.defaultModel) return null; return filterSelectableModels( selectableModelsForProvider(provider, [provider.defaultModel]) @@ -115,7 +188,7 @@ export default function useReviewerModelOptions() { // Local backend model lists come from the live runtime probe, so only show a // provider default when it is present in that authoritative list. const localDefault = (backend) => { - const candidate = providerFor((p) => p.id === backend)?.defaultModel; + const candidate = providersFor(backend)[0]?.defaultModel; return candidate && localIds(backend).includes(candidate) ? candidate : null; }; @@ -123,52 +196,58 @@ export default function useReviewerModelOptions() { const optionsByReviewer = { lmstudio: localIds('lmstudio'), ollama, - codex: providerTiers((p) => p.id === 'codex'), + codex: providerTiers('codex'), // Claude tiers first (the common case), then installed Ollama ids for an // Ollama-backed `claude`. Deduped, order-preserving. - claude: Array.from(new Set([...providerTiers((p) => p.id === 'claude-code'), ...ollama].filter(Boolean))), - antigravity: providerTiers(isAntigravityProvider), + claude: Array.from(new Set([...providerTiers('claude'), ...ollama].filter(Boolean))), + antigravity: providerTiers('antigravity'), // The shipped grok provider carries only the configured-default sentinel, // which `filterSelectableModels` strips — so this is legitimately `[]` until // the user lists real ids on the provider. The Model cell stays useful // regardless because grok, like every CLI reviewer, is free-text. - grok: providerTiers((p) => p.id === 'grok-cli', isGrokBuildCli), - cursor: providerTiers((p) => p.id === 'cursor-cli', (p) => p.id === 'cursor-tui'), + grok: providerTiers('grok'), + cursor: providerTiers('cursor'), // Legitimately empty, for grok's documented reason: the shipped kimi // provider carries only the configured-default sentinel, which // `filterSelectableModels` strips. Free-text keeps the cell usable. - kimi: providerTiers((p) => p.id === 'kimi-cli', isKimiProvider), - // Deliberately NOT sourced from the `opencode-` presets. Those - // enumerate ids that only resolve under the `OPENCODE_CONFIG_CONTENT` a - // PortOS-spawned provider injects, and the reviewer runs a bare `opencode` - // against the user's OWN config — so listing them would offer picks that - // silently fail. `opencode -m` takes a `provider/model` id the user types. - opencode: [], - mtplx: providerTiers((p) => p.id === 'mtplx'), + kimi: providerTiers('kimi'), + // The namespaced `opencode/*` ids the seeded Zen CLI/TUI records carry, + // which the Harnesses page refreshes from `opencode models` — so the cell + // is a dropdown of what this account can actually run instead of the plain + // text input it used to be. Still free-text underneath: `opencode -m` takes + // any `provider/model` the user's own config resolves. + opencode: providerTiers('opencode'), + mtplx: providerTiers('mtplx'), }; const defaultModels = { lmstudio: localDefault('lmstudio'), ollama: localDefault('ollama'), - codex: providerDefault((p) => p.id === 'codex'), - claude: providerDefault((p) => p.id === 'claude-code'), - antigravity: providerDefault(isAntigravityProvider), - grok: providerDefault((p) => p.id === 'grok-cli', isGrokBuildCli), - cursor: providerDefault((p) => p.id === 'cursor-cli', (p) => p.id === 'cursor-tui'), - kimi: providerDefault((p) => p.id === 'kimi-cli', isKimiProvider), + codex: providerDefault('codex'), + claude: providerDefault('claude'), + antigravity: providerDefault('antigravity'), + grok: providerDefault('grok'), + cursor: providerDefault('cursor'), + kimi: providerDefault('kimi'), + // Deliberately null even though the Zen records carry one: the reviewer + // spawns a BARE `opencode`, which falls back to whatever the user's own + // config names — not to the PortOS record's default. Naming a model here + // would claim a default the run won't use. opencode: null, - mtplx: providerDefault((p) => p.id === 'mtplx'), + mtplx: providerDefault('mtplx'), }; - // The agy provider's RAW catalog — one id per effort tier + // The agy providers' RAW catalog — one id per effort tier // (`gemini-3.6-flash-low|-medium|-high`), which is exactly what the narrowing // reads. Deliberately NOT `optionsByReviewer.antigravity`: that list has // already had the suffixes collapsed away, so it carries no tier information. - const antigravityCatalog = (providers || []).find(isAntigravityProvider)?.models || []; + const antigravityCatalog = Array.from(new Set( + providersFor('antigravity').flatMap((provider) => provider.models || []) + )); // The effort ladder a reviewer offers ONCE ITS MODEL IS PINNED. `agy` validates // the pair, so a model with no `-medium` sibling must not offer `medium` - // (#3733). `antigravityModelEffortLevels` returns null for "can't tell" — empty - // catalog, unset model, or the configured-default sentinel — and the full + // (#3733). `antigravityModelEffortLevels` returns null for "can't tell" — + // empty catalog, unset model, or the configured-default sentinel — and the full // static ladder stands there, the same null-means-fall-back contract // `effortLevelsForProvider` uses. `[]` is a real answer: that model has no // effort tiers at all. @@ -191,8 +270,9 @@ export default function useReviewerModelOptions() { // because their catalogs are stored snapshots that can lag a newly-released // tier — grok's shipped catalog holds no real id at all, so a typed id is the // ONLY way to pin one (an agy pin may also be typed effort-suffixed — the - // server splits it). Derived from the rosters so a reviewer added to either - // one can't silently default to the wrong control. + // server splits it) — and `opencode` because a user's own config can declare + // namespaces the Zen catalog never lists. Derived from the rosters so a + // reviewer added to either one can't silently default to the wrong control. freeText: Object.fromEntries( MODEL_SELECTABLE_REVIEWERS.map((r) => [r, !PROBED_LOCAL_BACKENDS.includes(r)]) ), @@ -200,6 +280,19 @@ export default function useReviewerModelOptions() { lmstudio: localStatus?.lmstudio?.available === false, ollama: localStatus?.ollama?.available === false, }, + // `every` over a NON-EMPTY match list, so the two ways to have no enabled + // record stay apart: an install that switched every Kimi record off is + // `true` (hide it), while a reviewer with no records at all — or a fetch + // that failed or hasn't landed — is `false` (nothing is known, so hide + // nothing). `enabled === false` rather than falsiness, which deliberately + // reads a record with no `enabled` key as ON — the opposite of + // `providerCardState`'s stricter test, because this answer HIDES a control + // and incomplete data must never do that. + providerDisabled: Object.fromEntries( + Object.entries(providersByReviewer).map(([reviewer, matched]) => + [reviewer, matched.length > 0 && matched.every((p) => p.enabled === false)] + ) + ), loaded, // Exposed so a consumer can assert it covers every model-taking reviewer. reviewers: MODEL_SELECTABLE_REVIEWERS, diff --git a/client/src/hooks/useReviewerModelOptions.test.jsx b/client/src/hooks/useReviewerModelOptions.test.jsx index 06503d1637..394ae6120d 100644 --- a/client/src/hooks/useReviewerModelOptions.test.jsx +++ b/client/src/hooks/useReviewerModelOptions.test.jsx @@ -24,12 +24,21 @@ const providers = [ 'gemini-3.1-pro-high', ], }, - // One `grok` binary ships as both a TUI and a CLI provider; the reviewer is - // spawned non-interactively, so the CLI's catalog is the one it should offer. - { id: 'grok-tui', type: 'tui', command: 'grok', models: ['stale-tui-id'] }, + // One `grok` binary ships as both a TUI and a CLI provider. Both list ids that + // binary accepts, so the picker unions them; the CLI's record still owns the + // shown DEFAULT, since the reviewer is spawned non-interactively. + { id: 'grok-tui', type: 'tui', command: 'grok', models: ['tui-only-id'] }, { id: 'grok-cli', type: 'cli', command: 'grok', models: ['grok-configured-default', 'grok-code-fast-1'] }, { id: 'cursor-cli', type: 'cli', command: 'cursor-agent', models: ['auto', 'gpt-5'] }, { id: 'mtplx', type: 'api', models: ['mtplx-qwen38-27b-optimized-speed'], defaultModel: 'mtplx-qwen38-27b-optimized-speed' }, + // The seeded OpenCode Zen wrappers, whose namespaced ids the Harnesses page + // refreshes from `opencode models` — the reviewer's dropdown source. + { id: 'opencode-zen-cli', type: 'cli', command: 'opencode', models: ['opencode/big-pickle'], defaultModel: 'opencode/big-pickle' }, + { id: 'opencode-zen-tui', type: 'tui', command: 'opencode', models: ['opencode/big-pickle', 'opencode/mimo-v2.5-free'] }, + // An OpenCode wrapper pointed at a local runtime: its ids resolve only under + // the config PortOS injects, so the reviewer (a BARE `opencode`) must not + // offer them. + { id: 'opencode-ollama', type: 'cli', command: 'opencode', ollamaBacked: true, models: ['qwen3-coder:30b'] }, ]; describe('useReviewerModelOptions', () => { @@ -69,6 +78,28 @@ describe('useReviewerModelOptions', () => { expect(result.current.defaultModels.antigravity).toBeNull(); }); + // The reported defect: `claude-code` (CLI) still listed the retired + // `claude-sonnet-4-6` while `claude-code-tui` had moved to `claude-sonnet-5`, + // and sourcing the picker from the CLI record alone showed the retired tier + // and hid the current one. `claude` has no `models` subcommand, so nothing can + // refresh that record in place. + it('unions the Claude CLI and TUI catalogs so one stale record can\'t hide a live tier', async () => { + getProviders.mockResolvedValue({ providers: [ + { id: 'claude-code', type: 'cli', command: 'claude', models: ['claude-haiku-4-5', 'claude-sonnet-4-6'], defaultModel: 'claude-haiku-4-5' }, + { id: 'claude-code-tui', type: 'tui', command: 'claude', models: ['claude-sonnet-5', 'claude-opus-5'] }, + // Bedrock ids resolve only under that record's own environment. + { id: 'claude-code-bedrock', type: 'cli', command: 'claude', models: ['us.anthropic.claude-sonnet-5'] }, + ] }); + const { result } = renderHook(() => useReviewerModelOptions()); + await waitFor(() => expect(result.current.loaded).toBe(true)); + expect(result.current.optionsByReviewer.claude).toEqual([ + 'claude-haiku-4-5', 'claude-sonnet-4-6', 'claude-sonnet-5', 'claude-opus-5', 'qwen2.5:7b', + ]); + // The CLI record still owns the shown default — the reviewer is spawned + // non-interactively. + expect(result.current.defaultModels.claude).toBe('claude-haiku-4-5'); + }); + // #3728: `agy --model ` is real, so the antigravity row gets a Model cell. // Its ids arrive effort-suffixed and would otherwise duplicate the Effort cell // (and hand agy a `--model X-high --effort high` pair it rejects). @@ -82,10 +113,14 @@ describe('useReviewerModelOptions', () => { }); // #3729: `grok --model ` is real, so the grok row gets a Model cell too. - it('sources grok options from the CLI provider, not the TUI, and drops the sentinel', async () => { + // Both records front the same binary, so the option list unions them (CLI + // first) rather than letting one record's staleness hide the other's ids — + // sourcing from a single record is exactly what hid `claude-sonnet-5` behind + // the CLI record's retired `claude-sonnet-4-6`. + it('unions the grok CLI and TUI catalogs, CLI first, and drops the sentinel', async () => { const { result } = renderHook(() => useReviewerModelOptions()); await waitFor(() => expect(result.current.loaded).toBe(true)); - expect(result.current.optionsByReviewer.grok).toEqual(['grok-code-fast-1']); + expect(result.current.optionsByReviewer.grok).toEqual(['grok-code-fast-1', 'tui-only-id']); // Free-text: the shipped grok catalog is sentinel-only, so a typed id is // often the only way to pin one. expect(result.current.freeText.grok).toBe(true); @@ -102,7 +137,70 @@ describe('useReviewerModelOptions', () => { getProviders.mockResolvedValue({ providers: [providers.find((p) => p.id === 'grok-tui')] }); const { result } = renderHook(() => useReviewerModelOptions()); await waitFor(() => expect(result.current.loaded).toBe(true)); - expect(result.current.optionsByReviewer.grok).toEqual(['stale-tui-id']); + expect(result.current.optionsByReviewer.grok).toEqual(['tui-only-id']); + }); + + // The Model cell used to be a bare text input for opencode: the reviewer runs a + // BARE `opencode`, and the only catalogs on hand were the `opencode-` + // presets, whose ids resolve solely under the config a PortOS-spawned provider + // injects. The seeded Zen wrappers are the exception — their namespaced ids are + // what `opencode models` prints, and the Harnesses page refreshes them from + // exactly that probe — so those, and only those, feed the dropdown. + describe('opencode', () => { + it('offers the Zen wrappers\' namespaced ids, unioned across CLI and TUI', async () => { + const { result } = renderHook(() => useReviewerModelOptions()); + await waitFor(() => expect(result.current.loaded).toBe(true)); + expect(result.current.optionsByReviewer.opencode) + .toEqual(['opencode/big-pickle', 'opencode/mimo-v2.5-free']); + // Still free-text underneath: a user's own config can declare namespaces + // the Zen catalog never lists. + expect(result.current.freeText.opencode).toBe(true); + }); + + it('never offers a local-runtime wrapper\'s ids', async () => { + const { result } = renderHook(() => useReviewerModelOptions()); + await waitFor(() => expect(result.current.loaded).toBe(true)); + expect(result.current.optionsByReviewer.opencode).not.toContain('qwen3-coder:30b'); + }); + + // The reviewer spawns a bare `opencode`, which falls back to whatever the + // user's OWN config names — not to the PortOS record's default. Showing the + // record's default would claim a model the run won't use. + it('shows no default even though the Zen record carries one', async () => { + const { result } = renderHook(() => useReviewerModelOptions()); + await waitFor(() => expect(result.current.loaded).toBe(true)); + expect(result.current.defaultModels.opencode).toBeNull(); + }); + }); + + // The Add row hides what this machine can't run, so "every record switched + // off" has to stay apart from "no record at all" and from "fetch failed". + describe('providerDisabled', () => { + it('is true only when every record fronting the reviewer is switched off', async () => { + getProviders.mockResolvedValue({ providers: [ + { id: 'kimi-cli', type: 'cli', command: 'kimi', enabled: false, models: [] }, + { id: 'kimi-tui', type: 'tui', command: 'kimi', enabled: false, models: [] }, + { id: 'grok-cli', type: 'cli', command: 'grok', enabled: false, models: [] }, + { id: 'grok-tui', type: 'tui', command: 'grok', enabled: true, models: [] }, + // No `enabled` key at all — a record written before the flag existed + // must not read as off. + { id: 'codex', type: 'cli', command: 'codex', models: [] }, + ] }); + const { result } = renderHook(() => useReviewerModelOptions()); + await waitFor(() => expect(result.current.loaded).toBe(true)); + expect(result.current.providerDisabled.kimi).toBe(true); + expect(result.current.providerDisabled.grok).toBe(false); + expect(result.current.providerDisabled.codex).toBe(false); + }); + + it('reports nothing disabled when the reviewer has no records, or the fetch failed', async () => { + getProviders.mockResolvedValue({ providers: [] }); + const { result } = renderHook(() => useReviewerModelOptions()); + await waitFor(() => expect(result.current.loaded).toBe(true)); + for (const reviewer of MODEL_SELECTABLE_REVIEWERS) { + expect(result.current.providerDisabled[reviewer], reviewer).toBe(false); + } + }); }); // #3733: `agy` validates the model/effort PAIR, so the Effort cell's ladder has diff --git a/scripts/migrations/337-claude-sonnet-5-additive.js b/scripts/migrations/337-claude-sonnet-5-additive.js new file mode 100644 index 0000000000..d02b6d03af --- /dev/null +++ b/scripts/migrations/337-claude-sonnet-5-additive.js @@ -0,0 +1,82 @@ +/** + * Offer `claude-sonnet-5` on a Claude CLI/TUI record that still lists only the + * retired `claude-sonnet-4-6` sonnet tier. + * + * Migration 153 already made this swap, but ONLY for a `models` array matching + * the prior seeded trio exactly — a user who had appended an id to the list (a + * Fable tier, say) was classified as "customized" and skipped, and their record + * kept the 4-6 tier while the shipped seed and their other Claude records moved + * on. `claude` has no `models` subcommand, so nothing in the app can refresh + * that record: the reviewer/task model pickers reading it offer the retired + * sonnet and cannot offer the current one at all. + * + * ADDITIVE, deliberately — the opposite policy from 153/206 and from + * `makeSeededProviderTierMigration`, because this one runs against lists the + * user curated: + * + * - `claude-sonnet-5` is INSERTED right after `claude-sonnet-4-6`, and the + * retired id is KEPT. `claude-sonnet-4-6` still resolves for the CLI, so + * dropping an id a user chose to list would remove a working pin; the defect + * is the new tier being absent, not the old one being present. + * - Tier pointers (`defaultModel`/`lightModel`/`mediumModel`/`heavyModel`) are + * left ALONE. They point at an id that still works, and a curated list is + * exactly where re-pointing would override a deliberate choice. + * + * Idempotent by the same condition either way: a record already listing + * `claude-sonnet-5` is untouched, so this is a no-op on a seeded install (153/206 + * or a fresh `data.reference` seed already put it there) and on a second run. + */ + +import { readProvidersDoc, writeJsonAtomic } from './_lib.js'; + +const PROVIDERS_REL_PATH = 'data/providers.json'; + +// The four seeded Claude records and the sonnet id each one spells. The Bedrock +// pair uses the region-qualified form its own environment resolves — inserting a +// bare `claude-sonnet-5` there would offer an id that record cannot run. +const TARGETS = [ + { id: 'claude-code', retired: 'claude-sonnet-4-6', current: 'claude-sonnet-5' }, + { id: 'claude-code-tui', retired: 'claude-sonnet-4-6', current: 'claude-sonnet-5' }, + { id: 'claude-code-bedrock', retired: 'us.anthropic.claude-sonnet-4-6', current: 'us.anthropic.claude-sonnet-5' }, + { id: 'claude-code-tui-bedrock', retired: 'us.anthropic.claude-sonnet-4-6', current: 'us.anthropic.claude-sonnet-5' }, +]; + +export default { + async up({ rootDir }) { + const doc = await readProvidersDoc({ rootDir }); + if (!doc.ok) { + if (doc.reason === 'no-file') console.log(`📄 ${PROVIDERS_REL_PATH} not present — skipping (fresh install seeds claude-sonnet-5 from data.reference)`); + else if (doc.reason === 'unreadable') console.log(`⚠️ ${PROVIDERS_REL_PATH}: invalid JSON, skipping (${doc.err.message})`); + else console.log(`⚠️ ${PROVIDERS_REL_PATH}: unexpected shape, skipping`); + return { ok: false, reason: doc.reason, updated: 0 }; + } + + const { config, providers, path: providersPath } = doc; + const touched = []; + + for (const { id, retired, current } of TARGETS) { + const provider = providers[id]; + if (!provider || !Array.isArray(provider.models)) continue; + const at = provider.models.indexOf(retired); + // Nothing to repair unless the retired id is listed AND the current one + // isn't: an already-current record (seeded, or bumped by 153) is a no-op, + // and a record that never listed the retired tier is not this bug. + if (at === -1 || provider.models.includes(current)) continue; + provider.models = [ + ...provider.models.slice(0, at + 1), + current, + ...provider.models.slice(at + 1), + ]; + touched.push(id); + } + + if (touched.length === 0) { + console.log(`✅ ${PROVIDERS_REL_PATH}: Claude sonnet tier already current — no change`); + return { ok: true, reason: 'already-current', updated: 0 }; + } + + await writeJsonAtomic(providersPath, config); + console.log(`📝 ${PROVIDERS_REL_PATH}: offered claude-sonnet-5 on ${touched.join(', ')}`); + return { ok: true, reason: 'updated', updated: touched.length }; + }, +}; diff --git a/scripts/migrations/337-claude-sonnet-5-additive.test.js b/scripts/migrations/337-claude-sonnet-5-additive.test.js new file mode 100644 index 0000000000..47316977fe --- /dev/null +++ b/scripts/migrations/337-claude-sonnet-5-additive.test.js @@ -0,0 +1,125 @@ +/** + * Test for migration 337 — offer `claude-sonnet-5` on a Claude CLI/TUI record + * that still lists only the retired `claude-sonnet-4-6` tier. + */ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtempSync, rmSync, writeFileSync, readFileSync, mkdirSync } from 'fs'; +import { tmpdir } from 'os'; +import { join } from 'path'; + +import migration from './337-claude-sonnet-5-additive.js'; + +const writeJson = (path, value) => writeFileSync(path, JSON.stringify(value, null, 2) + '\n'); +const readJson = (path) => JSON.parse(readFileSync(path, 'utf-8')); + +describe('migration 337 — claude-sonnet-5 additive repair', () => { + let rootDir; + let providersPath; + + beforeEach(() => { + rootDir = mkdtempSync(join(tmpdir(), 'portos-337-')); + mkdirSync(join(rootDir, 'data')); + providersPath = join(rootDir, 'data', 'providers.json'); + }); + + afterEach(() => rmSync(rootDir, { recursive: true, force: true })); + + const seed = (providers) => writeJson(providersPath, { activeProvider: 'claude-code', providers }); + + it('inserts claude-sonnet-5 after the retired tier on a CURATED list 153 skipped', async () => { + seed({ + 'claude-code': { + models: ['claude-haiku-4-5', 'claude-sonnet-4-6', 'claude-opus-5', 'claude-fable-5'], + defaultModel: 'claude-opus-5', + mediumModel: 'claude-sonnet-4-6', + }, + }); + + const result = await migration.up({ rootDir }); + + expect(result).toMatchObject({ ok: true, reason: 'updated', updated: 1 }); + const after = readJson(providersPath).providers['claude-code']; + expect(after.models).toEqual([ + 'claude-haiku-4-5', 'claude-sonnet-4-6', 'claude-sonnet-5', 'claude-opus-5', 'claude-fable-5', + ]); + // Additive: the retired id and every tier pointer survive untouched. + expect(after.mediumModel).toBe('claude-sonnet-4-6'); + expect(after.defaultModel).toBe('claude-opus-5'); + }); + + it('uses each Bedrock record\'s own region-qualified sonnet spelling', async () => { + seed({ + 'claude-code-bedrock': { + models: ['us.anthropic.claude-haiku-4-5', 'us.anthropic.claude-sonnet-4-6', 'global.anthropic.claude-opus-5'], + }, + 'claude-code-tui-bedrock': { + models: ['us.anthropic.claude-sonnet-4-6'], + }, + }); + + const result = await migration.up({ rootDir }); + + expect(result.updated).toBe(2); + const { providers } = readJson(providersPath); + expect(providers['claude-code-bedrock'].models).toEqual([ + 'us.anthropic.claude-haiku-4-5', + 'us.anthropic.claude-sonnet-4-6', + 'us.anthropic.claude-sonnet-5', + 'global.anthropic.claude-opus-5', + ]); + expect(providers['claude-code-tui-bedrock'].models).toEqual([ + 'us.anthropic.claude-sonnet-4-6', + 'us.anthropic.claude-sonnet-5', + ]); + // The bare id must never leak into a Bedrock record — its environment + // resolves only the region-qualified form. + expect(providers['claude-code-bedrock'].models).not.toContain('claude-sonnet-5'); + }); + + it('is a no-op on an already-current record and on a second run', async () => { + seed({ + 'claude-code': { models: ['claude-haiku-4-5', 'claude-sonnet-5', 'claude-opus-5'] }, + 'claude-code-tui': { models: ['claude-haiku-4-5', 'claude-sonnet-4-6', 'claude-sonnet-5'] }, + }); + + const first = await migration.up({ rootDir }); + expect(first).toMatchObject({ ok: true, reason: 'already-current', updated: 0 }); + + // And a record it DID repair stays repaired rather than gaining a duplicate. + seed({ 'claude-code': { models: ['claude-sonnet-4-6'] } }); + expect((await migration.up({ rootDir })).updated).toBe(1); + const second = await migration.up({ rootDir }); + expect(second.updated).toBe(0); + expect(readJson(providersPath).providers['claude-code'].models) + .toEqual(['claude-sonnet-4-6', 'claude-sonnet-5']); + }); + + it('leaves records outside the four seeded Claude ids alone', async () => { + seed({ + 'claude-ollama': { models: ['claude-sonnet-4-6'] }, + 'antigravity-cli': { models: ['claude-sonnet-4-6'] }, + }); + + expect((await migration.up({ rootDir })).updated).toBe(0); + const { providers } = readJson(providersPath); + expect(providers['claude-ollama'].models).toEqual(['claude-sonnet-4-6']); + expect(providers['antigravity-cli'].models).toEqual(['claude-sonnet-4-6']); + }); + + it('skips a missing or malformed providers file without throwing', async () => { + expect(await migration.up({ rootDir })).toMatchObject({ ok: false, reason: 'no-file' }); + + writeFileSync(providersPath, '{not json'); + expect(await migration.up({ rootDir })).toMatchObject({ ok: false, reason: 'unreadable' }); + + writeJson(providersPath, { activeProvider: 'claude-code' }); + expect(await migration.up({ rootDir })).toMatchObject({ ok: false, reason: 'bad-shape' }); + }); + + it('skips a record whose models field is not an array', async () => { + seed({ 'claude-code': { models: 'claude-sonnet-4-6', defaultModel: 'claude-sonnet-4-6' } }); + + expect((await migration.up({ rootDir })).updated).toBe(0); + expect(readJson(providersPath).providers['claude-code'].models).toBe('claude-sonnet-4-6'); + }); +}); From ec456631bc5bd51e30d2ec62d357d6d464973771 Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:12:56 +0000 Subject: [PATCH 004/178] docs: mark port 5564 as assigned to slotstream in PORTS reserved ranges (#6031) --- docs/PORTS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PORTS.md b/docs/PORTS.md index 4a3d0da99a..9deca01efc 100644 --- a/docs/PORTS.md +++ b/docs/PORTS.md @@ -132,7 +132,7 @@ PortOS automatically detects ports from env vars: | Range | Purpose | |-------|---------| | 5553-5561 | PortOS core services (includes the `:5553` loopback mirror and the `portos-db` Docker container on `:5561`) | -| 5562-5569 | Reserved for PortOS extensions. Assigned: 5562 whisper, 5563 Eidoverse bridge (on demand), 5568 llama-server. Unassigned but still reserved: 5564-5567, 5569 | +| 5562-5569 | Reserved for PortOS extensions. Assigned: 5562 whisper, 5563 Eidoverse bridge (on demand), 5564 slotstream (on demand), 5568 llama-server. Unassigned but still reserved: 5565-5567, 5569 | | 5570-5599 | User applications — **put managed apps here** | > **A collision inside `5553-5569` is silent, not loud.** The natural assumption is From f4dc1917e18154fd9e3bf307261df28859ec8f8c Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:13:42 +0000 Subject: [PATCH 005/178] docs: include required providerId in Execute AI Run curl example (#6030) --- docs/API.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/API.md b/docs/API.md index 119123f06d..0a304b8ac8 100644 --- a/docs/API.md +++ b/docs/API.md @@ -869,6 +869,7 @@ curl -X POST http://localhost:5555/api/apps \ curl -X POST http://localhost:5555/api/runs \ -H "Content-Type: application/json" \ -d '{ + "providerId": "claude-code", "prompt": "List all files in the current directory", "workspacePath": "/path/to/workspace" }' From c6d579ec4ca2e941cbd7683f48de4d068b740da2 Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:14:17 +0000 Subject: [PATCH 006/178] fix: contain throwing onRunStarted hook in emitRunStarted so TUI PTY runs can't orphan (#6002) --- server/services/runner.js | 15 ++++++++++----- server/services/runner.test.js | 19 +++++++++++++++++++ 2 files changed, 29 insertions(+), 5 deletions(-) diff --git a/server/services/runner.js b/server/services/runner.js index 4cd3eb42ba..285f4abed6 100644 --- a/server/services/runner.js +++ b/server/services/runner.js @@ -244,11 +244,16 @@ export async function resolveRunCwd({ runId, workspacePath, label, startTime = D * run tracking sees TUI runs as active. */ export function emitRunStarted({ runId, provider, model }) { - runnerConfig.hooks?.onRunStarted?.({ - runId, - provider: provider?.name || provider?.id, - model: model ?? provider?.defaultModel, - }); + // Fire-and-forget lifecycle notification — a throwing hook must not take + // down the already-registered PTY run (same orphaned-run shape as #5792). + safeSettle( + () => runnerConfig.hooks?.onRunStarted?.({ + runId, + provider: provider?.name || provider?.id, + model: model ?? provider?.defaultModel, + }), + `Run ${runId} onRunStarted hook`, + ); } /** diff --git a/server/services/runner.test.js b/server/services/runner.test.js index e5fc76ea33..426fb5e1d8 100644 --- a/server/services/runner.test.js +++ b/server/services/runner.test.js @@ -1205,6 +1205,25 @@ describe('emitRunStarted — payload-flattening contract', () => { model: 'gpt-4', })).not.toThrow(); }); + + it('contains a throwing onRunStarted hook instead of rejecting the TUI spawn path (#6002)', () => { + // emitRunStarted fires after the PTY is already registered in the + // active-run map (tuiPromptRunner.js), so a throw here would reject the + // spawn with a live PTY and no terminal settlement — the same orphaned + // shape #5792 closed for the CLI path. + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + setAIToolkit(fakeToolkit(), { + dataDir: '/tmp/test-runner', + hooks: { onRunStarted: () => { throw new Error('tui started hook boom'); } }, + }); + expect(() => emitRunStarted({ + runId: 'r-tui-throw', + provider: { name: 'codex', defaultModel: 'gpt-5' }, + model: 'gpt-4o', + })).not.toThrow(); + expect(errorSpy).toHaveBeenCalledWith(expect.stringContaining('onRunStarted hook threw during recovery')); + errorSpy.mockRestore(); + }); }); describe('executeCliRun — workspace validation (#3180)', () => { From 6b977ff405d89a83380aefe727b38e27de5357eb Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:15:08 +0000 Subject: [PATCH 007/178] fix: guard overlapping ImageGen picks with per-target sequence token (#5944) --- client/src/pages/ImageGen.jsx | 17 ++++++ client/src/pages/ImageGen.objectUrls.test.jsx | 57 ++++++++++++++++++- 2 files changed, 72 insertions(+), 2 deletions(-) diff --git a/client/src/pages/ImageGen.jsx b/client/src/pages/ImageGen.jsx index cb369c9fd3..2d31896522 100644 --- a/client/src/pages/ImageGen.jsx +++ b/client/src/pages/ImageGen.jsx @@ -186,6 +186,13 @@ export default function ImageGen() { // updater React skips once unmounted — an implementation detail to guard // against, not to rely on.) const mountedRef = useMounted(); + // Per-target pick sequence: two overlapping upload picks both survive the + // EXIF-normalization await, read the same stale preview ref, and each mint a + // url — only the last setState survives, orphaning the loser's url. Each + // handler bumps its slot's counter and bails when superseded, BEFORE minting, + // so a losing pick never creates a url (same token idiom as + // statusRequestToken below). + const pickSeqRef = useRef({ init: 0, refs: [] }); const [initImageStrength, setInitImageStrength] = useState(0.4); // Visual gallery picker target: null (closed), { kind: 'init' }, or // { kind: 'reference', slot: i }. The search/browse alternative to the plain @@ -550,13 +557,18 @@ export default function ImageGen() { const handlePickInitImage = async (e) => { const raw = e.target.files?.[0]; if (!raw) return; + const myPick = ++pickSeqRef.current.init; const file = await normalizeImageOrientation(raw); if (!mountedRef.current) return; + // A newer pick started while this one normalized — it owns the slot now. + // Bail before minting so this pick never creates an unreachable url. + if (myPick !== pickSeqRef.current.init) return; revokeIfBlob(initImagePreviewRef.current); setInitImage({ source: 'upload', file, name: file.name, previewUrl: URL.createObjectURL(file) }); // Default the output resolution to the uploaded image's dimensions, clamped // to the server's edge/pixel caps so a large phone photo doesn't 400 on Generate. const dims = await readImageDimensions(file); + if (myPick !== pickSeqRef.current.init) return; const clamped = dims && clampImageDimensions(dims.width, dims.height); if (clamped) { setWidth(clamped.width); setHeight(clamped.height); } }; @@ -582,8 +594,13 @@ export default function ImageGen() { const handlePickReferenceImage = async (slotIndex, e) => { const raw = e.target.files?.[0]; if (!raw) return; + const seqs = pickSeqRef.current.refs; + const myPick = seqs[slotIndex] = (seqs[slotIndex] ?? 0) + 1; const file = await normalizeImageOrientation(raw); if (!mountedRef.current) return; + // Superseded by a newer pick on this slot — bail before minting so the + // losing pick never creates an unreachable url. + if (myPick !== pickSeqRef.current.refs[slotIndex]) return; // Mint the url OUTSIDE the updater. StrictMode invokes a functional updater // twice in dev, and a url created inside it on the discarded pass is never // stored — so it can never be revoked. (The revoke stays inside, where it diff --git a/client/src/pages/ImageGen.objectUrls.test.jsx b/client/src/pages/ImageGen.objectUrls.test.jsx index b663316992..7fadc79468 100644 --- a/client/src/pages/ImageGen.objectUrls.test.jsx +++ b/client/src/pages/ImageGen.objectUrls.test.jsx @@ -7,7 +7,7 @@ import { MemoryRouter } from 'react-router'; // init image plus all four reference slots without needing a FLUX.2 install. const MODEL = { id: 'dev', name: 'FLUX.1 Dev', runner: 'mflux', steps: 20, guidance: 3.5 }; -const state = vi.hoisted(() => ({ created: [], revoked: [], fileSeq: 0 })); +const state = vi.hoisted(() => ({ created: [], createdFiles: [], revoked: [], fileSeq: 0 })); const nextFile = () => new File(['x'], `photo-${++state.fileSeq}.jpg`, { type: 'image/jpeg' }); @@ -139,12 +139,14 @@ describe('ImageGen object-URL lifecycle', () => { beforeEach(() => { state.created = []; + state.createdFiles = []; state.revoked = []; state.fileSeq = 0; restore = [ - stub(URL, 'createObjectURL', vi.fn(() => { + stub(URL, 'createObjectURL', vi.fn((file) => { const url = `blob:portos/${state.created.length + 1}`; state.created.push(url); + state.createdFiles.push(file?.name ?? ''); return url; })), stub(URL, 'revokeObjectURL', vi.fn((url) => { state.revoked.push(url); })), @@ -283,4 +285,55 @@ describe('ImageGen object-URL lifecycle', () => { expect(state.revoked).toEqual([blobUrl]); }); + + // Two picks that overlap in the EXIF-normalization await must not each mint + // a url: only the last setState survives, so the loser's url would be + // unreachable from state, the clear path, and the unmount sweep. + it('creates exactly one url when two init picks overlap, keeping the last pick', async () => { + // Defer only the normalization calls (two-arg form); the dims probe + // (one-arg form) keeps its immediate fallback so it can't hold a handler. + const pending = []; + window.createImageBitmap = vi.fn((...args) => (args.length === 2 + ? new Promise((_, reject) => { pending.push(() => reject(new Error('no decoder'))); }) + : Promise.reject(new Error('no decoder')))); + + await mount(); + await click('pick-init'); + await click('pick-init'); + expect(pending).toHaveLength(2); + expect(state.created).toEqual([]); + + await act(async () => { + pending.forEach((fail) => fail()); + await new Promise((r) => setTimeout(r, 0)); + }); + + await waitFor(() => expect(state.created).toHaveLength(1)); + expect(state.createdFiles).toEqual(['photo-2.jpg']); + expect(liveUrls()).toHaveLength(1); + expect(screen.getByTestId('init-url')).toHaveTextContent(state.created[0]); + }); + + it('creates exactly one url when two picks overlap on one reference slot, keeping the last pick', async () => { + const pending = []; + window.createImageBitmap = vi.fn(() => new Promise((_, reject) => { + pending.push(() => reject(new Error('no decoder'))); + })); + + await mount(); + await click('pick-ref-0'); + await click('pick-ref-0'); + expect(pending).toHaveLength(2); + expect(state.created).toEqual([]); + + await act(async () => { + pending.forEach((fail) => fail()); + await new Promise((r) => setTimeout(r, 0)); + }); + + await waitFor(() => expect(state.created).toHaveLength(1)); + expect(state.createdFiles).toEqual(['photo-2.jpg']); + expect(liveUrls()).toHaveLength(1); + expect(screen.getByTestId('ref-url-0')).toHaveTextContent(state.created[0]); + }); }); From a203ef924736c28e0878cdf97127e5c8e0747f8c Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:16:02 +0000 Subject: [PATCH 008/178] refactor: break taskLearning metrics<->routing static import cycle (#5916) Move resetTaskTypeLearning, removeTaskTypeFromLearningData, and purgeEnvironmentalFailuresForType into a new store-only leaf (taskLearning/reset.js). routing.js imports the leaf instead of metrics.js, so the two-module ESM cycle is gone; metrics.js and the barrel re-export the moved symbols for back-compat. Drops the #5916 entry from KNOWN_CYCLIC_COMPONENTS. --- server/services/serviceImportCycles.test.js | 2 - server/services/taskLearning/index.js | 10 +- server/services/taskLearning/metrics.js | 191 +---------------- server/services/taskLearning/reset.js | 203 +++++++++++++++++++ server/services/taskLearning/routing.js | 2 +- server/services/taskLearning/routing.test.js | 4 +- 6 files changed, 222 insertions(+), 190 deletions(-) create mode 100644 server/services/taskLearning/reset.js diff --git a/server/services/serviceImportCycles.test.js b/server/services/serviceImportCycles.test.js index 75583fca0e..301f36bd5d 100644 --- a/server/services/serviceImportCycles.test.js +++ b/server/services/serviceImportCycles.test.js @@ -83,8 +83,6 @@ const KNOWN_CYCLIC_COMPONENTS = [ { issue: 5918, members: ['pipeline/manuscriptFix.js', 'pipeline/manuscriptReview.js'] }, // #5919 — the receive path reaches subscription state back through the barrel. { issue: 5919, members: ['sharing/peerSync.js', 'sharing/peerSyncReceive.js'] }, - // #5916 — routing imports the metrics reset; metrics imports routing's predicates. - { issue: 5916, members: ['taskLearning/metrics.js', 'taskLearning/routing.js'] }, ]; // A component as one comparable string. Members arrive sorted from diff --git a/server/services/taskLearning/index.js b/server/services/taskLearning/index.js index febec8eb6f..4dbc204096 100644 --- a/server/services/taskLearning/index.js +++ b/server/services/taskLearning/index.js @@ -8,6 +8,8 @@ * The implementation is split by concern: * - store.js — shared persistence, cache, mutex, pure helpers * - metrics.js — recording completions + rebuilding aggregates + * - reset.js — destructive per-task-type reset leaf (imported by + * metrics + routing; owns no cycle) * - routing.js — heuristic routing, cooldown, skip, confidence * - safetyKind.js — outward-facing/irreversible safety-kind classifier * - correlationQuality.js — enriched-signal ↔ outcome correlation window @@ -48,14 +50,18 @@ export { computeLatencySplit, recordFailureSignature, recordEnvironmentalFailure, - purgeEnvironmentalFailuresForType, ENVIRONMENTAL_ERROR_CATEGORIES, - resetTaskTypeLearning, recalculateModelTierMetrics, recalculateDurationStats, getWindowedStats } from './metrics.js'; +export { + purgeEnvironmentalFailuresForType, + resetTaskTypeLearning, + removeTaskTypeFromLearningData +} from './reset.js'; + export { getTaskTypePriorityMultiplier, suggestModelTier, diff --git a/server/services/taskLearning/metrics.js b/server/services/taskLearning/metrics.js index 050e5b7cd2..2ffdb16ab1 100644 --- a/server/services/taskLearning/metrics.js +++ b/server/services/taskLearning/metrics.js @@ -92,27 +92,10 @@ export function shouldDivertToEnvironmental(outcomeSuccess, category, origin) { return origin == null || DIVERTIBLE_ORIGINS.has(origin); } -/** - * Remove a task type's contribution from every environmental bucket (issue - * #2618 reset parity): decrement each bucket's count by the type's share, - * delete the per-type entry, and drop a bucket left with nothing — so a reset - * type's old outages stop appearing in insights and error-share denominators. - * Pure — mutates `data`. Returns the number of events removed. - */ -export function purgeEnvironmentalFailuresForType(data, taskType) { - let removed = 0; - for (const [category, bucket] of Object.entries(data.environmentalFailures || {})) { - const typeCount = bucket.taskTypes?.[taskType] || 0; - if (typeCount === 0) continue; - removed += typeCount; - bucket.count = Math.max(0, (Number(bucket.count) || 0) - typeCount); - delete bucket.taskTypes[taskType]; - if (bucket.count <= 0 && Object.keys(bucket.taskTypes || {}).length === 0) { - delete data.environmentalFailures[category]; - } - } - return removed; -} +// `purgeEnvironmentalFailuresForType` lives in the reset leaf (`reset.js`, issue +// #5916) so routing.js can reach the reset path without closing a +// metrics⇄routing static import cycle; re-exported here for back-compat. +export { purgeEnvironmentalFailuresForType } from './reset.js'; /** * Milliseconds between two ISO timestamps. Pure. Returns null (not 0) when @@ -733,168 +716,10 @@ export async function getWindowedStats(taskType, { return computeWindowedStats(metrics?.recentOutcomes, { maxCount, maxAgeMs }); } -/** - * Reset learning data for a specific task type - * Used when a previously-failing task type has been fixed and should be retried - * Subtracts the task type's metrics from totals and removes the task type entry - * @param {string} taskType - The task type to reset (e.g., 'self-improve:ui') - * @returns {Object} Summary of what was reset - */ -export async function resetTaskTypeLearning(taskType) { - return withLock(async () => { - const data = await loadLearningData(); - - // Purge this type from the environmental buckets FIRST (#2618): an - // outage-only type has no byTaskType bucket, so the purge must not sit - // behind the task-type-not-found early return below. - // - // Deliberately part of the RESET path, not of removeTaskTypeFromLearningData: - // this function is "the user says this type is fixed — forget all of it", so - // dropping its outage history is intended. A caller repairing mis-recorded - // BUCKET data (e.g. migration 197) must not purge outages, which are recorded - // from real errors and are true regardless of any bucket-level bug. - const environmentalRemoved = purgeEnvironmentalFailuresForType(data, taskType); - - const metrics = data.byTaskType[taskType]; - if (!metrics) { - if (environmentalRemoved > 0) { - await saveLearningData(data); - emitLog('info', `Reset environmental-only learning data for ${taskType} (${environmentalRemoved} outage events purged)`, { taskType, environmentalRemoved }, '📚 TaskLearning'); - return { reset: true, reason: 'environmental-only', taskType, environmentalRemoved }; - } - return { reset: false, reason: 'task-type-not-found', taskType }; - } - - const previousMetrics = removeTaskTypeFromLearningData(data, taskType); - - await saveLearningData(data); - - emitLog('info', `Reset learning data for ${taskType} (was ${metrics.successRate}% success after ${metrics.completed} attempts)`, { - taskType, - previousSuccessRate: metrics.successRate, - previousAttempts: metrics.completed - }, '📚 TaskLearning'); - - return { reset: true, taskType, previousMetrics }; - }); -} - -/** - * Remove one task type's contribution from every learning aggregate, in place. - * Pure (mutates `data`, no I/O) so both the runtime reset and offline repairs - * (migrations) can share ONE definition of "what a task type contributes to" — - * a second, hand-rolled version would silently drift as aggregates are added. - * - * Unwinds: `totals` (+ recomputed max/ETA), `errorPatterns`, `byModelTier` (via - * `routingAccuracy`, which must be read BEFORE it is deleted), `routingAccuracy`, - * `byTaskType`, `failureSignatures` (#2619), and `correlationWindow` (#2619). - * - * Does NOT touch `environmentalFailures` — that is a separate ledger fed only by - * real outages, so removing it is a policy decision belonging to the caller (see - * `resetTaskTypeLearning`, which purges it; migration 197, which must not). - * - * @param {Object} data - the loaded learning store, mutated in place - * @param {string} taskType - e.g. 'self-improve:layered-intelligence' - * @returns {{ completed:number, succeeded:number, failed:number, successRate:number }|null} - * the removed bucket's headline metrics, or null when the type had no bucket. - */ -export function removeTaskTypeFromLearningData(data, taskType) { - const metrics = data?.byTaskType?.[taskType]; - if (!metrics) return null; - - // Subtract this task type's contribution from totals. Guarded because this helper - // also runs OFFLINE against a raw on-disk store (migrations), where the defaults - // loadLearningData applies at runtime haven't been layered on — and an aggregate - // this function throws on would block boot rather than repair anything. - if (data.totals && typeof data.totals === 'object') { - data.totals.completed -= metrics.completed; - data.totals.succeeded -= metrics.succeeded; - data.totals.failed -= metrics.failed; - data.totals.totalDurationMs -= metrics.totalDurationMs; - if (data.totals.successDurationMs) { - data.totals.successDurationMs = Math.max(0, data.totals.successDurationMs - (metrics.successDurationMs || 0)); - } - // Recalculate max from remaining task types (we can't subtract a max) - const remainingTypes = Object.entries(data.byTaskType).filter(([t]) => t !== taskType); - data.totals.successMaxDurationMs = remainingTypes.reduce((max, [, m]) => Math.max(max, m.successMaxDurationMs || 0), 0); - Object.assign(data.totals, calculateDurationETA(data.totals)); - } - - // Clean up error patterns referencing this task type - for (const [category, pattern] of Object.entries(data.errorPatterns || {})) { - const taskTypeCount = pattern?.taskTypes?.[taskType] || 0; - if (taskTypeCount > 0) { - pattern.count -= taskTypeCount; - delete pattern.taskTypes[taskType]; - } - // Remove empty error categories - if (pattern.count <= 0) { - delete data.errorPatterns[category]; - } - } - - // Subtract model tier contributions using routing accuracy data (before deleting it) - data.byModelTier ??= {}; - const routingData = data.routingAccuracy?.[taskType]; - if (routingData) { - for (const [tier, counts] of Object.entries(routingData)) { - const tierMetrics = data.byModelTier[tier]; - if (tierMetrics) { - const tierTotal = counts.succeeded + counts.failed; - tierMetrics.completed = Math.max(0, tierMetrics.completed - tierTotal); - tierMetrics.succeeded = Math.max(0, tierMetrics.succeeded - counts.succeeded); - tierMetrics.failed = Math.max(0, tierMetrics.failed - counts.failed); - // Estimate duration contribution using task type's avg duration per agent - if (tierTotal > 0 && metrics.avgDurationMs > 0) { - tierMetrics.totalDurationMs = Math.max(0, tierMetrics.totalDurationMs - (metrics.avgDurationMs * tierTotal)); - } - tierMetrics.avgDurationMs = tierMetrics.completed > 0 - ? Math.round(tierMetrics.totalDurationMs / tierMetrics.completed) - : 0; - // Clean up empty tiers - if (tierMetrics.completed <= 0) { - delete data.byModelTier[tier]; - } - } - } - delete data.routingAccuracy[taskType]; - } - - // Remove the task type entry - delete data.byTaskType[taskType]; - - // Purge the task type's samples from the enriched failure signatures (#2619). - // Each `recent[]` sample carries its `taskType` and drives - // deriveFailureSignalAvoidance, so a leftover sample would keep steering a - // just-reset type off a tier. Decrement each bucket's rolling count by the - // samples removed and drop a bucket left with nothing. - if (data.failureSignatures && typeof data.failureSignatures === 'object') { - for (const [category, bucket] of Object.entries(data.failureSignatures)) { - if (!Array.isArray(bucket?.recent)) continue; - const kept = bucket.recent.filter((s) => s?.taskType !== taskType); - const removed = bucket.recent.length - kept.length; - if (removed === 0) continue; - bucket.recent = kept; - bucket.count = Math.max(0, (Number(bucket.count) || 0) - removed); - if (bucket.recent.length === 0 && bucket.count <= 0) { - delete data.failureSignatures[category]; - } - } - } - - // Drop the task type's rows from the cross-type correlation window (#2619) so a - // rehabilitated type contributes no stale prediction/outcome pairs to the gauge. - if (Array.isArray(data.correlationWindow)) { - data.correlationWindow = data.correlationWindow.filter((row) => row?.taskType !== taskType); - } - - return { - completed: metrics.completed, - succeeded: metrics.succeeded, - failed: metrics.failed, - successRate: metrics.successRate - }; -} +// `resetTaskTypeLearning` + `removeTaskTypeFromLearningData` live in the reset +// leaf (`reset.js`, issue #5916); re-exported here so existing direct +// importers (`./metrics.js`, migrations) keep working. +export { resetTaskTypeLearning, removeTaskTypeFromLearningData } from './reset.js'; /** * Recalculate byModelTier from routingAccuracy data. diff --git a/server/services/taskLearning/reset.js b/server/services/taskLearning/reset.js new file mode 100644 index 0000000000..56a40b63e1 --- /dev/null +++ b/server/services/taskLearning/reset.js @@ -0,0 +1,203 @@ +/** + * Task Learning — reset leaf + * + * Owns the destructive per-task-type reset path (`resetTaskTypeLearning` plus + * the pure unwind helpers it composes). Split out of `metrics.js` (issue + * #5916) so `routing.js` — which calls the reset from its rehabilitation path — + * can import it without closing a metrics⇄routing static import cycle. This + * module imports only from the shared persistence leaf (`store.js`), so it + * sits at the bottom of the taskLearning dependency graph alongside it. + */ + +import { + withLock, + calculateDurationETA, + emitLog, + loadLearningData, + saveLearningData +} from './store.js'; + +/** + * Remove a task type's contribution from every environmental bucket (issue + * #2618 reset parity): decrement each bucket's count by the type's share, + * delete the per-type entry, and drop a bucket left with nothing — so a reset + * type's old outages stop appearing in insights and error-share denominators. + * Pure — mutates `data`. Returns the number of events removed. + */ +export function purgeEnvironmentalFailuresForType(data, taskType) { + let removed = 0; + for (const [category, bucket] of Object.entries(data.environmentalFailures || {})) { + const typeCount = bucket.taskTypes?.[taskType] || 0; + if (typeCount === 0) continue; + removed += typeCount; + bucket.count = Math.max(0, (Number(bucket.count) || 0) - typeCount); + delete bucket.taskTypes[taskType]; + if (bucket.count <= 0 && Object.keys(bucket.taskTypes || {}).length === 0) { + delete data.environmentalFailures[category]; + } + } + return removed; +} + +/** + * Reset learning data for a specific task type + * Used when a previously-failing task type has been fixed and should be retried + * Subtracts the task type's metrics from totals and removes the task type entry + * @param {string} taskType - The task type to reset (e.g., 'self-improve:ui') + * @returns {Object} Summary of what was reset + */ +export async function resetTaskTypeLearning(taskType) { + return withLock(async () => { + const data = await loadLearningData(); + + // Purge this type from the environmental buckets FIRST (#2618): an + // outage-only type has no byTaskType bucket, so the purge must not sit + // behind the task-type-not-found early return below. + // + // Deliberately part of the RESET path, not of removeTaskTypeFromLearningData: + // this function is "the user says this type is fixed — forget all of it", so + // dropping its outage history is intended. A caller repairing mis-recorded + // BUCKET data (e.g. migration 197) must not purge outages, which are recorded + // from real errors and are true regardless of any bucket-level bug. + const environmentalRemoved = purgeEnvironmentalFailuresForType(data, taskType); + + const metrics = data.byTaskType[taskType]; + if (!metrics) { + if (environmentalRemoved > 0) { + await saveLearningData(data); + emitLog('info', `Reset environmental-only learning data for ${taskType} (${environmentalRemoved} outage events purged)`, { taskType, environmentalRemoved }, '📚 TaskLearning'); + return { reset: true, reason: 'environmental-only', taskType, environmentalRemoved }; + } + return { reset: false, reason: 'task-type-not-found', taskType }; + } + + const previousMetrics = removeTaskTypeFromLearningData(data, taskType); + + await saveLearningData(data); + + emitLog('info', `Reset learning data for ${taskType} (was ${metrics.successRate}% success after ${metrics.completed} attempts)`, { + taskType, + previousSuccessRate: metrics.successRate, + previousAttempts: metrics.completed + }, '📚 TaskLearning'); + + return { reset: true, taskType, previousMetrics }; + }); +} + +/** + * Remove one task type's contribution from every learning aggregate, in place. + * Pure (mutates `data`, no I/O) so both the runtime reset and offline repairs + * (migrations) can share ONE definition of "what a task type contributes to" — + * a second, hand-rolled version would silently drift as aggregates are added. + * + * Unwinds: `totals` (+ recomputed max/ETA), `errorPatterns`, `byModelTier` (via + * `routingAccuracy`, which must be read BEFORE it is deleted), `routingAccuracy`, + * `byTaskType`, `failureSignatures` (#2619), and `correlationWindow` (#2619). + * + * Does NOT touch `environmentalFailures` — that is a separate ledger fed only by + * real outages, so removing it is a policy decision belonging to the caller (see + * `resetTaskTypeLearning`, which purges it; migration 197, which must not). + * + * @param {Object} data - the loaded learning store, mutated in place + * @param {string} taskType - e.g. 'self-improve:layered-intelligence' + * @returns {{ completed:number, succeeded:number, failed:number, successRate:number }|null} + * the removed bucket's headline metrics, or null when the type had no bucket. + */ +export function removeTaskTypeFromLearningData(data, taskType) { + const metrics = data?.byTaskType?.[taskType]; + if (!metrics) return null; + + // Subtract this task type's contribution from totals. Guarded because this helper + // also runs OFFLINE against a raw on-disk store (migrations), where the defaults + // loadLearningData applies at runtime haven't been layered on — and an aggregate + // this function throws on would block boot rather than repair anything. + if (data.totals && typeof data.totals === 'object') { + data.totals.completed -= metrics.completed; + data.totals.succeeded -= metrics.succeeded; + data.totals.failed -= metrics.failed; + data.totals.totalDurationMs -= metrics.totalDurationMs; + if (data.totals.successDurationMs) { + data.totals.successDurationMs = Math.max(0, data.totals.successDurationMs - (metrics.successDurationMs || 0)); + } + // Recalculate max from remaining task types (we can't subtract a max) + const remainingTypes = Object.entries(data.byTaskType).filter(([t]) => t !== taskType); + data.totals.successMaxDurationMs = remainingTypes.reduce((max, [, m]) => Math.max(max, m.successMaxDurationMs || 0), 0); + Object.assign(data.totals, calculateDurationETA(data.totals)); + } + + // Clean up error patterns referencing this task type + for (const [category, pattern] of Object.entries(data.errorPatterns || {})) { + const taskTypeCount = pattern?.taskTypes?.[taskType] || 0; + if (taskTypeCount > 0) { + pattern.count -= taskTypeCount; + delete pattern.taskTypes[taskType]; + } + // Remove empty error categories + if (pattern.count <= 0) { + delete data.errorPatterns[category]; + } + } + + // Subtract model tier contributions using routing accuracy data (before deleting it) + data.byModelTier ??= {}; + const routingData = data.routingAccuracy?.[taskType]; + if (routingData) { + for (const [tier, counts] of Object.entries(routingData)) { + const tierMetrics = data.byModelTier[tier]; + if (tierMetrics) { + const tierTotal = counts.succeeded + counts.failed; + tierMetrics.completed = Math.max(0, tierMetrics.completed - tierTotal); + tierMetrics.succeeded = Math.max(0, tierMetrics.succeeded - counts.succeeded); + tierMetrics.failed = Math.max(0, tierMetrics.failed - counts.failed); + // Estimate duration contribution using task type's avg duration per agent + if (tierTotal > 0 && metrics.avgDurationMs > 0) { + tierMetrics.totalDurationMs = Math.max(0, tierMetrics.totalDurationMs - (metrics.avgDurationMs * tierTotal)); + } + tierMetrics.avgDurationMs = tierMetrics.completed > 0 + ? Math.round(tierMetrics.totalDurationMs / tierMetrics.completed) + : 0; + // Clean up empty tiers + if (tierMetrics.completed <= 0) { + delete data.byModelTier[tier]; + } + } + } + delete data.routingAccuracy[taskType]; + } + + // Remove the task type entry + delete data.byTaskType[taskType]; + + // Purge the task type's samples from the enriched failure signatures (#2619). + // Each `recent[]` sample carries its `taskType` and drives + // deriveFailureSignalAvoidance, so a leftover sample would keep steering a + // just-reset type off a tier. Decrement each bucket's rolling count by the + // samples removed and drop a bucket left with nothing. + if (data.failureSignatures && typeof data.failureSignatures === 'object') { + for (const [category, bucket] of Object.entries(data.failureSignatures)) { + if (!Array.isArray(bucket?.recent)) continue; + const kept = bucket.recent.filter((s) => s?.taskType !== taskType); + const removed = bucket.recent.length - kept.length; + if (removed === 0) continue; + bucket.recent = kept; + bucket.count = Math.max(0, (Number(bucket.count) || 0) - removed); + if (bucket.recent.length === 0 && bucket.count <= 0) { + delete data.failureSignatures[category]; + } + } + } + + // Drop the task type's rows from the cross-type correlation window (#2619) so a + // rehabilitated type contributes no stale prediction/outcome pairs to the gauge. + if (Array.isArray(data.correlationWindow)) { + data.correlationWindow = data.correlationWindow.filter((row) => row?.taskType !== taskType); + } + + return { + completed: metrics.completed, + succeeded: metrics.succeeded, + failed: metrics.failed, + successRate: metrics.successRate + }; +} diff --git a/server/services/taskLearning/routing.js b/server/services/taskLearning/routing.js index 703bf58b19..4c39773cec 100644 --- a/server/services/taskLearning/routing.js +++ b/server/services/taskLearning/routing.js @@ -10,7 +10,7 @@ */ import { loadLearningData, emitLog, isSandboxedTaskType, computeEffectiveSuccessRate, computeWindowedStats, isSkipCandidate, DEFAULT_WINDOW_MAX_AGE_MS, ENVIRONMENTAL_ERROR_CATEGORIES } from './store.js'; -import { resetTaskTypeLearning } from './metrics.js'; +import { resetTaskTypeLearning } from './reset.js'; import { computeCorrelationQuality, isCorrelationProven } from './correlationQuality.js'; /** diff --git a/server/services/taskLearning/routing.test.js b/server/services/taskLearning/routing.test.js index 21ab087705..f18bb41569 100644 --- a/server/services/taskLearning/routing.test.js +++ b/server/services/taskLearning/routing.test.js @@ -11,7 +11,7 @@ import { getPerformanceSummary } from './routing.js'; import { loadLearningData } from './store.js'; -import { resetTaskTypeLearning } from './metrics.js'; +import { resetTaskTypeLearning } from './reset.js'; // Stub ONLY the persistence + log surface of the store; every pure helper // (computeEffectiveSuccessRate, computeWindowedStats, isSandboxedTaskType, …) @@ -20,7 +20,7 @@ vi.mock('./store.js', async (importOriginal) => { const actual = await importOriginal(); return { ...actual, loadLearningData: vi.fn(), emitLog: vi.fn() }; }); -vi.mock('./metrics.js', () => ({ resetTaskTypeLearning: vi.fn() })); +vi.mock('./reset.js', () => ({ resetTaskTypeLearning: vi.fn() })); // deriveFailureSignalAvoidance is the pure "routing consumes the enriched // failure signatures" core added for issue #2329. It takes learning data + a From cbe5cb45b90ca5238f0d1ac5cad9f9f7c5de1800 Mon Sep 17 00:00:00 2001 From: Adam Eivy Date: Thu, 3 Sep 2026 06:47:36 +0000 Subject: [PATCH 009/178] fix: paint the Video Gen model picker without waiting on the python probe (#5835) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Opening Video Gen in a new tab left the Model field on a "Loading models…" placeholder for a second or two before the picker appeared. The list rode along on GET /api/video-gen/status, which shells out to python on every call to check package imports — so the form waited on an interpreter probe it did not need in order to render a dropdown. The model list and the three numbers its auto-select reads (defaultModel, systemMemoryGb, fflfLtx2PixelBudget) now come from a new probe-free GET /api/video-gen/model-context, fetched alongside /status on mount. Both routes build that payload from one shared helper, so they cannot drift, and /status keeps returning the fields for its existing readers. /status is now responsible for connectivity alone, and the form hook takes a `modelContext` argument instead of `status` to make that split structural. With the list no longer behind the probe, the session cache that used to paint a stale picker on a revisit has no work left to do and is deleted — along with its `stale` flag and the freshness gate the connectivity UI needed to avoid reporting python health from a stored answer. Claude-Session: https://claude.ai/code/session_012krsCGMnjFmoeY9uyi4Ypf --- client/src/hooks/README.md | 2 +- client/src/hooks/useVideoGenForm.js | 57 +++++++------- client/src/hooks/useVideoGenForm.test.jsx | 34 ++++----- client/src/lib/README.md | 1 - client/src/lib/index.js | 1 - client/src/lib/videoGenStatusCache.js | 43 ----------- .../pages/VideoGen.composeWhileBusy.test.jsx | 2 + .../pages/VideoGen.federatedTarget.test.jsx | 2 + client/src/pages/VideoGen.jsx | 74 +++++++++++-------- .../src/pages/VideoGen.modelLoading.test.jsx | 68 ++++++++--------- client/src/pages/VideoGen.terms.test.jsx | 10 ++- .../VideoGen.textEncoderAutoDownload.test.jsx | 2 + client/src/services/apiImageVideo.js | 7 ++ client/src/test/videoGenPageMocks.jsx | 21 +++++- server/lib/apiRouteCatalog.generated.json | 12 ++- server/routes/videoGen.js | 56 ++++++++++---- server/routes/videoGen.test.js | 29 ++++++++ 17 files changed, 240 insertions(+), 181 deletions(-) delete mode 100644 client/src/lib/videoGenStatusCache.js diff --git a/client/src/hooks/README.md b/client/src/hooks/README.md index 4e1b0cc107..284fe9240d 100644 --- a/client/src/hooks/README.md +++ b/client/src/hooks/README.md @@ -206,7 +206,7 @@ grep -i "what you want to do" client/src/hooks/README.md | `useYoutubeIngest` | One YouTube brain-ingest job slot: start/cancel + SSE progress + terminal-frame handling via `POST /api/brain/youtube/ingest`. Returns `{ active, percent, stage, start(body), cancel }` — `start` takes the whole payload (`{ url, captureTranscript, downloadVideo, ingestAudio, note, agentPrompt, tags }`), not a bare URL. `onComplete(ingest)` fires with the stored ingest record; non-fatal `warnings[]` on the terminal frame are toasted automatically. | Quick Capture's YouTube path, and any other surface that ingests a video into the brain. | | `useYoutubeTrackImport` | One YouTube-audio-import job slot (#1945): start/cancel + SSE progress + terminal-frame handling via `POST /api/tracks/import/youtube`. Returns `{ active, percent, start(url, context), cancel }`; `onComplete(track, context)` fires with the finished Track. Call once per UI surface that can independently kick off an import — a shared slot would let one surface's kickoff orphan another's in-flight job. | Any picker that lets a user paste a YouTube URL to create a Track (Music Video's create form and track-change row). | | `useVideoGenFieldState` | Mutable VideoGen field values, setters, and lifecycle refs initialized from URL handoff values. | Internal state slice composed by `useVideoGenForm`; keep model reconciliation and submission behavior in their focused units. | -| `useVideoGenForm` | `useVideoGenForm({ models, status, availableLoras, grokEnabled })` → every VideoGen form field + setter, the URL-param prefill paths (ImageGen handoff, Continue, Remix, `?lora=`), the mode/backend transitions that clear stale inputs, the derived model/keyframe/IC gates (`extendModeBlocked`, `a2vModeBlocked`, `icLoraModeBlocked`, `keyframesError`), `applyRemix` / `applyResumedParams`, and `buildGeneratePayload()`. | The /media/video form — the single client-side source of truth for the payload `server/routes/videoGen.js` validates. Reuse it for any second entry point that generates video. | +| `useVideoGenForm` | `useVideoGenForm({ models, modelContext, availableLoras, grokEnabled })` → every VideoGen form field + setter, the URL-param prefill paths (ImageGen handoff, Continue, Remix, `?lora=`), the mode/backend transitions that clear stale inputs, the derived model/keyframe/IC gates (`extendModeBlocked`, `a2vModeBlocked`, `icLoraModeBlocked`, `keyframesError`), `applyRemix` / `applyResumedParams`, and `buildGeneratePayload()`. | The /media/video form — the single client-side source of truth for the payload `server/routes/videoGen.js` validates. Reuse it for any second entry point that generates video. | | `useVideoGenSubmitFlow` | Builds the current VideoGen wire payload and enveloped prompt from a form-state snapshot. | Internal submission slice composed by `useVideoGenForm`; all local, Grok, and federated request shaping remains in `videoGenSubmission`. | | `useVideoGenValidation` | Derives keyframe errors and all VideoGen submit-blocking predicates from the current fields and model capabilities. | Internal validation slice composed by `useVideoGenForm`; also exports `validateVideoKeyframes` for focused tests. | | `useVideoDownload` | One full-video-download job slot (#1946): start/cancel + SSE progress + terminal-frame handling via `POST /api/devtools/video-download`. Returns `{ active, percent, stage, context, start(url, context), cancel }`; `onComplete(video)` fires with the finished video-history entry. | The Dev Tools Video Downloader page. | diff --git a/client/src/hooks/useVideoGenForm.js b/client/src/hooks/useVideoGenForm.js index e8ff7988fd..79be036303 100644 --- a/client/src/hooks/useVideoGenForm.js +++ b/client/src/hooks/useVideoGenForm.js @@ -53,12 +53,14 @@ const editableRemixModel = (models, defaultModelId) => { * that clear now-irrelevant inputs, the derived model/keyframe/IC gates, and * `buildGeneratePayload()` — the single client-side source of truth for the * shape `server/routes/videoGen.js` validates. `VideoGen.jsx` keeps the - * fetching (status/models/history/gallery), the SSE run pipeline, the batch - * queue, and the rendering. + * fetching (status/model-context/history/gallery), the SSE run pipeline, the + * batch queue, and the rendering. * * The caller supplies the fetched context the form has to react to: - * - `models` / `status` — from `getVideoGenStatus()`; drive the model - * dropdown, the default-model seed, and the mode-compatibility fallback. + * - `models` / `modelContext` — from `getVideoGenModelContext()`; drive the + * model dropdown, the default-model seed, and the mode-compatibility + * fallback. Deliberately NOT `getVideoGenStatus()`: that route shells out + * to python, and the picker must not wait on the interpreter probe. * - `availableLoras` — the installed LoRA library, for name resolution. * - `grokEnabled` — the Settings → Image Gen toggle that reveals the * Local/Grok backend switch. @@ -69,7 +71,7 @@ const editableRemixModel = (models, defaultModelId) => { * wire accepts — kept here rather than in the page so there stays exactly * one builder for what `server/routes/videoGen.js` validates. */ -export function useVideoGenForm({ models, status, availableLoras, grokEnabled, remoteSubmissionFields = null }) { +export function useVideoGenForm({ models, modelContext, availableLoras, grokEnabled, remoteSubmissionFields = null }) { const [searchParams, setSearchParams] = useSearchParams(); const incomingSourceImage = searchParams.get('sourceImageFile'); const incomingAudioFilename = searchParams.get('audioFilename'); @@ -163,11 +165,11 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r }); return () => { cancelled = true; }; }, [incomingAudioFilename, setSearchParams]); - // Seed the model dropdown from the server's default once /status lands, - // without clobbering a Remix/deep-link/user pick that already set it. + // Seed the model dropdown from the server's default once the model context + // lands, without clobbering a Remix/deep-link/user pick that already set it. useEffect(() => { - if (status?.defaultModel) setModelId((prev) => prev || status.defaultModel); - }, [status?.defaultModel]); + if (modelContext?.defaultModel) setModelId((prev) => prev || modelContext.defaultModel); + }, [modelContext?.defaultModel]); // Re-sync when ImageGen pipes a new image via ?sourceImageFile=... useEffect(() => { @@ -370,7 +372,8 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r // server would 400 on submit; we proactively swap to a compatible model. // a2v fallback preference: highest-memory model that fits this machine // (leaving headroom for the OS + text encoder) > the largest if none fit. - // Other modes: status.defaultModel (if compatible) > first compatible model. + // Other modes: the context's defaultModel (if compatible) > first + // compatible model. useEffect(() => { if (!modelId || models.length === 0) return; const current = models.find((m) => m.id === modelId); @@ -385,13 +388,13 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r // so the user can at least try, and the install banner / OOM surfaces // the real constraint instead of a silent dropdown change. const reserveGb = 16; - // typeof === 'number' (not `status?.systemMemoryGb ? ...`) so a server - // legitimately reporting a tiny number (0 GB after rounding on a - // sub-GB box) flows through the `fits` check and lands on the - // smallest model. The truthiness shortcut would collapse 0 with - // "absent" and pick the LARGEST model on a tiny machine. - const budget = typeof status?.systemMemoryGb === 'number' - ? Math.max(0, status.systemMemoryGb - reserveGb) + // typeof === 'number' (not `modelContext?.systemMemoryGb ? ...`) so a + // server legitimately reporting a tiny number (0 GB after rounding on a + // sub-GB box) flows through the `fits` check and lands on the smallest + // model. The truthiness shortcut would collapse 0 with "absent" and pick + // the LARGEST model on a tiny machine. + const budget = typeof modelContext?.systemMemoryGb === 'number' + ? Math.max(0, modelContext.systemMemoryGb - reserveGb) : Number.POSITIVE_INFINITY; const sortedDesc = [...visibleModels].sort( (a, b) => videoModelMemoryGb(b) - videoModelMemoryGb(a), @@ -399,18 +402,18 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r const fits = sortedDesc.find((m) => videoModelMemoryGb(m) <= budget); fallback = (fits || sortedDesc[sortedDesc.length - 1])?.id || ''; } else { - const defaultModel = models.find((m) => m.id === status?.defaultModel); + const defaultModel = models.find((m) => m.id === modelContext?.defaultModel); if (defaultModel && isModelAllowedForMode(defaultModel, mode)) { fallback = defaultModel.id; } else { - fallback = visibleModels[0]?.id || status?.defaultModel || models[0]?.id || ''; + fallback = visibleModels[0]?.id || modelContext?.defaultModel || models[0]?.id || ''; } } if (!fallback || fallback === modelId) return; // Toast only for the stale-id case (model removed from catalog). The // mode-incompatibility swap is expected behavior after a mode change — // no need to surface it. Name the destination model so users on a2v - // don't think they landed on `status.defaultModel` (they may not have — + // don't think they landed on `modelContext.defaultModel` (they may not have — // a2v picks the largest-fits model, which is often a dgrauet entry). if (!current && staleModelToastRef.current !== modelId) { staleModelToastRef.current = modelId; @@ -418,7 +421,7 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r toast(`Original model "${modelId}" is no longer available — switched to "${fallbackName}"`); } applyModelSelection(fallback); - }, [modelId, models, status?.defaultModel, status?.systemMemoryGb, mode, visibleModels, applyModelSelection]); + }, [modelId, models, modelContext?.defaultModel, modelContext?.systemMemoryGb, mode, visibleModels, applyModelSelection]); const currentModel = models.find((m) => m.id === modelId); @@ -435,9 +438,9 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r setNumFrames(frames); }, [audioDurationSec, currentModel, fps, mode]); - // A source model can reach this hook either through a URL handoff before - // /status has populated `models`, or from the in-page gallery after it has. - // Resolve both cases here. The fallback is deliberately limited to models + // A source model can reach this hook either through a URL handoff before the + // model context has populated `models`, or from the in-page gallery after it + // has. Resolve both cases here. The fallback is deliberately limited to models // that can run a text remix and expose all restored prompt/sampler controls; // if no such model is installed we leave the source selected rather than // silently changing a faithful re-render. @@ -445,7 +448,7 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r if (!remixSourceModel || models.length === 0) return; const source = models.find((model) => model.id === remixSourceModel.id); if (source && !remixSourceModel.preserveConditioning && !hasEditableRemixControls(source)) { - const target = editableRemixModel(models, status?.defaultModel); + const target = editableRemixModel(models, modelContext?.defaultModel); if (target) { setModelId(target.id); setRemixModelFallback({ @@ -459,7 +462,7 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r setRemixModelFallback(null); } setRemixSourceModel(null); - }, [remixSourceModel, models, status?.defaultModel]); + }, [remixSourceModel, models, modelContext?.defaultModel]); // Until the user deliberately chooses a size, model changes carry their own // native default canvas. This is material for H3: the shared 768x512 default @@ -712,7 +715,7 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r keyframesActive, mode, numFrames, - pixelBudget: status?.fflfLtx2PixelBudget, + pixelBudget: modelContext?.fflfLtx2PixelBudget, sourceImageFile, sourceImageUpload, width, diff --git a/client/src/hooks/useVideoGenForm.test.jsx b/client/src/hooks/useVideoGenForm.test.jsx index 6723f7b3bc..6f4ae7b66f 100644 --- a/client/src/hooks/useVideoGenForm.test.jsx +++ b/client/src/hooks/useVideoGenForm.test.jsx @@ -67,13 +67,13 @@ const H3_REF2VA = { defaultFrames: 124, }; const MODELS = [MLX, LTX2]; -const STATUS = { connected: true, defaultModel: MLX.id }; +const MODEL_CONTEXT = { defaultModel: MLX.id }; -const render = ({ models = MODELS, status = STATUS, availableLoras = [], grokEnabled = false, url = '/media/video' } = {}) => { +const render = ({ models = MODELS, modelContext = MODEL_CONTEXT, availableLoras = [], grokEnabled = false, url = '/media/video' } = {}) => { const wrapper = ({ children }) => {children}; return renderHook( (props) => useVideoGenForm(props), - { wrapper, initialProps: { models, status, availableLoras, grokEnabled } }, + { wrapper, initialProps: { models, modelContext, availableLoras, grokEnabled } }, ); }; @@ -94,7 +94,7 @@ describe('useVideoGenForm', () => { vi.unstubAllGlobals(); }); - it('seeds the model from status.defaultModel without clobbering a URL pick', async () => { + it('seeds the model from modelContext.defaultModel without clobbering a URL pick', async () => { const { result } = render(); await waitFor(() => expect(result.current.modelId).toBe(MLX.id)); @@ -324,7 +324,7 @@ describe('useVideoGenForm', () => { it('clears sampler overrides on an automatic mode-compatible model fallback', async () => { const { result } = render({ models: [WAN_T2V, WAN_TI2V], - status: { connected: true, defaultModel: WAN_T2V.id }, + modelContext: { defaultModel: WAN_T2V.id }, }); await waitFor(() => expect(result.current.modelId).toBe(WAN_T2V.id)); act(() => { @@ -340,7 +340,7 @@ describe('useVideoGenForm', () => { it('does not submit chunks for a T2V-only Wan profile', async () => { const { result } = render({ models: [WAN_T2V], - status: { connected: true, defaultModel: WAN_T2V.id }, + modelContext: { defaultModel: WAN_T2V.id }, }); await waitFor(() => expect(result.current.modelId).toBe(WAN_T2V.id)); act(() => { @@ -353,7 +353,7 @@ describe('useVideoGenForm', () => { it('normalizes MiniMax H3 to its fixed temporal and sampler contract', async () => { const { result } = render({ models: [MLX, H3], - status: { connected: true, defaultModel: MLX.id }, + modelContext: { defaultModel: MLX.id }, }); await waitFor(() => expect(result.current.modelId).toBe(MLX.id)); act(() => { @@ -388,7 +388,7 @@ describe('useVideoGenForm', () => { // Substitutable prompt conditioner (#4081). describe('text encoder selection', () => { const renderWithH3 = async () => { - const rendered = render({ models: [MLX, H3], status: { connected: true, defaultModel: MLX.id } }); + const rendered = render({ models: [MLX, H3], modelContext: { defaultModel: MLX.id } }); await waitFor(() => expect(rendered.result.current.modelId).toBe(MLX.id)); act(() => rendered.result.current.handleModelChange(H3.id)); await waitFor(() => expect(rendered.result.current.modelId).toBe(H3.id)); @@ -398,7 +398,7 @@ describe('useVideoGenForm', () => { // An empty list is what hides the picker; a model with substitutions // exposes them straight off the server-decorated entry. it('exposes only the selected model’s options', async () => { - const { result } = render({ models: [MLX, H3], status: { connected: true, defaultModel: MLX.id } }); + const { result } = render({ models: [MLX, H3], modelContext: { defaultModel: MLX.id } }); await waitFor(() => expect(result.current.modelId).toBe(MLX.id)); expect(result.current.textEncoderOptions).toEqual([]); act(() => result.current.handleModelChange(H3.id)); @@ -465,7 +465,7 @@ describe('useVideoGenForm', () => { it('preserves H3 native 32px-grid geometry in the submitted payload', async () => { const { result } = render({ models: [MLX, H3], - status: { connected: true, defaultModel: MLX.id }, + modelContext: { defaultModel: MLX.id }, }); await waitFor(() => expect(result.current.modelId).toBe(MLX.id)); act(() => result.current.handleModelChange(H3.id)); @@ -482,7 +482,7 @@ describe('useVideoGenForm', () => { it('offers MiniMax H3 image mode, chaining and a non-advisory last frame', async () => { const { result } = render({ models: [MLX, H3], - status: { connected: true, defaultModel: MLX.id }, + modelContext: { defaultModel: MLX.id }, }); await waitFor(() => expect(result.current.modelId).toBe(MLX.id)); act(() => { @@ -791,7 +791,7 @@ describe('useVideoGenForm', () => { it('moves a fixed-profile remix to an editable model while preserving its restored controls', async () => { const { result } = render({ models: [MLX, H3], - status: { connected: true, defaultModel: MLX.id }, + modelContext: { defaultModel: MLX.id }, }); await waitFor(() => expect(result.current.modelId).toBe(MLX.id)); @@ -818,7 +818,7 @@ describe('useVideoGenForm', () => { it('uses the same editable-model fallback for a cross-page Remix handoff', async () => { const { result } = render({ models: [MLX, H3], - status: { connected: true, defaultModel: MLX.id }, + modelContext: { defaultModel: MLX.id }, url: `/media/video?modelId=${H3.id}&numFrames=124&steps=9&guidanceScale=0`, }); @@ -938,10 +938,10 @@ describe('useVideoGenForm — i2v reference mode (#4874)', () => { lastFrameAnchored: true, supportedModes: RUNTIME_MODES, }; const LTX25_MODELS = [LTX25, LTX2]; - const LTX25_STATUS = { connected: true, defaultModel: LTX25.id }; + const LTX25_MODEL_CONTEXT = { defaultModel: LTX25.id }; const inImageMode = async (opts = {}) => { - const { result } = render({ models: LTX25_MODELS, status: LTX25_STATUS, ...opts }); + const { result } = render({ models: LTX25_MODELS, modelContext: LTX25_MODEL_CONTEXT, ...opts }); await act(async () => { result.current.handleModeChange('image'); }); return result; }; @@ -982,7 +982,7 @@ describe('useVideoGenForm — i2v reference mode (#4874)', () => { (props) => useVideoGenForm(props), { wrapper: ({ children }) => {children}, - initialProps: { models: [], status: LTX25_STATUS, availableLoras: [], grokEnabled: false }, + initialProps: { models: [], modelContext: LTX25_MODEL_CONTEXT, availableLoras: [], grokEnabled: false }, }, ); await act(async () => { @@ -990,7 +990,7 @@ describe('useVideoGenForm — i2v reference mode (#4874)', () => { }); expect(result.current.i2vReferenceMode).toBe('inspire'); - rerender({ models: LTX25_MODELS, status: LTX25_STATUS, availableLoras: [], grokEnabled: false }); + rerender({ models: LTX25_MODELS, modelContext: LTX25_MODEL_CONTEXT, availableLoras: [], grokEnabled: false }); await waitFor(() => expect(result.current.currentModel?.id).toBe(LTX25.id)); expect(result.current.i2vReferenceMode).toBe('inspire'); }); diff --git a/client/src/lib/README.md b/client/src/lib/README.md index 2455ae4ef8..bbea8b4bce 100644 --- a/client/src/lib/README.md +++ b/client/src/lib/README.md @@ -80,7 +80,6 @@ grep -i "what you want to do" client/src/lib/README.md | `videoFinish.js` | Finish-a-draft gate (#3696). `isReproducibleTextToVideo(record)` is true only for a single text-to-video render whose history record already carries everything a re-render needs — `renderInputsVersion` (the positive marker that its conditioning inventory is trustworthy, so legacy records degrade to not-finishable rather than reading as unconditioned), an empty `conditioning` array, a resolved seed, a real prompt, and no stitched/chained/upscaled provenance. `finishTargetForRecord(record, models)` additionally resolves the draft model's server-declared `finishModelId` against the models this install can actually run, returning the delivery model entry or `null`. `isDeliveryVideoModel(model, models)` is the other end of the same graph — true when some entry names `model` as its `finishModelId`, which is what makes a delivery render always decode on the full decoder (#5423). Mirrors `finishTargetForModel` / `isDeliveryVideoModel` in `server/lib/videoFinishProfiles.js`; the pair table itself stays server-side. | | `videoGenParams.js` | Pure VideoGen param helpers: `FRAME_OPTIONS`/`FPS_OPTIONS`/`VIDEO_EDGE_BOUNDS`/`MAX_CHUNKS`/`CHUNK_OPTIONS`/`DEFAULT_CONTEXT_FRAMES`/`CONTEXT_FRAME_OPTIONS` constants, model-aware frame/fps/resolution-grid normalization, separate mute vs prompt-audio capability checks, `supportsContextWindow(model)` (does this runtime have an extend pipeline to feed a continuation window to — display mirror of `server/lib/videoContinuity.js`, pinned by `server/lib/videoContinuity.parity.test.js`), `videoModelMemoryGb()` (model memory footprint), `selectVideoMemoryProfile(model, systemMemoryGb)` + `VIDEO_MEMORY_RESERVE_GB` (#5420 — which declared weight-placement profile this machine can actually hold, out of the `memoryProfiles` the server decorates onto the entry; the floors ride on the model so only the reserve is mirrored, and an unmeasured `systemMemoryGb` returns a `null` usable figure rather than reading as a box that is too small), `computeFflfSafeFrames()` (FFLF/ltx2 pixel-budget back-solve, mirrors `server/services/videoGen/local.js`), and `isModelAllowedForMode()` (a2v uses the shared audio-runtime capability; IC remix remains LTX-only). Speed profiles (#4875) mirror the conditioner shape: `DEFAULT_SPEED_PROFILE_ID` (must equal `SPEED_PROFILE_DEFAULT_ID` in `server/lib/videoSpeedProfiles.js` — absence and `'quality'` are the same request), `isDefaultSpeedProfileId()` mirrors the server's absence-is-the-default rule, `speedProfilesForModel()` reads the server-decorated `speedProfiles` off the entry, `speedProfilesForMode(model, mode)` applies the SAME mode gate the server's `speedProfileDeclineReason` does (so a profile the server would decline is never offered, nor allowed to lock the dials), `normalizeSpeedProfileForModel()` snaps a selection onto what a just-switched model declares (mode-independent, so switching to fflf hides the picker without rewriting the choice), `speedProfileIdFromRecord()` reads one back out of a history entry / resumed job, `selectedSpeedProfile(id, model, mode)` resolves the profile actually driving the render — what the picker shows and what disables Steps+CFG — and `videoChainChunkModes({ model, mode, chaining, contextFrames, hasSourceImage })` derives the modes a CHAINED request’s chunks will run in (chunk 0 the request’s, chunks 1+ `extend` on a window-continuity chain or `image` on a frame hop), mirroring `generateChainedVideo`’s dispatch and `resolveContinuityStrategy`; `resolveContextFramesForDisplay()` is the absent/invalid→`DEFAULT_CONTEXT_FRAMES` half of `resolveContextFrames` that gate depends on. All pinned by `server/lib/videoSpeedProfiles.parity.test.js`. Also mirrors the IC-LoRA remix registry (`IC_LORA_MODES`/`IC_LORA_MODE_VALUES`/`isIcLoraMode()`/`icLoraSpecForMode()`/`icResolutionIssue()`) from `server/lib/icLoraWeights.js` so the form validates a reference render pre-submit — pinned by `server/lib/icLoraWeights.parity.test.js`. | | `videoGenResolutions.js` | Shared resolution presets/default for video generation, model-specific preset/default resolvers (for native canvases such as MiniMax H3), and `snapAspectToImage()` to pick the closest-aspect preset for an I2V source. | -| `videoGenStatusCache.js` | Session-scoped cache of the model-shaping half of `GET /api/video-gen/status` (`readCachedVideoGenStatus` / `writeCachedVideoGenStatus` / `VIDEO_GEN_STATUS_CACHE_KEY`), so the Video Gen Model picker paints from the previous answer instead of waiting on the python probe behind that route. Stores only the model list plus `defaultModel` / `systemMemoryGb` and hands them back marked `stale: true`; every python-health field is dropped rather than guarded, so a stored answer can never report connectivity. | | `videoGenSubmission.js` | Builds the local, Grok, and federated video-generation request bodies from validated form state, including prompt envelopes and empty-value wire sentinels. | | `videoReferenceModes.js` | Mirror of `server/lib/videoReferenceModes.js` (parity enforced by `server/lib/videoReferenceModes.mirror.test.js`) — the i2v reference-mode contract: `I2V_REFERENCE_MODES`, `I2V_REFERENCE_MODE_OPTIONS` (label + the promise sentence `AdvancedParamsPanel` and the source-frame note print), `runtimeSupportsI2vReferenceMode` (gates which options the picker offers), `resolveI2vReferenceStrength` (the effective strength the panel displays), and `i2vReferenceModeViolation` for pre-submit feedback. | | `videoRenderPhase.js` | Video render phase → named progress step (#5872). `resolveVideoRenderSteps({ generating, phase, progressPct })` → `{ activeId, steps: [{ id, label, state: 'done'|'active'|'pending' }] }`, collapsing the runners' fine-grained `STAGE:` vocabulary (`load-transformer`, `encode-prompt`, `sampling`, `mux`, …) onto six steps a person can read — the queue's own `queued` is one of them, so a caller needs no separate flag. Family prefixes (`download-*`, `load-*`, `wan-*`, …) absorb markers a future runner adds; `videoRenderStepFor(phase)` returns `null` for a genuinely unknown one, never step 0. Consumed by `components/videoGen/RenderStatusCard.jsx`. | diff --git a/client/src/lib/index.js b/client/src/lib/index.js index 59de4bc9e0..d08cdcc4c1 100644 --- a/client/src/lib/index.js +++ b/client/src/lib/index.js @@ -61,7 +61,6 @@ export * from './slashdoCatalog.js'; export * from './videoFinish.js'; export * from './videoGenParams.js'; export * from './videoGenResolutions.js'; -export * from './videoGenStatusCache.js'; export * from './videoGenSubmission.js'; export * from './videoReferenceModes.js'; export * from './videoRenderPhase.js'; diff --git a/client/src/lib/videoGenStatusCache.js b/client/src/lib/videoGenStatusCache.js deleted file mode 100644 index d8fd1c6211..0000000000 --- a/client/src/lib/videoGenStatusCache.js +++ /dev/null @@ -1,43 +0,0 @@ -/** - * Session-scoped cache of the model-shaping half of `GET /api/video-gen/status`. - * - * That probe shells out to python and rebuilds the hardware-aware model list on - * every call, so a cold Video Gen page load leaves the Model picker with nothing - * to render for a second or two. Caching lets the picker paint from the previous - * answer while the live probe revalidates behind it. - * - * Only `CACHED_FIELDS` is stored, and the read hands it back marked - * `stale: true`. Everything the payload says about python health — `connected`, - * `reason`, `missingPackages`, `pythonPath`, `byovRuntimes`, `runtime` — is - * deliberately dropped rather than guarded, because an interpreter the user just - * fixed (or just broke) must never be reported from a stored answer, and a field - * that isn't there can't be read by mistake. - * - * Session, not local: the model registry and the python environment both move - * with an upgrade or an install, and a payload kept for weeks would outlive - * both. - */ -import { safeReadJsonSession, safeWriteJsonSession } from './safeStorage.js'; - -// Bump the suffix when `CACHED_FIELDS` changes, so an older tab's entry is -// ignored rather than half-read. -export const VIDEO_GEN_STATUS_CACHE_KEY = 'portos.videoGenStatus.v1'; - -// The model list plus the numbers that decide which model is selected for it. -const CACHED_FIELDS = ['models', 'defaultModel', 'systemMemoryGb']; - -// Returns the cached fields with `stale: true`, or null when nothing usable is -// stored. An entry with no `models` array is worthless here — painting the -// picker is the whole point — so it reads as absent. -export const readCachedVideoGenStatus = () => { - const cached = safeReadJsonSession(VIDEO_GEN_STATUS_CACHE_KEY); - if (!cached || typeof cached !== 'object' || !Array.isArray(cached.models)) return null; - return { ...cached, stale: true }; -}; - -// Store the cacheable slice of a freshly fetched payload. -export const writeCachedVideoGenStatus = (status) => { - if (!status || typeof status !== 'object' || !Array.isArray(status.models)) return; - const slice = Object.fromEntries(CACHED_FIELDS.map((field) => [field, status[field]])); - safeWriteJsonSession(VIDEO_GEN_STATUS_CACHE_KEY, slice); -}; diff --git a/client/src/pages/VideoGen.composeWhileBusy.test.jsx b/client/src/pages/VideoGen.composeWhileBusy.test.jsx index 415c4aecfb..c5d9f854db 100644 --- a/client/src/pages/VideoGen.composeWhileBusy.test.jsx +++ b/client/src/pages/VideoGen.composeWhileBusy.test.jsx @@ -7,6 +7,7 @@ import { resetVideoGenMockState, state, videoGenModel, + videoGenModelContext, videoGenStatus, videoGenTermsGate, } from '../test/videoGenPageMocks.jsx'; @@ -20,6 +21,7 @@ describe('VideoGen compose-while-busy', () => { beforeEach(() => { resetVideoGenMockState(); state.getVideoGenStatus.mockResolvedValue(videoGenStatus([MODEL])); + state.getVideoGenModelContext.mockResolvedValue(videoGenModelContext([MODEL])); state.modelStatuses = { [MODEL.id]: { id: MODEL.id, repo: MODEL.repo, cached: true, sizeBytes: 100 } }; state.generateVideo.mockReturnValue(new Promise(() => {})); state.attach.mockReturnValue(new Promise(() => {})); diff --git a/client/src/pages/VideoGen.federatedTarget.test.jsx b/client/src/pages/VideoGen.federatedTarget.test.jsx index 69954f679f..110e32501f 100644 --- a/client/src/pages/VideoGen.federatedTarget.test.jsx +++ b/client/src/pages/VideoGen.federatedTarget.test.jsx @@ -7,6 +7,7 @@ import { resetVideoGenMockState, state, videoGenModel, + videoGenModelContext, videoGenStatus, videoGenTermsGate, } from '../test/videoGenPageMocks.jsx'; @@ -49,6 +50,7 @@ describe('VideoGen federated render target', () => { resetVideoGenMockState(); state.peers = [PEER]; state.getVideoGenStatus.mockResolvedValue(videoGenStatus([MODEL])); + state.getVideoGenModelContext.mockResolvedValue(videoGenModelContext([MODEL])); state.modelStatuses = { [MODEL.id]: { id: MODEL.id, repo: MODEL.repo, cached: true, sizeBytes: 100 } }; state.generateVideo.mockReturnValue(new Promise(() => {})); state.attach.mockReturnValue(new Promise(() => {})); diff --git a/client/src/pages/VideoGen.jsx b/client/src/pages/VideoGen.jsx index 73fb67267b..d2cd41680a 100644 --- a/client/src/pages/VideoGen.jsx +++ b/client/src/pages/VideoGen.jsx @@ -25,9 +25,9 @@ * * Form state, the URL-param prefill paths, the mode/backend transitions, and * `buildGeneratePayload()` live in `useVideoGenForm` (issue #3291) — this page - * owns the fetching (status/models/history/gallery), the SSE run pipeline, the - * the rendering. The durable server queue owns queued work; each render target - * drains through its own lane. + * owns the fetching (status/model-context/history/gallery), the SSE run + * pipeline, and the rendering. The durable server queue owns queued work; + * each render target drains through its own lane. * * "Add to queue" submits immediately to the durable server queue. That is * important for mixed-target work: a Grok submission can start in its cloud @@ -82,7 +82,7 @@ import { useVideoGenForm } from '../hooks/useVideoGenForm.js'; import { useFederatedMediaTarget } from '../hooks/useFederatedMediaTarget'; import RemoteMediaTargetPicker from '../components/federatedMedia/RemoteMediaTargetPicker'; import { - getVideoGenStatus, generateVideo, cancelVideoGen, + getVideoGenStatus, getVideoGenModelContext, generateVideo, cancelVideoGen, listVideoHistory, deleteVideoHistoryItem, setVideoHidden, upscaleVideo, patchSettingsSlice, @@ -98,7 +98,6 @@ import ResolutionField from '../components/media/ResolutionField'; import { VIDEO_EDGE_BOUNDS, videoEdgeBoundsForModel, IC_LORA_MODES } from '../lib/videoGenParams.js'; import { finishTargetForRecord, isDeliveryVideoModel } from '../lib/videoFinish.js'; import { peerModelRequiresInput } from '../lib/federatedMediaReadiness.js'; -import { readCachedVideoGenStatus, writeCachedVideoGenStatus } from '../lib/videoGenStatusCache.js'; const MODES = [ { id: 'text', label: 'Text', icon: Type, desc: 'Text-to-video' }, { id: 'image', label: 'Image', icon: ImageIcon, desc: 'Image-to-video (start frame)' }, @@ -121,12 +120,16 @@ export default function VideoGen() { refreshGrokEnabled(); }; - // Paint the model picker from the previous /status answer while the live - // probe runs. The cached entry carries `stale: true` and holds nothing but - // the model-shaping fields (see lib/videoGenStatusCache.js); connectivity UI - // below gates on `statusFresh`. - const [status, setStatus] = useState(readCachedVideoGenStatus); + // `/status` owns connectivity ONLY. It shells out to python on every call + // (~1-2s), so nothing the form needs to render may wait on it. + const [status, setStatus] = useState(null); + // The model list plus the numbers its auto-select reads, off the probe-free + // `/model-context`. Fetched alongside /status on mount, it lands first — so + // the Model picker paints on a cold load instead of holding a placeholder + // through the interpreter probe. + const [modelContext, setModelContext] = useState(null); const [statusLoading, setStatusLoading] = useState(true); + const [modelContextLoading, setModelContextLoading] = useState(true); // Grok Build CLI video backend (#2859 phase 2) — surfaced only when the // user enabled Grok in Settings → Image Gen (one toggle covers image + // video). 'local' keeps every existing flow untouched. @@ -134,7 +137,7 @@ export default function VideoGen() { // The jobId of the render this tab's Generate button currently owns — // threaded into cancelVideoGen so cancellation is job-scoped. const activeJobIdRef = useRef(null); - const [models, setModels] = useState(() => status?.models || []); + const models = useMemo(() => modelContext?.models || [], [modelContext]); const refreshGrokEnabled = useCallback(() => { getSettings({ silent: true }) .then((sv) => setGrokEnabled(sv?.imageGen?.grok?.enabled === true)) @@ -189,7 +192,7 @@ export default function VideoGen() { icStrength, setIcStrength, icSkipStage2, setIcSkipStage2, applyRemix, applyFinish, applyResumedParams, buildGeneratePayload, } = useVideoGenForm({ - models, status, availableLoras, grokEnabled, + models, modelContext, availableLoras, grokEnabled, remoteSubmissionFields: remoteTarget.isRemote ? remoteTarget.submissionFields : null, }); @@ -403,19 +406,28 @@ export default function VideoGen() { const refreshStatus = useCallback(() => { setStatusLoading(true); getVideoGenStatus() - .then((s) => { - setStatus(s); - setModels(s.models || []); - writeCachedVideoGenStatus(s); - }) + .then(setStatus) .catch(() => setStatus({ connected: false, reason: 'Status check failed' })) .finally(() => setStatusLoading(false)); }, []); + // Kept separate from refreshStatus so the picker never inherits the python + // probe's latency. A failure leaves `modelContext` null, which reads exactly + // like "no model to offer" — the Model field takes itself away rather than + // holding a placeholder forever. + const refreshModelContext = useCallback(() => { + setModelContextLoading(true); + getVideoGenModelContext({ silent: true }) + .then(setModelContext) + .catch(() => {}) + .finally(() => setModelContextLoading(false)); + }, []); + useEffect(() => { refreshStatus(); + refreshModelContext(); return () => eventSourceRef.current?.close(); - }, [refreshStatus, eventSourceRef]); + }, [refreshStatus, refreshModelContext, eventSourceRef]); // SSE subscriber shared by the in-flight POST path and the mount-time // resume path. `withToast: false` on resume suppresses the success/error @@ -811,16 +823,13 @@ export default function VideoGen() { // `byovRuntimeMissing` for those models. Without this, a user who installed // ONLY a BYOV runtime via the modal would stay stuck behind a "not // configured" error from the unrelated legacy probe. - // A cached entry says nothing about python health, so the connectivity UI - // waits for the live probe rather than reporting the interpreter state of - // whenever the last visit happened. - const statusFresh = !!status && !status.stale; // The Model field renders as soon as there is anything to say — the list, or - // the fact that it is still being probed. Only a finished probe that named no - // model at all takes the field away. + // the fact that it is still being fetched. Only a finished fetch that named + // no model at all takes the field away. That fetch no longer waits on the + // python probe, so on a cold load the list itself is normally what lands. const modelsLoading = models.length === 0; - const modelFieldVisible = !modelsLoading || statusLoading; - const notConnected = statusFresh && status.connected === false && !needsByovProbe; + const modelFieldVisible = !modelsLoading || modelContextLoading; + const notConnected = !!status && status.connected === false && !needsByovProbe; // A federated render answers to the PEER’s readiness, not to this machine’s // runtime gates — none of the local probes below describe the hardware it @@ -834,7 +843,7 @@ export default function VideoGen() { return (
- {statusFresh ? ( + {status ? ( - {statusFresh && status.connected === false && (() => { + {status && status.connected === false && (() => { const missingCount = status.missingPackages?.length || 0; const hasPath = !!status.pythonPath; return ( @@ -1392,7 +1401,7 @@ export default function VideoGen() { backend={backend} backendDisclosures={status?.backendDisclosures} model={isGrok ? null : currentModel} - systemMemoryGb={status?.systemMemoryGb} + systemMemoryGb={modelContext?.systemMemoryGb} /> {!isGrok && ( @@ -1563,10 +1572,11 @@ export default function VideoGen() { onClose={() => setInstallModalOpen(false)} onComplete={() => { refreshByovStatus(); - // The capability probe is part of /video-gen/status's model - // decoration. Refresh it after install/repair so H3's LoRA picker - // and warning react without a manual page reload. + // The capability probe decorates the model list, and the install + // also moves python health. Refresh both after install/repair so + // H3's LoRA picker and warning react without a manual page reload. refreshStatus(); + refreshModelContext(); }} />
diff --git a/client/src/pages/VideoGen.modelLoading.test.jsx b/client/src/pages/VideoGen.modelLoading.test.jsx index 9840dece63..a6636bbc46 100644 --- a/client/src/pages/VideoGen.modelLoading.test.jsx +++ b/client/src/pages/VideoGen.modelLoading.test.jsx @@ -7,90 +7,82 @@ import { resetVideoGenMockState, state, videoGenModel, + videoGenModelContext, videoGenStatus, } from '../test/videoGenPageMocks.jsx'; /** - * The Model picker paints before /status lands. + * The Model picker does not wait on /status. * - * /status shells out to python and rebuilds the hardware-aware model list on - * every call, so the field used to be absent for a second or two and then pop - * into the middle of the form. It now holds its place with a loading - * placeholder, and a session-cached payload paints the real list immediately — - * while every connectivity claim keeps waiting for the live probe. + * /status shells out to python on every call, so a cold load used to leave the + * field absent for a second or two and then pop it into the middle of the form. + * The list now comes from the probe-free /video-gen/model-context, which lands + * on its own — while every connectivity claim keeps waiting for the live probe. + * The loading placeholder remains for the window before either answers. */ const MODEL_ONE = videoGenModel('example-one'); const MODEL_TWO = videoGenModel('example-two'); const statusPayload = (overrides = {}) => videoGenStatus([MODEL_ONE, MODEL_TWO], overrides); +const modelContextPayload = (overrides = {}) => videoGenModelContext([MODEL_ONE, MODEL_TWO], overrides); await loadVideoGenPage(); -const { VIDEO_GEN_STATUS_CACHE_KEY } = await import('../lib/videoGenStatusCache.js'); -// A /status call the test settles by hand, so the page can be asserted mid-probe. -const deferredStatus = () => { +// A call the test settles by hand, so the page can be asserted mid-flight. +const deferred = (mock) => { let settle; - state.getVideoGenStatus.mockReturnValue(new Promise((resolve) => { settle = resolve; })); + mock.mockReturnValue(new Promise((resolve) => { settle = resolve; })); return async (payload) => { await act(async () => { settle(payload); }); }; }; +const deferredStatus = () => deferred(state.getVideoGenStatus); +const deferredModelContext = () => deferred(state.getVideoGenModelContext); -describe('VideoGen model picker while /status is in flight', () => { +describe('VideoGen model picker vs the /status python probe', () => { beforeEach(() => { localStorage.clear(); sessionStorage.clear(); resetVideoGenMockState(); state.getVideoGenStatus.mockResolvedValue(statusPayload()); + state.getVideoGenModelContext.mockResolvedValue(modelContextPayload()); state.attach.mockResolvedValue({ filename: 'example.mp4' }); }); it('keeps the Model field with a loading placeholder until the model list lands', async () => { - const resolveStatus = deferredStatus(); + const resolveModelContext = deferredModelContext(); await renderVideoGenPage(); const field = screen.getByLabelText('Model'); expect(field).toBeDisabled(); expect(field).toHaveTextContent('Loading models…'); - await resolveStatus(statusPayload()); + await resolveModelContext(modelContextPayload()); await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(MODEL_ONE.id)); expect(screen.getByLabelText('Model')).toBeEnabled(); }); - it('paints the cached model list on the next load instead of waiting for the probe', async () => { - const first = await renderVideoGenPage(); - await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(MODEL_ONE.id)); - // Only the model-shaping slice is persisted — python health never is. - expect(Object.keys(JSON.parse(sessionStorage.getItem(VIDEO_GEN_STATUS_CACHE_KEY))).sort()) - .toEqual(['defaultModel', 'models', 'systemMemoryGb']); - first.unmount(); - + it('paints the model list on a cold load while every python claim waits', async () => { + // No prior visit and no priming — the regression this pins is the Model + // field sitting on its placeholder for the whole /status round trip, and + // the converse: nothing may report the interpreter before it answers. const resolveStatus = deferredStatus(); await renderVideoGenPage(); const field = screen.getByLabelText('Model'); expect(field).toBeEnabled(); expect(field).toHaveValue(MODEL_ONE.id); - await resolveStatus(statusPayload()); - }); - - it('never reports python health from a cached entry', async () => { - // A hand-written entry carrying a FAILED probe — the belt to the - // projection's braces. The model list may come from storage; the diagnosis - // may not, because the interpreter can have been fixed since. - sessionStorage.setItem(VIDEO_GEN_STATUS_CACHE_KEY, JSON.stringify(statusPayload({ - connected: false, - reason: 'Python probe failed', - missingPackages: ['torch'], - }))); - const resolveStatus = deferredStatus(); - await renderVideoGenPage(); - - expect(screen.getByLabelText('Model')).toHaveValue(MODEL_ONE.id); expect(screen.getByText('Checking…')).toBeInTheDocument(); expect(screen.queryByText(/Install missing Python packages/)).toBeNull(); - expect(screen.queryByText(/Python probe failed/)).toBeNull(); await resolveStatus(statusPayload({ connected: true, pythonVersion: '3.12.1' })); await waitFor(() => expect(screen.getByText('Python 3.12.1')).toBeInTheDocument()); }); + + it('takes the Model field away when the context fetch names no model at all', async () => { + // A failed /model-context leaves nothing to offer. The field must not hold + // its placeholder forever — the rest of the form closes over the gap. + state.getVideoGenModelContext.mockRejectedValue(new Error('offline')); + await renderVideoGenPage(); + + await waitFor(() => expect(screen.queryByLabelText('Model')).toBeNull()); + }); }); diff --git a/client/src/pages/VideoGen.terms.test.jsx b/client/src/pages/VideoGen.terms.test.jsx index 76c5d4ece8..e3fc1f78d6 100644 --- a/client/src/pages/VideoGen.terms.test.jsx +++ b/client/src/pages/VideoGen.terms.test.jsx @@ -7,6 +7,7 @@ import { resetVideoGenMockState, state, videoGenModel, + videoGenModelContext, videoGenStatus, videoGenTermsGate, } from '../test/videoGenPageMocks.jsx'; @@ -33,6 +34,7 @@ describe('VideoGen MiniMax H3 orchestration', () => { state.generateVideo.mockResolvedValue({ jobId: 'job-1' }); state.repair.mockResolvedValue({ ok: true }); state.getVideoGenStatus.mockResolvedValue(videoGenStatus([H3_ONE, H3_TWO])); + state.getVideoGenModelContext.mockResolvedValue(videoGenModelContext([H3_ONE, H3_TWO])); state.attach.mockImplementation(async (_jobId, handlers) => { handlers.onComplete({ result: { filename: 'example.mp4' } }); return { filename: 'example.mp4' }; @@ -98,10 +100,14 @@ describe('VideoGen MiniMax H3 orchestration', () => { it('refreshes the model capability payload after runtime setup completes', async () => { await renderVideoGenPage(); await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3_ONE.id)); - const before = state.getVideoGenStatus.mock.calls.length; + const beforeStatus = state.getVideoGenStatus.mock.calls.length; + const beforeContext = state.getVideoGenModelContext.mock.calls.length; await act(async () => { await state.runtimeInstallComplete(); }); - await waitFor(() => expect(state.getVideoGenStatus).toHaveBeenCalledTimes(before + 1)); + // The install moves BOTH halves: the hardware decoration on the model list + // and the python health the connectivity banner reads. + await waitFor(() => expect(state.getVideoGenModelContext).toHaveBeenCalledTimes(beforeContext + 1)); + expect(state.getVideoGenStatus).toHaveBeenCalledTimes(beforeStatus + 1); }); }); diff --git a/client/src/pages/VideoGen.textEncoderAutoDownload.test.jsx b/client/src/pages/VideoGen.textEncoderAutoDownload.test.jsx index b22a213f35..78ab72171d 100644 --- a/client/src/pages/VideoGen.textEncoderAutoDownload.test.jsx +++ b/client/src/pages/VideoGen.textEncoderAutoDownload.test.jsx @@ -17,6 +17,7 @@ import { resetVideoGenMockState, state, videoGenModel, + videoGenModelContext, videoGenStatus, videoGenTermsGate, } from '../test/videoGenPageMocks.jsx'; @@ -44,6 +45,7 @@ describe('VideoGen substitute text-encoder auto-download', () => { beforeEach(() => { resetVideoGenMockState(); state.getVideoGenStatus.mockResolvedValue(videoGenStatus([MODEL])); + state.getVideoGenModelContext.mockResolvedValue(videoGenModelContext([MODEL])); // The substitute is never resident: what these cases pin down is the // request, and a cached encoder would short-circuit it. state.getModelStatus = (id) => (String(id).startsWith('__text_encoder_option__:') diff --git a/client/src/services/apiImageVideo.js b/client/src/services/apiImageVideo.js index 6dd2bc59ba..6690b48d45 100644 --- a/client/src/services/apiImageVideo.js +++ b/client/src/services/apiImageVideo.js @@ -165,6 +165,13 @@ export const getVideoGenStatus = (options = {}) => request('/video-gen/status', export const listVideoModels = ({ includeUnavailable = false, ...options } = {}) => request('/video-gen/models', options) .then((models) => filterHardwareCompatibleModels(models, { includeUnavailable })); +// `{ models, defaultModel, systemMemoryGb, fflfLtx2PixelBudget }` — the model +// list plus the numbers its auto-select reads, with no python probe behind it. +// getVideoGenStatus() returns the same fields, but only after shelling out to +// the interpreter; fetch this alongside it so the Model picker paints first. +export const getVideoGenModelContext = (options = {}) => + request('/video-gen/model-context', options) + .then((ctx) => ({ ...ctx, models: filterHardwareCompatibleModels(ctx?.models) })); // `{ models: [...], textEncoder: { repo, cached, sizeBytes } }`. Same shape // contract as the image variant + a text-encoder block since the active // encoder is a separate multi-GB pull. diff --git a/client/src/test/videoGenPageMocks.jsx b/client/src/test/videoGenPageMocks.jsx index 937fe98776..cf09048e3d 100644 --- a/client/src/test/videoGenPageMocks.jsx +++ b/client/src/test/videoGenPageMocks.jsx @@ -42,6 +42,11 @@ export const state = { peers: [], /** `getVideoGenStatus`; a spy so a suite can defer it, count calls or vary the payload. */ getVideoGenStatus: vi.fn(), + /** + * `getVideoGenModelContext`; the probe-free half the Model picker reads (#5835). + * A spy for the same reasons — a suite defers it to assert the page mid-flight. + */ + getVideoGenModelContext: vi.fn(), generateVideo: vi.fn(), attach: vi.fn(), eventSourceRef: { current: null }, @@ -61,7 +66,7 @@ export const state = { universeStyle: DEFAULT_UNIVERSE_STYLE, }; -const SPIES = ['getVideoGenStatus', 'generateVideo', 'attach', 'start', 'startWhenIdle', 'repair', 'cancel', 'refresh']; +const SPIES = ['getVideoGenStatus', 'getVideoGenModelContext', 'generateVideo', 'attach', 'start', 'startWhenIdle', 'repair', 'cancel', 'refresh']; /** Restore every documented default, including fresh spies. Call it first in `beforeEach`. */ export function resetVideoGenMockState() { @@ -105,6 +110,19 @@ export const videoGenTermsGate = (termsId) => ({ licenseUrl: 'https://example.com/license', }); +/** + * A `/model-context` payload over `models` — the model list plus the three + * shaping numbers the picker's auto-select reads, with no python probe behind + * them. The first model is the default unless overridden. + */ +export const videoGenModelContext = (models, overrides = {}) => ({ + models, + defaultModel: models[0]?.id ?? null, + systemMemoryGb: 128, + fflfLtx2PixelBudget: 8_000_000, + ...overrides, +}); + /** A `/status` payload over `models`; the first model is the default unless overridden. */ export const videoGenStatus = (models, overrides = {}) => ({ connected: true, @@ -122,6 +140,7 @@ vi.mock('../services/api', () => ({ // a media provider the picker renders nothing and every local path is unchanged. getInstances: vi.fn(async () => ({ peers: state.peers })), getVideoGenStatus: (...args) => state.getVideoGenStatus(...args), + getVideoGenModelContext: (...args) => state.getVideoGenModelContext(...args), generateVideo: (...args) => state.generateVideo(...args), cancelVideoGen: vi.fn(async () => ({})), listVideoHistory: vi.fn(async () => []), diff --git a/server/lib/apiRouteCatalog.generated.json b/server/lib/apiRouteCatalog.generated.json index 9227c19c57..02ca6e6af8 100644 --- a/server/lib/apiRouteCatalog.generated.json +++ b/server/lib/apiRouteCatalog.generated.json @@ -16454,6 +16454,14 @@ "server/routes/videoGen.js" ] }, + { + "method": "GET", + "path": "/api/video-gen/model-context", + "mountPath": "/api/video-gen", + "sources": [ + "server/routes/videoGen.js" + ] + }, { "method": "GET", "path": "/api/video-gen/model-terms", @@ -17497,8 +17505,8 @@ ], "stats": { "mounts": 147, - "operations": 2167, - "declarations": 2175, + "operations": 2168, + "declarations": 2176, "sourceFiles": 230 } } diff --git a/server/routes/videoGen.js b/server/routes/videoGen.js index 7e4e5cfc88..659dea1dac 100644 --- a/server/routes/videoGen.js +++ b/server/routes/videoGen.js @@ -88,6 +88,33 @@ const hardwareAwareVideoModels = async () => { }; }; +// The model list plus the three numbers that decide which entry the picker +// auto-selects. Deliberately free of any python probe: /status shells out to +// the interpreter on every call (~1-2s) and the Model field used to wait on it, +// so `/model-context` serves the same fields off the registry and the cached +// hardware probe alone. /status keeps returning them for its other readers — +// this is the single builder both routes share, so the two can't drift. +const videoModelContext = async () => { + const { capabilities, models } = await hardwareAwareVideoModels(); + return { + // Each entry carries its optional `disclosure` block (provenance, weights/ + // runtime licenses, pinned-snapshot download size) straight off the + // registry — absent for custom models, which the UI renders as Unknown. + models, + defaultModel: defaultVideoModelId(capabilities), + // Total system memory in GB — the client uses this to auto-select the + // highest-memory mode-compatible model that fits on this machine. + // Rounded to nearest GB; sub-GB precision isn't useful for the + // model-size comparison and reads more cleanly in the UI. + systemMemoryGb: Math.round(os.totalmem() / 1024 ** 3), + // Effective FFLF/ltx2 stage-2 pixel-frame budget (honors + // FFLF_LTX2_PIXEL_BUDGET). The multi-keyframe picker mirrors the + // back-solve so it can reject out-of-budget keyframe indices before + // submit instead of letting the worker 400 mid-render. + fflfLtx2PixelBudget: resolveFflfLtx2PixelBudget(), + }; +}; + // M4A files are stored in an MP4 container. Browsers and OS file pickers // label them inconsistently: Safari uses `video/mp4`, Chrome/Firefox use // `audio/mp4`, and some platforms emit `audio/x-m4a` or `audio/aac`. @@ -400,18 +427,17 @@ router.get('/status', asyncHandler(async (_req, res) => { const s = await getSettings(); const py = s.imageGen?.local?.pythonPath || null; const { connected, reason, missing, pythonVersion } = await resolveLocalPythonHealth(py); - const { capabilities, models } = await hardwareAwareVideoModels(); res.json({ connected, pythonPath: py, pythonVersion: pythonVersion || null, reason, missingPackages: missing, - // Each entry carries its optional `disclosure` block (provenance, weights/ - // runtime licenses, pinned-snapshot download size) straight off the - // registry — absent for custom models, which the UI renders as Unknown. - models, - defaultModel: defaultVideoModelId(capabilities), + // `models` / `defaultModel` / `systemMemoryGb` / `fflfLtx2PixelBudget` — + // kept here for the callers that already read them off /status. The Video + // Gen page takes them from GET /model-context instead, so its Model picker + // never waits on the python probe above. + ...(await videoModelContext()), // Server-owned execution + policy scope per render backend (#3674). The // client renders these strings verbatim so the wording can't drift between // the two surfaces. @@ -419,16 +445,6 @@ router.get('/status', asyncHandler(async (_req, res) => { // Authoritative list of bring-your-own-venv runtimes — lets the client // gate the install-banner probe without hardcoding the same Set. byovRuntimes: Object.keys(BYOV_RUNTIME_INFO), - // Total system memory in GB — the client uses this to auto-select the - // highest-memory mode-compatible model that fits on this machine. - // Rounded to nearest GB; sub-GB precision isn't useful for the - // model-size comparison and reads more cleanly in the UI. - systemMemoryGb: Math.round(os.totalmem() / 1024 ** 3), - // Effective FFLF/ltx2 stage-2 pixel-frame budget (honors - // FFLF_LTX2_PIXEL_BUDGET). The multi-keyframe picker mirrors the - // back-solve so it can reject out-of-budget keyframe indices before - // submit instead of letting the worker 400 mid-render. - fflfLtx2PixelBudget: resolveFflfLtx2PixelBudget(), // Runtime fingerprint — host chip/os + resolved ltx/mlx/torch versions per // installed BYOV runtime — so the UI can show the exact numerical stack and // bug reports for garbled/"mosaic" output carry the version info that makes @@ -530,6 +546,14 @@ router.get('/models', asyncHandler(async (_req, res) => { res.json(models); })); +// Everything the Model picker needs to render AND auto-select, with no python +// probe in the way. A sibling route rather than a wrapper around /models so the +// bare-array shape that route has always returned stays intact for its existing +// callers (and for an older client talking to a newer server). +router.get('/model-context', asyncHandler(async (_req, res) => { + res.json(await videoModelContext()); +})); + router.get('/models/status', asyncHandler(async (_req, res) => { // Text encoder is shared across all video renders. A registry entry with // `localPath` (e.g. an LM Studio install) trumps the HF cache check, so diff --git a/server/routes/videoGen.test.js b/server/routes/videoGen.test.js index 457c94ceb9..d4dba66796 100644 --- a/server/routes/videoGen.test.js +++ b/server/routes/videoGen.test.js @@ -567,6 +567,35 @@ describe('videoGen routes', () => { }); }); + describe('GET /model-context', () => { + it('serves the model list and its auto-select numbers without probing python', async () => { + const { checkPackages } = await import('../lib/pythonSetup.js'); + checkPackages.mockClear(); + + const r = await request(app).get('/api/video-gen/model-context'); + expect(r.status).toBe(200); + // The picker's whole input set, in one probe-free answer. + expect(r.body.models.map((m) => m.id)).toEqual(['ltx2_unified']); + expect(r.body.defaultModel).toBe('ltx2_unified'); + expect(typeof r.body.systemMemoryGb).toBe('number'); + expect(r.body.systemMemoryGb).toBeGreaterThan(0); + expect(typeof r.body.fflfLtx2PixelBudget).toBe('number'); + expect(r.body.fflfLtx2PixelBudget).toBeGreaterThan(0); + // The point of the route: no interpreter subprocess in the request path. + expect(checkPackages).not.toHaveBeenCalled(); + }); + + it('agrees with /status on every field the two share', async () => { + const [context, status] = await Promise.all([ + request(app).get('/api/video-gen/model-context'), + request(app).get('/api/video-gen/status'), + ]); + for (const field of ['models', 'defaultModel', 'systemMemoryGb', 'fflfLtx2PixelBudget']) { + expect(status.body[field]).toEqual(context.body[field]); + } + }); + }); + describe('GET /models/:modelId/download — restricted terms', () => { const h3CheckpointFiles = ['LICENSE', 'FL2VA/model_index.json', 'FL2VA/video_vae/source/model.safetensors']; const h3 = { From 4fcaa9b56cfcb40baf9c4154121023b5f5e41b10 Mon Sep 17 00:00:00 2001 From: "[._.]/ Adam Eivy" Date: Thu, 3 Sep 2026 15:26:21 +0000 Subject: [PATCH 010/178] fix: sweep sub-44px icon buttons tree-wide and widen tap-target guard (#5904) Every icon-only button with tight p-0.5/p-1/p-1.5 padding now carries min-h-[44px] min-w-[44px] inline-flex items-center justify-center, and action rows holding two of them widen from gap-1 to gap-2. Icon sizes are untouched. The a11yConventions tap-target rule moves from src/components/meatspace/ to the whole client tree (same padding-shape filter and self-probe); sibling-owned SongBook surfaces are excluded for the parallel change that owns them. --- client/src/a11yConventions.test.js | 34 +++++++++++-------- client/src/components/CatalogCastPanel.jsx | 2 +- client/src/components/FolderPicker.jsx | 2 +- client/src/components/Layout.jsx | 2 +- client/src/components/ThemeSwitcher.jsx | 2 +- .../components/apps/AppOperationBanner.jsx | 2 +- client/src/components/apps/EditAppDrawer.jsx | 2 +- .../src/components/brain/links/LinkChip.jsx | 2 +- client/src/components/brain/tabs/InboxTab.jsx | 22 ++++++------ client/src/components/brain/tabs/LinksTab.jsx | 8 ++--- .../src/components/brain/tabs/MemoryTab.jsx | 10 +++--- client/src/components/brain/tabs/NotesTab.jsx | 8 ++--- client/src/components/brain/tabs/TrustTab.jsx | 2 +- client/src/components/calendar/ConfigTab.jsx | 2 +- client/src/components/calendar/ReviewTab.jsx | 10 +++--- .../cos/ActionableInsightsBanner.jsx | 2 +- client/src/components/cos/JobCard.jsx | 8 ++--- client/src/components/cos/TaskAddForm.jsx | 2 +- client/src/components/cos/tabs/AgentCard.jsx | 10 +++--- .../components/cos/tabs/MemoryEditModal.jsx | 2 +- client/src/components/cos/tabs/MindTab.jsx | 2 +- client/src/components/cos/tabs/RunsTab.jsx | 8 ++--- client/src/components/cos/tabs/TaskItem.jsx | 12 +++---- .../cos/tabs/schedule/AppTaskTypeSection.jsx | 2 +- .../src/components/dashboard/LayoutEditor.jsx | 6 ++-- .../dashboard/builtins/DailyDriverWidget.jsx | 2 +- .../digital-twin/ListEnrichment.jsx | 2 +- .../digital-twin/NextActionBanner.jsx | 4 +-- .../digital-twin/tabs/AccountsTab.jsx | 2 +- .../digital-twin/tabs/AppearanceTab.jsx | 2 +- .../digital-twin/tabs/AutobiographyTab.jsx | 2 +- .../fableloom/LoomEpisodeOutlinePlanner.jsx | 2 +- .../fableloom/LoomHostedSessionModal.jsx | 2 +- .../components/fableloom/LoomSeriesPlan.jsx | 6 ++-- .../components/feature-agents/OutputTab.jsx | 2 +- .../components/goals/GoalLinkedActivities.jsx | 2 +- .../components/goals/GoalLinkedCalendars.jsx | 2 +- .../src/components/goals/GoalProgressLog.jsx | 4 +-- client/src/components/goals/GoalTodoList.jsx | 2 +- client/src/components/goals/GoalsListView.jsx | 4 +-- .../src/components/media/MediaJobsQueue.jsx | 2 +- client/src/components/media/MediaLightbox.jsx | 10 +++--- .../src/components/media/PromptFromMedia.jsx | 2 +- .../components/media/VideoTimelineLanes.jsx | 2 +- .../media/VideoTimelineLanes.test.jsx | 6 ++-- client/src/components/messages/DraftsTab.jsx | 10 +++--- .../src/components/messages/IMessageTab.jsx | 4 +-- client/src/components/music/AlbumsManager.jsx | 8 ++--- .../src/components/musicVideo/SceneCard.jsx | 8 ++--- .../components/onboarding/FirstRunCard.jsx | 2 +- .../persona/PersonaMasterDetail.jsx | 2 +- client/src/components/pipeline/CanonCard.jsx | 16 ++++----- .../pipeline/ImagePromptCandidates.jsx | 2 +- .../pipeline/arcCanvas/IssueRow.jsx | 2 +- .../pipeline/arcCanvas/SeasonRow.jsx | 4 +-- .../pipeline/arcCanvas/TickingClockEditor.jsx | 2 +- .../manuscript/ManuscriptCommentCard.jsx | 4 +-- .../pipeline/stages/ComicPagesStage.jsx | 2 +- .../pipeline/stages/ComicScriptStage.jsx | 2 +- .../pipeline/stages/PovRewritePanel.jsx | 2 +- .../pipeline/stages/StoryboardsStage.jsx | 4 +-- .../components/privacy/PrivacyChangesTab.jsx | 8 ++--- client/src/components/quotaBurn/JobRow.jsx | 2 +- .../src/components/settings/ApiAccessTab.jsx | 2 +- .../src/components/settings/DatabaseTab.jsx | 2 +- .../settings/LocalLlmRuntimesView.jsx | 2 +- .../settings/LocalModelAssessments.jsx | 8 ++--- .../components/settings/LocalSetupPanel.jsx | 2 +- .../settings/ModelCapabilityTests.jsx | 2 +- .../components/settings/MtplxServerCard.jsx | 2 +- .../settings/RuntimeServersCard.jsx | 2 +- .../settings/SlotstreamServerCard.jsx | 2 +- .../src/components/shell/ShellImageDrop.jsx | 2 +- .../components/songs/MidiVisualization.jsx | 8 ++--- .../components/songs/ReferenceAnalysis.jsx | 14 ++++---- client/src/components/songs/RoundEditForm.jsx | 6 ++-- .../src/components/songs/SongRecordings.jsx | 6 ++-- .../src/components/songs/SongScoreParts.jsx | 4 +-- .../components/sprites/AssetPromptSection.jsx | 2 +- client/src/components/sprites/LoopTrimmer.jsx | 6 ++-- .../components/sprites/ReferenceWorkflow.jsx | 2 +- .../src/components/sprites/SpriteCatalog.jsx | 6 ++-- .../components/sprites/SpriteDetailHeader.jsx | 4 +-- .../src/components/sync/SyncDetailDrawer.jsx | 2 +- .../universe/VisionDescribeModal.jsx | 2 +- .../universeBuilder/CompositeSheetsEditor.jsx | 12 +++---- .../universeBuilder/UniverseBibleTab.jsx | 4 +-- .../UniverseCategoryEditor.jsx | 24 ++++++------- .../UniverseStyleReferences.jsx | 2 +- .../src/components/videoGen/IcLoraPanel.jsx | 2 +- .../src/components/videoGen/KeyframePanel.jsx | 2 +- client/src/components/voice/VoicePicker.jsx | 2 +- client/src/components/voice/VoiceWidget.jsx | 4 +-- client/src/components/wiki/tabs/BrowseTab.jsx | 8 ++--- client/src/components/wiki/tabs/GraphTab.jsx | 4 +-- client/src/hooks/useAgentFeedbackToast.jsx | 6 ++-- client/src/pages/ApiExplorer.jsx | 4 +-- client/src/pages/Browser.jsx | 2 +- client/src/pages/ChiefOfStaff.jsx | 6 ++-- client/src/pages/FableLoom.jsx | 2 +- client/src/pages/HistoryPage.jsx | 2 +- client/src/pages/ImageGen.jsx | 2 +- client/src/pages/Instances.jsx | 8 ++--- client/src/pages/Loops.jsx | 10 +++--- client/src/pages/Loras.jsx | 4 +-- client/src/pages/MediaCollectionDetail.jsx | 2 +- client/src/pages/MediaHistory.jsx | 2 +- client/src/pages/MoodBoardDetail.jsx | 6 ++-- client/src/pages/MoodBoards.jsx | 2 +- client/src/pages/PipelineSeries.jsx | 4 +-- client/src/pages/PromptManager.jsx | 6 ++-- client/src/pages/Review.jsx | 18 +++++----- client/src/pages/RoundEditor.jsx | 2 +- client/src/pages/RunnerPage.jsx | 2 +- client/src/pages/ThreejsModelDetail.jsx | 2 +- client/src/pages/Tribe.jsx | 2 +- client/src/pages/VideoGen.jsx | 2 +- client/src/pages/VideoTimeline.jsx | 2 +- client/src/pages/VideoTimelineEditor.jsx | 2 +- client/src/pages/WritersRoom.jsx | 2 +- 120 files changed, 284 insertions(+), 280 deletions(-) diff --git a/client/src/a11yConventions.test.js b/client/src/a11yConventions.test.js index df241a30f7..3f345c99d5 100644 --- a/client/src/a11yConventions.test.js +++ b/client/src/a11yConventions.test.js @@ -32,9 +32,9 @@ * 6. An `` with no `alt`, which is announced by its `src` — a hashed * filename or a blob URL. `alt=""` is the correct spelling for a * decorative image and passes; only the omission is the bug. - * 7. An icon-only `
-
+
-
+
{(activeType === 'projects' || activeType === 'ideas' || activeType === 'admin') && record.status !== 'done' && ( @@ -512,7 +512,7 @@ export default function NotesTab() { )} diff --git a/client/src/components/calendar/ReviewTab.jsx b/client/src/components/calendar/ReviewTab.jsx index 39ad3569a1..4a9339e2f8 100644 --- a/client/src/components/calendar/ReviewTab.jsx +++ b/client/src/components/calendar/ReviewTab.jsx @@ -111,7 +111,7 @@ export default function ReviewTab() { {/* Date Navigation */}
- - {!isToday && ( @@ -240,11 +240,11 @@ export default function ReviewTab() {
{!isReviewed && !isEditing && ( -
+
-
+
diff --git a/client/src/components/cos/tabs/MindTab.jsx b/client/src/components/cos/tabs/MindTab.jsx index 3066f99667..cc8e34b4d0 100644 --- a/client/src/components/cos/tabs/MindTab.jsx +++ b/client/src/components/cos/tabs/MindTab.jsx @@ -706,7 +706,7 @@ export default function MindTab() { {messageImages.map((image) => (
  • -
    • diff --git a/client/src/components/cos/tabs/RunsTab.jsx b/client/src/components/cos/tabs/RunsTab.jsx index e62495b4b6..33f7cbcc30 100644 --- a/client/src/components/cos/tabs/RunsTab.jsx +++ b/client/src/components/cos/tabs/RunsTab.jsx @@ -331,7 +331,7 @@ export default function RunsTab() { {run.success === false && ( diff --git a/client/src/components/dashboard/LayoutEditor.jsx b/client/src/components/dashboard/LayoutEditor.jsx index 0adb2ba0db..0043877561 100644 --- a/client/src/components/dashboard/LayoutEditor.jsx +++ b/client/src/components/dashboard/LayoutEditor.jsx @@ -327,9 +327,9 @@ export default function LayoutEditor({ layouts, activeLayoutId, limits, onClose, {meta?.label ?? id} {!meta && (unknown — skipped)} - - - + + + ); })} diff --git a/client/src/components/dashboard/builtins/DailyDriverWidget.jsx b/client/src/components/dashboard/builtins/DailyDriverWidget.jsx index 553e627ae8..a9e87ea96d 100644 --- a/client/src/components/dashboard/builtins/DailyDriverWidget.jsx +++ b/client/src/components/dashboard/builtins/DailyDriverWidget.jsx @@ -118,7 +118,7 @@ export default function DailyDriverWidget({ dashboardState }) { disabled={dismissing} aria-label="Dismiss for today" title="Dismiss for today" - className="ml-auto p-1 rounded text-gray-500 hover:text-white hover:bg-port-border/60 disabled:opacity-50" + className="min-h-[44px] min-w-[44px] inline-flex items-center justify-center ml-auto p-1 rounded text-gray-500 hover:text-white hover:bg-port-border/60 disabled:opacity-50" > diff --git a/client/src/components/digital-twin/ListEnrichment.jsx b/client/src/components/digital-twin/ListEnrichment.jsx index a812b0cefc..5a6afc2038 100644 --- a/client/src/components/digital-twin/ListEnrichment.jsx +++ b/client/src/components/digital-twin/ListEnrichment.jsx @@ -194,7 +194,7 @@ export default function ListEnrichment({ {items.length > 1 && ( diff --git a/client/src/components/digital-twin/tabs/AutobiographyTab.jsx b/client/src/components/digital-twin/tabs/AutobiographyTab.jsx index 15a290b81b..1bdb270e15 100644 --- a/client/src/components/digital-twin/tabs/AutobiographyTab.jsx +++ b/client/src/components/digital-twin/tabs/AutobiographyTab.jsx @@ -559,7 +559,7 @@ export default function AutobiographyTab({ onRefresh }) {
    -
    diff --git a/client/src/components/fableloom/LoomHostedSessionModal.jsx b/client/src/components/fableloom/LoomHostedSessionModal.jsx index 6e9d1a6349..7e3f7f417e 100644 --- a/client/src/components/fableloom/LoomHostedSessionModal.jsx +++ b/client/src/components/fableloom/LoomHostedSessionModal.jsx @@ -219,7 +219,7 @@ export default function LoomHostedSessionModal({ - - + + +