Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions server/lib/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub
| `mediaModels.js` | Single source of truth for image/video model metadata. Entry fields are documented in the module docblock — note `repoFiles[]`, which narrows a model's own repo to an explicit file list for an aggregate repo that holds far more than the runner loads (MiniMax H3 CUDA). |
| `minimaxH3Memory.js` | The declared weight-placement table for the three MiniMax H3 entries (#5420) — H3 is the one video model family whose components fit nowhere unassisted, so "does this box have enough" is a render gate, not a UI fact. `MINIMAX_H3_MEMORY_PROFILES` maps entry id → `{ shippedRepo, shippedRevision, profiles }`, each profile carrying an honest `minMemoryGb` host floor and (CUDA only) a `minVramGb` device floor, ordered best-first. Every capacity number is HOISTED from what already existed — the CUDA tiers are `resolve_offload_profile()`'s own thresholds in `scripts/generate_minimax_h3_cuda.py`, the host floors are the entries' `memoryGb` — the sole new number being `MINIMAX_H3_HOST_RESERVE_GB`, a policy reserve held back for the OS. `applyMiniMaxH3MemoryProfiles(list)` is the load-time backfill (twin of migration 317) and guards BOTH `repo` and `revision`, like the speed-profile decorator. `selectMiniMaxH3MemoryProfile({ model, totalMemoryGb })` picks the best profile the HOST can hold (VRAM is the runner's call — the server has no synchronous device view); `miniMaxH3MemoryDeclineReason()` RETURNS the fail-closed reason so the render path can 400 it and a status surface can show it, and returns `null` on an UNMEASURED host — "not measured" is a deferral to the runner, never the same as zero. `validateMiniMaxH3MemoryProfileTable` / `sanitizeMiniMaxH3MemoryProfiles` warn + strip a hand-edited table (NaN floor, duplicate/reserved id, mis-ordered tiers) at load. |
| `videoContinuity.js` | How chunk N+1 of a chained video render is conditioned on chunk N. `resolveContextFrames(requested)` normalizes the tail-window size (absent → `DEFAULT_CONTEXT_FRAMES` = 22 ≈ 1s @ 24fps; an explicit `0` is preserved as "last frame only", NOT collapsed into the default) and clamps to `MIN_CONTEXT_FRAMES`..`MAX_CONTEXT_FRAMES`. `resolveContinuityStrategy({model, contextFrames})` picks `'window'` (LTX-2 `extend_from_video` conditioned on the prior chunk's last N frames — motion, not just a pose) or `'frame'` (extract the last frame, run i2v), degrading to `'frame'` on any runtime outside `CONTEXT_WINDOW_RUNTIMES` rather than rejecting. `extendLatentFrames` / `extendedPixelFrames` convert across the VAE's `LATENT_FRAME_STRIDE` (8 pixel frames per latent), `contextPrefixFrames({totalFrames, extendLatents})` measures how much of an extend render is echoed context to trim back off before stitching (0 = leave it alone), and `tailWindowStartFrame` gives the cut index for the window itself. Pure — importable from `prepareParams.js` without dragging in `local.js`. Mirrored for the picker in `client/src/lib/videoGenParams.js`, pinned by `videoContinuity.parity.test.js`. |
| `videoPromptLinter.js` | Deterministic lint pass for continuous-video clip prompts (part of #6217, independent of `scriptVideoCompiler.js` #6225 by interface — lints already-built prompt strings plus caller-supplied framing/reference metadata rather than a compiler-shaped clip). `lintClipPrompt(clip, {bible, maxLength})` checks a single clip: a `cutType: 'continue'` clip must open with `"Hard cut to <framing>:"` and use a framing distinct from `previousFraming`; every `references` entry (`{kind: 'cast'|'locations', id}`) must resolve to a bible descriptor present VERBATIM in the prompt (never paraphrased); the prompt must not contain the banned cross-clip referents (same/still/again/continues/as before), negatives (no/without/never), or UI-overlay terms (text/caption/overlay), matched on word boundaries so substrings like "against" or "context" don't false-positive; and the prompt must be `<= MAX_CLIP_PROMPT_LENGTH` (800) characters. Returns `{pass, reasons[]}`, never throws. `lintClips(clips, {bible, maxLength})` lints an ordered chain, deriving each clip's `previousFraming` from the preceding array element's `framing` when not set explicitly. |
| `videoDisclosure.js` | Video Gen provenance/licensing + backend policy-scope facts (#3674). `VIDEO_MODEL_DISCLOSURES` maps each shipped video model id to `{ shippedRepo, disclosure }` (model card URL, weights license, runtime license, pinned-snapshot download size in decimal GB, review date) — every value checked against a primary upstream source, and any fact that could not be established is OMITTED so the UI renders "Unknown" instead of guessing. `applyVideoDisclosures(list)` is the load-time backfill (twin of migration 237) with the same preservation guards: an existing `disclosure` key wins, a custom id is skipped, and a `repo` pointed at a fork keeps Unknown. `VIDEO_BACKEND_DISCLOSURES` / `videoBackendDisclosure(id)` state where inference runs (`execution: local or hosted`) and whose policy applies — execution facts only, never a restrictiveness ranking. `APACHE_2` / `GEMMA_TERMS` are the shared license descriptors `videoTextEncoders.js` reuses, so a license-text correction reaches both tables from one edit. |
| `videoDraftDecoders.js` | Preview-fidelity ("draft") video decode (#5423). `VIDEO_DRAFT_DECODERS` declares each entry id → a separately downloaded decoder asset, pin-guarded on repo AND revision; `applyVideoDraftDecoders(list)` is the load-time backfill and `validateDraftDecoderTable` / `sanitizeDraftDecoders` warn + strip a hand-edited row (missing pin, a multi-shard or path-traversing file list, a runtime whose builder emits no draft flags) so a full decode can never report itself as a draft one. `DRAFT_DECODE_FULL` (`'full'`) is a deliberate NO-OP — `isFullDecode(id)` makes absence and that sentinel the same request, so a full-decode render builds byte-identical spawn args. `draftDecodeDeclineReason({ model, decodeId, models, runtimeRevision, assetCached })` RETURNS (never throws) the reason a draft decode does not apply — the model is a delivery target in the finish graph (`isDeliveryVideoModel`), it declares no decoder, the installed runner checkout is not the revision the asset was verified against, or the weights are not downloaded — because a knob that only makes a render cheaper must degrade rather than 400 a submitted job; `resolveVideoDraftDecoder()` returns the concrete asset or `null`. `publicVideoDraftDecodeOptions(model)` is the picker payload (empty for a model with no decoder, so the client renders no control) and `downloadableVideoDraftDecoders(list)` the download targets. The table ships NO entry, and as of 2026-08-30 that is a decision rather than a gap ([ADR](../../docs/decisions/2026-08-30-h3-draft-decoder-asset.md)): the shim substitutes one file into upstream’s checkpoint root under a STRICT key match, so a candidate must be a complete unquantized full-`VideoVAE` checkpoint — which every genuinely light decoder (TAE, quantized repack, decoder-only head) is not. |
| `videoFinishProfiles.js` | Draft → delivery ("Finish") relationships between video models (#3696). `VIDEO_FINISH_PROFILES` declares each fast draft entry id → `{ shippedRepo, finishModelId }`, only for pairs that share a runtime, base repo and supported modes (the same weights at a different step budget), so re-rendering the draft's seed reproduces its composition instead of re-rolling it. `applyVideoFinishProfiles(list)` is the load-time backfill (twin of migration 238) with the usual preservation guards (existing key wins, custom id skipped, forked `repo` skipped); `validateFinishProfileGraph(list)` returns the graph problems (missing / self-referencing / chained target, runtime / repo / supportedModes mismatch) and `sanitizeFinishProfiles(list)` warns + strips them at load so a typo can never surface a Finish button targeting nothing. `finishTargetForModel(model, availableModels)` resolves the delivery entry scoped to what this install can run. |
Expand Down
1 change: 1 addition & 0 deletions server/lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ export * from './mediaModels.js';
export * from './minimaxH3Memory.js';
export * from './videoContinuity.js';
export * from './videoDisclosure.js';
export * from './videoPromptLinter.js';
export * from './videoDraftDecoders.js';
export * from './videoFinishProfiles.js';
export * from './videoSpeedProfiles.js';
Expand Down
116 changes: 116 additions & 0 deletions server/lib/videoPromptLinter.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
/**
* Deterministic lint pass for continuous-video clip prompts (part of #6217's
* chained-generation feature). Catches the prompt patterns that cause visual
* smear/collapse when a video model chains one clip onto the next: a missing
* hard-cut opener, a re-used camera framing, a paraphrased bible descriptor,
* or language that reads fine in prose but breaks a video prompt — referring
* back to "the same" shot, negating something, or naming on-screen text.
*
* Pure text analysis — no I/O, no video-backend awareness. Independent of
* `scriptVideoCompiler.js` (#6225) by interface: it lints already-built clip
* prompt strings plus caller-supplied framing/reference metadata, not a
* compiler-shaped clip object. `continuousVideo.js` (#6227) composes this
* with the compiler's output.
*
* Returns structured per-clip results rather than throwing — callers decide
* how to surface a failing lint (block submission, warn, retry the prompt).
*/

import { resolveBibleDescriptor } from './scriptVideoCompiler.js';

export const MAX_CLIP_PROMPT_LENGTH = 800;
const HARD_CUT_PREFIX = 'Hard cut to';

const BANNED_REFERENTS = ['same', 'still', 'again', 'continues', 'as before'];
const BANNED_NEGATIVES = ['no', 'without', 'never'];
const BANNED_OVERLAY_TERMS = ['text', 'caption', 'overlay'];

const escapeRegExp = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');

// \b (not a hand-rolled [^a-z0-9] boundary) so "same_text" isn't flagged as containing
// the standalone word "same" — \w already includes '_', matching how prose reads a word.
const bannedTermPattern = (term) => ({ term, regex: new RegExp(`\\b${escapeRegExp(term).replace(/\s+/g, '\\s+')}\\b`, 'i') });
const BANNED_REFERENT_PATTERNS = BANNED_REFERENTS.map(bannedTermPattern);
const BANNED_NEGATIVE_PATTERNS = BANNED_NEGATIVES.map(bannedTermPattern);
const BANNED_OVERLAY_PATTERNS = BANNED_OVERLAY_TERMS.map(bannedTermPattern);

const findBannedTerms = (prompt, patterns) => patterns.filter(({ regex }) => regex.test(prompt)).map(({ term }) => term);

const asString = (value) => (typeof value === 'string' ? value : '');

/**
* Lint a single clip prompt against every rule.
*
* @param {object} clip
* @param {string} clip.prompt
* @param {'fresh'|'continue'} clip.cutType
* @param {string} [clip.framing] - this clip's camera framing/angle, required to check the hard-cut opener on a 'continue' clip
* @param {string} [clip.previousFraming] - the preceding chained clip's framing/angle, checked only when cutType === 'continue'
* @param {Array<{kind: 'cast'|'locations', id: string}>} [clip.references] - bible entries this clip's prompt must carry verbatim
* @param {object} [options]
* @param {object} [options.bible]
* @param {number} [options.maxLength]
* @returns {{pass: boolean, reasons: string[]}}
*/
export function lintClipPrompt(clip, { bible, maxLength = MAX_CLIP_PROMPT_LENGTH } = {}) {
const prompt = asString(clip?.prompt);
const { cutType } = clip || {};
const framing = asString(clip?.framing).trim() || null;
const previousFraming = asString(clip?.previousFraming).trim() || null;
const references = Array.isArray(clip?.references) ? clip.references : [];
const reasons = [];

if (cutType === 'continue') {
const opener = framing ? `${HARD_CUT_PREFIX} ${framing}:` : null;
if (!opener || !prompt.trimStart().startsWith(opener)) {
reasons.push(opener
? `missing hard-cut opener "${opener}"`
: 'missing hard-cut opener: clip.framing was not provided');
}
if (framing && previousFraming && framing.toLowerCase() === previousFraming.toLowerCase()) {
reasons.push(`framing "${framing}" repeats the preceding clip's framing — a continuing clip needs a distinct camera framing/angle`);
}
}

for (const ref of references) {
const descriptor = resolveBibleDescriptor(bible, ref?.kind, ref?.id);
if (!descriptor) {
reasons.push(`no bible descriptor found for ${ref?.kind}/${ref?.id}`);
} else if (!prompt.includes(descriptor)) {
reasons.push(`prompt is missing the verbatim bible descriptor for ${ref.kind}/${ref.id}`);
}
}

for (const term of findBannedTerms(prompt, BANNED_REFERENT_PATTERNS)) {
reasons.push(`banned cross-clip referent "${term}" — describe what is visible in THIS clip instead of referring back to a previous one`);
}
for (const term of findBannedTerms(prompt, BANNED_NEGATIVE_PATTERNS)) {
reasons.push(`banned negative construction "${term}" — video models tend to render negated content instead of omitting it`);
}
for (const term of findBannedTerms(prompt, BANNED_OVERLAY_PATTERNS)) {
reasons.push(`banned UI overlay language "${term}" — video models tend to render literal on-screen text/captions from this`);
}

if (prompt.length > maxLength) {
reasons.push(`prompt is ${prompt.length} characters, over the ${maxLength} character limit`);
}

return { pass: reasons.length === 0, reasons };
}

/**
* Lint an ordered array of clips (chain order, same as
* `scriptVideoCompiler.compileScriptToClips` emits them). Each clip's
* `previousFraming` defaults to the preceding array element's `framing` when
* not set explicitly.
*
* @returns {{pass: boolean, results: Array<{index: number, pass: boolean, reasons: string[]}>}}
*/
export function lintClips(clips, { bible, maxLength = MAX_CLIP_PROMPT_LENGTH } = {}) {
const clipList = Array.isArray(clips) ? clips : [];
const results = clipList.map((clip, index) => {
const previousFraming = clip?.previousFraming ?? clipList[index - 1]?.framing ?? null;
return { index, ...lintClipPrompt({ ...clip, previousFraming }, { bible, maxLength }) };
});
return { pass: results.every((r) => r.pass), results };
}
Loading