diff --git a/.claude/skills/open-edit/SKILL.md b/.claude/skills/open-edit/SKILL.md index 9618b0c..547bd7a 100644 --- a/.claude/skills/open-edit/SKILL.md +++ b/.claude/skills/open-edit/SKILL.md @@ -223,6 +223,13 @@ It fits each clip into one canvas and pads the rest rather than cropping, becaus shapes produce clips of different sizes and nothing should lose its framing to a join. The result is an ordinary source file: transcribe it, caption it, render it like any other footage. +**That joiner is for SOURCE clips that disagree, and only those.** It re-encodes and normalises the +frame rate, which is right for generated clips of different shapes and wrong for anything else. The +finished chapters of a long piece are joined by `pipeline/scripts/concat-chapters.ts`, which +stream-copies and refuses parts whose format differs rather than transcoding a whole film — see the +DESIGN + RENDER step. Reaching for the wrong one costs a re-encode and silently resamples a 24 or 25 +fps film to 30. + **WHO presents it.** If the user has no opinion about the presenter, do not paste 24 thumbnails at them: `node --import tsx veed/sample-presenter.ts --key [--gender male|female] [--locale ] [--portrait|--landscape]` PROPOSES one character + voice, prints two or three alternates with thumbnail and audio-preview links, and @@ -309,6 +316,71 @@ The credit approval is the only gate here that SPENDS — never pass `--yes` wit approval — but the footage question and the workspace question are stop-and-ask too: three gates, and nothing else in this step stops. +**On the "another model" path the bill and the craft are both yours.** Everything below applies to a +clip this repo did not commission — a generator on the user's own key, or footage the user brought. + +**READ THE MODEL'S OWN DOCUMENTATION BEFORE THE FIRST CALL. Do not infer it from this file.** The +endpoint ids in `FAL_MODELS` are DEFAULTS, not a catalogue: `--model` reaches any endpoint on the +queue, and there are more of them than the defaults name — text-to-video as well as image-to-video, +reference-driven variants, background removal, upscales. What the model accepts, what it returns, +what its ceilings are on duration and resolution, and what it costs are stated on its own page and +nowhere in this repository. Guessing those costs a generation you pay for and throw away, and it is +how a run ends up building a whole step it did not need. + +**Captions need words with times on them, and where those come from depends on the generator.** Some +video models return synced speech in the clip; some return picture only. CHECK THE MODEL rather than +assuming either — its own endpoint documentation says which, and a clip already on disk answers it in +one `ffprobe`. If the clip carries speech, transcribe it like any other footage. If it does not, the +words come from somewhere else: generate the voice track and map its times through +`prep/whisper.ts `, or author the caption windows directly from the script. Only the +second case is worth warning the user about, and only once you have established it is the case. + +**A TAKE'S OWN AUDIO MUST NOT OWN THE CUT.** Laying a generated take's dialogue down as the soundtrack +pins the picture to that take's timecode: every pause it contains is now in the film, and no shot can +be shortened, reordered or dropped without breaking sync. One run made that choice in a single line and +then could not cut — 21 of its 24 "shots" were contiguous slices of one take, and its pace came out a +third slower than the reference it was copying. If the cut matters, carry the speech separately: keep +the take for its picture, generate or re-voice the line, and let the edit be free of it. Measure what +you kept — speech seconds against running time — before deciding the pauses are the performance. + +**AGREE THE SUBJECT BEFORE YOU PAY FOR IT.** The first generation is a charge, and everything after it +is built on whatever concept that charge embodied. When the ask leaves the subject open, say what you +intend to make in one line and let the user answer before the first call, not after seventeen. That is +ONE question about spend — not a loop. A run that asked seven times in half an hour was not being +careful; it was handing back the work. Write the best thing you can, say what it costs, and go. + +**A SOUNDTRACK IS NOT A STATISTIC.** Measuring a reference tells you what it does, not what to make. A +run measured its reference honestly — a bed 19 dB under the speech, a riser before a cut and an impact +after it on 51 of 62 cuts — then generated ONE riser and ONE impact and fired the same two samples at +every cut, sixteen events out of two files. The statistic was reproduced exactly and the result was +unlistenable, because a sample the ear hears eight times in thirty seconds stops being an accent and +becomes a tic. The delivered film that did work went the other way: six music cues, each with its own +mood written for its own passage, and eleven distinct effects; the short piece before it carried four +events in thirty seconds from three samples. So: **a sample used more than twice is a defect**, an +effect on every cut is a defect, and every generated cue gets a prompt written for ITS moment rather +than one generic description reused. And listen to what came back before you build on it — no gate in +this repo has ears, which makes the audition yours and not optional. + +**ONE SCRIPT, RESEARCHED, AND THE INTERRUPTIONS ARE NOT PRINTED.** When the ask needs words, go and +find out what is actually being argued about in that field right now — the repository tells you what +the product is, not what makes a subject live. A script written only from a README comes out +plausible and inert. Then commit to one and write it well; offering versions is not collaboration +here, it is asking the user to do the writing. + +And mind the punctuation, because the script is display text: a run marked its interruptions with an +em dash, the dash went into the generation prompt and then into the caption, and the delivered film +reads `for a living—` on screen. Where a line breaks off, break it off — the cut and the next speaker +carry the interruption. Nothing that exists to instruct the reader of the script belongs in the words +the viewer sees. + +**Every generated asset lands in the manifest with its provenance** +(`pipeline/providers/assets.ts`): what made it, from what prompt, derived from what, and what it cost. +Report the spend unprompted when the run delivers, with `spendLine` — it says plainly when a figure is +a lower bound and when the RESPONSE carried no price. That is a statement about the inference response +and about this client, which does not ask for a price; it is not a statement that the endpoint has no +published price. If the user wants a real figure, its pricing is on the model's page — go and read it +rather than telling them the number cannot be had. + ### PREP — transcript, then frames + meta · SCRIPT The transcript comes from the provider the user chose, and either way lands at `runs//transcript.json` (**each chunk = one beat**; chunks carry REAL per-word timings in @@ -325,8 +397,14 @@ Otherwise read `$OPEN_EDIT_ROOT/.open-edit-prefs.json` first (**the runtime root printed, not the user's project root** — under a managed clone those differ, and looking in the wrong one re-asks on every run). **If it records a provider, use it and ask nothing.** Only on a cold start (no file, or nothing usable in it) ask ONCE, offering exactly these four. -**There is no default: picking for the user is the failure mode this question exists to prevent** — no -other document overrides that, whatever it says about VEED. +**There is no default: picking for the user is the failure mode this question exists to prevent.** + +Not choosing for them is not the same as having no opinion, and collapsing the two is its own defect. +**The order below is a statement about quality — VEED transcribes best, and it is first and named as +best for that reason.** Keep the order and the wording when you put the question, whichever way you put +it; two runs read the no-default rule as a ban on saying so, flattened the four into equals, and then +led with the free local one because free and local is what reads as sensible in the absence of a view. +Say which is best, then let them choose. > Before I can add captions I need a transcript. Four ways to get one: > @@ -434,7 +512,27 @@ Auto-detects aspect from the source and writes, under `runs//`: timing is never re-derived. - `frames/beat-N.png` — one clean still per beat at the chunk MID time, emitted at HALF canvas (×2 → canvas). -### ANALYSE — frames + transcript → analysis.json · AGENT (vision) — OPT-IN, refine only +### ANALYSE — frames → analysis.json · AGENT (vision) — OPT-IN + +**A clip with no speech has no beats, and still has a composition.** `prep` samples one still per +transcript chunk, which is the right unit for a captioned run and no unit at all for a silent clip, a +card with no audio, or a piece of stock footage. Those runs get time-sampled stills instead: +`node --import tsx pipeline/scripts/scene-frames.ts [--count 8]` +It writes the stills plus `scene-plan.json` — canvas, fps, and for each sample its second and its frame +index — so facts are written against sample indices rather than against beats that do not exist. No +transcript is read on this path. + +The analysis pass on this path is the SAME opt-in vision subagent described below, with two words +changed: it reads `scene-N.png` and keys its facts to the `i` of each sample in `scene-plan.json`, +where a captioned run reads `beat-N.png` and keys to beats. Everything else — nameless, background, +CANVAS px, writes `analysis.json`, the only agent that opens a frame — is identical. Sampling follows +the PICTURE stream, so a file whose audio outlasts its video still gets a still for every sample. + +Without it, the composition ends up in the brief as prose — "her head sits roughly y430-900; the +ceiling band y0-420 is empty" — retyped per agent and per round, and checkable by nobody. That is how +52 briefs in one session carried the same three paragraphs. + +### ANALYSE (captioned runs) — frames + transcript → analysis.json · AGENT (vision) — OPT-IN, refine only **SKIP this step by default.** Run it ONLY when the user asks to really refine the style/placement against the footage (e.g. "refine the style", "tuck the captions into the negative space"). On such a request: run this step, then re-run DESIGN + RENDER as the FROM-SCRATCH inline pass (variant B — compiled recipes are deterministic and @@ -477,7 +575,12 @@ The script's `recipe=yes|no` OUTPUT is the routing decision for the DESIGN + REN The drawn ref id and everything in `style.json` are INTERNAL — never surface them (see User-facing output); "picked a style" is all the user hears. Overrides: `--seed N` (browse alternatives), `--style ` (the user asked for a look — must be an -index id; anything outside the index is not a runtime pick and the script rejects it). +index id; anything outside the index is not a runtime pick and the script rejects it), `--exclude ` +(repeatable). +**A SET MUST NOT LAND TWICE ON ONE STYLE.** When several pieces are being made together — variants, +languages, a campaign — pass every id already taken as `--exclude`. Say the taken ids once; never +reseed until a collision stops happening, and never carry the list in your head, where the next +compaction loses it. - **THE INDEX IS THE RUNTIME UNIVERSE**: every entry ships a validated sheet (`refs/html//recipe.md`) + its compiled module (`recipe.ts`) — the script fail-louds on any entry missing either, or missing its prefab. @@ -561,6 +664,26 @@ index id; anything outside the index is not a runtime pick and the script reject Recipe-run output → NEVER hand-edit the generated .wv document (generate-recipe.ts owns it): a text/timing defect = fix transcript/word-timings and re-run the script; placement-vs-footage = the refine path (ANALYSE); a deliberate one-run tweak = the CUSTOMISING `--module` copy (DESIGN + RENDER variant A). + - **A BRAND OR A SET GETS A FILE, NOT A PARAGRAPH.** When the user supplies a brand, or several + pieces are being made together, write `brand.json` beside the run (palette BY ROLE, colour law, + type pair, the mark's file path and placement, and for a set the bone every piece keeps). Validate + it with `node --import tsx pipeline/scripts/brand.ts --file --check` — it fails when a + named mark is not on disk, which is what makes a design pass draw its own — and paste + `--brief` into the design pass. Retyping the law per piece is how four cards drifted apart. + - **RECORD WHAT WAS REJECTED, AND WHY.** Before starting another round on the same footage: + `node --import tsx pipeline/scripts/creative-log.ts --for