[feat] agent: add H3 cinematography compiler, storyboard audit, contact-sheet QC, and shot revision

apps/api/src/opendirector_api/creation.py

@@ -1,4 +1,5 @@
 """Authoritative project endpoints for the canvas creation workflow."""
+from typing import Literal
 from uuid import UUID
 from pathlib import Path
 from base64 import b64encode
@@ -185,13 +186,33 @@ async def query(project_id: UUID, value: CreationQuery):
         result = await canvas.query(value)
         if value.type == 'asset.preview' and 'preview' in result:
             result['preview'].pop('path')
-            result['preview']['url'] = f'/api/creation/projects/{project_id}/assets/{value.objectId}/preview'
+            base_url = f'/api/creation/projects/{project_id}/assets/{value.objectId}/preview'
+            if value.videoScope == 'contact_sheet':
+                result['preview']['url'] = (
+                    f'{base_url}?video_scope=contact_sheet&columns={value.columns}&rows={value.rows}'
+                )
+            else:
+                result['preview']['url'] = base_url
         return result
 
     @router.get('/projects/{project_id}/assets/{asset_id}/preview')
-    async def asset_preview(project_id: UUID, asset_id: UUID):
+    async def asset_preview(
+        project_id: UUID,
+        asset_id: UUID,
+        video_scope: Literal['first_frame', 'contact_sheet'] = Query(default='first_frame'),
+        columns: int = Query(default=4, ge=1, le=8),
+        rows: int = Query(default=3, ge=1, le=8),
+    ):
         target = projects.local.state_dir / 'previews' / str(project_id)
-        result = await run_in_threadpool(assets.preview, project_id, asset_id, target)
+        result = await run_in_threadpool(
+            assets.preview,
+            project_id,
+            asset_id,
+            target,
+            video_scope=video_scope,
+            columns=columns,
+            rows=rows,
+        )
         if 'preview' not in result:
             return result
         return FileResponse(result['preview']['path'], media_type=result['preview']['mimeType'])

docs/guides/cli.md

@@ -42,6 +42,46 @@ CLI 输出 JSON。修改回执附带对应的 assetId、itemId、specId 及其
 Agent 的 Shell 通过 PATH 使用同一个 `opendirector`。宿主提供受令牌保护的本轮连接、工程范围、编辑会话和执行目录，文件登记限制在本轮执行目录。每次调用产生业务活动及回执，供聊天与轨迹页展示。
 
 
+## H3 专业分镜提示词编排、连续性审计、多帧质检与单镜头局部修订
+
+`opendirector video` 支持首帧（`--first-frame`）、尾帧（`--last-frame`）、首尾帧插值（同时传 `--first-frame` 与 `--last-frame`）、多模态参考（`--reference-item`）与续写模式（`--continue-from`），并提供 H3 分镜计划校验、结构化提示词编译/标签校验、工程级跨镜头连续性审计（`storyboard.audit`）、视频多帧拼图预览与客观测量质检（`measuredFacts`），以及不破坏相邻轨道与字幕的时间线单镜头局部替换：
+
+```bash
+# 校验多镜头景别与运镜多样性（防止同质化）及 4—15 秒时长约束
+opendirector video lint-shot-plan --file ./shot-plan.json
+
+# 编译并校验符合 MiniMax-H3 规范的 T2VA / I2VA / FL2VA / L2VA / Ref2VA 英文提示词
+opendirector video compile-h3-prompt --file ./shot-01-spec.json
+
+# 创建首尾帧（FL2VA）或续写视频节点，并结合工程实际参考顺序校验 <Picture N>/<Video N>/<Audio N> 序号
+opendirector --project PROJECT_UUID video create \
+  --model MiniMax-H3 --prompt "..." --duration 6 --x 480 --y 0 \
+  --first-frame START_ITEM_UUID --last-frame END_ITEM_UUID --title "Shot 01"
+opendirector --project PROJECT_UUID video lint-h3-prompt --spec-id SPEC_UUID
+
+# 审计当前工程跨镜头角色锚点（Subject）、上游关键帧就绪度与试拍门控预算
+opendirector --project PROJECT_UUID video audit-storyboard --timeline-editor TIMELINE_EDITOR_UUID
+
+# 按工程视频 assetId 或执行目录文件做带时间戳十二宫格审看与客观静帧/黑场测量（不污染工程素材库）
+opendirector --project PROJECT_UUID video inspect-shot --asset VIDEO_ASSET_UUID --contact-sheet ./shot-01-qc.jpg
+opendirector video inspect-shot ./shot-01.mp4 --contact-sheet ./shot-01-qc.jpg
+
+# 时间线单镜头局部重拍替换：可显式指定片段与素材，也可直接按已关联的上游画布节点同步最新采纳结果
+opendirector --project PROJECT_UUID video revise-shot \
+  --timeline-editor TIMELINE_EDITOR_UUID --source-item SHOT_2_ITEM_UUID
+```
+
+### H3 shot orchestration, continuity audit, multi-frame QC, and single-shot timeline revision
+
+`opendirector video` supports first-frame (`--first-frame`), last-frame (`--last-frame`), first-and-last keyframe (`FL2VA` with both options), reference (`--reference-item`), and continuation (`--continue-from`) creation, paired with H3 shot-plan diversity checks, structured prompt compilation/linting, live project continuity auditing (`storyboard.audit`), multi-frame contact-sheet QC (`measuredFacts`), and surgical single-clip timeline replacement:
+
+- `video lint-shot-plan` checks 4–15s H3 bounds, canonical framing/camera vocabulary, and consecutive `(shotSize, cameraMotion)` repetition before creating nodes.
+- `video compile-h3-prompt` builds canonical `T2VA`, `I2VA`, `FL2VA`, `L2VA`, or 6-section `Ref2VA` prompts; `video lint-h3-prompt --spec-id` verifies alignment headers, `[Shot N] At MM:SS.mmm` cut timestamps, and `<Picture N>` / `<Video N>` / `<Audio N>` ordinals against the node's live active references (including embedded video audio offsets).
+- `video audit-storyboard` (`query storyboard.audit`) inspects the live `ProjectSnapshot` to detect unanchored `T2VA` shots alongside `Subject` anchors, unrendered upstream keyframes, excessive continuation chains, unlinked timeline clips, and pilot-gate pending video duration.
+- `video inspect-shot` (and `query asset.preview` with `videoScope="contact_sheet"`) renders a timestamped JPEG contact sheet for a registered project video `assetId` or local file and reports deterministic `measuredFacts` (`suspectedFrozen`, `blackFrameIndices`, `matchesH3Grid`, `hasAudio`) separately from creative judgment.
+- `video revise-shot` replaces only the target clip's `AssetRef` inside a `timeline` `EditorDocument` (by `--clip-id`/`--asset` or automatically following `--source-item` via `inputConnections`) using its exact `expectedVersions`, preserving neighbor clips, audio tracks, fonts, and timed text overlays.
+
+
 ## 检查已有生成结果
 
 `job show` / `job wait` 在任务成功后返回 `delivery`：已登记的 AssetRef、当前采用状态及展示该素材的画布项。生成成功不等于质量验收；Agent 应检查现有素材。复制到执行目录做媒体探测、抽帧和审看不需要重新发布，也不应为同一次交付新增画布项。编辑产生的新交付文件才需要登记；独立副本展示应有用户要求。

packages/agent/skills/h3-shot-director/SKILL.md

@@ -0,0 +1,21 @@
+---
+name: h3-shot-director
+description: Direct single-shot and multi-shot MiniMax-H3 videos using canonical cinematography vocabulary, structured T2VA/I2VA/FL2VA/L2VA/Ref2VA prompt compilation, pilot-anchor gates, contact-sheet QC, and surgical single-shot timeline revision.
+---
+
+# H3 Shot Director
+
+Translate creative requests or reference videos into executable, visually diverse MiniMax-H3 shot plans, canonical H3 prompts, verified media deliveries, and editable timelines through `opendirector` CLI commands on `PATH`.
+
+Planning or prompt drafting alone never authorizes submitting remote generation jobs. Separate planning receipts, generation execution facts, technical/visual QC, and user adoption.
+
+## End-to-End Workflow
+
+1. **Inspect Project & Reference Footage**: Run `opendirector project show` and `opendirector model list --kind video` first. When adapting a reference video, copy or read it inside the authorized execution directory and run `opendirector video inspect-shot <file> --contact-sheet ./ref-sheet.jpg` to extract stream facts (`durationSeconds`, `width`, `height`, `fps`, `hasAudio`, `matchesH3Grid`) and a timestamped visual contact sheet before planning shots.
+2. **Draft & Lint Anti-Homogenization Shot Plan**: Read [shot-orchestration.md](shot-orchestration.md). Produce a shot table specifying `shotSize`, `cameraMotion`, `durationSeconds` (`4–15s`), `mode` (`t2va`, `i2va`, `fl2va`, `l2va`, `ref2va`, or `continuation`), anchor assets, and audiovisual action. Validate the plan with `opendirector video lint-shot-plan --file ./shot-plan.json` so 3+ consecutive shots never repeat the same `(shotSize, cameraMotion)` pair.
+3. **Pilot Anchor & Project Continuity Gate**: For multi-shot sequences (3+ shots) or costly cloud runs, bind shared `Subject` or scene anchor assets across shots, generate/adopt keyframe anchors first, and audit the live project graph with `opendirector video audit-storyboard` (`query storyboard.audit`) to catch unanchored `T2VA` islands, unrendered upstream keyframes, or monotonous shot streaks before submitting batch video jobs.
+4. **Compile & Lint Canonical H3 Prompts**: Read [h3-vocabulary.md](h3-vocabulary.md). Write prompts strictly in H3's English section structure while preserving verbatim original language inside `<d>[Language] ...</d>` and quoted on-screen text. Use `opendirector video compile-h3-prompt --file ./shot-spec.json` to build structured prompts and `opendirector video lint-h3-prompt --spec-id <SPEC_UUID>` after `video create` to verify alignment headers, cut timestamps, and `<Picture N>` / `<Video N>` / `<Audio N>` ordinals (accounting for embedded audio tracks on reference videos).
+5. **Execute, Inspect Across Time & Assemble**: Create nodes with `opendirector video create` (`--first-frame`, `--last-frame`, `--reference-item`, or `--continue-from`), submit authorized jobs, wait with `job wait`, and inspect delivered project video assets across time via `opendirector video inspect-shot --asset <ASSET_UUID>` (or `query asset.preview` with `videoScope="contact_sheet"`). Report deterministic `measuredFacts` (`suspectedFrozen`, `blackFrameIndices`, `matchesH3Grid`, `hasAudio`) separately from your visual/creative review, without importing inspection contact sheets as project assets. Assemble verified clips onto an editable timeline (`editor create` / `editor save`) with `inputConnections` linking each `slotId` to its source shot item.
+6. **Surgical Single-Shot Local Revision**: When a user or QC review asks to reshoot or fix one shot in an assembled timeline, regenerate and adopt only that shot's replacement candidate, then run `opendirector video revise-shot --timeline-editor <EDITOR_UUID> --source-item <SHOT_ITEM_UUID>` (or `--clip-id <CLIP_ID> --asset <NEW_ASSET_UUID>`) so neighbor clips, audio tracks, fonts, and text overlays remain untouched.
+
+Resolve relative links (`h3-vocabulary.md`, `shot-orchestration.md`) from this skill directory. Keep user model, resolution, aspect-ratio, and audio choices intact unless explicitly asked to change them.

packages/agent/skills/h3-shot-director/h3-vocabulary.md

@@ -0,0 +1,71 @@
+# MiniMax-H3 Cinematography & Prompt Grammar
+
+MiniMax-H3 (`MiniMax-H3`, `MiniMax-H3-Max`, `MiniMax-H3-Highspeed`, `MiniMax-H3-Lite`, and `vllm_omni` / FastH3 deployments) expects structured English prompts with exact section order, natural-English camera actions, sequential `[Shot N]` markers, and strict per-modality reference ordinals.
+
+## 1. Choosing the H3 Mode
+
+| Mode | CLI `video create` flags | Prompt structure | Best use case |
+| --- | --- | --- | --- |
+| `t2va` | *(no media flags)* | 3 core sections (no alignment header) | Pure text-to-audiovisual shot when no visual anchor exists yet |
+| `i2va` | `--first-frame <ITEM>` | `I2VA` alignment line + blank line + 3 core sections | Forward action starting from an anchor keyframe at `0.00s` |
+| `fl2va` | `--first-frame <ITEM> --last-frame <ITEM>` | `FL2VA` alignment line + blank line + 3 core sections | Controlled state/pose/camera transition bridging two keyframes (prefer single shot) |
+| `l2va` | `--last-frame <ITEM>` | `L2VA` alignment line + blank line + 3 core sections | Plausible preceding action converging onto a target landing frame at `S.SSs` |
+| `ref2va` | `--reference-item <ITEM> ...` | 6 full-reference sections | Multi-subject identity consistency, voice timbre reference, or source-video structure/editing |
+| `continuation` | `--continue-from <VIDEO_ITEM>` | Continuity prompt extending prior latent tail | Seamless multi-segment extension on `vllm_omni` (`17n+5` frame grid at 24 fps) |
+
+Duration bounds are `4.0–15.0` seconds per H3 generation. On 24-fps FastH3 / `vllm_omni`, frames snap up to `17n + 5` (for example `5s -> 124 frames`, `7s -> 175 frames`).
+
+## 2. Exact Alignment Headers (`I2VA` / `FL2VA` / `L2VA`)
+
+Place the alignment instruction on line 1, followed by **one blank line**, where `N` is the final `[Shot N]` index and `S.SS` is the target duration to two decimals:
+
+- **I2VA**:
+  `For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.`
+- **FL2VA**:
+  `How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot N) aligns with the S.SS-second mark of the target video.`
+- **L2VA**:
+  `How the reference pictures align with the target video — <Picture 1> (from [Shot N]) aligns with the S.SS-second mark of the target video.`
+
+`T2VA` and `Ref2VA` never include a keyframe alignment header.
+
+## 3. Core Sections: Base (`T2VA` / `I2VA` / `FL2VA` / `L2VA`) vs `Ref2VA`
+
+### Base 3-Section Order
+1. `integrated_multimodal_description:` — Style + `[Shot 1]` (never timestamped) + optional `[Shot 2] At MM:SS.mmm, the camera cuts to ...`, including subject actions, camera motion, `(S1)` dialogue `<d>[Language] ...</d>`, and synchronized diegetic sound.
+2. `overall_soundscape:` — 1–4 sentences of continuous room tone, weather, footsteps, cloth, and non-verbal human sound across the clip (`N/A` only when explicit complete silence is requested). Never put `<d>` here.
+3. `non_diegetic_music:` — 1–3 sentences describing instrumentation, tempo, and dynamics of audience-only score (avoid abstract mood words; use `N/A` when unscored).
+
+### Ref2VA 6-Section Order
+1. `subject_definitions:` — One line per tracked `<Subject N>`, frame-anchor `<Picture N>`, structural `<Video N>`, or `<Audio N>`. Cite source images inside `<Subject N>` definitions when used only for identity/style.
+2. `summary:` — Must begin with `[task type]` combining `keyframe completion`, `reference generation`, `video editing`, `video continuation`, `audio reuse`, or `audio reference` with ` + `.
+3. `retention_analysis:` — One line per label using fixed markers (`fully_preserved`, `partially_preserved`, `attribute_transfer`, `weak_reference` for visual labels; `fully_copy`, `partially_copy`, `reference`, `weak_reference` for `<Audio N>`). Never write `(S1)` speaker IDs in `retention_analysis`.
+4. `detailed_description:` — 1–2 style sentences **before** `[Shot 1]`, followed by shot-by-shot playback description weaving `<Subject N>`, `<Picture N>`, `<Video N>`, `<Audio N>`, and `(Sx)` `<d>[Language] ...</d>`.
+5. `overall_soundscape:`
+6. `non_diegetic_music:`
+
+## 4. Reference Ordinal Rule (`<Audio N>` Offset)
+
+OpenDirector's FastH3 adapter numbers each modality in reference order, **reserving `<Audio 1..K>` for reference videos that contain an audio stream (`hasAudio: true`) before standalone audio references**:
+
+- Example: Reference 1 = `hero.mp4` (`hasAudio: true`), Reference 2 = `voice.wav` (`audio`).
+  - `hero.mp4` -> `<Video 1>` (and embedded soundtrack `<Audio 1>`)
+  - `voice.wav` -> `<Audio 2>` (not `<Audio 1>`!)
+- Always run `opendirector video lint-h3-prompt --spec-id <SPEC_UUID>` before `generation submit` to inspect `referenceOrdinals` and catch off-by-one `<Audio N>` tags automatically.
+
+## 5. Camera Vocabulary & Framing Variety
+
+Express camera movement as a **natural English action** inside the shot sentence rather than comma-separated tags at the end:
+
+- **Shot sizes**: `extreme wide shot`, `wide shot`, `medium-wide shot`, `medium shot`, `medium close-up`, `close shot`, `close-up`, `extreme close-up`, `over-the-shoulder shot`, `two-shot`, `low-angle shot`, `high-angle shot`, `overhead shot`, `bird's-eye shot`.
+- **12 H3 motion families**: `Zoom In` / `Zoom Out`, `Push In` / `Pull Out`, `Pan Left` / `Pan Right`, `Truck Left` / `Truck Right`, `Tilt Up` / `Tilt Down`, `Pedestal Up` / `Pedestal Down`, `Arc Shot`, `Tracking Shot`, `Static Shot`, `Shake Slightly` / `Shake Strongly`, `POV`, `Roll Clockwise` / `Roll Counterclockwise`.
+- **Amplitude & speed** (omit medium/normal): `with small amplitude` / `with large amplitude`, `at slow speed` / `at fast speed`.
+- Example: `The camera pushes in with small amplitude at slow speed toward the brass key in her palm.`
+
+## 6. Dialogue, Voiceover, and On-Screen Text
+
+- Assign `(S1)`, `(S2)` in order of first vocal event; reuse across shots. Non-speaking characters receive no speaker ID.
+- Keep verbatim spoken words and original language inside `<d>[Language] ...</d>`:
+  `The calm detective (S1) says: <d>[Chinese] 先别动那封信。</d>`
+- Off-screen voiceover must use `says in an off-screen voiceover:` and state closed lips immediately after `</d>`:
+  `The man (S1) says in an off-screen voiceover: <d>[English] We never returned.</d> while his lips remain completely closed.`
+- Use `<scenetrans>` on both sides when speech crosses an internal `[Shot N]` cut, and `<cutoff>` when truncated by clip end. Quote visible signage/text verbatim in English double quotes (for example `"24H OPEN"`).