Repository navigation
[feat] agent: add H3 cinematography compiler, storyboard audit, contact-sheet QC, and shot revision - #342
Open
BeatSeat wants to merge 2 commits into
Open
[feat] agent: add H3 cinematography compiler, storyboard audit, contact-sheet QC, and shot revision#342BeatSeat wants to merge 2 commits into
BeatSeat wants to merge 2 commits into
Conversation
…ct-sheet QC, and shot revision ## summary - Add `opendirector.domain.h3_cinematography` encoding MiniMax-H3's canonical cinematography vocabulary (`SHOT_SIZES`, 20 `CAMERA_MOTIONS` across 12 families, natural-English amplitude/speed phrasing, dialogue and `Overall Soundscape` / `Non-Diegetic Music` structure, `17n+5` latent frame grid math, and `<Picture N>` / `<Video N>` / `<Audio N>` ordinal alignment with `fasth3.native_prompt` embedded video soundtracks), plus `compile_h3_prompt`, `lint_h3_prompt`, and anti-homogenization `lint_shot_plan`. - Extend `asset.preview` (`videoScope="contact_sheet"`) and `opendirector.media.shot_inspection` to render timestamped JPEG contact sheets for registered project video assets and local files while reporting deterministic `measuredFacts` (`meanFrameDelta`, `maxFrameDelta`, `suspectedFrozen`, `blackFrameIndices`, `matchesH3Grid`, `hasAudio`) separate from Agent creative review. - Add `storyboard.audit` (`opendirector.application.creation_storyboard`) across HTTP `/queries`, Agent `query`, and `opendirector video audit-storyboard` to audit cross-shot `Subject` anchor continuity, upstream keyframe readiness, continuation chain depth, unlinked timeline clips, and pilot-gate budgets on live `ProjectSnapshot` state. - Extend `opendirector video` (`create --last-frame/--continue-from`, `compile-h3-prompt`, `lint-h3-prompt`, `lint-shot-plan`, `audit-storyboard`, `inspect-shot --asset`, and lineage-aware `revise-shot`), ship the built-in `h3-shot-director` Agent skill, regenerate creation contracts, and update bilingual CLI documentation. ## validation - `python scripts/generate_contracts.py --check` - `lint-imports` - `pytest tests/test_h3_shot_orchestration.py tests/test_creation_contracts.py tests/test_skill_snapshot.py tests/test_agent_metadata.py tests/test_agent_gateway.py tests/test_cli.py -q` - `npm run check:i18n` Fixes: ThinkFlowLab#338 Refs: ThinkFlowLab#107, ThinkFlowLab#191, ThinkFlowLab#261, ThinkFlowLab#263
…contracts ## summary - Detect contradictory single-shot camera motions and CLI/SD pseudo-parameter tags outside `<d>...</d>` dialogue in `lint_h3_prompt`, extract compiled H3 camera motion verb phrases in `extract_prompt_shot_features`, and publish `vllmOmniFrames` alongside `gridFrames` in `h3_frame_grid`. - Unify H3 reference-source resolution across `storyboard.audit` and `opendirector video lint-h3-prompt` in `resolve_h3_spec_reference_sources`, preserving `promptContent` mention order, `first_frame` `context_only` filtering, and `assetId` deduplication. - Respect group-inherited canvas visibility in `audit_project_storyboard`, enforce mutually exclusive scopes and unique `specIds` in `StoryboardAuditQuery`, and regenerate TypeScript and JSON contracts. - Reuse `TIMELINE_DURATION_EPSILON`, preserve existing `EditorInputConnection` IDs on repeated slot revisions, standardize domain error messages, and gate real FFmpeg inspection with `@requires_ffmpeg()`. ## validation - `.venv/bin/python3 scripts/generate_contracts.py --check` - `PYTHONPATH=packages/core/src:apps/api/src .venv/bin/lint-imports` - `PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python3 -m pytest tests/test_h3_shot_orchestration.py tests/test_creation_contracts.py tests/test_skill_snapshot.py tests/test_agent_metadata.py tests/test_agent_gateway.py tests/test_cli.py -q` - `npm run check:i18n` Refs: ThinkFlowLab#338
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
opendirector.domain.h3_cinematographyfor canonical HappyHorse-1.0 (H3) shot compilation and deterministic linting across 7 canonical shot sizes, 20 camera motions in 12 motion families, natural-English speed/amplitude phrasing,17n+5frame-grid helpers, dialogue + soundscape linting,<Picture N>/<Video N>/<Audio N>ordinal alignment (accounting for embedded video audio tracks as wired inhappyhorse_local), and multi-shot anti-homogenization sequence checks.asset.preview(AssetPreviewQuery,CreationAssets.preview,/projects/{id}/assets/{asset_id}/preview, andAgentTools) withvideoScope="contact_sheet"(columns,rows), rendering timestamped multi-frame JPEG contact sheets (work/previews/{asset_id}-sheet-{columns}x{rows}.jpg) plus objectivemeasuredFacts(meanFrameDelta,maxFrameDelta,firstLastSimilarity,suspectedFrozen,blackFrameIndices,blackFrameRatio,matchesH3Grid,hasAudio) so Agents are no longer limited to Frame-0 video previews while keeping technical measurements strictly separate from creative acceptance.storyboard.audit(StoryboardAuditQuery+opendirector.application.creation_storyboard.audit_project_storyboard) exposed uniformly across HTTP/queries,AgentTools, andopendirector video audit-storyboardto check cross-shotSubjectcontinuity gaps, unrendered upstream reference keyframes, video continuation depth (> 3), unlinked timeline visual clips, live H3 prompt/ordinal regressions, sequence diversity, and pilot-gate pending video duration.opendirector videoCLI (createwith--last-frame,--continue-from,--aspect-ratio,--resolution,--audio,--title;compile-h3-prompt;lint-h3-prompt;lint-shot-plan;audit-storyboard;inspect-shotfor local files or--asset UUID; and lineage-awarerevise-shotresolving target timeline clips automatically viaTimelineDocument.inputConnectionsand returning non-destructivecanvas.materializecommands).h3-shot-directorskill (SKILL.md,h3-vocabulary.md,shot-orchestration.md), regeneratesrc/contracts/creation.schema.jsonandsrc/contracts/generated.ts, and document the new workflows bilingually indocs/guides/cli.md.Linked issue
Fixes #338
Refs #107, #191, #261, #263
Fixes #NorRefs #Nis included, or no issue appliesLabels
Suggested labels (external contributor cannot apply labels directly):
type:feat,area:agent,area:assets,area:timeline,area:docs,modality:video.Test plan
python3 scripts/generate_contracts.py --check(passed)PYTHONPATH=packages/core/src:apps/api/src lint-imports(passed: 137 files, 4 contracts kept, 0 broken)PYTHONPATH=packages/core/src:apps/api/src pytest tests/test_h3_shot_orchestration.py -q(10 passed, 1 skipped becauseffmpeg/ffprobebinaries are not installed in this container)PYTHONPATH=packages/core/src:apps/api/src pytest tests/test_creation_contracts.py tests/test_skill_snapshot.py tests/test_agent_metadata.py tests/test_agent_gateway.py tests/test_cli.py -q(141 passed)npm run check:i18n(passed: 3014 keys, 0 problems)npm run build(node_modules/tscnot installed in local environment)ffmpeg/ffprobeexecution (test_real_ffmpeg_shot_inspectionintests/test_h3_shot_orchestration.py)Evidence that it works
ToolGateway+ real PIL contact-sheet JPEG rendering +CliRunnerintests/test_h3_shot_orchestration.py.Provider verification
FakeHappyHorseAdapter/ deterministic frame probe & extractor); no live GPUhappyhorse_localworker or external commercial video API calls were executed in this environment.Bilingual UI
docs/guides/cli.mdupdated in bothzh-CNanden-US;npm run check:i18npassed with 0 problems).