feat(video): accept a start frame image for fal.ai video models by myxamediyar · Pull Request #9761 · simstudioai/sim

myxamediyar · 2026-11-07T01:03:51Z

Summary

Adds start frame (image-to-video) support to the fal.ai provider in the Video Generator block. 94 of the 112 catalog models have an image-to-video sibling endpoint, which is now recorded under an image key in fal-video-models.json along with its image field and the settings it accepts.

Request routing: when a start frame is attached, buildFalVideoRequest targets the model's image-to-video endpoint. The client downloads the image and sends it as a data URI in the model's field (image_url or start_image_url), with the same 5 MB limit Runway uses (now in a shared readReferenceImageDataUri helper).
Validation: a start frame on a model without an image-to-video endpoint is rejected with a clear error instead of "Unknown model". Legacy IDs that point to a catalog model use its image-to-video endpoint; old IDs outside the catalog reject a start frame.
Block UI: the start frame upload/input shows for fal models that support it. The duration, aspect ratio, resolution, audio and prompt optimizer dropdowns follow the endpoint that will actually run, and params the image endpoint doesn't accept are dropped.
Pricing: image-to-video endpoints now use the video fallback cost floor, not the image floor.
Tool: video_falai takes an optional visualReference file param. Tool metadata is regenerated.
Schema check: validate-fal-video-models.ts now checks each model with and without a start frame against fal's published schemas.

Type of Change

Testing

New client.test.ts cases: a start frame goes to the image-to-video endpoint for Kling v3 Pro (start_image_url) and Veo 3.1 (image_url), and models without one are rejected
New falai-pricing.test.ts cases for image-to-video fallback floors
vitest for lib/internal/video and falai-pricing (65 passing) and tsc --noEmit both pass

Reviewers: please check the conditional field visibility in video_generator.ts, especially legacy fal models and switching models after a frame is attached.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

vercel · 2026-11-07T01:03:56Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Nov 7, 2026 2:10am UTC

myxamediyar · 2026-11-07T01:06:38Z

@greptile

myxamediyar · 2026-11-07T01:06:40Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T01:06:44Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

greptile-apps · 2026-11-07T01:10:37Z

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no new blocking issues found.

Summary

Adds optional start frame images to supported Fal.ai video models.

Fal.ai videos can start from an image on supported models.
The Video Generator shows frame inputs and settings for the selected endpoint.
Fal.ai image-to-video endpoints use the video fallback cost floor.
The Fal.ai video tool accepts an optional start-frame file.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Video request] --> B{Start frame attached?}
  B -->|No| C[Text-to-video endpoint]
  B -->|Yes| D[Check image type and file access]
  D --> E[Read image within size limit]
  E --> F[Image-to-video endpoint]
  C --> G[Poll and store video]
  F --> G

_{Reviews (9) · Last reviewed commit: "fix(video): reject a reference file that..." · Reviewed by Greptile}

cubic-dev-ai

1 issue found across 11 files

Confidence score: 4/5

In video_generator.ts, legacy Fal IDs are treated as start-frame-capable, so the frame fields can appear for the image variant. Base the capability check on the resolved model variant.

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/sim/blocks/blocks/video_generator.ts">

<violation number="1" location="apps/sim/blocks/blocks/video_generator.ts:349">
P2: Legacy Fal model IDs are still treated as start-frame-capable. Because `getFalVideoModel` preserves `legacyId` while exposing the image variant, this condition shows the frame fields, and the V3 parameter guard keeps the frame for submission. Exclude `model.legacyId` from both predicates and clear/reject the frame for those IDs.</violation>
</file>

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

myxamediyar · 2026-11-07T01:21:30Z

@greptile

myxamediyar · 2026-11-07T01:21:31Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T01:21:37Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 12 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Turn on auto-fix | Re-trigger cubic}

myxamediyar · 2026-11-07T01:29:48Z

@greptile

myxamediyar · 2026-11-07T01:29:49Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T01:29:54Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

greptile-apps · 2026-11-07T01:33:37Z

Want your agent to iterate on Greptile's feedback? Try greploops.

cubic-dev-ai

All reported issues were addressed across 13 files

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

myxamediyar · 2026-11-07T01:42:14Z

@greptile

myxamediyar · 2026-11-07T01:42:15Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T01:42:20Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 13 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 · 2026-11-07T01:48:59Z

@greptile

waleedlatif1 · 2026-11-07T01:49:00Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T01:49:04Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 15 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

myxamediyar · 2026-11-07T01:54:14Z

@greptile

myxamediyar · 2026-11-07T01:54:15Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T01:54:20Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 15 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

Fal models with an image-to-video sibling endpoint now take an optional start frame. When one is attached, requests go to the image-to-video endpoint with the image as a data URI in the model's image field, and the block's settings dropdowns follow that endpoint's options. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- Drop the optional start frame fields from `dependsOn`: an array there disables the field until every entry is filled, which greyed out the settings for text-only runs. Options and conditions already re-read all block values. - Pass the run's abort signal through to the start frame download. - The fal tool now offers `visualReference` to agents for models with an image-to-video endpoint; update the provider selection test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…validates Attaching or removing a start frame switches a fal model between its text-to-video and image-to-video endpoints, and the settings picked for one can be invalid on the other (Seedance 2.5 takes 16:9 from a prompt but only auto from a frame). The V3 block now drops any stored setting the selected endpoint does not accept, so the run uses the model default instead of failing validation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Runway models also switch endpoints when a start frame is attached, and their stored settings can be valid on one and not the other (Kling 4 takes 1920:1080 from a prompt but only auto ratios from a frame). Both fal and direct providers now drop a setting only when the selected endpoint rejects it and the model's other endpoint accepts it. Values neither endpoint lists still reach the request builder, which maps legacy Runway ratios such as 16:9 and rejects anything else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… model A fal start frame now passes the same opaque model-input provenance check as a Runway reference before its bytes reach the model.

The Reference Image is a basic upload / advanced reference pair, and the serializer sends only the active member. The fal settings fields counted a file left in the inactive member too, so switching to an empty advanced input kept showing the start frame endpoint's settings while the run called the prompt endpoint. Resolve the active member with `resolveActiveCanonicalValue`, and pass `__canonicalModes` to dropdown option functions as the editor already does for conditions, so a field's visibility and its options agree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

myxamediyar · 2026-11-07T02:03:43Z

@greptile

myxamediyar · 2026-11-07T02:03:44Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T02:03:50Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 17 files

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

…tting An agent can pass any file as the start frame or Runway reference, and nothing checked its type before the paid request. Validation now returns a 400 for a non-image `visualReference` before the file is downloaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

myxamediyar · 2026-11-07T02:10:18Z

@greptile

myxamediyar · 2026-11-07T02:10:20Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-11-07T02:10:25Z

@cubic-dev-ai review this PR

@myxamediyar I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 17 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic}

greptile-apps Bot reviewed Nov 7, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/video_generator.ts Resolved

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

Comment thread apps/sim/lib/internal/video/client.ts Resolved

Comment thread apps/sim/lib/media/fal-video-models.json Resolved

Comment thread apps/sim/blocks/blocks/video_generator.ts Resolved

vercel Bot temporarily deployed to Preview November 7, 2026 01:19 Inactive

greptile-apps Bot reviewed Nov 7, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/video_generator.ts Resolved

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

myxamediyar marked this pull request as ready for review November 7, 2026 01:28

myxamediyar assigned mzxchandra Nov 7, 2026

vercel Bot temporarily deployed to Preview November 7, 2026 01:29 Inactive

waleedlatif1 self-assigned this Nov 7, 2026

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/video_generator.ts Resolved

vercel Bot temporarily deployed to Preview November 7, 2026 01:42 Inactive

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview November 7, 2026 01:48 Inactive

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

myxamediyar force-pushed the feat/fal-video-start-frame branch from d877e42 to e2e424c Compare November 7, 2026 01:54

vercel Bot temporarily deployed to Preview November 7, 2026 01:54 Inactive

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

greptile-apps Bot reviewed Nov 7, 2026

View reviewed changes

Comment thread apps/sim/blocks/blocks/video_generator.ts Resolved

myxamediyar and others added 6 commits November 6, 2026 17:59

fix(video): check fal start frame provenance before sending it to the…

b548760

… model A fal start frame now passes the same opaque model-input provenance check as a Runway reference before its bytes reach the model.

myxamediyar force-pushed the feat/fal-video-start-frame branch from e2e424c to 0dbb2a6 Compare November 7, 2026 02:03

vercel Bot temporarily deployed to Preview November 7, 2026 02:03 Inactive

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

Comment thread apps/sim/lib/internal/video/client.ts Resolved

vercel Bot temporarily deployed to Preview November 7, 2026 02:10 Inactive

cubic-dev-ai Bot reviewed Nov 7, 2026

View reviewed changes

Conversation

myxamediyar commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 7, 2026

Uh oh!

greptile-apps Bot commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 7, 2026

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 7, 2026

Uh oh!

greptile-apps Bot commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Nov 7, 2026

Uh oh!

waleedlatif1 commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

myxamediyar commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot commented Nov 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

myxamediyar commented Nov 7, 2026 •

edited

Loading

vercel Bot commented Nov 7, 2026 •

edited

Loading

greptile-apps Bot commented Nov 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading