Repository navigation
Commit e820b58
feat(video): accept a start frame image for fal.ai video models (#9761)
* feat(video): accept a start frame image for fal.ai video models
Fal models with an image-to-video sibling endpoint now take an optional
start frame. When one is attached, requests go to the image-to-video
endpoint with the image as a data URI in the model's image field, and the
block's settings dropdowns follow that endpoint's options.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(video): keep fal settings enabled without a start frame
- Drop the optional start frame fields from `dependsOn`: an array there
disables the field until every entry is filled, which greyed out the
settings for text-only runs. Options and conditions already re-read
all block values.
- Pass the run's abort signal through to the start frame download.
- The fal tool now offers `visualReference` to agents for models with an
image-to-video endpoint; update the provider selection test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(video): send the model default for fal settings a frame change invalidates
Attaching or removing a start frame switches a fal model between its
text-to-video and image-to-video endpoints, and the settings picked for
one can be invalid on the other (Seedance 2.5 takes 16:9 from a prompt
but only auto from a frame). The V3 block now drops any stored setting
the selected endpoint does not accept, so the run uses the model
default instead of failing validation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(video): drop only settings a start frame change made stale
Runway models also switch endpoints when a start frame is attached, and
their stored settings can be valid on one and not the other (Kling 4
takes 1920:1080 from a prompt but only auto ratios from a frame).
Both fal and direct providers now drop a setting only when the selected
endpoint rejects it and the model's other endpoint accepts it. Values
neither endpoint lists still reach the request builder, which maps
legacy Runway ratios such as 16:9 and rejects anything else.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(video): check fal start frame provenance before sending it to the model
A fal start frame now passes the same opaque model-input provenance check as a
Runway reference before its bytes reach the model.
* fix(video): pick fal settings from the active start frame input
The Reference Image is a basic upload / advanced reference pair, and the
serializer sends only the active member. The fal settings fields counted
a file left in the inactive member too, so switching to an empty
advanced input kept showing the start frame endpoint's settings while
the run called the prompt endpoint. Resolve the active member with
`resolveActiveCanonicalValue`, and pass `__canonicalModes` to dropdown
option functions as the editor already does for conditions, so a field's
visibility and its options agree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(video): reject a reference file that is not an image before submitting
An agent can pass any file as the start frame or Runway reference, and
nothing checked its type before the paid request. Validation now returns
a 400 for a non-image `visualReference` before the file is downloaded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Waleed Latif <walif6@gmail.com>1 parent 07a8d0c commit e820b58
17 files changed
Lines changed: 1304 additions & 189 deletions
File tree
- apps/sim
- app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/dropdown
- blocks/blocks
- lib
- internal/video
- media
- tools
- providers
- scripts
- serializer
- tools
- generated
- video
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
167 | | - | |
| 167 | + | |
| 168 | + | |
168 | 169 | | |
169 | 170 | | |
170 | 171 | | |
171 | 172 | | |
172 | 173 | | |
173 | | - | |
174 | | - | |
175 | | - | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
176 | 180 | | |
177 | 181 | | |
178 | 182 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
7 | | - | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
8 | 19 | | |
9 | 20 | | |
10 | 21 | | |
| |||
42 | 53 | | |
43 | 54 | | |
44 | 55 | | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
50 | 90 | | |
51 | 91 | | |
52 | 92 | | |
53 | 93 | | |
54 | 94 | | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
62 | 100 | | |
63 | 101 | | |
64 | 102 | | |
65 | 103 | | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
76 | 116 | | |
77 | 117 | | |
78 | | - | |
| 118 | + | |
79 | 119 | | |
80 | 120 | | |
81 | 121 | | |
| |||
109 | 149 | | |
110 | 150 | | |
111 | 151 | | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
112 | 195 | | |
113 | 196 | | |
114 | 197 | | |
| |||
141 | 224 | | |
142 | 225 | | |
143 | 226 | | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
144 | 238 | | |
145 | 239 | | |
146 | 240 | | |
147 | 241 | | |
148 | 242 | | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
| 243 | + | |
155 | 244 | | |
156 | 245 | | |
157 | 246 | | |
| |||
168 | 257 | | |
169 | 258 | | |
170 | 259 | | |
171 | | - | |
| 260 | + | |
172 | 261 | | |
173 | 262 | | |
174 | 263 | | |
| |||
302 | 391 | | |
303 | 392 | | |
304 | 393 | | |
305 | | - | |
| 394 | + | |
306 | 395 | | |
307 | | - | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
308 | 406 | | |
309 | 407 | | |
310 | 408 | | |
| |||
1950 | 2048 | | |
1951 | 2049 | | |
1952 | 2050 | | |
1953 | | - | |
1954 | | - | |
| 2051 | + | |
| 2052 | + | |
| 2053 | + | |
| 2054 | + | |
| 2055 | + | |
| 2056 | + | |
| 2057 | + | |
| 2058 | + | |
| 2059 | + | |
| 2060 | + | |
| 2061 | + | |
| 2062 | + | |
| 2063 | + | |
| 2064 | + | |
1955 | 2065 | | |
1956 | 2066 | | |
1957 | 2067 | | |
1958 | 2068 | | |
1959 | 2069 | | |
| 2070 | + | |
1960 | 2071 | | |
1961 | 2072 | | |
1962 | 2073 | | |
1963 | 2074 | | |
1964 | 2075 | | |
1965 | | - | |
| 2076 | + | |
| 2077 | + | |
| 2078 | + | |
| 2079 | + | |
| 2080 | + | |
| 2081 | + | |
1966 | 2082 | | |
1967 | 2083 | | |
1968 | 2084 | | |
1969 | 2085 | | |
1970 | 2086 | | |
1971 | 2087 | | |
1972 | 2088 | | |
1973 | | - | |
1974 | | - | |
| 2089 | + | |
| 2090 | + | |
| 2091 | + | |
| 2092 | + | |
| 2093 | + | |
| 2094 | + | |
| 2095 | + | |
| 2096 | + | |
| 2097 | + | |
| 2098 | + | |
1975 | 2099 | | |
1976 | 2100 | | |
1977 | 2101 | | |
| |||
0 commit comments