Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions packages/sdk/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@
- `lucy-2.1` - Real-time video editing (supports reference image)
- `lucy-2.5` - Real-time video editing (supports reference image)
- `lucy-vton-3.5` - Real-time virtual try-on 3.5
- `lucy-vton-3.6` - Real-time virtual try-on 3.6 (opt-in by name; no `speed: "fast"` tier)
- `lucy-restyle-2` - Real-time video restyling
- `lucy-latest`, `lucy-vton-latest`, `lucy-restyle-latest` - Server-resolved aliases for the latest stable version (`lucy-vton-latest` → `lucy-vton-3.5`)

Expand All @@ -84,9 +85,71 @@
- `lucy-2.1` - long-form video editing (720p)
- `lucy-2.5` - long-form video editing (720p)
- `lucy-vton-3.5` - virtual try-on 3.5 video editing
- `lucy-vton-3.6` - virtual try-on 3.6 video editing (opt-in by name)
- `lucy-restyle-2` - video restyling
- `lucy-latest`, `lucy-vton-latest`, `lucy-restyle-latest`, `lucy-clip-latest` - Server-resolved aliases (`lucy-vton-latest` → `lucy-vton-3.5`)

### Image Models (Process API)
- `lucy-image-2` - image-to-image editing
- `lucy-image-latest` - Server-resolved alias

## Adding a Model

`src/shared/model.ts` is the shared source of truth for the Decart client SDKs — the Python, Swift, Kotlin and
C++ SDKs each carry a test asserting their registry matches this one. Land a new model here first, release, then
mirror it.

### Before writing any code: confirm what the API actually serves

Every registry value must be read off the deployed model, not copied from a sibling:

| Registry field | Must match |
| --- | --- |
| `width` / `height` | the resolution the API serves for that model |
| realtime vs video entry | which surfaces the model is enabled on |
| `queueUrlPath` | the live `/v1/jobs/<name>` route |
| `supportedSpeeds` | the speed tiers the API actually offers — **omit the field when there are none** |
| `inputSchema` | the input types the model accepts |

`lucy-vton-3.5` was landed twice because the first attempt advertised a resolution that did not match how the
model was served; it was reverted across all five SDK repos the next day. Confirm geometry before pinning it.

### The nine sites in `src/shared/model.ts`

1. `CANONICAL_MODEL_NAMES`
2. `CANONICAL_REALTIME_MODEL_NAMES` (realtime models only)
3. `CANONICAL_VIDEO_MODEL_NAMES` (video models only)
4. the `realtimeModels` zod union
5. the `videoModels` zod union
6. `modelInputSchemas`
7. the `_models.realtime` registry entry
8. the `_models.video` registry entry
9. the JSDoc option lists on `models.realtime()` / `models.video()`

Then `src/process/types.ts` — add the name to the right branch of `ModelSpecificInputs`, and `AGENTS.md`
(Supported Models) + `index.html` (playground `<select>`).

Do **not** add to `modelAliases` — that map is only for deprecated names that log a warning. `-latest` names are
deliberately absent from it, and a unit test asserts they never warn.

### `supportedSpeeds` must match the service

`supportedSpeeds: ["fast"]` is a claim that the API will serve that model on the `fast` tier and price it
accordingly. Setting it for a model that does not offer the tier is not a harmless hint — the connection is
rejected at billing time. Only list a tier once the model is confirmed to be served on it.

### `-latest` aliases

`lucy-latest` / `lucy-vton-latest` / `lucy-restyle-latest` / `lucy-clip-latest` resolve **server-side**. A new
model is opt-in by explicit name until the service repoints the alias. When that happens, update the alias
entry's geometry here in the same release, or the SDK advertises a capture size the alias is not serving.

### Checklist

- [ ] `pnpm typecheck && pnpm test && pnpm build && pnpm test:package && pnpm format:check`
- [ ] `listModels()` length assertion in `tests/unit.test.ts` bumped (+1 per surface)
- [ ] canonical option arrays in `tests/unit.test.ts` updated
- [ ] `tests/e2e.test.ts` / `tests/e2e-realtime.test.ts` coverage added
- [ ] `chore: release vX.Y.Z` commit (version in `packages/sdk/package.json` only) + `vX.Y.Z` tag — there are no
changesets in this repo; `release.yml` publishes and runs `changelogithub` off conventional-commit titles
- [ ] sibling PRs in `decart-python`, `decart-ios`, `decart-android`, `decart-cpp` (same title)
3 changes: 2 additions & 1 deletion packages/sdk/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,7 @@ <h3>Configuration</h3>
<option value="lucy-2.1" selected>Lucy 2.1</option>
<option value="lucy-2.5">Lucy 2.5</option>
<option value="lucy-vton-3.5">Lucy VTON 3.5</option>
<option value="lucy-vton-3.6">Lucy VTON 3.6</option>
<option value="lucy-restyle-2">Lucy Restyle 2</option>
</optgroup>
<optgroup label="Override">
Expand All @@ -353,7 +354,7 @@ <h3>Configuration</h3>
</div>
<div class="control-group" id="custom-model-group" style="display: none;">
<label for="custom-model-name">Custom model name:</label>
<input type="text" id="custom-model-name" placeholder="e.g. lucy-vton-3.6">
<input type="text" id="custom-model-name" placeholder="e.g. lucy-vton-4">
<div class="inline-controls">
<label style="margin: 0; font-weight: normal;">Width <input type="number" id="custom-model-width" min="1" step="1" value="1280" style="width: 90px;"></label>
<label style="margin: 0; font-weight: normal;">Height <input type="number" id="custom-model-height" min="1" step="1" value="720" style="width: 90px;"></label>
Expand Down
2 changes: 1 addition & 1 deletion packages/sdk/src/process/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ export type ModelSpecificInputs<T extends ModelDefinition> = T["name"] extends "
? ImageEditingInputs
: T["name"] extends "lucy-restyle-v2v" | "lucy-restyle-2"
? VideoRestyleInputs
: T["name"] extends "lucy-2.1" | "lucy-2.5" | "lucy-vton-3.5" | "lucy-vton-latest"
: T["name"] extends "lucy-2.1" | "lucy-2.5" | "lucy-vton-3.5" | "lucy-vton-3.6" | "lucy-vton-latest"
? VideoEdit2Inputs
: T["name"] extends "lucy-pro-v2v" | "lucy-clip"
? VideoEditInputs
Expand Down
44 changes: 40 additions & 4 deletions packages/sdk/src/shared/model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,27 @@ const CANONICAL_MODEL_NAMES = [
"lucy-2.1",
"lucy-2.5",
"lucy-vton-3.5",
"lucy-vton-3.6",
"lucy-restyle-2",
"lucy-clip",
"lucy-image-2",
] as const;

const CANONICAL_REALTIME_MODEL_NAMES = ["lucy-2.1", "lucy-2.5", "lucy-vton-3.5", "lucy-restyle-2"] as const;
const CANONICAL_VIDEO_MODEL_NAMES = ["lucy-clip", "lucy-2.1", "lucy-2.5", "lucy-vton-3.5", "lucy-restyle-2"] as const;
const CANONICAL_REALTIME_MODEL_NAMES = [
"lucy-2.1",
"lucy-2.5",
"lucy-vton-3.5",
"lucy-vton-3.6",
"lucy-restyle-2",
] as const;
const CANONICAL_VIDEO_MODEL_NAMES = [
"lucy-clip",
"lucy-2.1",
"lucy-2.5",
"lucy-vton-3.5",
"lucy-vton-3.6",
"lucy-restyle-2",
] as const;
const CANONICAL_IMAGE_MODEL_NAMES = ["lucy-image-2"] as const;

export const canonicalRealtimeModels = z.enum(CANONICAL_REALTIME_MODEL_NAMES);
Expand Down Expand Up @@ -53,6 +67,7 @@ export const realtimeModels = z.union([
z.literal("lucy-2.1"),
z.literal("lucy-2.5"),
z.literal("lucy-vton-3.5"),
z.literal("lucy-vton-3.6"),
z.literal("lucy-restyle-2"),
// Latest aliases (server-side resolution)
z.literal("lucy-latest"),
Expand All @@ -65,6 +80,7 @@ export const videoModels = z.union([
z.literal("lucy-2.1"),
z.literal("lucy-2.5"),
z.literal("lucy-vton-3.5"),
z.literal("lucy-vton-3.6"),
z.literal("lucy-restyle-2"),
// Latest aliases (server-side resolution)
z.literal("lucy-latest"),
Expand Down Expand Up @@ -240,6 +256,7 @@ export const modelInputSchemas = {
"lucy-2.1": videoEdit2Schema,
"lucy-2.5": videoEdit2Schema,
"lucy-vton-3.5": videoEdit2Schema,
"lucy-vton-3.6": videoEdit2Schema,
// Latest aliases (server-side resolution)
"lucy-latest": videoEdit2Schema,
"lucy-vton-latest": videoEdit2Schema,
Expand Down Expand Up @@ -354,6 +371,14 @@ const _models = {
inputSchema: z.object({}),
supportedSpeeds: ["fast"],
},
"lucy-vton-3.6": {
urlPath: "/v1/stream",
name: "lucy-vton-3.6" as const,
fps: { ideal: 30, max: 30 },
width: 1280,
height: 720,
inputSchema: z.object({}),
},
"lucy-restyle-2": {
urlPath: "/v1/stream",
name: "lucy-restyle-2" as const,
Expand Down Expand Up @@ -461,6 +486,15 @@ const _models = {
height: 720,
inputSchema: modelInputSchemas["lucy-vton-3.5"],
},
"lucy-vton-3.6": {
urlPath: "/v1/generate/lucy-vton-3.6",
queueUrlPath: "/v1/jobs/lucy-vton-3.6",
name: "lucy-vton-3.6" as const,
fps: 20,
width: 1280,
height: 720,
inputSchema: modelInputSchemas["lucy-vton-3.6"],
},
"lucy-restyle-2": {
urlPath: "/v1/generate/lucy-restyle-2",
queueUrlPath: "/v1/jobs/lucy-restyle-2",
Expand Down Expand Up @@ -565,7 +599,8 @@ export const models = {
* Available options:
* - `"lucy-2.1"` - Lucy 2.1 realtime video editing
* - `"lucy-2.5"` - Lucy 2.5 realtime video editing
* - `"lucy-vton-3.5"` - Lucy virtual try-on 3.5 (latest)
* - `"lucy-vton-3.5"` - Lucy virtual try-on 3.5
* - `"lucy-vton-3.6"` - Lucy virtual try-on 3.6
* - `"lucy-restyle-2"` - Realtime video restyling
*/
realtime: <T extends RealTimeModels>(model: T): ModelDefinition<T> => {
Expand All @@ -583,7 +618,8 @@ export const models = {
* - `"lucy-clip"` - Video-to-video editing
* - `"lucy-2.1"` - Long-form video editing (Lucy 2.1)
* - `"lucy-2.5"` - Long-form video editing (Lucy 2.5)
* - `"lucy-vton-3.5"` - Virtual try-on 3.5 video editing (latest)
* - `"lucy-vton-3.5"` - Virtual try-on 3.5 video editing
* - `"lucy-vton-3.6"` - Virtual try-on 3.6 video editing
* - `"lucy-restyle-2"` - Video restyling
*/
video: <T extends VideoModels>(model: T): ModelDefinition<T> & { fps: number; queueUrlPath: string } => {
Expand Down
1 change: 1 addition & 0 deletions packages/sdk/tests/e2e-realtime.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ const REALTIME_MODELS: RealTimeModels[] = [
"lucy-2.1",
"lucy-2.5",
"lucy-vton-3.5",
"lucy-vton-3.6",
"lucy-latest",
];

Expand Down
12 changes: 12 additions & 0 deletions packages/sdk/tests/e2e.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,18 @@ describe.concurrent("E2E Tests", { timeout: TIMEOUT, retry: 2 }, () => {
await expectResult(result, "lucy-vton-3.5", ".mp4");
});

it("lucy-vton-3.6: virtual try-on (prompt + reference_image)", async () => {
const result = await client.queue.submitAndPoll({
model: models.video("lucy-vton-3.6"),
prompt: "Wearing the garment from the reference image",
reference_image: garmentBlob,
data: videoBlob,
seed: 42,
});

await expectResult(result, "lucy-vton-3.6", ".mp4");
});

it("lucy-latest: video editing", async () => {
const result = await client.queue.submitAndPoll({
model: models.video("lucy-latest"),
Expand Down
32 changes: 30 additions & 2 deletions packages/sdk/tests/unit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2586,12 +2586,19 @@ describe("Canonical Model Names", () => {
expect(canonicalModelSchema.safeParse(alias).success).toBe(false);
}

expect(canonicalRealtimeModels.options).toEqual(["lucy-2.1", "lucy-2.5", "lucy-vton-3.5", "lucy-restyle-2"]);
expect(canonicalRealtimeModels.options).toEqual([
"lucy-2.1",
"lucy-2.5",
"lucy-vton-3.5",
"lucy-vton-3.6",
"lucy-restyle-2",
]);
expect(canonicalVideoModels.options).toEqual([
"lucy-clip",
"lucy-2.1",
"lucy-2.5",
"lucy-vton-3.5",
"lucy-vton-3.6",
"lucy-restyle-2",
]);
expect(canonicalImageModels.options).toEqual(["lucy-image-2"]);
Expand Down Expand Up @@ -2651,7 +2658,7 @@ describe("Canonical Model Names", () => {
it("lists all models when called without options", () => {
const listedModels = listModels();

expect(listedModels).toHaveLength(21);
expect(listedModels).toHaveLength(23);
expect(listedModels.some((model) => model.kind === "realtime" && model.name === "lucy-2.1")).toBe(true);
expect(listedModels.some((model) => model.kind === "video" && model.name === "lucy-clip")).toBe(true);
expect(listedModels.some((model) => model.kind === "image" && model.name === "lucy-image-2")).toBe(true);
Expand Down Expand Up @@ -2734,6 +2741,17 @@ describe("Canonical Model Names", () => {
expect(model.height).toBe(720);
});

it("lucy-vton-3.6 canonical name works", () => {
const model = models.realtime("lucy-vton-3.6");
expect(model.name).toBe("lucy-vton-3.6");
expect(model.urlPath).toBe("/v1/stream");
expect(model.fps).toEqual({ ideal: 30, max: 30 });
expect(model.width).toBe(1280);
expect(model.height).toBe(720);
// 3.6 is not offered on the fast tier, unlike 3.5.
expect(model.supportedSpeeds).toBeUndefined();
});

it("lucy-restyle-2 canonical name works", () => {
const model = models.realtime("lucy-restyle-2");
expect(model.name).toBe("lucy-restyle-2");
Expand Down Expand Up @@ -2777,6 +2795,16 @@ describe("Canonical Model Names", () => {
expect(model.height).toBe(720);
});

it("lucy-vton-3.6 as video model works", () => {
const model = models.video("lucy-vton-3.6");
expect(model.name).toBe("lucy-vton-3.6");
expect(model.urlPath).toBe("/v1/generate/lucy-vton-3.6");
expect(model.queueUrlPath).toBe("/v1/jobs/lucy-vton-3.6");
expect(model.fps).toBe(20);
expect(model.width).toBe(1280);
expect(model.height).toBe(720);
});

it("lucy-restyle-2 as video model works", () => {
const model = models.video("lucy-restyle-2");
expect(model.name).toBe("lucy-restyle-2");
Expand Down
Loading