Decart iOS SDK

Native Swift SDK for Decart AI - Real-time video processing and AI generation for iOS & macOS.

Overview

Decart iOS SDK provides three primary APIs:

• RealtimeManager - Real-time video processing with LiveKit streaming, automatic reconnection, and generation tracking
• QueueClient - Async video generation via the Queue API (/v1/jobs/*) with submit, poll, and download
• ProcessClient - Synchronous image generation

All APIs leverage modern Swift concurrency (async/await) with type-safe interfaces and comprehensive error handling.

Features

• Real-time video processing with LiveKit
• Automatic reconnection with exponential backoff
• Generation tick events for billing/usage tracking
• Async Queue API for video generation (submit → poll → download)
• Synchronous image generation
• Native Swift with modern concurrency (async/await)
• AsyncStream events for reactive state management
• Type-safe API with compile-time guarantees
• iOS 15+ and macOS 12+ support
• SwiftUI ready

Installation

Swift Package Manager

Add to your Package.swift:

dependencies: [
    .package(url: "https://github.com/decartai/decart-ios.git", from: "0.7.1")
]

Or in Xcode:

1. File → Add Package Dependencies
2. Enter: https://github.com/decartai/decart-ios
3. Select version and add to target

Usage Examples

1. Real-time Video Processing

Stream video with real-time AI processing using LiveKit:

import DecartSDK
import LiveKit

let config = DecartConfiguration(apiKey: "your-api-key")
let client = DecartClient(decartConfiguration: config)

let model: RealtimeModel = .lucyRestyle2
let modelConfig = Models.realtime(model)

let realtimeManager = try client.createRealtimeManager(
    options: RealtimeConfiguration(
        model: modelConfig,
        initialPrompt: DecartPrompt(text: "Lego World")
    )
)

// Listen to connection events
Task {
    for await state in realtimeManager.events {
        print("State: \(state.connectionState)")
        if let tick = state.generationTick {
            print("Generation: \(tick)s")
        }
        if let sessionId = state.sessionId {
            print("Session: \(sessionId)")
        }
    }
}

// Rebind tracks after automatic reconnection
Task {
    for await newRemoteStream in realtimeManager.remoteStreamUpdates {
        print("Got new remote stream after reconnect")
        // Update your UI with newRemoteStream.videoTrack
    }
}

// Create a LiveKit camera track and connect
let captureOptions = CameraCaptureOptions(
    position: .front,
    dimensions: Dimensions(width: Int32(modelConfig.height), height: Int32(modelConfig.width)),
    fps: modelConfig.fps
)
// `mirror: .auto` pre-flips the front camera so the server receives frames in
// display orientation (see "Front-camera mirroring" below). Pass `.off` to disable.
let mirror = MirroringVideoProcessor(mode: .auto)
let videoTrack = LocalVideoTrack.createCameraTrack(name: "video0", options: captureOptions, processor: mirror)
let localStream = RealtimeMediaStream(videoTrack: videoTrack, id: .localStream)
let remoteStream = try await realtimeManager.connect(localStream: localStream)

// Update prompt in real-time (suspends until the server acks)
do {
    try await realtimeManager.setPrompt(DecartPrompt(text: "Anime World"))
} catch {
    // ack failure, timeout, or websocket disconnect while the call was pending
}

// Cleanup
try? await videoTrack.stop()
await realtimeManager.disconnect()

Front-camera mirroring

Front cameras look natural only when mirrored, but mirroring just the local preview leaves the AI output reversed — and flipping the output would also flip any server-baked content (e.g. watermarks). MirroringVideoProcessor pre-flips frames before encoding, so the server works in display orientation and both the local preview and remote stream render as-is:

let mirror = MirroringVideoProcessor(mode: .auto)
let videoTrack = LocalVideoTrack.createCameraTrack(name: "video0", options: captureOptions, processor: mirror)

• .off (default) — never mirror.
• .auto — mirror only when the active camera is .front. When switching cameras, update mirror.cameraPosition = cameraCapturer.position so it follows the camera.
• .on — always mirror.

Render both streams with RTCMLVideoViewWrapper(track:) — no mirror: argument, since the frames are already in display orientation:

RTCMLVideoViewWrapper(track: localStream.videoTrack)
RTCMLVideoViewWrapper(track: remoteStream.videoTrack)

Output resolution

Realtime models default to a 720p remote stream. Pass resolution: .p1080 on the RealtimeConfiguration to request 1080p instead.

let realtimeManager = try client.createRealtimeManager(
    options: RealtimeConfiguration(
        model: modelConfig,
        resolution: .p1080 // default: nil (server-side default of 720p)
    )
)

Fast mode

Fast mode (speed: .fast) serves the session from a higher-compute tier for lower latency and higher throughput; output quality is unchanged. It is currently available for lucy-2.5 / lucy-latest and lucy-vton-3.5 / lucy-vton-latest, in the US region only, and is billed at 2x the standard realtime rate for those models. Other models ignore the option. Omit it (the default) for standard mode.

Models that offer fast mode list it in ModelDefinition.supportedSpeeds; passing speed for any other model logs a warning and the server serves the session in standard mode. See the platform docs for details.

let modelConfig = Models.realtime(.lucy2_5)
let fastModeAvailable = modelConfig.supportedSpeeds.contains(.fast)

let realtimeManager = try client.createRealtimeManager(
    options: RealtimeConfiguration(
        model: modelConfig,
        speed: fastModeAvailable ? .fast : nil // default: nil (standard mode)
    )
)

2. Image-to-Image Generation

Transform images with AI:

import DecartSDK

let config = DecartConfiguration(apiKey: "your-api-key")
let client = DecartClient(decartConfiguration: config)

let imageData = try Data(contentsOf: referenceImageURL)
let fileInput = try FileInput.image(data: imageData)

let input = try ImageToImageInput(prompt: "Make it cyberpunk", data: fileInput)

let processClient = try client.createProcessClient(
    model: .lucy_pro_i2i,
    input: input
)

let resultData = try await processClient.process()
let image = UIImage(data: resultData)

3. Video Generation (Queue API)

Generate videos using the Queue API — handles long-running jobs without HTTP timeouts.

import DecartSDK

let config = DecartConfiguration(apiKey: "your-api-key")
let client = DecartClient(decartConfiguration: config)

// Video to video
let videoData = try Data(contentsOf: referenceVideoURL)
let fileInput = try FileInput.video(data: videoData)
let input = try VideoToVideoInput(prompt: "Apply anime style", data: fileInput)

let result = try await client.queue.submitAndPoll(model: .lucy_pro_v2v, input: input) { status in
    print("Job \(status.jobId): \(status.status)")
}

switch result {
case .completed(let jobId, let data):
    try data.write(to: outputURL)
case .failed(let jobId, let error):
    print("Failed: \(error)")
}

Step-by-step control

// Submit and manage the job manually
let submitResponse = try await client.queue.submit(model: .lucy_pro_v2v, input: input)
print("Job ID: \(submitResponse.jobId)")

// Poll status
let statusResponse = try await client.queue.status(jobId: submitResponse.jobId)

// Download result when complete
if statusResponse.status == .completed {
    let videoData = try await client.queue.result(jobId: submitResponse.jobId)
}

Check status of any job

let status = try await client.queue.status(jobId: "some-job-id")
let data = try await client.queue.result(jobId: "some-job-id")

5. Video Edit (lucy-2.1)

Edit a video with a prompt and optional reference image:

let videoData = try Data(contentsOf: videoURL)
let videoFile = try FileInput.video(data: videoData)

let input = try VideoEditInput(prompt: "Add snow", data: videoFile)
let result = try await client.queue.submitAndPoll(model: .lucy2_1, input: input)

6. Video Restyle (lucy-restyle-v2v)

Restyle a video using either a text prompt or a reference image (mutually exclusive):

let videoData = try Data(contentsOf: videoURL)
let videoFile = try FileInput.video(data: videoData)

// With prompt
let input = try VideoRestyleInput(prompt: "Studio Ghibli style", data: videoFile)

// Or with reference image
let refData = try Data(contentsOf: referenceURL)
let refImage = try FileInput.image(data: refData)
let input = try VideoRestyleInput(data: videoFile, referenceImage: refImage)

let result = try await client.queue.submitAndPoll(model: .lucy_restyle_v2v, input: input)

API Reference

DecartConfiguration

let config = DecartConfiguration(
    baseURL: "https://api3.decart.ai",  // Optional
    apiKey: "your-api-key"
)

DecartClient

let client = DecartClient(decartConfiguration: config)

// Create realtime manager
func createRealtimeManager(options: RealtimeConfiguration) throws -> RealtimeManager

// Create process clients (synchronous image generation)
func createProcessClient(model: ImageModel, input: ImageToImageInput) throws -> ProcessClient

// Queue client for async video generation (recommended)
var queue: QueueClient { get }

QueueClient

// Submit a job (one overload per input type)
func submit(model: VideoModel, input: VideoToVideoInput) async throws -> JobSubmitResponse
func submit(model: VideoModel, input: VideoEditInput) async throws -> JobSubmitResponse
func submit(model: VideoModel, input: VideoRestyleInput) async throws -> JobSubmitResponse

// Poll / download
func status(jobId: String) async throws -> JobStatusResponse
func result(jobId: String) async throws -> Data

// Submit + poll until terminal state (one overload per input type)
func submitAndPoll(model: VideoModel, input: ..., onStatusChange: ((JobStatusResponse) -> Void)?) async throws -> QueueJobResult

RealtimeManager

func connect(localStream: RealtimeMediaStream) async throws -> RealtimeMediaStream
func disconnect() async
// Suspends until the server acks. Throws on ack failure, timeout (15s for
// prompts, 30s for reference-image set_image), or websocket disconnect
// while the call is pending. Wrap in `do/try await/catch` to surface
// per-call failures.
func setPrompt(_ prompt: DecartPrompt) async throws

// State events (connection, queue position, generation ticks, session ID)
let events: AsyncStream<DecartRealtimeState>

// New remote stream after automatic reconnection
let remoteStreamUpdates: AsyncStream<RealtimeMediaStream>

// Connection states: .idle, .connecting, .connected, .generating, .reconnecting, .disconnected, .error

ProcessClient

func process() async throws -> Data

Available Models

Realtime Models:

• RealtimeModel.lucy2_1 - Realtime video editing with reference image support
• RealtimeModel.lucy2_5 - Realtime video editing with reference image support
• RealtimeModel.lucyVton3_5 - Virtual try-on
• RealtimeModel.lucyVton3_6 - Virtual try-on 3.6 (opt-in by name; no speed: .fast tier)
• RealtimeModel.lucyRestyle2 - Realtime video restyling

Image Models:

• ImageModel.lucyImage2 - Image to image

Video Models:

• VideoModel.lucyClip - Video to video
• VideoModel.lucy2_1 - Video edit with optional reference image
• VideoModel.lucy2_5 - Video edit with optional reference image
• VideoModel.lucyVton3_5 - Virtual try-on video edit
• VideoModel.lucyVton3_6 - Virtual try-on 3.6 video edit (opt-in by name)
• VideoModel.lucyRestyle2 - Video restyle (prompt or reference image)

Input Types

// File-based inputs
ImageToImageInput(prompt: String, data: FileInput, seed: Int?)
VideoToVideoInput(prompt: String, data: FileInput, seed: Int?)
VideoEditInput(prompt: String, data: FileInput, referenceImage: FileInput?, seed: Int?)
VideoRestyleInput(prompt: String?, data: FileInput, referenceImage: FileInput?, seed: Int?)

// File input helpers
FileInput.image(data: Data, filename: String)
FileInput.video(data: Data, filename: String)
FileInput.from(data: Data, uniformType: UTType?)

Queue Types

enum JobStatus: String, Codable {
    case pending, processing, completed, failed
}

struct JobSubmitResponse: Codable { let jobId: String; let status: JobStatus }
struct JobStatusResponse: Codable { let jobId: String; let status: JobStatus }

enum QueueJobResult {
    case completed(jobId: String, data: Data)
    case failed(jobId: String, error: String)
}

RealtimeConfiguration

RealtimeConfiguration(
    model: ModelDefinition,
    initialPrompt: DecartPrompt,           // Optional
    resolution: Resolution?,               // Optional: .p720 / .p1080 (default: nil, server default 720p)
    speed: Speed?,                         // Optional: .fast (default: nil, standard mode)
    connection: ConnectionConfig,          // Optional
    media: MediaConfig,                    // Optional
    observability: ObservabilityConfig,    // Optional
    debugQuality: Bool                     // Optional, diagnostic only (default: false)
)

// Compute tier (fast mode). Advertised per model via `ModelDefinition.supportedSpeeds`.
enum Speed: String { case fast }

// Connection config
ConnectionConfig(
    connectionTimeout: TimeInterval,
    reconnectAttempts: Int
)

// Media config
MediaConfig(
    video: VideoConfig
)

// Video config
VideoConfig(
    maxBitrate: Int,     // default: 3_500_000
    maxFramerate: Int,   // default: 30
    preferredCodec: String, // "h264", "vp8", "vp9", or "av1"
    simulcast: Bool      // default: true
)

Requirements

• iOS 17.0+ / macOS 12.0+
• Swift 6.2.1
• Xcode with the Swift 6.2.1 toolchain

Environment Variables

Configure these in your Xcode scheme (Edit Scheme → Run → Environment Variables):

Variable	Required	Description
DECART_API_KEY	Yes	Your Decart API key from platform.decart.ai
DECART_DEFAULT_PROMPT	No	Default prompt for realtime sessions (defaults to "Simpsons")
ENABLE_DECART_SDK_DUBUG_LOGS	No	Set to YES to enable verbose SDK logging

Dependencies

• LiveKit Swift SDK - Realtime media transport for iOS/macOS

License

MIT License - see LICENSE for details.

Links

• Decart Website
• Platform
• Documentation
• GitHub Issues