feat: add VideoPrism temporal video search

INSTALLATION_GUIDE.md

@@ -14,7 +14,7 @@ shape needs.
 | Desktop app | Install the native package | Adopt a compatible local installation or create a private Desktop-managed runtime |
 | Browser UI in Docker | Published `vidxp` image | One CPU worker/UI container |
 | Public/self-hosted service | `compose.coolify.yaml` | API/MCP control plane, CPU worker, PostgreSQL, Chroma, tusd |
-| Embed one capability | `dialogue`, `scene`, or `actor` extra | Python indexing/retrieval code |
+| Embed one capability | `dialogue`, `scene`, `actor`, or `videoprism` extra | Python indexing/retrieval code |
 
 Do not install the bare package and expect it to index video. Base `vidxp`
 provides the lightweight command shell, configuration, and typed contracts.
@@ -227,7 +227,8 @@ Extras are composable:
 | `dialogue` | Storage, transcription, dialogue embeddings | Scene/actor providers |
 | `scene` | Storage, PyTorch, Transformers, OpenCV, Pillow | Dialogue/actor providers |
 | `actor` | Storage, OpenCV, YuNet/SFace support | Dialogue/scene providers |
-| `all` | Dialogue, scene, and actor | Grounded-query model client and UI |
+| `videoprism` | Storage, VideoPrism, PyTorch, Transformers, Torchvision | Other model providers |
+| `all` | Every built-in search capability | Grounded-query model client and UI |
 | `local-worker` | `all` plus grounded-query client | Browser UI, MCP SDK, HTTP server |
 | `frontend` | Streamlit | Worker providers |
 | `mcp` | MCP SDK | Worker providers |

README.md

@@ -15,7 +15,7 @@
 </p>
 
 <p align="center">
-  <strong>Dialogue search · Scene search · Actor grouping</strong>
+  <strong>Dialogue search · Scene search · Temporal video search · Actor grouping</strong>
 </p>
 
 <p align="center">
@@ -43,9 +43,13 @@ VidXP makes one video—or an entire collection—searchable by meaning:
   matching moments.
 - **Scene search:** describe what appeared on screen and find the closest
   visual matches.
+- **VideoPrism search:** describe an action or event spanning a short clip.
 - **Actor matching:** find recurring faces within a video and export a
   highlighted video for a selected group.
 
+Existing indexes remain compatible. Index media with the `videoprism`
+capability selected before using temporal video search.
+
 Use it to search years of family videos, add video search to an editing
 workflow, or let an AI agent answer questions using evidence from your own
 video library. Your videos can stay on your machine.
@@ -138,6 +142,9 @@ vidxp index create <media-id>
 # Find a visual moment
 vidxp search scene "a yellow taxi on a city street"
 
+# Find an action spanning several frames
+vidxp search videoprism "a person opens a door and walks outside"
+
 # Find something that was said
 vidxp search dialogue "the bread just came out of the oven"
 ```
@@ -197,10 +204,11 @@ approximately 3 GiB.
 |---|---:|
 | Dialogue search | 2.64 GiB |
 | Scene search | 1.43 GiB |
+| VideoPrism temporal search | 0.93 GiB |
 | Actor matching | 37 MiB |
 
 A full local Desktop setup with every search capability uses approximately
-7.1 GiB. Leave additional temporary space during installation and for indexes,
+8.1 GiB. Leave additional temporary space during installation and for indexes,
 source videos, and exported results.
 
 By default, the CLI and desktop app share the same VidXP data directory:

desktop/model-cache-catalog.json

@@ -9,6 +9,11 @@
     "label": "google/siglip2-base-patch16-224",
     "relative_artifact": "models--google--siglip2-base-patch16-224/snapshots/75de2d55ec2d0b4efc50b3e9ad70dba96a7b2fa2/model.safetensors"
   },
+  {
+    "id": "google/videoprism-lvt-base-f16r288",
+    "label": "google/videoprism-lvt-base-f16r288",
+    "relative_artifact": "models--google--videoprism-lvt-base-f16r288/snapshots/fb6de9f0eb7bc285be86bdca1cf7daa3e3ef51ff/model.safetensors"
+  },
   {
     "id": "Qwen/Qwen3-Embedding-0.6B",
     "label": "Qwen/Qwen3-Embedding-0.6B",

desktop/src/components/TargetSummary.tsx

@@ -48,6 +48,7 @@ const CAPABILITY_LABELS: Record<string, string> = {
   dialogue: 'Dialogue search',
   media: 'Video tools',
   scene: 'Visual scene search',
+  videoprism: 'Temporal video search',
 };
 
 interface WorkerFailure {

docs/desktop.md

@@ -135,13 +135,13 @@ Target profiles use a versioned desktop-private schema. Profile content and the
 selected profile identity are stored separately. No credentials or remote tokens
 are stored; remote targets are intentionally outside this release.
 
-Users select dialogue, scene, and actor search features independently. Product
-choices map to package extras as follows: **Local video processing** adds
-`local-worker` and includes all built-in search features, **Browser interface**
-adds `frontend`, **AI assistant integration** adds the stdio `mcp` transport,
-and **App integration service** adds the loopback API plus Streamable HTTP MCP
-through `server`. These package names stay out of the normal product flow. Model preparation
-can be deferred, and a native folder picker
+Users select dialogue, scene, VideoPrism temporal search, and actor features
+independently. Product choices map to package extras as follows: **Local video
+processing** adds `local-worker` and includes all built-in search features,
+**Browser interface** adds `frontend`, **AI assistant integration** adds the
+stdio `mcp` transport, and **App integration service** adds the loopback API
+plus Streamable HTTP MCP through `server`. These package names stay out of the
+normal product flow. Model preparation can be deferred, and a native folder picker
 can select a model-cache directory before any model is downloaded.
 The managed runtime acquires the exact VidXP package from the wheel embedded in
 the Desktop installer with dependency resolution disabled, then resolves that

plugins/vidxp/skills/vidxp-ingest-video/SKILL.md

@@ -10,8 +10,9 @@ description: Use VidXP to upload, import, register, and automatically index vide
 1. Resolve the `vidxp` MCP tools and call `get_workspace`. Do not import a video
    that is already registered or indexed.
 2. Choose indexable modalities from the workspace. Use `dialogue` and `scene`
-   for ordinary content retrieval. Add `actor` only when anonymous recurring-face
-   clusters are wanted; it does not identify people by name.
+   for ordinary content retrieval. Add `videoprism` when the request depends on
+   actions or events spanning multiple frames. Add `actor` only when anonymous
+   recurring-face clusters are wanted; it does not identify people by name.
 3. Call `get_runtime_readiness`. If selected models are missing, submit
    `prepare_models`, use `wait_job` with its observation token for subsequent
    bounded waits, then fetch `get_job` once when terminal.

pyproject.toml

@@ -95,18 +95,24 @@ actor = { file = [
   "src/vidxp/requirements/storage.txt",
   "src/vidxp/capabilities/actor/requirements.txt",
 ] }
+videoprism = { file = [
+  "src/vidxp/requirements/storage.txt",
+  "src/vidxp/capabilities/videoprism/requirements.txt",
+] }
 all = { file = [
   "src/vidxp/requirements/storage.txt",
   "src/vidxp/capabilities/dialogue/requirements.txt",
   "src/vidxp/capabilities/scene/requirements.txt",
   "src/vidxp/capabilities/actor/requirements.txt",
+  "src/vidxp/capabilities/videoprism/requirements.txt",
 ] }
 local-worker = { file = [
   "src/vidxp/requirements/storage.txt",
   "src/vidxp/requirements/slm.txt",
   "src/vidxp/capabilities/dialogue/requirements.txt",
   "src/vidxp/capabilities/scene/requirements.txt",
   "src/vidxp/capabilities/actor/requirements.txt",
+  "src/vidxp/capabilities/videoprism/requirements.txt",
 ] }
 mcp = { file = ["src/vidxp/requirements/mcp.txt"] }
 slm = { file = ["src/vidxp/requirements/slm.txt"] }
@@ -121,6 +127,7 @@ server-worker = { file = [
   "src/vidxp/capabilities/dialogue/requirements.txt",
   "src/vidxp/capabilities/scene/requirements.txt",
   "src/vidxp/capabilities/actor/requirements.txt",
+  "src/vidxp/capabilities/videoprism/requirements.txt",
 ] }
 test = { file = ["src/vidxp/requirements/test.txt"] }
 frontend = { file = ["src/vidxp/requirements/frontend.txt"] }
@@ -136,6 +143,9 @@ select = ["E4", "E7", "E9", "F"]
 torch = [
   { index = "pytorch-cpu", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
 ]
+torchvision = [
+  { index = "pytorch-cpu", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+]
 
 [[tool.uv.index]]
 name = "pytorch-cpu"

src/vidxp/capabilities/registry.py

@@ -503,8 +503,9 @@ def _builtin_plugins() -> tuple[CapabilityPlugin, ...]:
     from vidxp.capabilities.actor.definition import PLUGIN as actor
     from vidxp.capabilities.dialogue.definition import PLUGIN as dialogue
     from vidxp.capabilities.scene.definition import PLUGIN as scene
+    from vidxp.capabilities.videoprism.definition import PLUGIN as videoprism
 
-    return dialogue, scene, actor
+    return dialogue, scene, actor, videoprism
 
 
 def _external_entry_points(allowlist: tuple[str, ...]) -> tuple[EntryPoint, ...]:

src/vidxp/capabilities/videoprism/__init__.py

@@ -0,0 +1 @@
+"""VideoPrism temporal video search capability."""

src/vidxp/capabilities/videoprism/config.py

@@ -0,0 +1,15 @@
+from __future__ import annotations
+
+from pydantic import Field
+
+from vidxp.capabilities.contracts import CapabilityConfig
+from vidxp.core.contracts import IndexConfig
+
+
+class VideoPrismConfig(CapabilityConfig):
+    batch_size: int = Field(default=1, gt=0)
+    sample_fps: float = Field(default=2.0, gt=0)
+
+
+def videoprism_config(config: IndexConfig) -> VideoPrismConfig:
+    return VideoPrismConfig.model_validate(config.options_for("videoprism"))

src/vidxp/capabilities/videoprism/definition.py

@@ -0,0 +1,95 @@
+from __future__ import annotations
+
+from typing import Any, Mapping
+
+from vidxp.application_models import CapabilityRole
+from vidxp.capabilities.contracts import (
+    CapabilityDefinition,
+    CapabilityExecutor,
+    CapabilityPlugin,
+    OperationDefinition,
+    PreparationContext,
+    module_import_check,
+)
+from vidxp.capabilities.schemas import SearchInput, SearchResult
+from vidxp.capabilities.videoprism.config import VideoPrismConfig
+from vidxp.capabilities.videoprism.indexing import VISUAL_PROCESSOR
+from vidxp.capabilities.videoprism.models import get_videoprism_model
+from vidxp.capabilities.videoprism.operations import search_operation
+from vidxp.capabilities.videoprism.specs import VIDEOPRISM_MODEL
+from vidxp.capabilities.visual import index_capabilities
+from vidxp.core.contracts import IndexConfig, VideoSource
+from vidxp.core.indexing_common import ProgressCallback, report_preparation
+
+
+def prepare_models(
+    context: PreparationContext,
+    progress: ProgressCallback | None,
+) -> tuple[str, ...]:
+    VideoPrismConfig.model_validate(context.settings)
+    report_preparation(
+        progress,
+        "videoprism_model",
+        f"Preparing VideoPrism {VIDEOPRISM_MODEL.model_id}",
+    )
+    get_videoprism_model(context.runtime, download=True, progress=progress)
+    return (VIDEOPRISM_MODEL.model_id,)
+
+
+def model_manifest(
+    config: IndexConfig,
+    _sources: tuple[VideoSource, ...],
+) -> Mapping[str, Any]:
+    return {"videoprism": VIDEOPRISM_MODEL.identity()}
+
+
+DEFINITION = CapabilityDefinition(
+    name="videoprism",
+    description="Index and search temporal video clips with VideoPrism.",
+    extra="videoprism",
+    config_model=VideoPrismConfig,
+    collection_name="videoprism",
+    index_stage="visual_indexing",
+    execution_group="visual",
+    prepares_models=True,
+    roles=(CapabilityRole.searchable, CapabilityRole.queryable),
+    model_specs=(VIDEOPRISM_MODEL,),
+    operations={
+        "search": OperationDefinition(
+            input_model=SearchInput,
+            output_model=SearchResult,
+        )
+    },
+)
+
+
+def create_executor() -> CapabilityExecutor:
+    return CapabilityExecutor(
+        indexer=index_capabilities,
+        index_processor=VISUAL_PROCESSOR,
+        operations={"search": search_operation},
+        prepare=prepare_models,
+        model_manifest=model_manifest,
+        runtime_checks=(
+            module_import_check("OpenCV import", "cv2", "VideoCapture"),
+            module_import_check("Torch import", "torch"),
+            module_import_check("Torchvision import", "torchvision"),
+            module_import_check(
+                "Transformers VideoPrism import",
+                "transformers",
+                "VideoPrismClipModel",
+                "VideoPrismProcessor",
+            ),
+            module_import_check(
+                "Hugging Face Hub import",
+                "huggingface_hub",
+                "snapshot_download",
+            ),
+        ),
+    )
+
+
+PLUGIN = CapabilityPlugin(
+    definition=DEFINITION,
+    executor_factory=create_executor,
+)