framewise

Turn a video into a lean, deduplicated, high-quality set of frames — ideal for feeding vision LLMs within a token budget.

framewise is a smarter successor to naive frame extraction. Instead of uniform sampling plus a crude pixel diff, it combines perceptual-hash deduplication, blur filtering, and optional scene-cut detection so you get the fewest, most representative frames — perfect for vision/multimodal LLM prompts where every image costs tokens.

Features

• Perceptual-hash dedup — dHash + Hamming distance drops near-duplicate frames, not just pixel-identical ones.
• Blur filtering — optional Laplacian-variance sharpness gate discards out-of-focus / motion-blurred frames.
• Scene detection — optionally force-keep frames at detected scene cuts so transitions are never lost.
• VLM token-budget friendly — maxFrames caps the kept set with deterministic, evenly-spaced sampling.
• Zero-config ffmpeg — uses system ffmpeg if present, otherwise falls back to the optional ffmpeg-static binary.
• Safe by construction — shells out via execFile with an argument array (no shell), so paths can never inject commands.
• TypeScript-native — full types, ESM, works as a library and a CLI.

Install

pnpm add framewise

framewise needs an ffmpeg binary. Either install ffmpeg and put it on your PATH, or install the optional bundled binary:

pnpm add ffmpeg-static

Quick start

Library

import { extractFrames } from 'framewise';

const result = await extractFrames('demo.mp4', './frames', {
  fps: 2,
  maxFrames: 40,
  blurThreshold: 50,
  sceneDetection: true,
});

console.log(`kept ${result.keptCount} of ${result.extractedCount} frames`);
for (const frame of result.frames) {
  console.log(frame.index, frame.path, frame.keptReason, frame.timestampSec);
}

CLI

# Simplest form — writes to ./demo-frames next to the video
framewise demo.mp4

# Sample at 2 fps, cap at 40 frames
framewise demo.mp4 ./frames --fps 2 --max-frames 40

# Drop blurry frames and keep scene cuts
framewise demo.mp4 --blur-threshold 50 --scene-detection

# WebP output, machine-readable result
framewise demo.mp4 --format webp --quality 70 --json

API reference

extractFrames(videoPath, outputDir, options?)

Extracts, filters and renumbers frames into outputDir (created recursively). Returns an ExtractResult.

Options

Option	Type	Default	Description
fps	number	5	Frames per second to sample from the video.
format	'jpeg' | 'png' | 'webp'	'jpeg'	Output image format.
quality	number (0–100)	85	Output quality, mapped to ffmpeg -q:v.
maxFrames	number	undefined	Caps discretionary frames via even sampling; always retains first, last and scene-cut transitions.
dedupe	boolean	true	Enable dHash + Hamming-distance deduplication.
dedupeThreshold	number	8	Min Hamming distance from the last kept frame required to keep.
blurThreshold	number	undefined	Min Laplacian variance to keep; disabled when unset.
sceneDetection	boolean	false	Force-keep frames at detected scene cuts.
sceneThreshold	number	0.08	Normalized mean-abs-diff threshold for a scene cut.
includeFirstLast	boolean	true	Always include the video's first and last frames.
ffmpegPath	string	undefined	Override the ffmpeg binary path.
onProgress	(e) => void	undefined	Progress callback fired at extracting/analyzing/selecting/writing.

ExtractResult

Field	Type	Description
outputDir	string	Directory the final frames were written to.
frames	FrameInfo[]	Kept frames, renumbered frame-0001….
extractedCount	number	Raw frames pulled from ffmpeg.
keptCount	number	Frames kept after selection.
droppedDuplicates	number	Frames dropped as near-duplicates.
droppedBlurry	number	Frames dropped by the blur gate.

Each FrameInfo has index, path, timestampSec, dHash (16-hex-char), sharpness (Laplacian variance) and keptReason ('first' | 'last' | 'scene-cut' | 'unique' | 'sampled').

Building blocks

Individually exported and unit-tested:

• isVideoFile(filename): boolean
• checkFfmpeg(ffmpegPath?): Promise<boolean>
• resolveFfmpegBinary(ffmpegPath?): Promise<string | null>
• computeDHash(imagePath): Promise<string>
• hammingDistance(hexA, hexB): number
• computeSharpness(imagePath): Promise<number>

How selection works

1. Extract — one ffmpeg pass samples frames at fps; first/last frames are pulled separately when includeFirstLast is on.
2. Analyze — compute a dHash and a Laplacian-variance sharpness score for every extracted frame.
3. Blur gate — if blurThreshold is set, drop frames below it (first/last are never dropped).
4. Scene cuts — if sceneDetection is on, mark frames whose downscaled mean-abs-diff from the previous frame ≥ sceneThreshold as force-keeps.
5. Dedup — sequentially keep a frame only if its Hamming distance from the last kept frame ≥ dedupeThreshold; force-keeps always survive.
6. Budget — if maxFrames is set, it caps only discretionary frames via evenly-spaced sampling; first, last and scene-cut transitions are always retained (so the kept count may exceed maxFrames when forced frames alone do).
7. Write — kept frames are renumbered to frame-0001.<ext> and all raw files are removed, so the output directory holds only the final set.

Requirements

• Node.js >= 20
• An ffmpeg binary on PATH, or the optional ffmpeg-static dependency.

Contributing

Contributions are welcome! Please read the contributing guide for development setup and the PR process, and note the Code of Conduct. To report a security issue, see the security policy.

License

MIT © Dominik Rycharski

See LICENSE for details.