Image.decode returns the first frame of animations

Author: matmen

CHANGELOG.md

@@ -35,6 +35,8 @@ Upgrading from 3.x? Upgrade to the latest 3.x release first, fix the deprecation
 
 - **The positional arguments of `new Frame()`, `Frame.from()` and `new Animation()` are removed, and `Animation.decode()` takes no flag**: `new Frame(width, height, duration, xOffset, yOffset, disposalMode)` is `new Frame(width, height, { duration, xOffset, yOffset, disposalMode })`, `Frame.from(image, duration, xOffset, yOffset, disposalMode)` is `Frame.from(image, { duration, xOffset, yOffset, disposalMode })`, `new Animation(frames, loopCount)` is `new Animation(frames, { loopCount })`, and `Animation.decode(data, true)` is `Animation.decode(data, { onlyFirstFrame: true })`.
 
+- **`Image.decode()` returns the first frame of an animation**: a GIF or an animated WebP, which threw in 3.x, is an `Image` of its first frame, composed onto the canvas as `Animation.decode()` composes it, and decoding stops after that frame, so it is fast on long animations. Animated JPEG XLs and AVIF sequences were already their first frame. An APNG is still its default image, what viewers without APNG support show. Use `Animation.decode()` for all the frames.
+
 - **`decode()` returns an `Animation` only for files with several frames**: a GIF, APNG, animated WebP or JPEG XL with more than one frame is an `Animation` (`decode(gif)` used to be a `GIF`), and one with a single frame is an `Image`, like a still image (a one-frame GIF used to be a `GIF`). With `onlyFirstFrame`, the first frame comes back as an `Image`, since the frame count isn't read.
 
 - **The `GIF` class is removed**: `new GIF(frames, loopCount)` is `new Animation(frames, { loopCount })`, `GIF.decode(data)` is `Animation.decode(data)` and `gif.encode(quality)` is `animation.encodeGIF({ quality })`. `Animation` decodes and encodes GIFs the same way it does APNGs and WebPs.

guides/Migrating to 4.0.md

@@ -41,8 +41,13 @@ if (decoded instanceof Animation) {
 }
 ```
 
-With `onlyFirstFrame` on `decode()`, the file isn't read past its first frame, so the frame count is not known. The
-first frame comes back as an `Image`.
+`Image.decode()` returns the first frame of an animated GIF or WebP, which threw in 3.x. The frame is an `Image` composed
+onto the canvas like the first frame of `Animation.decode()`, and decoding stops after it, so it is fast for a long
+animation. Use `Animation.decode()` for all the frames. An APNG still gives its default image, as before.
+
+```ts
+const first = await Image.decode(data); // the first frame of an animated GIF or WebP
+```
 
 Search your code for: `decode(` where the result is used for `encode`, `.length`, indexing or `instanceof GIF`, and
 `onlyFirstFrame`.

src/image.ts

@@ -2726,9 +2726,10 @@ export class Image {
 
     /**
      * Decodes an image (PNG, JPEG, TIFF, WebP, BMP, ICO, CUR, QOI, PBM, PGM, PPM, PAM, PFM, Radiance HDR, TGA, AVIF or
-     * JPEG XL). For APNGs this is the default image; animated WebPs and GIFs have no single image, decode those with
-     * {@link Animation.decode}; for icons and cursors it's their largest image, for AVIF image sequences and JPEG XL
-     * animations their first frame
+     * JPEG XL). For animations (GIFs, animated WebPs, AVIF image sequences, JPEG XL animations) this is the first frame,
+     * composed onto the canvas as {@link Animation.decode} has it, and decoding stops after it: use
+     * {@link Animation.decode} for all frames. For APNGs this is the default image, which is what viewers without APNG
+     * support show; for icons and cursors it's their largest image
      * @param input The binary data to decode
      * @param options See {@link DecodeOptions}. JPEGs, TIFFs and PNGs are rotated/mirrored as their orientation tag
      *     (Exif, for PNGs also XMP) says by default: photos taken with a rotated camera are stored sideways
@@ -2748,8 +2749,10 @@ export class Image {
             if (ImageType.isJPEG(dataView)) return jpeg.decode(data, { maxPixels });
             if (ImageType.isTIFF(dataView)) return tiff.decode(data, maxPixels);
             if (ImageType.isWebP(dataView)) {
-                if (webp.isAnimated(data)) throw new Error('Animated WebPs have to be decoded with Animation.decode()');
-                return webp.decode(data, maxPixels);
+                if (!webp.isAnimated(data)) return webp.decode(data, maxPixels);
+                // the first frame only: the rest of the file isn't decoded
+                const animation = webp.decodeAnimation(data, true, maxPixels);
+                return { width: animation.width, height: animation.height, pixels: webp.composite(animation)[0].pixels, orientation: animation.orientation, icc: animation.icc };
             }
             // bitmaps can hold a JPEG or PNG, icons PNGs
             const bitmap = async (decoded: bmp.Decoded): Promise<{ width: number, height: number, pixels: Uint8Array }> => {
@@ -2766,7 +2769,13 @@ export class Image {
             if (ImageType.isPNM(dataView)) return netpbm.decode(data, maxPixels, toneMapping);
             if (ImageType.isHDR(dataView)) return hdr.decode(data, maxPixels, toneMapping);
             if (ImageType.isTGA(dataView)) return tga.decode(data, maxPixels);
-            if (ImageType.isGIF(dataView)) throw new Error('GIFs have to be decoded with Animation.decode() (or decode(), which returns an image or an animation)');
+            if (ImageType.isGIF(dataView)) {
+                // the first frame only: the rest of the file isn't decoded
+                const gifData = gif.decode(data, true, maxPixels);
+                const [first] = gif.composite(gifData);
+                if (!first) throw new Error('GIF: no frames');
+                return { width: gifData.width, height: gifData.height, pixels: first.pixels };
+            }
             throw new Error('Unsupported image type');
         });
 

tests/decode-frames.js

@@ -144,4 +144,87 @@ const throwsA = async (fn, type, label) => {
     const oneImage = await decode(oneApng);
     if (oneImage instanceof Animation || !(oneImage instanceof Image)) panic('one-frame APNG with acTL: decode() should give an Image');
     if (sha256(oneImage.bitmap) !== sha256(oneAnimation.frames[0].bitmap)) panic('one-frame APNG with acTL: decode() pixels differ from Animation.decode');
+
+    // Image.decode of an animation is its first frame, an Image composed like Animation.decode's first frame: frames that
+    // differ, with offsets and disposal modes, from our encoders (GIF, APNG, animated WebP, lossless and lossy)
+    const moving = new Animation([
+        new Frame(12, 8, {duration: 50}).fill((x, y) => Colors.fromRGBA(x * 20, y * 30, 90, 255)),
+        new Frame(5, 4, {duration: 70, xOffset: 3, yOffset: 2, disposalMode: 'background'}).fill((x, y) => Colors.fromRGBA(255, x * 50, y * 60, 128)),
+        new Frame(6, 3, {duration: 40, xOffset: 1, yOffset: 4, disposalMode: 'previous'}).fill(0x00ff00ff),
+        new Frame(4, 4, {duration: 60, xOffset: 8, yOffset: 4}).fill((x, y) => Colors.fromRGBA(y * 90, 0, 200, 255 - x * 60))
+    ]);
+    const movingFiles = {
+        GIF: await moving.encodeGIF(),
+        'GIF without optimization': await moving.encodeGIF({optimize: false}),
+        APNG: await moving.encodeAPNG(),
+        'APNG without optimization': await moving.encodeAPNG({optimize: false}),
+        'animated WebP': await moving.encodeWebP(),
+        'lossy animated WebP': await moving.encodeWebP({quality: 80})
+    };
+    for (const [name, bytes] of Object.entries(movingFiles)) {
+        const image = await Image.decode(bytes);
+        if (image.constructor !== Image) panic(`${name}: Image.decode gives ${image.constructor.name}, not an Image`);
+        const frames = (await Animation.decode(bytes)).frames;
+        if (frames.length !== 4) panic(`${name}: ${frames.length} frames instead of 4`);
+        const reference = frames[0];
+        if (image.width !== reference.width || image.height !== reference.height) panic(`${name}: ${image.width}x${image.height} instead of ${reference.width}x${reference.height}`);
+        if (sha256(image.bitmap) !== sha256(reference.bitmap)) panic(`${name}: Image.decode is not the first frame of Animation.decode`);
+        if (sha256(image.bitmap) === sha256(frames[1].bitmap)) panic(`${name}: the frames should differ`);
+    }
+    // options apply to the first frame: maxPixels counts the canvas
+    await throwsA(() => Image.decode(movingFiles.GIF, {maxPixels: 10}), RangeError, 'Image.decode maxPixels of an animated GIF');
+    if ((await Image.decode(movingFiles.GIF, {maxPixels: 96})).width !== 12) panic('Image.decode: a GIF whose canvas fits maxPixels should decode');
+
+    // an APNG whose default image (IDAT) isn't part of the animation: Image.decode gives that image, what viewers without
+    // APNG support show, and Animation.decode only the frames of the animation
+    const pngList = bytes => {
+        const list = [], view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
+        for (let at = 8; at < bytes.length;) {
+            const length = view.getUint32(at);
+            list.push({type: String.fromCharCode(...bytes.subarray(at + 4, at + 8)), data: bytes.subarray(at + 8, at + 8 + length)});
+            at += 12 + length;
+        }
+        return list;
+    };
+    // (more colors than a palette holds, so both are truecolor PNGs without a PLTE chunk to copy)
+    const defaultPNG = new Uint8Array(await new Image(32, 32).fill((x, y) => Colors.fromRGBA(x * 7, y * 7, (x * y) & 255, 255)).encodePNG());
+    const framePNG = new Uint8Array(await new Image(32, 32).fill((x, y) => Colors.fromRGBA(0, x * 7, y * 7, 255)).encodePNG());
+    const ihdr = pngList(defaultPNG).find(c => c.type === 'IHDR').data;
+    const defaultIDAT = pngList(defaultPNG).find(c => c.type === 'IDAT').data;
+    const frameIDAT = pngList(framePNG).find(c => c.type === 'IDAT').data;
+    const actl = new Uint8Array(8);
+    new DataView(actl.buffer).setUint32(0, 1); // one frame in the animation, which the IDAT isn't part of
+    const fcTL = new Uint8Array(26), fcView = new DataView(fcTL.buffer);
+    fcView.setUint32(0, 0); // sequence number
+    fcView.setUint32(4, 32); // width
+    fcView.setUint32(8, 32); // height
+    fcView.setUint16(20, 100); // delay: 100/1000 s
+    fcView.setUint16(22, 1000);
+    const fdAT = new Uint8Array(4 + frameIDAT.length);
+    new DataView(fdAT.buffer).setUint32(0, 1); // sequence number
+    fdAT.set(frameIDAT, 4);
+    const separate = Buffer.concat([
+        defaultPNG.subarray(0, 8), pngChunk('IHDR', ihdr), pngChunk('acTL', actl), pngChunk('IDAT', defaultIDAT),
+        pngChunk('fcTL', fcTL), pngChunk('fdAT', fdAT), pngChunk('IEND', new Uint8Array(0))
+    ]);
+    const shown = await Image.decode(separate);
+    if (shown.constructor !== Image || sha256(shown.bitmap) !== sha256((await Image.decode(defaultPNG)).bitmap)) panic('APNG with a separate default image: Image.decode should give the default image');
+    const animated = await Animation.decode(separate);
+    if (animated.length !== 1 || sha256(animated.frames[0].bitmap) !== sha256((await Image.decode(framePNG)).bitmap)) panic('APNG with a separate default image: Animation.decode should give only the animation\'s frame');
+
+    // a large animation: Image.decode stops after its first frame, so it takes a fraction of the time of all 50 frames
+    const many = new Animation(Array.from({length: 50}, (_, i) => new Frame(480, 360, {duration: 40}).fill((x, y) => Colors.fromRGBA((x * 7 + i * 13) & 255, (y * 5 + x * i) & 255, (x ^ y) & 255, 255))));
+    const large = await many.encodeGIF();
+    const median = async fn => {
+        const times = [];
+        for (let i = 0; i < 3; i++) {
+            const start = process.hrtime.bigint();
+            await fn();
+            times.push(Number(process.hrtime.bigint() - start) / 1e6);
+        }
+        return times.sort((a, b) => a - b)[1];
+    };
+    const firstMs = await median(() => Image.decode(large));
+    const allMs = await median(() => Animation.decode(large));
+    if (firstMs * 3 > allMs) panic(`Image.decode of 50 frames took ${firstMs.toFixed(1)} ms, Animation.decode ${allMs.toFixed(1)} ms: not a third`);
 })().catch(error => panic(error.stack || String(error)));

tests/option-checks.js

@@ -61,9 +61,10 @@ const rejects = async (name, fn, Type, pattern) => {
     }
     await rejects('Animation.decode threads 0', () => Animation.decode(png, {threads: 0}), RangeError, /threads must be/);
 
-    // GIFs are refused by Image.decode() with a hint at Animation.decode()
+    // Image.decode takes the options object of a GIF too: maxPixels counts its first frame
     const gif = await new Animation([new Frame(4, 4).fill(0xff0000ff)]).encodeGIF();
-    await rejects('Image.decode GIF', () => Image.decode(gif), Error, /Animation\.decode\(\)/);
+    await rejects('Image.decode GIF maxPixels', () => Image.decode(gif, {maxPixels: 15}), RangeError, /maxPixels/);
+    if ((await Image.decode(gif, {maxPixels: 16})).width !== 4) panic('Image.decode GIF: the first frame should fit maxPixels 16');
 
     // valid values keep working
     for (const effort of [0, 1, 2]) await image.encodeAVIF({effort});

tests/webp.js

@@ -36,14 +36,11 @@ const animated = [
         if (sha256(image.bitmap) !== hash) panic(`${name}: decoded pixels don't match libwebp`);
     }
 
-    // animated WebPs: Image.decode refuses them, Animation.decode composites the frames
+    // animated WebPs: Image.decode gives the first frame as libwebp composites it, Animation.decode all of them
     const data = await fs.readFile('./tests/webp/animated.webp');
-    try {
-        await Image.decode(data);
-        panic('Image.decode should reject animated WebPs');
-    } catch (err) {
-        if (!/Animation\.decode/.test(err.message)) panic(`unexpected error ${err.message}`);
-    }
+    const firstFrame = await Image.decode(data);
+    if (firstFrame.constructor !== Image || firstFrame.width !== 96 || firstFrame.height !== 72) panic('animated WebP: Image.decode should give an Image of the canvas');
+    if (sha256(firstFrame.bitmap) !== animated[0]) panic('animated WebP: Image.decode is not the first frame libwebp composites');
     const animation = await Animation.decode(data);
     if (!(await decode(data) instanceof Animation)) panic('decode() should return an Animation for animated WebPs');
     if (animation.length !== 4 || animation.loopCount !== Infinity) panic('animated WebP: wrong frame or loop count');