Skip to content

Commit befbac4

Browse files
matmenmatmen
committed
benchmark: more formats and operations, uniform charts, startup measurement
1 parent 49b829d commit befbac4

19 files changed

Lines changed: 916 additions & 645 deletions

‎bench/README.md‎

Lines changed: 58 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -7,56 +7,54 @@ Photon is WebAssembly, and so is jsquash (Squoosh's libavif and libjxl, for AVIF
77

88
<img src="charts/overview-portable.svg" alt="Time per operation of the libraries that run everywhere, colored by the distance to the fastest">
99

10-
<img src="charts/vs-javascript.svg" alt="ImageScript's speed compared to the fastest other pure JavaScript library">
11-
1210
<img src="charts/overview-native.svg" alt="Time per operation of ImageScript and the native libraries">
1311

1412
<img src="charts/install.svg" alt="Install size of each library">
1513

16-
<img src="charts/codecs.svg" alt="Decoding and encoding speed">
17-
18-
<img src="charts/modern.svg" alt="AVIF and JPEG XL decoding and encoding speed">
19-
20-
<img src="charts/operations.svg" alt="Speed of image operations">
21-
22-
<img src="charts/rendering.svg" alt="Text and SVG rendering speed">
23-
24-
<img src="charts/animation.svg" alt="Animated GIF decoding and encoding speed">
25-
2614
<img src="charts/sizes-lossless.svg" alt="Lossless file sizes">
2715

2816
<img src="charts/sizes-lossy.svg" alt="Lossy file sizes with their quality">
2917

3018
## Numbers
3119

32-
Time per operation (less is faster), measured 2026-11-04 with Node.js v24.22.2 on Intel(R) Xeon(R) 6975P-C, ImageScript 3.14.0.
20+
Time per operation (less is faster), measured 2026-11-05 with Node.js v24.11.1 on AMD Ryzen 7 5800X 8-Core Processor, ImageScript 3.14.0.
3321

34-
| | **ImageScript 3** | **Jimp** | **image-js** | **PureJsImage** | **Photon** | **sharp** | **jsquash** | **@napi-rs/canvas** |
22+
| | **ImageScript** | **Jimp** | **image-js** | **PureJsImage** | **Photon** | **sharp** | **jsquash** | **@napi-rs/canvas** |
3523
|---|--:|--:|--:|--:|--:|--:|--:|--:|
36-
| decode JPEG | **55 ms** | 127 ms | 143 ms | 103 ms | 34 ms | 13 ms | – | 15 ms |
37-
| decode PNG | **28 ms** | 47 ms | 107 ms | 71 ms | 22 ms | 19 ms | – | 17 ms |
38-
| decode WebP (lossy) | **67 ms** | – | – | 83 ms | 65 ms | 26 ms | – | 13 ms |
39-
| encode JPEG (quality 80) | **45 ms** | 279 ms | 90 ms | 105 ms | 89 ms | 15 ms | – | 22 ms |
40-
| encode PNG | **144 ms** | 200 ms | 299 ms | 477 ms | 79 ms | 112 ms | – | 388 ms |
41-
| encode WebP (lossless) | **1.1 s** | – | – | 2.1 s | 33 ms | 1.2 s | – | 379 ms |
42-
| decode AVIF | **184 ms** | – | – | 1.4 s | – | 68 ms | 169 ms | 51 ms |
43-
| decode JPEG XL | **348 ms** | – | – | fails | – | – | 303 ms | 48 ms |
44-
| encode AVIF (quality 60) | **3.7 s** | – | – | 988 ms | – | 1.9 s | 1.8 s | – |
45-
| encode JPEG XL (quality 90) | **1.6 s** | – | – | – | – | – | 3.2 s | – |
46-
| resize to half | **11 ms** | 11 ms | 72 ms | 52 ms | 38 ms | 7.6 ms | – | 2.2 ms |
47-
| rotate by 30° | **41 ms** | 436 ms | 615 ms | 421 ms | 186 ms | 33 ms | – | 25 ms |
48-
| blur (radius 5) | **54 ms** | 57 s | 541 ms | – | 135 ms | 13 ms | – | 18 ms |
49-
| composite an overlay | **2.0 ms** | 22 ms | 8.3 ms | – | 5.0 ms | 4.1 ms | – | 2.4 ms |
50-
| crop | **1.3 ms** | 14 ms | 15 ms | 8.8 ms | 3.5 ms | 0.73 ms | – | 0.71 ms |
51-
| clone | **1.0 ms** | 1.9 ms | 1.6 ms | 1.1 ms | 2.5 ms | 1.1 ms | – | 2.3 ms |
52-
| render text | **0.42 ms** | – | – | – | – | 0.51 ms | – | 0.25 ms |
53-
| render SVG | **130 ms** | – | – | – | – | 30 ms | – | 9.8 ms |
54-
| decode animated GIF | **53 ms** | – | – | – | – | 57 ms | – | – |
55-
| encode animated GIF | **191 ms** | – | – | – | – | 877 ms | – | – |
24+
| decode JPEG | **46 ms** | 102 ms | 112 ms | 86 ms | 32 ms | 9.4 ms | – | 14 ms |
25+
| decode PNG | **18 ms** | 41 ms | 78 ms | 40 ms | 17 ms | 12 ms | – | 13 ms |
26+
| decode WebP (lossy) | **57 ms** | – | – | 67 ms | 60 ms | 24 ms | – | 12 ms |
27+
| decode AVIF | **156 ms** | – | – | 1.2 s | – | 60 ms | 146 ms | 43 ms |
28+
| decode JPEG XL | **262 ms** | – | – | fails | – | – | 316 ms | 48 ms |
29+
| decode animated GIF | **41 ms** | – | – | – | – | 37 ms | – | – |
30+
| decode animated WebP | **185 ms** | – | – | – | – | 257 ms | – | – |
31+
| encode JPEG (quality 80) | **41 ms** | 282 ms | 74 ms | 96 ms | 76 ms | 12 ms | – | 19 ms |
32+
| encode PNG | **113 ms** | 159 ms | 251 ms | 363 ms | 69 ms | 91 ms | – | 316 ms |
33+
| encode WebP (quality 80) | **135 ms** | – | – | 87 ms | – | 126 ms | – | 107 ms |
34+
| encode WebP (lossless) | **833 ms** | – | – | 1.8 s | 29 ms | 1.0 s | – | 322 ms |
35+
| encode AVIF (quality 60) | **1.7 s** | – | – | 965 ms | – | 684 ms | 1.6 s | – |
36+
| encode JPEG XL (quality 90) | **710 ms** | – | – | – | – | – | 1.7 s | – |
37+
| encode JPEG XL (lossless) | **265 ms** | – | – | – | – | – | 306 ms | – |
38+
| encode animated GIF | **149 ms** | – | – | – | – | 772 ms | – | – |
39+
| encode animated WebP (q 80) | **624 ms** | – | – | – | – | 519 ms | – | – |
40+
| resize to half | **9.2 ms** | 11 ms | 61 ms | 42 ms | 45 ms | 5.5 ms | – | 1.8 ms |
41+
| resize to half, bicubic | **43 ms** | 102 ms | 261 ms | – | 71 ms | 5.8 ms | – | – |
42+
| thumbnail (cover 400×400) | **16 ms** | 35 ms | – | 27 ms | – | 3.8 ms | – | 0.64 ms |
43+
| crop | **1.3 ms** | 12 ms | 11 ms | 6.6 ms | 3.4 ms | 0.52 ms | – | 0.42 ms |
44+
| rotate by 30° | **26 ms** | 402 ms | 501 ms | 339 ms | 146 ms | 11 ms | – | 18 ms |
45+
| rotate by 90° | **7.1 ms** | 34 ms | 15 ms | 84 ms | 98 ms | 1.5 ms | – | 8.5 ms |
46+
| flip | **1.4 ms** | 67 ms | 8.5 ms | 39 ms | 14 ms | 1.2 ms | – | 6.7 ms |
47+
| blur (radius 5) | **51 ms** | 54 s | 569 ms | – | 103 ms | 6.1 ms | – | 12 ms |
48+
| sharpen | **137 ms** | – | – | – | 71 ms | 116 ms | – | – |
49+
| composite an overlay | **1.6 ms** | 22 ms | 7.1 ms | – | 4.5 ms | 2.0 ms | – | 1.6 ms |
50+
| greyscale | **4.5 ms** | 84 ms | 13 ms | – | 4.7 ms | 9.7 ms | – | 5.1 ms |
51+
| brightness and contrast | **7.1 ms** | 185 ms | – | – | 19 ms | 4.2 ms | – | 9.3 ms |
52+
| render text | **0.33 ms** | – | – | – | – | 0.23 ms | – | 0.17 ms |
53+
| render SVG | **97 ms** | – | – | – | – | 25 ms | – | 8.3 ms |
5654

5755
Encoded sizes:
5856

59-
| | **ImageScript 3** | **Jimp** | **image-js** | **PureJsImage** | **Photon** | **sharp** | **jsquash** | **@napi-rs/canvas** |
57+
| | **ImageScript** | **Jimp** | **image-js** | **PureJsImage** | **Photon** | **sharp** | **jsquash** | **@napi-rs/canvas** |
6058
|---|--:|--:|--:|--:|--:|--:|--:|--:|
6159
| photo PNG | 2.0 MB | 2.7 MB | 2.8 MB | 2.1 MB | 2.8 MB | 2.8 MB | – | 2.1 MB |
6260
| photo JPEG | 218 kB, 40.6 dB | 260 kB, 40.9 dB | 260 kB, 40.9 dB | 222 kB, 40.5 dB | 257 kB, 40.4 dB | 218 kB, 40.6 dB | – | 218 kB, 40.6 dB |
@@ -65,26 +63,33 @@ Encoded sizes:
6563
| graphic JPEG | 276 kB, 37.3 dB | 315 kB, 37.4 dB | 315 kB, 37.4 dB | 286 kB, 37.3 dB | 314 kB, 37.4 dB | 275 kB, 37.3 dB | – | 276 kB, 37.3 dB |
6664
| graphic WebP | 277 kB | – | – | 758 kB | 856 kB | 239 kB | – | 631 kB |
6765
| animation GIF | 825 kB, identical pixels | – | – | – | – | 720 kB, 45.6 dB | – | – |
68-
| photo AVIF | 93 kB, 40.9 dB | – | – | 992 kB, 32.7 dB | – | 91 kB, 40.6 dB | 80 kB, 40.2 dB | – |
69-
| photo JPEG XL | 245 kB, 44.5 dB | – | – | – | – | – | 230 kB, 44.1 dB | – |
70-
| graphic AVIF | 147 kB, 41 dB | – | – | 709 kB, 38.5 dB | – | 114 kB, 38.7 dB | 136 kB, 40.5 dB | – |
71-
| graphic JPEG XL | 230 kB, 43.4 dB | – | – | – | – | – | 194 kB, 41.9 dB | – |
72-
73-
| | **ImageScript 3** | **Jimp** | **image-js** | **PureJsImage** | **Photon** | **sharp** | **jsquash** | **@napi-rs/canvas** |
66+
| photo AVIF | 93 kB, 40.9 dB | – | – | 992 kB, 32.7 dB | – | 92 kB, 40.6 dB | 80 kB, 40.2 dB | – |
67+
| photo JPEG XL | 245 kB, 44.5 dB | – | – | – | – | – | 227 kB, 44 dB | – |
68+
| graphic AVIF | 147 kB, 41 dB | – | – | 709 kB, 38.5 dB | – | 115 kB, 38.7 dB | 136 kB, 40.5 dB | – |
69+
| graphic JPEG XL | 230 kB, 43.4 dB | – | – | – | – | – | 193 kB, 41.8 dB | – |
70+
| photo WebP lossy | 123 kB, 40 dB | – | – | 153 kB, 38.6 dB | – | 118 kB, 40.4 dB | – | 118 kB, 40.4 dB |
71+
| photo JPEG XL lossless | 1.2 MB | – | – | – | – | – | 1.2 MB | – |
72+
| graphic WebP lossy | 241 kB, 38.4 dB | – | – | 301 kB, 38 dB | – | 233 kB, 39 dB | – | 234 kB, 39 dB |
73+
| graphic JPEG XL lossless | 663 kB | – | – | – | – | – | 666 kB | – |
74+
| animation WebP | 534 kB, 33.6 dB | – | – | – | – | 683 kB, 33.1 dB | – | – |
75+
76+
| | **ImageScript** | **Jimp** | **image-js** | **PureJsImage** | **Photon** | **sharp** | **jsquash** | **@napi-rs/canvas** |
7477
|---|---|---|---|---|---|---|---|---|
7578
| kind | JavaScript | JavaScript | JavaScript | JavaScript | WebAssembly | native (libvips) | WebAssembly | native (Skia) |
7679
| package | `imagescript@3.14.0` | `jimp@1.6.1` | `image-js@1.7.1` | `purejsimage@0.17.0` | `@silvia-odwyer/photon-node@0.3.4` | `sharp@0.35.5` | `@jsquash/avif@2.1.1 @jsquash/jxl@1.3.0` | `@napi-rs/canvas@1.0.11` |
77-
| install size | 2.4 MB | 26.8 MB | 47.8 MB | 6.7 MB | 2.3 MB | 20.3 MB | 14.2 MB | 35.0 MB |
78-
| packages | 1 | 64 | 55 | 1 | 1 | 6 | 3 | 2 |
80+
| install size | 2.4 MB | 26.8 MB | 47.8 MB | 6.7 MB | 2.3 MB | 30.2 MB | 14.2 MB | 35.0 MB |
81+
| packages | 1 | 64 | 55 | 1 | 1 | 9 | 3 | 2 |
7982

8083
## Method
8184

8285
- Inputs (in `inputs/`, sources in [inputs/SOURCES.md](inputs/SOURCES.md)): a 1920×1080 photo (JPEG, and made from it
83-
when the benchmark starts, PNG, lossy WebP, AVIF by sharp (libaom) and JPEG XL by jsquash (libjxl, effort 7), both
84-
at the qualities below), a 1434×1434 graphic with transparency (PNG), a 445×445 diagram with transparency placed on
85-
the photo (PNG), a 300×300 GIF with 101 frames.
86-
- AVIF and JPEG XL are encoded at the same nominal quality (60 and 90); as the libraries' scales differ,
87-
the file sizes come with their PSNR (of the color over black, decoded by libavif and libjxl). PureJsImage's AVIF
86+
when the benchmark starts, PNG, lossy WebP, AVIF by sharp (libaom) and JPEG XL by jsquash (libjxl), at the
87+
qualities below), a 1434×1434 graphic with transparency (PNG), a 445×445 diagram with transparency placed on
88+
the photo (PNG), a 300×300 GIF with 101 frames (and made from it, an animated WebP by sharp at quality 80).
89+
- Lossy WebP, AVIF and JPEG XL are encoded at the same nominal quality (80, 60 and 90); as the libraries'
90+
scales differ, the file sizes come with their PSNR (of the color over black, decoded by libwebp, libavif and
91+
libjxl). jsquash's libjxl runs at the effort ImageScript's default matches (cjxl's 5 lossy, 3 lossless; its own
92+
default is 7). PureJsImage's AVIF
8893
encoder has no quality option, and its JPEG XL decoder doesn't support everything libjxl's files use, which shows as
8994
failing. jsquash only encodes and decodes AVIF and JPEG XL (it gets the other inputs as pixels); @napi-rs/canvas
9095
decodes both with Skia's codecs, but doesn't encode them.
@@ -95,8 +100,10 @@ Encoded sizes:
95100
clone first, sharp runs a pipeline from raw pixels back to raw pixels, PureJsImage (which has no decoded images) one
96101
from a BMP back to a BMP, @napi-rs/canvas draws into a new canvas. PureJsImage has no blur, composite, rendering or
97102
animated GIFs.
98-
- Where the library lets you choose, resizing is bilinear, blurring is
99-
gaussian with a radius or sigma of 5, rotating grows the image to fit. Each library uses its own defaults
103+
- Where the library lets you choose, resizing is bilinear (or bicubic, in its own row), the thumbnail covers 400×400
104+
(cropping the sides), blurring is gaussian with a radius or sigma of 5, sharpening an unsharp mask of radius or
105+
sigma 1 (Photon: its 3×3 sharpening kernel), rotating by 30° grows the image to fit, brightness and contrast go up
106+
20% (Jimp's and Photon's scales converted). image-js's and sharp's greyscale make one grey channel (and alpha). Each library uses its own defaults
100107
otherwise, e.g. its PNG compression level.
101108
- Threads: ImageScript uses worker threads where they pay off (the processors available, at most 8), and sharp gets
102109
as many (`sharp.concurrency()`; its default on glibc Linux without jemalloc, like on CI, is one). The other
@@ -108,4 +115,4 @@ Encoded sizes:
108115
- Install sizes: `npm install <library>` in an empty project, the apparent size of `node_modules`.
109116

110117
Run it: `npm install` in this directory, then `node run.js` (on an idle machine; `IMAGESCRIPT=local` measures
111-
this checkout after `npm run build` instead of the version from npm), `node installs.js` and `node report.js`.
118+
this checkout after `npm run build` instead of the version from npm; it runs `startup.js` too), `node installs.js` and `node report.js`.

‎bench/bench.js‎

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
import fs from 'node:fs';
55
import sharp from 'sharp';
66
import { Bench } from 'tinybench';
7-
import { libraries, referenceDecode, AVIF_QUALITY } from './libs.js';
7+
import { libraries, referenceDecode, AVIF_QUALITY, WEBP_QUALITY } from './libs.js';
88

99
const lib = libraries.find(l => l.name === process.argv[2]);
1010
if (!lib) throw new Error(`unknown library ${process.argv[2]}, one of: ${libraries.map(l => l.name).join(', ')}`);
@@ -16,15 +16,17 @@ const files = {
1616
'animation.gif': fs.readFileSync(new URL('./inputs/animation.gif', import.meta.url))
1717
};
1818
files['photo.png'] = await sharp(files['photo.jpg']).png().toBuffer();
19-
files['photo.webp'] = await sharp(files['photo.jpg']).webp({ quality: 80 }).toBuffer();
19+
files['photo.webp'] = await sharp(files['photo.jpg']).webp({ quality: WEBP_QUALITY }).toBuffer();
20+
// (the animation as an animated WebP, by libwebp)
21+
files['animation.webp'] = await sharp(files['animation.gif'], { animated: true }).webp({ quality: WEBP_QUALITY }).toBuffer();
2022
// (and AVIF by libaom, JPEG XL by libjxl: sharp's and jsquash's encoders at the benchmark's qualities)
2123
files['photo.avif'] = await sharp(files['photo.jpg']).avif({ quality: AVIF_QUALITY }).toBuffer();
2224
const rawOf = async file => { const { data, info } = await sharp(files[file]).ensureAlpha().raw().toBuffer({ resolveWithObject: true }); return { data, width: info.width, height: info.height }; };
2325
const pixels = { 'photo.jpg': await rawOf('photo.jpg'), 'graphic.png': await rawOf('graphic.png'), 'overlay.png': await rawOf('overlay.png') };
2426
files['photo.jxl'] = new Uint8Array(await (libraries.find(l => l.name === 'jsquash').encode.jxl)({ ...pixels['photo.jpg'], data: new Uint8ClampedArray(pixels['photo.jpg'].data) }));
2527

2628
const speed = {}, sizes = {};
27-
const formatName = format => ({ jpeg: 'JPEG', png: 'PNG', webp: 'WebP', avif: 'AVIF', jxl: 'JPEG XL' })[format];
29+
const formatName = format => ({ jpeg: 'JPEG', png: 'PNG', webp: 'WebP', webpLossy: 'WebP lossy', avif: 'AVIF', jxl: 'JPEG XL', jxlLossless: 'JPEG XL lossless' })[format];
2830

2931
// collect: canvases (native memory) and Photon images (WebAssembly memory) hold their pixels outside the V8 heap and
3032
// only free them in finalizers that run on a later event-loop turn, which tinybench's loop never yields to. So between
@@ -54,7 +56,7 @@ for (const [task, file, format] of [['decode JPEG', 'photo.jpg'], ['decode PNG',
5456
}
5557
for (const [format, encode] of Object.entries(lib.encode))
5658
await measure(`encode ${formatName(format)}`, () => encode(photo));
57-
for (const op of ['resize', 'rotate', 'blur', 'composite', 'crop', 'clone'])
59+
for (const op of ['resize', 'resizeBicubic', 'cover', 'crop', 'rotate', 'rotate90', 'flip', 'blur', 'sharpen', 'composite', 'greyscale', 'adjust'])
5860
if (lib[op]) await measure(op, () => lib[op](photo, overlay));
5961
// a library that has the feature but fails on this input is recorded as failing
6062
for (const op of ['text', 'svg']) {
@@ -67,10 +69,12 @@ if (lib.animation) {
6769
const animation = await lib.animation.decode(files['animation.gif']);
6870
await measure('decode GIF', () => lib.animation.decode(files['animation.gif']));
6971
await measure('encode GIF', () => lib.animation.encode(animation));
72+
await measure('decode animated WebP', () => lib.animation.decode(files['animation.webp']));
73+
if (lib.animation.encodeWebP) await measure('encode animated WebP', () => lib.animation.encodeWebP(animation));
7074
}
7175

7276
// encoded sizes, with each library's default settings (JPEG at quality 80, WebP lossless, AVIF and JPEG XL at the same
73-
// quality), and the PSNR (of the color over black) of the lossy ones: JPEG, AVIF, JPEG XL and the GIF
77+
// quality), and the PSNR (of the color over black) of the lossy ones: JPEG, lossy WebP, AVIF, JPEG XL and the animations
7478
const quality = {};
7579
const sharpPixels = async (data, animated = false) => (await sharp(data, { animated }).ensureAlpha().raw().toBuffer());
7680
const psnr = (a, b) => {
@@ -88,8 +92,8 @@ for (const [name, file] of [['photo', 'photo.jpg'], ['graphic', 'graphic.png']])
8892
const encoded = new Uint8Array(await encode(image));
8993
sizes[`${name} ${formatName(format)}`] = encoded.length;
9094
if (format === 'avif' || format === 'jxl') quality[`${name} ${formatName(format)}`] = psnr(pixels[file].data, await referenceDecode(encoded, format));
91-
// (JPEG decoded by libjpeg-turbo, through sharp)
92-
if (format === 'jpeg') quality[`${name} JPEG`] = psnr(pixels[file].data, await sharpPixels(encoded));
95+
// (JPEG and WebP decoded by libjpeg-turbo and libwebp, through sharp)
96+
if (format === 'jpeg' || format === 'webpLossy') quality[`${name} ${formatName(format)}`] = psnr(pixels[file].data, await sharpPixels(encoded));
9397
}
9498
}
9599
if (lib.animation) {
@@ -98,6 +102,12 @@ if (lib.animation) {
98102
// (all frames, decoded by libnsgif through sharp; only if the file keeps every frame, encoders may merge equal ones)
99103
const [a, b] = await Promise.all([sharpPixels(files['animation.gif'], true), sharpPixels(encoded, true)]);
100104
if (a.length === b.length) quality['animation GIF'] = psnr(a, b);
105+
if (lib.animation.encodeWebP) {
106+
const webp = await lib.animation.encodeWebP(await lib.animation.decode(files['animation.gif']));
107+
sizes['animation WebP'] = webp.length;
108+
const c = await sharpPixels(webp, true);
109+
if (a.length === c.length) quality['animation WebP'] = psnr(a, c);
110+
}
101111
}
102112

103113
console.log(JSON.stringify({ speed, sizes, quality }));

0 commit comments

Comments
 (0)