Skip to content

Commit dc518c4

Browse files
matmenclaude
andcommitted
JXL: apply embedded ICC profiles
The ICC profile stream was read and dropped; now it's decoded (its header prediction, tag table and commands, ported from jxl-oxide's icc/decode.rs) and, for images whose pixels are in its space (not XYB: lossless ones, recompressed JPEGs), applied when decoding like PNGs' and JPEGs' (Image.decode and Animation.decode, applyColorProfile false keeps the stored values). XYB images already decode to sRGB; their profile only says what the image was. The decoded profiles are byte for byte libjxl's (djxl --icc_out) for lossless P3, Adobe RGB, Rec. 2020, table-curve and gray profiles. The eight conformance files this changes (cafe, spot, patches_lossless, ...) match djxl's output with its profile applied (within 1-4; up to 20 where ±1 of the stored value lands near the sRGB gamut's edge), and lossy XYB files with a profile stay within 1 of djxl's sRGB. New fixtures by cjxl in tests/icc (scripts/icc-fixtures.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsU2GAqbnFSNaYDBGD1Km2
1 parent 7381c94 commit dc518c4

10 files changed

Lines changed: 215 additions & 12 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
# Changelog
22

3+
## 3.11.0
4+
5+
### Changes
6+
7+
- JPEG XLs with an embedded ICC profile whose pixels are in its color space (lossless ones and recompressed JPEGs, not
8+
XYB) are converted to sRGB when decoding, like the other formats' (`applyColorProfile: false` keeps the stored
9+
values): the profile's stream is decoded (ported from jxl-oxide; the same profiles as libjxl's), then applied like
10+
PNGs' and JPEGs'. Lossy (XYB) JPEG XLs already decode to sRGB, their profiles only say what the image was before.
11+
312
## 3.10.0
413

514
### Performance

‎THIRD_PARTY_NOTICES.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ following projects, under their own licenses. Each derived file says so in its h
1616
| [dav1d](https://code.videolan.org/videolan/dav1d) by VideoLAN, the dav1d authors and Two Orioles | `src/codecs/av1/*` (the AV1 decoder, and the encoder's syntax, contexts and reconstruction mirrored from it: bitstream parsing, symbol decoding, prediction, inverse transforms, loop filters, film grain; the tables in `src/codecs/av1/data.ts`, generated from dav1d by `scripts/av1-tables.c`) | BSD-2-Clause |
1717
| [libaom](https://aomedia.googlesource.com/aom/) by the Alliance for Open Media | `src/codecs/av1/msac-enc.ts` (the symbol encoder), `src/codecs/av1/fdct.ts` (the forward Walsh-Hadamard transform), `src/codecs/av1/encode.ts` (the loop filter level from the quantizer), `src/codecs/avif-enc.ts` (the quantizer to qindex table) | BSD-2-Clause |
1818
| [libavif](https://github.com/AOMediaCodec/libavif) by Joe Drago | `src/codecs/avif.ts` (YUV to RGB conversion with bilinear chroma upsampling, alpha range expansion and unpremultiplication) | BSD-2-Clause |
19-
| [jxl-oxide](https://github.com/tirr-c/jxl-oxide) by Wonwoo Choi | `src/codecs/jxl/*` (the JPEG XL decoder: bit reader, header and frame header fields, entropy decoding, VarDCT, dequantization matrices, transforms, restoration filters, upsampling, the parsing of patches, splines and noise) | MIT (or Apache-2.0) |
19+
| [jxl-oxide](https://github.com/tirr-c/jxl-oxide) by Wonwoo Choi | `src/codecs/jxl/*` (the JPEG XL decoder: bit reader, header and frame header fields, entropy decoding, VarDCT, dequantization matrices, transforms, restoration filters, upsampling, the parsing of patches, splines and noise, the ICC profile stream) | MIT (or Apache-2.0) |
2020
| [libjxl](https://github.com/libjxl/libjxl) by the JPEG XL Project Authors | `src/codecs/jxl/modular.ts` (Modular decoding: MA trees, predictors, the self-correcting predictor, transforms), `src/codecs/jxl/frame.ts` (the sections of frames, the order of the rendering stages, reference frames), `src/codecs/jxl/blend.ts`, `src/codecs/jxl/color.ts` (XYB to RGB), `src/codecs/jxl/features.ts` (rendering patches, splines and noise), `src/codecs/jxl/fast-lossless.ts` (the lossless encoder: palette detection, YCoCg, the gradient predictor, prefix codes, runs as LZ77, the headers), `src/codecs/jxl/patches-enc.ts` (finding text-like patches, packing them into a reference frame, quantizing it like Modular XYB, the patch dictionary) | BSD-3-Clause |
2121
| [JPEG XL conformance](https://github.com/libjxl/conformance) by the JPEG XL Project Authors | `tests/jxl/alpha-*.jxl`, `animation-spline.jxl`, `blendmodes.jxl`, `mul-no-extra-channels.jxl` (test files) | BSD-3-Clause |
2222
| [Various smartphones - Android, Blackberry, Windows Phone, and iOS.jpg](https://commons.wikimedia.org/wiki/File:Various_smartphones_-_Android,_Blackberry,_Windows_Phone,_and_iOS.jpg) by Carlos Varela | `tests/targets/external.png` and `external.jpg` (a square of it, scaled to 638x638), the test files made from them (`tests/avif`, `tests/jxl`, `tests/ico`, `tests/hdr`, `tests/qoi`, `tests/tiffs`) and the test targets showing them; `bench/inputs/photo.jpg` (unmodified) | [CC BY 2.0](https://creativecommons.org/licenses/by/2.0/) |

‎scripts/icc-fixtures.py‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@
33
# ImageCms) converts them to, relative colorimetric to sRGB, as PNGs: from the source pixels for the lossless formats,
44
# from what Pillow decodes for JPEG and AVIF. The profiles are matrix/TRC ones written here (Display P3 with a
55
# parametric curve, Adobe RGB with a gamma, P3 with a table curve, Rec. 2020 with a type 4 parametric curve, gray), and
6-
# one that is sRGB (its pixels stay as they are), and a gray one on a color image (not applied). Needs Pillow (with AVIF) and numpy:
7-
# python scripts/icc-fixtures.py
8-
import io, struct
6+
# one that is sRGB (its pixels stay as they are), and a gray one on a color image (not applied). Also JPEG XLs by
7+
# cjxl: lossless ones (their pixels are in the profile's space) and a lossy one (XYB, decoded to sRGB: its profile is
8+
# only what the image was before). Needs Pillow (with AVIF), numpy and cjxl (libjxl):
9+
# CJXL=path/to/cjxl python scripts/icc-fixtures.py
10+
import io, os, struct, subprocess
911
from pathlib import Path
1012
import numpy as np
1113
from PIL import Image, ImageCms
@@ -129,3 +131,8 @@ def convert(im, icc):
129131
convert(gray, gray_profile).save(out / 'gray.ref.png')
130132
# a gray profile on a color image (invalid, like libpng says: not applied)
131133
source.save(out / 'gray-on-rgb.png', icc_profile=gray_profile)
134+
135+
# JPEG XLs: lossless (the pixels in the profile's space, compared with the PNGs' references) and lossy (XYB with the
136+
# profile, decoded to sRGB without it)
137+
for name, distance in [('display-p3', 0), ('gray', 0), ('rec2020', 1)]:
138+
subprocess.run([os.environ.get('CJXL', 'cjxl'), '-d', str(distance), str(out / f'{name}.png'), str(out / f'{name}{"-xyb" if distance else ""}.jxl')], check=True, capture_output=True)

‎src/animation.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -451,15 +451,16 @@ export class Animation extends Array<Frame> {
451451

452452
if (type === 'jxl') {
453453
const frames: Frame[] = [];
454-
let loopCount = -1;
454+
let loopCount = -1, icc: Uint8Array | undefined;
455455
for await (const decoded of jxl.frames(data, maxPixels, 'reinhard', threads)) {
456456
const frame = new Frame(decoded.width, decoded.height, decoded.duration, 0, 0, Frame.DISPOSAL_BACKGROUND);
457457
frame.bitmap = decoded.pixels;
458458
frames.push(frame.__orient__(decoded.orientation));
459459
loopCount = fromPlays(decoded.loopCount);
460+
icc = decoded.icc;
460461
if (onlyExtractFirstFrame) break;
461462
}
462-
return new Animation(frames, frames.length > 1 ? loopCount : -1);
463+
return new Animation(profile(frames, icc), frames.length > 1 ? loopCount : -1);
463464
}
464465

465466
if (type) return new Animation([Frame.from(await Image.decode(data, { maxPixels, applyColorProfile }), 0)]);

‎src/codecs/jxl.ts‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
import { startThreads } from '../util/threads.ts';
88
import { MAX_PIXELS, checkPixels } from '../util/limits.ts';
99
import { BitReader } from './jxl/bits.ts';
10-
import { imageHeader, skipICC } from './jxl/headers.ts';
10+
import { decodeICC, imageHeader, readICC } from './jxl/headers.ts';
1111
import { frameHeader, toc } from './jxl/frame-header.ts';
1212
import { decodeFrame, DecoderState, type FrameThreads } from './jxl/frame.ts';
1313
import { ColorOutput } from './jxl/color.ts';
@@ -73,12 +73,15 @@ function codestream(d: Uint8Array): Uint8Array {
7373
}
7474

7575
/** The frames shown (composited, as RGBA), their durations in milliseconds */
76-
export async function* frames(data: Uint8Array, maxPixels = MAX_PIXELS, toneMapping: ToneMapping = 'reinhard', threads = 1): AsyncGenerator<{ width: number, height: number, pixels: Uint8Array, duration: number, orientation: number, loopCount: number }> {
76+
export async function* frames(data: Uint8Array, maxPixels = MAX_PIXELS, toneMapping: ToneMapping = 'reinhard', threads = 1): AsyncGenerator<{ width: number, height: number, pixels: Uint8Array, duration: number, orientation: number, loopCount: number, icc?: Uint8Array }> {
7777
const d = codestream(data);
7878
const r = new BitReader(d);
7979
const image = imageHeader(r);
8080
checkPixels(image.width, image.height, maxPixels);
81-
if (image.colour.wantICC) skipICC(r);
81+
// the embedded ICC profile: what the pixels are in unless they're XYB (which decodes to sRGB here; its profile only
82+
// says what the image was before)
83+
const stream = image.colour.wantICC ? readICC(r) : null;
84+
const icc = stream && !image.xybEncoded ? decodeICC(stream) ?? undefined : undefined;
8285
r.zeroPadToByte();
8386
let at = r.bytePosition;
8487
if (image.preview) {
@@ -111,7 +114,7 @@ export async function* frames(data: Uint8Array, maxPixels = MAX_PIXELS, toneMapp
111114
shown++;
112115
yield {
113116
width: image.width, height: image.height, pixels: state.output.toRGBA(frame.canvas), duration: frame.header.duration * tick,
114-
orientation: image.orientation, loopCount: a ? a.numLoops : 1
117+
orientation: image.orientation, loopCount: a ? a.numLoops : 1, icc
115118
};
116119
}
117120
if (frame.header.isLast) return;
@@ -122,7 +125,7 @@ export async function* frames(data: Uint8Array, maxPixels = MAX_PIXELS, toneMapp
122125
}
123126

124127
/** Decodes the first frame shown */
125-
export async function decode(data: Uint8Array, maxPixels = MAX_PIXELS, toneMapping: ToneMapping = 'reinhard', threads = 1): Promise<{ width: number, height: number, pixels: Uint8Array, orientation: number }> {
128+
export async function decode(data: Uint8Array, maxPixels = MAX_PIXELS, toneMapping: ToneMapping = 'reinhard', threads = 1): Promise<{ width: number, height: number, pixels: Uint8Array, orientation: number, icc?: Uint8Array }> {
126129
for await (const frame of frames(data, maxPixels, toneMapping, threads)) return frame;
127130
throw new Error('JPEG XL: no frame shown');
128131
}

‎src/codecs/jxl/headers.ts‎

Lines changed: 172 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -241,12 +241,13 @@ export function imageHeader(r: BitReader): ImageHeader {
241241
}
242242

243243
/** Reads (and skips) the embedded ICC profile's stream */
244-
export function skipICC(r: BitReader): void {
244+
export function readICC(r: BitReader): Uint8Array {
245245
const size = r.u64();
246246
if (size > 1 << 28) throw new Error('JPEG XL: ICC profile too large');
247247
const dec = new EntropyDecoder(r, 41);
248248
dec.begin(r);
249249
const kind = (b: number): number => b >= 97 && b <= 122 || b >= 65 && b <= 90 ? 0 : b >= 48 && b <= 57 || b === 46 || b === 44 ? 1 : -1;
250+
const out = new Uint8Array(size);
250251
let b1 = 0, b2 = 0;
251252
for (let i = 0; i < size; i++) {
252253
let ctx = 0;
@@ -258,10 +259,180 @@ export function skipICC(r: BitReader): void {
258259
}
259260
const b = dec.read(r, ctx);
260261
if (b > 255) throw new Error('JPEG XL: invalid ICC profile');
262+
out[i] = b;
261263
b2 = b1;
262264
b1 = b;
263265
}
264266
dec.finish();
267+
return out;
268+
}
269+
270+
// the tags and tag types the ICC stream's commands abbreviate
271+
const COMMON_TAGS = ['rTRC', 'rXYZ', 'cprt', 'wtpt', 'bkpt', 'rXYZ', 'gXYZ', 'bXYZ', 'kXYZ', 'rTRC', 'gTRC', 'bTRC', 'kTRC', 'chad', 'desc', 'chrm', 'dmnd', 'dmdd', 'lumi'];
272+
const COMMON_DATA = ['XYZ ', 'desc', 'text', 'mluc', 'para', 'curv', 'sf32', 'gbd '];
273+
274+
/** The ICC profile an encoded ICC stream (see readICC) describes, null if the stream is invalid */
275+
export function decodeICC(stream: Uint8Array): Uint8Array | null {
276+
let at = 0;
277+
const fail = (): never => {
278+
throw new RangeError('invalid ICC stream');
279+
};
280+
const varint = (s: Uint8Array, pos: { at: number }): number => {
281+
let value = 0;
282+
for (let shift = 0; shift < 63; shift += 7) {
283+
if (pos.at >= s.length) fail();
284+
const b = s[pos.at++];
285+
value += (b & 127) * 2 ** shift;
286+
if (!(b & 128)) break;
287+
}
288+
return value;
289+
};
290+
try {
291+
const head = { at: 0 };
292+
const outputSize = varint(stream, head), commandsSize = varint(stream, head);
293+
at = head.at;
294+
if (at + commandsSize > stream.length || outputSize > 1 << 28) fail();
295+
const commands = stream.subarray(at, at + commandsSize), cmd = { at: 0 };
296+
let data = stream.subarray(at + commandsSize);
297+
const headerSize = Math.min(outputSize, 128);
298+
if (data.length < headerSize) fail();
299+
const out = new Uint8Array(outputSize);
300+
let o = 0;
301+
const push = (...bytes: number[]): void => pushBytes(bytes);
302+
const pushBytes = (bytes: ArrayLike<number>): void => {
303+
if (o + bytes.length > outputSize) fail();
304+
out.set(bytes, o);
305+
o += bytes.length;
306+
};
307+
const pushText = (t: string): void => pushBytes([...t].map(c => c.charCodeAt(0)));
308+
const u32 = (v: number): number[] => [v >>> 24 & 255, v >>> 16 & 255, v >>> 8 & 255, v & 255];
309+
const take = (n: number): Uint8Array => {
310+
if (n > data.length) fail();
311+
const bytes = data.subarray(0, n);
312+
data = data.subarray(n);
313+
return bytes;
314+
};
315+
316+
// the header: predicted, the stream has the differences
317+
const header = take(headerSize);
318+
const predict = (i: number): number => {
319+
const h40 = String.fromCharCode(header[40]), h41 = String.fromCharCode(header[41]);
320+
if (i <= 3) return u32(outputSize)[i];
321+
if (i === 8) return 4;
322+
if (i >= 12 && i <= 23) return 'mntrRGB XYZ '.charCodeAt(i - 12);
323+
if (i >= 36 && i <= 39) return 'acsp'.charCodeAt(i - 36);
324+
if (h40 === 'A' && i >= 41 && i <= 43) return 'PPL'.charCodeAt(i - 41);
325+
if (h40 === 'M' && i >= 41 && i <= 43) return 'SFT'.charCodeAt(i - 41);
326+
if (h40 === 'S' && h41 === 'G' && (i === 42 || i === 43)) return 'I '.charCodeAt(i - 42);
327+
if (h40 === 'S' && h41 === 'U' && (i === 42 || i === 43)) return 'NW'.charCodeAt(i - 42);
328+
if (i === 70) return 246;
329+
if (i === 71) return 214;
330+
if (i === 73) return 1;
331+
if (i === 78) return 211;
332+
if (i === 79) return 45;
333+
if (i >= 80 && i <= 83) return header[4 + i - 80];
334+
return 0;
335+
};
336+
for (let i = 0; i < headerSize; i++) push(predict(i) + header[i] & 255);
337+
if (outputSize <= 128) return out;
338+
339+
// the tag table
340+
const v = varint(commands, cmd);
341+
if (v > 0) {
342+
const numTags = v - 1;
343+
if ((outputSize - 128) / 12 < numTags) fail();
344+
push(...u32(numTags));
345+
let prevStart = numTags * 12 + 128, prevSize = 0;
346+
for (; ;) {
347+
// (like jxl-oxide: the commands may end here)
348+
if (cmd.at >= commands.length) return out.subarray(0, o);
349+
const command = commands[cmd.at++], code = command & 63;
350+
if (code === 0) break;
351+
let tag: string;
352+
if (code === 1) tag = String.fromCharCode(...take(4));
353+
else if (code <= 20) tag = COMMON_TAGS[code - 2];
354+
else return fail();
355+
const start = command & 64 ? varint(commands, cmd) : prevStart + prevSize;
356+
const size = command & 128 ? varint(commands, cmd) : ['rXYZ', 'gXYZ', 'bXYZ', 'kXYZ', 'wtpt', 'bkpt', 'lumi'].includes(tag) ? 20 : prevSize;
357+
if (start + size > outputSize) fail();
358+
prevStart = start;
359+
prevSize = size;
360+
pushText(tag);
361+
push(...u32(start), ...u32(size));
362+
if (code === 2) {
363+
pushText('gTRC');
364+
push(...u32(start), ...u32(size));
365+
pushText('bTRC');
366+
push(...u32(start), ...u32(size));
367+
} else if (code === 3) {
368+
pushText('gXYZ');
369+
push(...u32(start + size), ...u32(size));
370+
pushText('bXYZ');
371+
push(...u32(start + 2 * size), ...u32(size));
372+
}
373+
}
374+
}
375+
376+
// the rest: copies, shuffled copies, predicted numbers, abbreviated XYZ values and tag types
377+
const shuffle = (bytes: Uint8Array, width: number): Uint8Array => {
378+
const n = bytes.length, shuffled = new Uint8Array(n);
379+
if (width === 2) {
380+
const half = n >> 1, odd = n & 1;
381+
let k = 0;
382+
for (let i = 0; i < half; i++) {
383+
shuffled[k++] = bytes[i];
384+
shuffled[k++] = bytes[i + half + odd];
385+
}
386+
if (odd) shuffled[k] = bytes[half];
387+
} else {
388+
const step = n >> 2, wide = n & 3;
389+
let k = 0;
390+
for (let i = 0; i < step; i++) {
391+
let base = i;
392+
for (let j = 0; j < wide; j++, base += step + 1) shuffled[k++] = bytes[base];
393+
for (let j = wide; j < 4; j++, base += step) shuffled[k++] = bytes[base];
394+
}
395+
for (let i = 1; i <= wide; i++) shuffled[k++] = bytes[(step + 1) * i - 1];
396+
}
397+
return shuffled;
398+
};
399+
while (cmd.at < commands.length) {
400+
const command = commands[cmd.at++];
401+
if (command === 1) pushBytes(take(varint(commands, cmd)));
402+
else if (command === 2 || command === 3) pushBytes(shuffle(take(varint(commands, cmd)), command === 2 ? 2 : 4));
403+
else if (command === 4) {
404+
if (cmd.at >= commands.length) fail();
405+
const flags = commands[cmd.at++], width = (flags & 3) + 1, order = flags >> 2 & 3;
406+
if (width === 3 || order === 3) fail();
407+
const stride = flags & 16 ? varint(commands, cmd) : width;
408+
if (stride < width || stride * 4 >= o) fail();
409+
const num = varint(commands, cmd), raw = take(num), bytes = width === 1 ? raw : shuffle(raw, width);
410+
for (let i = 0; i < num; i += width) {
411+
const prev = [0, 0, 0];
412+
for (let j = 0; j <= order; j++) {
413+
const from = o - stride * (j + 1);
414+
let value = 0;
415+
for (let k = 0; k < width; k++) value = (value << 8 | out[from + k]) >>> 0;
416+
prev[j] = value;
417+
}
418+
const p = order === 0 ? prev[0] : order === 1 ? 2 * prev[0] - prev[1] >>> 0 : 3 * (prev[0] - prev[1]) + prev[2] >>> 0;
419+
for (let j = 0; j < Math.min(width, num - i); j++) push(bytes[i + j] + (p >>> 8 * (width - 1 - j)) & 255);
420+
}
421+
} else if (command === 10) {
422+
pushText('XYZ ');
423+
push(0, 0, 0, 0);
424+
pushBytes(take(12));
425+
}
426+
else if (command >= 16 && command <= 23) {
427+
pushText(COMMON_DATA[command - 16]);
428+
push(0, 0, 0, 0);
429+
} else fail();
430+
}
431+
return o === outputSize ? out : null;
432+
} catch (e) {
433+
if (e instanceof RangeError) return null;
434+
throw e;
435+
}
265436
}
266437

267438
// the default upsampling weights

0 commit comments

Comments
 (0)