Skip to content

Commit ae710fb

Browse files
matmenmatmen
committed
fix(gainmap): apply color operations correctly to HDR values above 1.0
1 parent 816cda8 commit ae710fb

5 files changed

Lines changed: 163 additions & 101 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,18 @@
88
each part gets on HDR displays) stay HDR. `Image.decode` reads the gain map into the new `image.gainMap` (a
99
`GainMap`: its pixels, decoded only when needed, and its parameters, from ISO 21496-1 or Adobe's hdrgm XMP), and
1010
`encodeJPEG()` writes it back with both kinds of metadata (Android, Chrome and Apple read them): decoding and
11-
re-encoding a photo keeps the gain map as it was. Before, it was dropped and HDR photos came out SDR only.
11+
re-encoding a photo keeps the gain map. Before, it was dropped and HDR photos came out SDR only.
1212
- The gain map follows the image: resizing, `cover`/`contain`/`fit`, cropping, `region`, trimming, rotating,
1313
flipping, `fisheye` and the Exif orientation transform it alike (at its own resolution, a quarter of the width
1414
and height in photos from phones).
1515
- Drawing keeps it right: what `composite()` puts over the image brings its own gain map (or none: no boost there),
1616
blended with the image's by the light of both; `drawBox`, `drawCircle` and `fill` leave no boost where they
1717
draw. Images without a gain map get one when an HDR image is composited onto them.
18-
- Color changes (`brightness`, `contrast`, `gamma`, `lightness`, `red`/`green`/`blue`, the inverts, and for maps
19-
with a gain per channel also `saturation`, `hueShift`, `greyscale`, `sepia`) are made to the HDR rendition too,
20-
with its brightest light as white, so the highlights change like the rest; `blur` blurs the gain map alike.
18+
- Color changes (`brightness`, `contrast`, `gamma`, `lightness`, `red`/`green`/`blue`, and for maps with a gain
19+
per channel also `saturation`, `hueShift`, `greyscale`, `sepia`) are made to the HDR rendition too, on values
20+
with SDR white as 1 and the highlights above it: what HDR displays don't boost changes like on SDR ones, the
21+
highlights change alike. The inverts leave no boost (the highlights would become the darkest parts); `blur`
22+
blurs the gain map alike.
2123
- `new GainMap(image, options)` makes gain maps (`image.gainMap = ...`, `null` removes it), `GainMap.decode()` and
2224
`gainMap.encode()` read and write one alone. `encodeJPEG({ gainMap: { quality, scale } })` re-encodes the map
2325
at a quality or size, `{ gainMap: false }` leaves it out; `DecodeOptions.gainMap: false` doesn't read it.

‎src/gainmap.ts‎

Lines changed: 41 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,7 @@ const perChannel = (value: number | number[], name: string): number[] => {
3636
* (JPEG-R) photos from phones carry it. HDR displays show the image boosted by it, the others the image as it is.
3737
*
3838
* {@link Image.decode} reads the gain maps of JPEGs, AVIFs and JPEG XLs into {@link Image.gainMap},
39-
* {@link Image.encodeJPEG}, {@link Image.encodeAVIF} and {@link Image.encodeJXL} write them;
40-
* the map's pixels are only decoded once they're needed. The map covers the whole image, at any resolution (phones
39+
* {@link Image.encodeJPEG}, {@link Image.encodeAVIF} and {@link Image.encodeJXL} write them. The map covers the whole image, at any resolution (phones
4140
* write a quarter of the width and height). Its value `v` (0-255, per channel or one for all) boosts the linear
4241
* light of the image by `2^(gainMapMin + (gainMapMax - gainMapMin) * (v / 255)^(1 / gamma))`
4342
*/
@@ -59,12 +58,8 @@ export class GainMap {
5958
/** whether the boost applies in the SDR image's color space (else in the HDR image's) */
6059
useBaseColorSpace: boolean;
6160

62-
/** the map's JPEG (without metadata) until its pixels are used */
63-
private __jpeg__: Uint8Array | null = null;
64-
/** its pixels from then on */
61+
/** its pixels */
6562
private __pixels__: Image | null = null;
66-
/** an Apple gain map's headroom: its values are converted when decoded (and the JPEG can't be kept as it is) */
67-
private __apple__ = 0;
6863

6964
/**
7065
* Creates a gain map
@@ -87,21 +82,38 @@ export class GainMap {
8782
this.__check__();
8883
}
8984

90-
/** @internal from a decoded file: the map's JPEG, its pixels decoded when first used */
85+
/**
86+
* @internal from a decoded file: the map's JPEG, decoded now (a damaged one throws here, not in a later
87+
* operation); `apple`: an Apple gain map's headroom, its values converted to the parameters' scale
88+
*/
9189
static __from_jpeg__(map: Uint8Array, params: codec.Params, apple = 0): GainMap {
92-
const gainMap = Object.create(GainMap.prototype) as GainMap;
93-
gainMap.__pixels__ = null;
94-
gainMap.__apple__ = apple;
95-
gainMap.__jpeg__ = codec.stripMetadata(map);
96-
gainMap.__set_params__(params);
97-
return gainMap;
90+
const { width, height, pixels } = jpeg.decode(map);
91+
const image = new Image(width, height);
92+
image.bitmap = pixels;
93+
if (apple) {
94+
const table = new Uint8Array(256);
95+
for (let v = 0; v < 256; v++) table[v] = Math.round(255 * codec.appleValue(v / 255, apple));
96+
for (let i = 0; i < pixels.length; i += 4) {
97+
pixels[i] = table[pixels[i]];
98+
pixels[i + 1] = table[pixels[i + 1]];
99+
pixels[i + 2] = table[pixels[i + 2]];
100+
}
101+
}
102+
return GainMap.__from_pixels__(image, params, false);
98103
}
99104

100-
/** @internal from decoded pixels (an AVIF's or JPEG XL's map) */
101-
static __from_pixels__(image: Image, params: codec.Params): GainMap {
105+
/**
106+
* @internal from decoded pixels (an AVIF's or JPEG XL's map); `grey`: a map whose channels differ by at most 2
107+
* levels anywhere is made grey (a grey map stored in color, like libjxl writes them: one channel when written)
108+
*/
109+
static __from_pixels__(image: Image, params: codec.Params, grey = true): GainMap {
110+
const px = image.bitmap;
111+
if (grey) {
112+
let spread = 0;
113+
for (let i = 0; i < px.length && spread <= 2; i += 4) spread = Math.max(spread, Math.abs(px[i] - px[i + 1]), Math.abs(px[i] - px[i + 2]));
114+
if (spread > 0 && spread <= 2) for (let i = 0; i < px.length; i += 4) px[i] = px[i + 1] = px[i + 2] = Math.round((px[i] + px[i + 1] + px[i + 2]) / 3);
115+
}
102116
const gainMap = Object.create(GainMap.prototype) as GainMap;
103-
gainMap.__jpeg__ = null;
104-
gainMap.__apple__ = 0;
105117
gainMap.__pixels__ = image;
106118
gainMap.__set_params__(params);
107119
return gainMap;
@@ -140,50 +152,29 @@ export class GainMap {
140152
if (p.capacityMin < 0 || p.capacityMax <= p.capacityMin) throw new RangeError('The gain map\'s hdrCapacityMax must be above its hdrCapacityMin, which can\'t be negative');
141153
}
142154

143-
/** The map's pixels (decoded when first read). Changes to them are written when the image is encoded */
155+
/** The map's pixels. Changes to them are written when the image is encoded */
144156
get image(): Image {
145-
if (!this.__pixels__) {
146-
const { width, height, pixels } = jpeg.decode(this.__jpeg__ as Uint8Array);
147-
const image = new Image(width, height);
148-
image.bitmap = pixels;
149-
if (this.__apple__) {
150-
// (Apple's values to the parameters' log2 scale)
151-
const table = new Uint8Array(256);
152-
for (let v = 0; v < 256; v++) table[v] = Math.round(255 * codec.appleValue(v / 255, this.__apple__));
153-
for (let i = 0; i < pixels.length; i += 4) {
154-
pixels[i] = table[pixels[i]];
155-
pixels[i + 1] = table[pixels[i + 1]];
156-
pixels[i + 2] = table[pixels[i + 2]];
157-
}
158-
this.__apple__ = 0;
159-
}
160-
this.__pixels__ = image;
161-
// (from now on the pixels can change: encoding writes them anew)
162-
this.__jpeg__ = null;
163-
}
164-
return this.__pixels__;
157+
return this.__pixels__ as Image;
165158
}
166159

167160
set image(image: Image) {
168161
if (!(image instanceof Image)) throw new TypeError('The gain map has to be an Image');
169162
this.__pixels__ = image;
170-
this.__jpeg__ = null;
171163
}
172164

173-
/** The map's width (without decoding it) */
165+
/** The map's width */
174166
get width(): number {
175-
return this.__pixels__ ? this.__pixels__.width : codec.dimensions(this.__jpeg__ as Uint8Array)?.width ?? 0;
167+
return (this.__pixels__ as Image).width;
176168
}
177169

178-
/** The map's height (without decoding it) */
170+
/** The map's height */
179171
get height(): number {
180-
return this.__pixels__ ? this.__pixels__.height : codec.dimensions(this.__jpeg__ as Uint8Array)?.height ?? 0;
172+
return (this.__pixels__ as Image).height;
181173
}
182174

183175
/** `1` for a grey map (one boost for red, green and blue), `3` for one per channel */
184176
get channels(): 1 | 3 {
185-
if (!this.__pixels__) return codec.componentCount(this.__jpeg__ as Uint8Array) === 1 ? 1 : 3;
186-
const bitmap = this.__pixels__.bitmap;
177+
const bitmap = (this.__pixels__ as Image).bitmap;
187178
for (let i = 0; i < bitmap.length; i += 4) if (bitmap[i] !== bitmap[i + 1] || bitmap[i] !== bitmap[i + 2]) return 3;
188179
return 1;
189180
}
@@ -192,18 +183,14 @@ export class GainMap {
192183
clone(): GainMap {
193184
const copy = Object.create(GainMap.prototype) as GainMap;
194185
copy.__set_params__(this.__params__());
195-
copy.__jpeg__ = this.__jpeg__;
196-
copy.__apple__ = this.__apple__;
197-
copy.__pixels__ = this.__pixels__ ? this.__pixels__.clone() : null;
186+
copy.__pixels__ = (this.__pixels__ as Image).clone();
198187
return copy;
199188
}
200189

201190
/**
202-
* @internal the map's JPEG without metadata: the decoded file's as it is if the pixels weren't used and no quality
203-
* or size is asked for, else encoded anew
191+
* @internal the map as a JPEG without metadata, at a quality (default 90) and size
204192
*/
205193
__body__(quality?: number, width?: number, height?: number): Uint8Array {
206-
if (this.__jpeg__ && !this.__apple__ && quality === undefined && (width === undefined || width === this.width && height === this.height)) return this.__jpeg__;
207194
let image = this.image;
208195
if (width !== undefined && height !== undefined && (width !== image.width || height !== image.height)) image = image.clone().resize(width, height);
209196
const bitmap = image.bitmap;
@@ -270,8 +257,7 @@ export class GainMap {
270257
/**
271258
* Encodes the gain map alone: a JPEG with its parameters (hdrgm XMP and ISO 21496-1), like the second image in an
272259
* Ultra HDR JPEG
273-
* @param quality The JPEG quality (1-100). Default: `90`, or the decoded file's map as it is if its pixels weren't
274-
* used
260+
* @param quality The JPEG quality (1-100). Default: `90`
275261
*/
276262
async encode(quality?: number): Promise<Uint8Array> {
277263
if (quality !== undefined) quality = Math.max(1, checkQuality(quality));
@@ -289,9 +275,6 @@ export class GainMap {
289275
if (found) return GainMap.__from_jpeg__(found.map, found.params, found.apple);
290276
const params = codec.readParams(bytes);
291277
if (!params) throw new Error('No gain map (or no gain map parameters) found');
292-
const gainMap = GainMap.__from_jpeg__(bytes, params);
293-
// (decoded now: a damaged JPEG throws here, not later)
294-
void gainMap.image;
295-
return gainMap;
278+
return GainMap.__from_jpeg__(bytes, params);
296279
}
297280
}

‎src/image.ts‎

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -114,8 +114,8 @@ export interface JPEGEncodeOptions {
114114
metadata?: Metadata;
115115
/**
116116
* Whether to write the image's {@link Image.gainMap} (an Ultra HDR JPEG, brighter on HDR displays), or how:
117-
* `quality` the map's JPEG quality (default: `90`, or the decoded map as it was if it's unchanged), `scale` its
118-
* size as a factor of the image's (0-1, default: the map's size). Default: `true` (written if there is one)
117+
* `quality` the map's JPEG quality (default: `90`), `scale` its size as a factor of the image's (0-1, default: the
118+
* map's size). Default: `true` (written if there is one)
119119
*/
120120
gainMap?: boolean | { quality?: number, scale?: number };
121121
}
@@ -233,7 +233,7 @@ export interface DecodeOptions {
233233
toneMapping?: ToneMapping;
234234
/**
235235
* Whether to read the gain maps of JPEGs (Ultra HDR, JPEG-R, ISO 21496-1, Apple's), AVIFs (tone mapped items) and
236-
* JPEG XLs (jhgm boxes) into {@link Image.gainMap}. A JPEG's map's pixels are only decoded when they're needed. Default: `true`
236+
* JPEG XLs (jhgm boxes) into {@link Image.gainMap}. Default: `true`
237237
*/
238238
gainMap?: boolean;
239239
/**
@@ -1640,7 +1640,7 @@ export class Image {
16401640
* Inverts the images colors
16411641
*/
16421642
invert(): this {
1643-
if (this.__gain_map__) return this.__recolor__(image => image.invert());
1643+
if (this.__gain_map__) return this.__unboost__(image => image.invert());
16441644
const bitmap = this.bitmap;
16451645
for (let i = 0; i < bitmap.length; i += 4) {
16461646
bitmap[i] = 255 - bitmap[i];
@@ -1655,7 +1655,7 @@ export class Image {
16551655
* Inverts the images lightness (HSL), keeping hue and saturation: dark colors become light and vice versa
16561656
*/
16571657
invertLightness(): this {
1658-
if (this.__gain_map__) return this.__recolor__(image => image.invertLightness());
1658+
if (this.__gain_map__) return this.__unboost__(image => image.invertLightness());
16591659
return this.__map_hsla__(c => { c.l = 1 - c.l; });
16601660
}
16611661

@@ -1664,7 +1664,7 @@ export class Image {
16641664
* and the others scale along, black becomes white
16651665
*/
16661666
invertValue(): this {
1667-
if (this.__gain_map__) return this.__recolor__(image => image.invertValue());
1667+
if (this.__gain_map__) return this.__unboost__(image => image.invertValue());
16681668
const bitmap = this.bitmap;
16691669
for (let i = 0; i < bitmap.length; i += 4) {
16701670
const max = Math.max(bitmap[i], bitmap[i + 1], bitmap[i + 2]);
@@ -1818,7 +1818,7 @@ export class Image {
18181818
* @param factor The factor (>= 0, 1 keeps the image)
18191819
*/
18201820
brightness(factor: number): this {
1821-
if (this.__gain_map__) return this.__recolor__(image => image.brightness(factor));
1821+
if (this.__gain_map__) return this.__recolor__(image => image.brightness(factor), false, v => v * factor);
18221822
factor = finite(factor, 'brightness factor');
18231823
if (factor < 0)
18241824
throw new RangeError('Invalid brightness factor');
@@ -1830,7 +1830,7 @@ export class Image {
18301830
* @param factor The factor (>= 0, 1 keeps the image, 0 makes it grey)
18311831
*/
18321832
contrast(factor: number): this {
1833-
if (this.__gain_map__) return this.__recolor__(image => image.contrast(factor));
1833+
if (this.__gain_map__) return this.__recolor__(image => image.contrast(factor), false, v => (v - 0.5) * factor + 0.5);
18341834
factor = finite(factor, 'contrast factor');
18351835
if (factor < 0)
18361836
throw new RangeError('Invalid contrast factor');
@@ -1842,7 +1842,7 @@ export class Image {
18421842
* @param gamma The gamma (> 0, 1 keeps the image)
18431843
*/
18441844
gamma(gamma: number): this {
1845-
if (this.__gain_map__) return this.__recolor__(image => image.gamma(gamma));
1845+
if (this.__gain_map__) return this.__recolor__(image => image.gamma(gamma), false, v => Math.max(0, v) ** (1 / gamma));
18461846
gamma = finite(gamma, 'gamma');
18471847
if (gamma <= 0)
18481848
throw new RangeError('Invalid gamma');
@@ -2127,14 +2127,31 @@ export class Image {
21272127
return new Uint8Array(this.bitmap.buffer, this.bitmap.byteOffset, this.bitmap.byteLength);
21282128
}
21292129

2130+
/**
2131+
* A change after which the image has no HDR highlights (the inverts: the boosted light would become the darkest):
2132+
* made to the pixels, the map then boosts nothing
2133+
* @internal
2134+
* @ignore
2135+
*/
2136+
protected __unboost__(apply: (image: Image) => void): this {
2137+
const map = this.__gain_map__ as GainMap;
2138+
this.__gain_map__ = null;
2139+
try {
2140+
apply(this);
2141+
} finally {
2142+
this.__gain_map__ = gain.neutralMap(map, this.width, this.width, this.height);
2143+
}
2144+
return this;
2145+
}
2146+
21302147
/**
21312148
* A color change with a gain map: made to the pixels (without the map), then to the map (the same change made to
2132-
* the HDR rendition, see util/gain.ts). Changes of only the colors' hue and saturation leave a single channel map
2133-
* as it is
2149+
* the HDR rendition, see util/gain.ts; `exact`: the change as a function of an encoded value, for values above SDR
2150+
* white too). Changes of only the colors' hue and saturation leave a single channel map as it is
21342151
* @internal
21352152
* @ignore
21362153
*/
2137-
protected __recolor__(apply: (image: Image) => void, chromaOnly = false): this {
2154+
protected __recolor__(apply: (image: Image) => void, chromaOnly = false, exact?: (value: number) => number): this {
21382155
const map = this.__gain_map__ as GainMap;
21392156
this.__gain_map__ = null;
21402157
try {
@@ -2144,7 +2161,7 @@ export class Image {
21442161
}
21452162
const before = this.__resize__(map.width, map.height);
21462163
apply(this);
2147-
gain.recolor(map, before, this.__resize__(map.width, map.height), apply);
2164+
gain.recolor(map, before, apply, exact);
21482165
return this;
21492166
} finally {
21502167
this.__gain_map__ = map;

0 commit comments

Comments
 (0)