Reduce memory use for Linear DNG decoding and CPU export

[hsnilsson]

Summary

Reduce temporary memory use for Linear DNG decoding and full-resolution CPU export without early downsampling or reduced arithmetic precision.

• Normalize Linear DNG RGB data in bounded float64 row blocks, preserving float32 output, LUT handling, crop, black/white levels, and white balance.
• Disable retention of CPU stage-cache arrays during export. Preview caching remains enabled by default.
• Process saturation/skin protection and USM sharpening in row blocks, with filter halos and the full frame's Numba kernel choice preserved for short tail blocks.
• Add exact-output regression tests for decoding, cache transitions, block seams, and serial/parallel short tails.

Validated candidate: 5b17c49, pushed to hsnilsson/NegPy on codex/reduce-linear-dng-memory-pr. All results below refer to this candidate. Baseline comparisons use upstream main at 8e7474f (2026-09-10), including incremental painted-heal bakes. No product code changed after the final validation.

Regression checks (2026-09-10)

• Windows: 5,530 tests and 71 subtests passed; 19 skipped, 14 deselected.
• Linux (Ubuntu 24.04/WSL2): 5,514 tests and 71 subtests passed; 35 skipped, 14 deselected.
• All 73 independent exact array/byte comparisons against upstream 8e7474f passed on each platform.
• Ruff formatting/lint and type checks passed on both platforms. The ten PR files match byte-for-byte across the tested checkouts.

Full-resolution comparison (2026-09-10)

The full-original Linux baseline/candidate comparison completed successfully for both JPEG and 16-bit TIFF. The input was a 19,200 x 12,752 JPEG XL DNG, exported at its normal 19,120 x 12,736 DNG crop without extra cropping or resizing. Color Negative, saturation, skin protection, and masked USM were enabled; each version reused one ImageProcessor for JPEG followed by TIFF.

• Rendered float32 SHA-256 and encoded-file SHA-256 matched between versions for each format. Dimensions, dtype, and encoded size also matched; TIFF uint16 depth was checked.
• Both runs completed successfully and verified unchanged copied inputs. The original archive file's checksum was also unchanged. No PR code changes were needed to pass this comparison.
• Both runs used an 8 GiB RAM / 24 GiB swap limit. Reported service swap peaks were 22.9 GiB for baseline and 6.9 GiB for candidate. These are constrained-run observations, not unconstrained RAM measurements or a general speedup claim.

Memory and batch validation (2026-09-10)

These checks use the same candidate and baseline identified above. Memory figures below are decimal GB.

• 48 sequential CPU exports of 33.56 MP crops from three real photographs covered three process modes and JPEG/16-bit TIFF on Windows and Linux. Corresponding baseline/candidate float32 arrays and encoded bytes matched exactly within each platform, including repeated exports.
• Three fresh-process Windows A/B pairs for one crop/JPEG configuration reduced median peak working set from 4.556 GB to 2.233 GB (about 51%); all output hashes matched.
• The Windows candidate completed full-original JPEG and 16-bit TIFF exports from a 244 MP JPEG XL DNG, with its normal crop and no resizing, at 14.953 GB peak working set. The external guard recorded a minimum of about 8.9 GB available RAM.

Limits

Full-original parity covers one image and one process mode on Linux. Full-original Windows baseline attempts stopped at the memory guard, so full-original Windows baseline/candidate parity remains unverified. The measurements are fixture-specific, not guarantees of a general reduction or the absence of leaks. Longer production batches and desktop queue/prefetch behavior were not validated. The default suite excludes slow tests and skips unavailable platform/sample/profile cases. GPU processing, Richardson-Lucy sharpening, and denoise are unchanged. Full-frame source/output buffers and other stages still use substantial memory.

[hsnilsson]

I'm doing 244MP DNG conversions with JPEG XL inside, and got issues with too high memory usage.

This PR reduces the memory use by 51% by processing data in chunks rather than all at once. This makes huge conversions possible on limited RAM (32GB), but may slow down the process by some amount which should probably be measured before taking this in.

Thumbs up if I can continue - then I'll try measure if it affects the processing time (for images I can fit in memory).

[hsnilsson]

AI-assisted comment (model: gpt-6-astra).

Windows CPU timing follow-up: compared 8e7474f with 32ac4a2 using seven alternating A/B pairs and 168 timed JPEG/16-bit TIFF exports from three 33.56 MP Linear DNG crop fixtures. Numba was warmed up, and hashing was outside the timed section.

Median paired changes:

• Mixed export sequence: 1.3% slower (individual pairs ranged from 3.5% faster to 2.2% slower).
• New-source JPEG exports with default settings: 2.6% faster.
• Exports reusing decoded sources: 2.6% slower.

All corresponding rendered-pixel and encoded-file hashes matched. These results are fixture-specific, not a full-resolution 244 MP speed claim.

Linux timing remains incomplete: the baseline repeatedly reached the host-memory safety limit, so no reliable Linux percentage is available yet.