Add From metadata lens correction for Sony ARW and DNG

CLAUDE.md

@@ -47,6 +47,7 @@ Migrations that rewrite *rows* rather than a config payload need a repository, s
 - **CPU**: `DarkroomEngine.process()` (`negpy/services/rendering/engine.py`) — base (geometry + normalization) → exposure (incl. dodge/burn) → clahe → lab → alt process → toning → crop → finish. The first four stages are cached per config-hash via `_run_stage()`; the rest run unconditionally. The alt-process stage (lith or cyanotype, never both) is B&W-only and off by default; when off, both engines skip it rather than run an identity pass.
 - **GPU**: `GPUEngine` (`negpy/services/rendering/gpu_engine.py`) — the same logical stages as WGSL compute shaders from `negpy/features/<name>/shaders/`, with its own config-diff change detection.
 - **Orchestration**: `ImageProcessor` (`image_processor.py`) tries GPU first and falls back to CPU. Export always runs full-res, with CPU stage caching off (`PipelineContext.cache_stages`). Linear DNG decode, CPU saturation and unsharp masking use row blocks to bound temporary storage. `PipelineContext` carries `scale_factor`, `process_mode`, `active_roi` and a `metrics` dict between stages.
+- **Embedded lens correction** (`features/lens`) is a single-file decode step shared by preview and export: flat-field, lens warp, then sensor unmix and user geometry. Its independent distortion and CA settings and flat-field token belong to the source identity. It is disabled for composite setup, composite assembly and RGB+IR sources.
 - **Source bakes** run before either engine, on the linear source: flat-field, sensor unmix and every defect repair (IR, detected specks, painted heal strokes). Both engines re-upload that source per frame, so a bake reaches them parity-free and needs no shader. Each bake folds a token into `source_hash` to invalidate the engine cache.
 - **Working space**: scene-linear internally; the working OETF (Adobe RGB 1998 TRC — a pure 563/256 power, no linear segment) is the final engine step. Lab/toning compute CIELAB directly from linear, D65. Adobe RGB rather than a wide gamut because ProPhoto's imaginary primaries inflate chroma in the saturation and toning stages.
 
@@ -63,6 +64,10 @@ Every feature lives in `negpy/features/<name>/`:
 
 One exception: `features/altprocess/` holds only `models.py`. Lith and cyanotype are mutually exclusive, so they share the Alternative Processes panel and one `AltProcessConfig`; their logic and shaders stay in `features/lith/` and `features/cyanotype/`.
 
+`features/lens/warps.py` holds frozen lens models with `has_distortion`, `has_ca`, and
+`remap(...)`, as defined by `LensWarp` in `models.py`. `logic.py` applies their maps in
+row blocks. File readers are registered in `infrastructure/loaders/lens_metadata.py`.
+
 ### Desktop (MVC)
 
 - `AppState` (`negpy/desktop/session.py`) — mutable session state

docs/USER_GUIDE.md

@@ -511,6 +511,8 @@ Where the frame gets its final shape: what is inside the print, and whether it s
     Both replicate a wedge along the squeezed edge, as Fine Rotation does; crop it off. Crop before correcting if you can, because the meters read the corrected frame: on an uncropped scan a big correction pulls rebate and surround into the metered area and the print darkens.
 
 *   **Distortion Correction** (-0.100 to 0.100, in steps of 0.001): radial lens distortion. Positive corrects barrel, negative pincushion. Use the film rebate as a straight-edge reference. Corrected before Tilt and Swing.
+*   **Metadata Distortion**: straighten curved lines with lens correction data stored in the file. Replaces manual distortion correction. Set before cropping or retouching. Available when the file contains supported distortion data.
+*   **Metadata CA**: reduce color fringes along edges with lens correction data stored in the file. Works independently of Metadata Distortion and can be used with manual distortion correction. Available when the file contains supported CA data.
 
 <!-- panel:flatfield -->
 ### 5.2 Flat Field: even out the light

negpy/desktop/controller.py

@@ -152,6 +152,7 @@
 from negpy.services.rendering.prefetch_policy import MIN_RAM_RESERVE_BYTES
 from negpy.services.rendering.preview_manager import PreviewManager
 from negpy.services.rendering.source_identity import source_token
+from negpy.services.rendering.lens import lens_decode_token, metadata_lens_corrections
 from negpy.services.view.coordinate_mapping import CoordinateMapping
 
 logger = get_logger(__name__)
@@ -236,6 +237,8 @@ def _autocrop_fingerprint(config: WorkspaceConfig, workspace_color_space: str) -
         int(geometry.autocrop_offset),
         round(float(geometry.autocrop_rebate_trim), 4),
         round(float(geometry.distortion_k1), 9),
+        bool(geometry.lens_distortion_from_metadata),
+        bool(geometry.lens_ca_from_metadata),
         bool(flatfield.apply),
         str(flatfield.profile_id),
         bool(config.process.linear_raw),
@@ -543,12 +546,12 @@ def __init__(self, session_manager: DesktopSessionManager):
         self._busy_toast = False
         self._pending_render_task: Any = None
 
-        # Last displayed render per frame, so navigate-back paints instantly while the
-        # authoritative render refreshes underneath.
-        self._render_memo = RenderMemo()
+        # Correction states share the bounded cache, so toggling back can paint immediately.
+        self._render_memo = RenderMemo(keep_variants=True)
         # (source_hash, memo_key, content_rect) of the on-screen GPU render; load_file
         # files its texture under this on the way out.
         self._last_render_identity: Optional[tuple] = None
+        self._expected_render_key = ""
         self._render_memo.large_entries = self.state.hq_preview
         # Test strips, keyed density/grade-blind (see _strip_memo_key). Four mosaics per
         # entry, hence the conservative budget.
@@ -1847,6 +1850,7 @@ def load_file(self, file_path: str, preserve_zoom: bool = False, force_detect: b
         self._foreground_preview_generation = self._prefetch_gen
         self._pause_background_thumbnails()
         self._preview_load_t0 = time.perf_counter()
+        keep_preview = preserve_zoom and self._requested_file_path == file_path
         self._requested_file_path = file_path
         # A strip belongs to one frame, and the memo fast path below repaints without
         # going through request_render, so drop it here too. Zone pins froze their sample
@@ -1860,11 +1864,12 @@ def load_file(self, file_path: str, preserve_zoom: bool = False, force_detect: b
         # shaped it has changed, since select_file already hydrated its config. Paint it
         # now, with no spinner and no toasts, and let the real render refresh the metrics.
         target_hash = self._file_hash_for_path(file_path)
-        memo = self._render_memo.get(target_hash, self._render_memo_key()) if target_hash else None
+        self._expected_render_key = self._render_memo_key()
+        memo = self._render_memo.get(target_hash, self._expected_render_key) if target_hash else None
 
         if not preserve_zoom:
             self.zoom_requested.emit(1.0)
-        if memo is None:
+        if memo is None and not keep_preview:
             self.loading_started.emit()
         self._thumb_config = None
 
@@ -1953,9 +1958,8 @@ def load_file(self, file_path: str, preserve_zoom: bool = False, force_detect: b
                 # The half suffix distinguishes the two halves' preview caches now
                 # that the slice happens pre-downsample (each half is its own buffer).
                 file_hash=self._file_hash_for_path(file_path),
-                # A memoized frame is already painted, so the embedded-JPEG splash would
-                # repaint stale pixels over it.
-                use_splash=memo is None,
+                # A reload or memo hit keeps the rendered frame until its replacement is ready.
+                use_splash=memo is None and not keep_preview,
                 detect_mode=(
                     pending_import.detect_mode
                     if pending_import is not None
@@ -1969,6 +1973,8 @@ def load_file(self, file_path: str, preserve_zoom: bool = False, force_detect: b
                 flatfield_profile_id=flatfield.profile_id if (stitch.stitch_enabled and flatfield.apply) else "",
                 half_slice=half_info,
                 demosaic=self.state.config.process.demosaic_preview,
+                lens_corrections=metadata_lens_corrections(self.state.config),
+                lens_flatfield=self.state.config.flatfield,
             )
         )
 
@@ -2028,6 +2034,9 @@ def _on_preview_loaded(
         if self._requested_file_path != file_path:
             return
         self._foreground_preview_generation = None
+        decoded_lens_token = cam_matrix[3] if cam_matrix and len(cam_matrix) > 3 else ""
+        if decoded_lens_token != lens_decode_token(metadata_lens_corrections(self.state.config), self.state.config.flatfield):
+            return
         logger.info(
             "load-timing preview_e2e %.0fms (load request -> decoded buffer) %s",
             (time.perf_counter() - self._preview_load_t0) * 1000,
@@ -2037,7 +2046,10 @@ def _on_preview_loaded(
         if ir_preview is not None:
             ir_preview, _ = self._split_active_half(ir_preview, None)
         self.state.preview_raw = raw
-        self.state.preview_cam_xyz, self.state.preview_camera_wb = cam_matrix or (None, None)
+        self.state.preview_cam_xyz, self.state.preview_camera_wb = cam_matrix[:2] if cam_matrix else (None, None)
+        self.state.preview_lens = cam_matrix[2] if cam_matrix and len(cam_matrix) > 2 else None
+        self.state.preview_lens_path = file_path
+        self.state.preview_lens_token = decoded_lens_token
         self.state.preview_proxy = _interactive_proxy(raw)
         self.state.preview_ir = ir_preview
         self.state.preview_ir_proxy = _interactive_ir_proxy(ir_preview, self.state.preview_proxy)
@@ -2118,6 +2130,8 @@ def _neighbor_prefetch_task(
             protected_file_hashes=protected_file_hashes,
             half_slice=self._half_slice_for_asset(asset["path"], file_hash),
             demosaic=saved.process.demosaic_preview if saved else self.state.config.process.demosaic_preview,
+            lens_corrections=metadata_lens_corrections(saved or self.state.config),
+            lens_flatfield=(saved or self.state.config).flatfield,
         )
 
     def _start_next_neighbor_prefetch(self) -> None:
@@ -4480,6 +4494,10 @@ def request_render(
         before/after split instead of being displayed.
         """
         self._render_debounce.stop()
+        lens_token = lens_decode_token(metadata_lens_corrections(self.state.config), self.state.config.flatfield)
+        if not ephemeral and self.state.current_file_path and lens_token != self.state.preview_lens_token:
+            self.load_file(self.state.current_file_path, preserve_zoom=True)
+            return
 
         # Any direct render exits the flat preview-peek.
         if config_override is None and self.state.flat_peek:
@@ -4530,6 +4548,7 @@ def request_render(
         memo_key = ""
         if config_override is None and not ephemeral and not crop_preview_full and not interactive:
             memo_key = self._render_memo_key()
+        self._expected_render_key = memo_key
 
         dip = self.active_diptych()
         cam_xyz, camera_wb = self._effective_cam_xyz()
@@ -5653,16 +5672,12 @@ def _clear_busy_toast(self) -> None:
             self.set_status("")
 
     def _renders_another_frame(self, metrics: Dict[str, Any]) -> bool:
-        """True when a render belongs to a frame that is no longer selected.
-
-        A render carries the hash it was dispatched for, and nothing cancels one that is
-        already in flight — click the next frame mid-render and it still lands. Its pixels
-        and its measurements describe the frame the user has left, so they must not reach
-        the canvas or ``last_metrics``. A task dispatched before the file had a hash
-        carries the same ``"preview"`` placeholder ``request_render`` gives it.
-        """
+        """Reject pixels and measurements for a different file or superseded edit."""
         src = metrics.get("source_hash")
-        return src is not None and src != (self.state.current_file_hash or "preview")
+        key = metrics.get("memo_key")
+        return (src is not None and src != (self.state.current_file_hash or "preview")) or bool(
+            key and self._expected_render_key and key != self._expected_render_key
+        )
 
     def _on_render_finished(self, _result: Any, metrics: Dict[str, Any]) -> None:
         self._is_rendering = False
@@ -5843,7 +5858,9 @@ def _on_metrics_updated(self, metrics: Dict[str, Any]) -> None:
                 # Move the frame's memo entry to the updated config's key so the first
                 # navigate-back after an initial render still hits. A GPU render is not filed
                 # until navigate-away, so its identity follows too.
-                self._render_memo.rekey(src or self.state.current_file_hash or "", self._render_memo_key())
+                self._render_memo.rekey(
+                    src or self.state.current_file_hash or "", self._render_memo_key(), old_key=metrics.get("memo_key", "")
+                )
                 if self._last_render_identity is not None:
                     self._last_render_identity = (
                         self._last_render_identity[0],

negpy/desktop/render_memo.py

@@ -22,14 +22,18 @@
 
 
 class RenderMemo:
-    """LRU of the last render per file: file_hash -> (memo_key, payload)."""
+    """Bounded render LRU, optionally retaining multiple settings per file."""
 
-    def __init__(self, app_config: Any = None) -> None:
+    def __init__(self, app_config: Any = None, *, keep_variants: bool = False) -> None:
         self._app = app_config or APP_CONFIG
-        self._entries: "OrderedDict[str, tuple[str, dict]]" = OrderedDict()
+        self._keep_variants = keep_variants
+        self._entries: "OrderedDict[tuple[str, str], tuple[str, dict]]" = OrderedDict()
         # Hundreds of MB an entry (HQ renders, strip mosaics): use the full-res knob.
         self.large_entries = False
 
+    def _entry_key(self, file_hash: str, memo_key: str) -> tuple[str, str]:
+        return file_hash, memo_key if self._keep_variants else ""
+
     def _budget(self) -> int:
         if self.large_entries:
             return max(2, int(getattr(self._app, "preview_cache_max_full_res_entries", 2)))
@@ -50,28 +54,33 @@ def _dispose(self, entry: "Optional[tuple[str, dict]]", keep: "Optional[dict]" =
     def store(self, file_hash: str, memo_key: str, payload: dict) -> None:
         if not file_hash or not memo_key:
             return
-        self._dispose(self._entries.pop(file_hash, None), keep=payload)
-        self._entries[file_hash] = (memo_key, payload)
+        key = self._entry_key(file_hash, memo_key)
+        self._dispose(self._entries.pop(key, None), keep=payload)
+        self._entries[key] = (memo_key, payload)
         while len(self._entries) > self._budget():
             self._dispose(self._entries.popitem(last=False)[1], keep=payload)
 
     def get(self, file_hash: str, memo_key: str) -> Optional[dict]:
-        entry = self._entries.get(file_hash)
+        key = self._entry_key(file_hash, memo_key)
+        entry = self._entries.get(key)
         if entry is None or entry[0] != memo_key:
             return None
-        self._entries.move_to_end(file_hash)
+        self._entries.move_to_end(key)
         return entry[1]
 
-    def rekey(self, file_hash: str, new_key: str) -> None:
+    def rekey(self, file_hash: str, new_key: str, *, old_key: str = "") -> None:
         """Follow a render-neutral config change (e.g. measured bounds persisted
         after the render, with render=False): the stored pixels are still valid,
         only their identity moved."""
-        entry = self._entries.get(file_hash)
-        if entry is not None and new_key:
-            self._entries[file_hash] = (new_key, entry[1])
+        if not new_key or (self._keep_variants and not old_key):
+            return
+        entry = self._entries.pop(self._entry_key(file_hash, old_key), None)
+        if entry is not None:
+            self.store(file_hash, new_key, entry[1])
 
     def invalidate(self, file_hash: str) -> None:
-        self._dispose(self._entries.pop(file_hash, None))
+        for key in [key for key in self._entries if key[0] == file_hash]:
+            self._dispose(self._entries.pop(key))
 
     def clear(self) -> None:
         for entry in self._entries.values():

negpy/desktop/session.py

@@ -27,6 +27,7 @@
 from negpy.features.hdr.logic import resolve_anchor, seed_shadow_density
 from negpy.features.hdr.models import ANCHOR_EV_UNSET, HdrConfig, hdr_frame_paths
 from negpy.features.stitch.models import StitchConfig
+from negpy.features.lens.models import LensMetadata
 from negpy.infrastructure.display.color_spaces import WORKING_COLOR_SPACE
 from negpy.infrastructure.storage.repository import StorageRepository
 from negpy.kernel.system.config import APP_CONFIG
@@ -85,6 +86,9 @@ class AppState:
     # None for sources that carry no camera matrix (scanner TIFF, JPEG).
     preview_cam_xyz: Optional[list] = None
     preview_camera_wb: Optional[list] = None
+    preview_lens: Optional[LensMetadata] = None
+    preview_lens_path: str = ""
+    preview_lens_token: str = ""
     # Preview-resolution stand-in for preview_raw while HQ is on. Interactive frames render
     # against it. None when preview_raw is already small enough.
     preview_proxy: Optional[Any] = None

negpy/desktop/settings_catalog.py

@@ -133,7 +133,8 @@ def _fmt_gear(values: tuple) -> str:
         _row("Fine Rotation", "geometry", "fine_rotation"),
         _row("Easel Tilt", "geometry", "converge_v"),
         _row("Easel Swing", "geometry", "converge_h"),
-    _row("Distortion Correction", "geometry", "distortion_k1", sticky=True),
+        _row("Lens Correction", "geometry", "distortion_k1", "lens_distortion_from_metadata", sticky=True),
+        _row("Metadata CA", "geometry", "lens_ca_from_metadata", sticky=True),
         _row("Flip Horizontal", "geometry", "flip_horizontal", sticky=True),
         _row("Flip Vertical", "geometry", "flip_vertical", sticky=True),
     )),

negpy/desktop/view/keyboard_shortcuts.py

@@ -189,6 +189,8 @@ def _build_actions(self) -> dict[str, Callable[[], None]]:
             "crop_guide_next": lambda: controls.geometry_sidebar.cycle_guide(),
             "crop_guide_orient": controller.cycle_crop_guide_orientation,
             "auto_crop": lambda: controls.geometry_sidebar.reset_crop_btn.toggle(),
+            "lens_distortion_from_metadata": lambda: controls.geometry_sidebar.metadata_distortion_btn.click(),
+            "lens_ca_from_metadata": lambda: controls.geometry_sidebar.metadata_ca_btn.click(),
             "pick_dust": lambda: _toggle_tool_button(self.window, "finish", controls.retouch_sidebar.pick_dust_btn),
             "pick_scratch": lambda: _toggle_tool_button(self.window, "finish", controls.retouch_sidebar.pick_scratch_btn),
             "pick_scratch_line": lambda: _toggle_tool_button(self.window, "finish", controls.retouch_sidebar.pick_line_btn),

negpy/desktop/view/shortcut_registry.py

@@ -49,6 +49,8 @@ class ShortcutEntry:
     "manual_crop": ShortcutEntry("Shift+C", "Toggle manual crop", "Tools"),
     "crop_guide_next": ShortcutEntry("O", "Next crop guide overlay", "Geometry"),
     "crop_guide_orient": ShortcutEntry("Shift+O", "Rotate crop guide orientation", "Geometry"),
+    "lens_distortion_from_metadata": ShortcutEntry("", "Metadata Distortion", "Geometry"),
+    "lens_ca_from_metadata": ShortcutEntry("", "Metadata CA", "Geometry"),
     "auto_crop": ShortcutEntry("Shift+A", "Toggle autocrop", "Geometry"),
     "pick_dust": ShortcutEntry("Shift+D", "Toggle heal tool", "Tools"),
     "pick_scratch": ShortcutEntry("Shift+S", "Toggle scratch tool", "Tools"),