EXIF orientation is not applied on decode (Android rotates, iOS disagrees with itself)

[mrousavy]

Version

react-native-nitro-image@0.15.2, react-native-nitro-modules@0.37.0, RN 0.86.2, Expo SDK 57.

What happens

ImageFactory.loadFromFile / loadFromFileAsync do not apply the EXIF orientation tag, so a
JPEG that stores its rotation in EXIF (which is what essentially every camera pipeline writes)
decodes sideways on Android. On iOS it does not decode sideways, but the orientation leaks in a
different way: Image.width/height and Image.toRawPixelData() disagree with each other for
the same image.

This makes the same call return meaningfully different results per platform, and the difference
cannot be corrected in JS without a Platform.OS branch whose two arms do opposite things.

Android: pixels are not rotated, and the tag is then dropped

HybridImageFactory.kt:94

override fun loadFromFile(filePath: String): HybridImageSpec {
    val cleanPath = filePath.toFilePath()
    val bitmap = BitmapFactory.decodeFile(cleanPath)
    ...

BitmapFactory has never applied EXIF orientation, and there is no ExifInterface anywhere in
the package (grep -rni exif android/src returns nothing). loadFromEncodedImageData has the
same problem via BitmapFactory.decodeByteArray.

The tag is not preserved either. toEncodedImageData goes through Bitmap.compress
(Bitmap+compressInMemory.kt:19), which writes no orientation tag, so the re-encoded JPEG does
not even carry a hint that a downstream consumer could act on. The orientation is not just
ignored, it is destroyed.

iOS: orientation is respected in some places and not others

UIImage(contentsOfFile:) reads EXIF into imageOrientation without baking the pixels. From
there the package is inconsistent about which one it means:

accessor	source	orientation applied?
Image.width / Image.height	uiImage.size (NativeImage.swift:40-41)	yes, UIImage.size accounts for imageOrientation
resize(width:height:)	uiImage.draw(in:) (NativeImage.swift:104-121)	yes, draw(in:) honours it
toRawPixelData()	self.cgImage, reporting cg.width/cg.height (UIImage+toRawPixelData.swift:17-45)	no
toEncodedImageData()	jpegData(compressionQuality:)	re-emits it as an EXIF tag on un-rotated pixels

So for a photo carrying EXIF orientation 6, on iOS today:

const image = await Images.loadFromFileAsync(path)
image.width                     // 1440
image.height                    // 1920
image.toRawPixelData().width    // 1920  <- disagrees with image.width
image.toRawPixelData().height   // 1440  <- disagrees with image.height

Two properties of one object describing the same image, transposed relative to each other. That
one is reproducible without any of the context below.

How to reproduce

Any EXIF-oriented JPEG will do, but the easy source is VisionCamera, since it is the sibling
library and react-native-nitro-image is its peer dependency:

const { filePath } = await photoOutput.capturePhotoToFile({}, {})
const image = await Images.loadFromFileAsync(filePath)
await image.saveToFileAsync(somewhereVisible, 'jpg', 80)

Hold the phone upright in a portrait-locked app. On Android the saved file is rotated 90 degrees.
On iOS it is upright, but image.toRawPixelData() reports transposed dimensions.

This is not an exotic input. react-native-vision-camera deliberately writes rotation as an EXIF
tag rather than rotating the buffer, and says so in its own source
(HybridPhoto.kt:191):

// JPEG buffers already carry imageInfo.rotationDegrees in EXIF. Exif.rotate(...)
// composes with the existing orientation tag, so rotating here would apply it twice.

and its CameraOutput spec documents the strategy as "A Photo output might apply orientation via
EXIF flags". Photo.toImage() in that library physically rotates the bitmap by
orientation.counterRotated() before handing it over, which is VisionCamera telling us directly
that the buffer behind a captured file is not upright. So the file path and the toImage() path
currently disagree about orientation for the same photo.

On a phone with a landscape-mounted sensor in a portrait-locked app, imageInfo.rotationDegrees
is 90 for an ordinary capture, so on Android this is the default path rather than an edge case.

Prior art: this is the thing decoders normally do for you

• Glide applies it during decode. Verified in the glide-5.0.5.aar bytecode rather than from the
  docs - Downsampler calls TransformationUtils.getExifOrientationDegrees,
  isExifOrientationRequired, then rotateImageExif(BitmapPool, Bitmap, int).
• expo-image-manipulator normalises explicitly on iOS: it installs an
  ImageFixOrientationTransformer the moment the image loads, whose own doc says it "guarantees
  that the original pixel data matches the displayed orientation".
• UIImage itself is orientation-aware; the gap here is only that some accessors in this package
  read uiImage and others read cgImage.

Why this cannot be worked around downstream

The correction needed is the opposite on each platform: Android needs the rotation applied, iOS
has already applied it in size and resize and would double-rotate. So a consumer has to write
a Platform.OS branch, in which each arm is only ever exercised on one platform, sitting on top
of a JS EXIF parser they had to write themselves, to undo a per-platform difference in a
cross-platform library. That is a lot of unverifiable correctness for something the decoder is
better placed to do once.

Nothing is exposed that would even make that possible cheaply today: there is no
Image.orientation and no EXIF accessor, so the consumer has to re-read and parse the file
header separately from the decode that just read it.

Suggested fix

The behaviour I would expect, in rough priority order:

1. loadFromFile* and loadFromEncodedImageData* return upright pixels on both platforms.
   On Android, read ExifInterface and rotate/flip after BitmapFactory (Glide's
   TransformationUtils.rotateImageExif is the reference implementation). On iOS, normalise to
   .up at load, the way expo-image-manipulator does, so cgImage and size can never
   disagree again.
2. Make toRawPixelData() agree with width/height regardless. Even without (1), those two
   describing the same image differently is a bug on its own, and it is the part most likely to
   silently corrupt someone's GPU or ML pipeline.
3. Preserve orientation through the encoders, or state clearly that output is always upright
   with no tag. Right now Android drops the tag and iOS writes one, so round-tripping a file
   through this library changes its meaning per platform.
4. If (1) is considered a breaking change, an opt-out (loadFromFileAsync(path, { applyExif: false }))
   plus a readable Image.orientation would at least make the current behaviour intentional and
   correctable.

I would suggest (1) as the default: it matches Glide, SDWebImage, UIKit and every other image
pipeline a consumer is likely to be migrating from, and the surprising behaviour is the one that
needs the opt-in.

Context

Found while migrating Foodr off expo-image-manipulator
onto this library (mrousavy/Foodr#19). The app photographs restaurant menus and sends them to a
vision model, so a 90-degree rotation is a direct hit to OCR accuracy on the app's only
important request. The migration PR is open but blocked on this, with the affected call site
marked TODO rather than guessed at.

[mrousavy]

Downstream PR that is blocked on this: mrousavy/Foodr#65. It is open as a draft with the affected call site marked TODO rather than guessed at, so it should be correct as written the moment this lands - a useful smoke test for the fix.