Skip to content

fix(geotiff): decode big endian tiles in the platform's byte order - #687

Open
james-willis wants to merge 2 commits into
developmentseed:mainfrom
james-willis:jw/big-endian-decode
Open

james-willis wants to merge 2 commits into
developmentseed:mainfrom
james-willis:jw/big-endian-decode

Conversation

@james-willis

@james-willis james-willis commented Oct 1, 2026 •

Copy link
Copy Markdown

Closes #686.

Big endian (MM) TIFFs decoded to byte-swapped pixel values: toTypedArray views the decoded bytes in the platform's byte order, and DecoderMetadata didn't carry the file's.

Changes

  • DecoderMetadata gains an optional littleEndian (default true, so existing callers of decode() are unaffected). fetch.ts sets it from image.tiff.isLittleEndian for both data and mask tiles.
  • decode() byte-swaps samples wider than 8 bits from big endian files, before applyPredictor, since horizontal differencing (predictor 2) has to accumulate native-order integers.
  • Predictor 3 is excluded: its byte planes are most significant byte first in every file (TIFF Technical Note 3), and decodeRowFloatingPoint already reassembles them in platform order.
  • The swap writes to a new buffer, because the uncompressed decoder returns the caller's bytes, which may be shared with the block cache.
  • Codecs that return typed pixels (LERC, JPEG, WebP) don't go through this path and are unchanged.

Tests

Three unit tests in decode.test.ts build big endian tiles from known values:

  • float32, no predictor (also asserts the input buffer is untouched)
  • uint16, predictor 2
  • float32, predictor 3, decoded as both II and MM, which must agree

The first two fail on main; the third fails if the swap is applied to predictor 3. pnpm typecheck and biome check are clean.

The second commit bumps fixtures/geotiff-test-data to pick up its big endian fixtures (developmentseed/geotiff-test-data#73, #77) and adds both to integration-rasterio.test.ts:

  • uint16_1band_lzw_block128_predictor2_big_endian: fails tile pixel data matches on main, passes here.
  • float32_1band_deflate_block64_predictor3_big_endian: passes either way, and guards against swapping predictor 3 byte planes.

With pixi run generate-npy as in CI, the integration suite passes 19/19 here and fails only the uint16 big endian case against main's decode.ts / fetch.ts.

@github-actions github-actions Bot added the fix label Oct 1, 2026
@james-willis
james-willis marked this pull request as ready for review October 1, 2026 18:18
@kylebarron

Copy link
Copy Markdown
Member

Thanks!

Could you make a PR to add a big-endian TIFF to https://github.com/developmentseed/geotiff-test-data so that we have a known TIFF to test against?

@kylebarron

Copy link
Copy Markdown
Member

Thanks for the associated PRs into geotiff-test-data; now we can update the submodule here and test against those files

Bump geotiff-test-data to pick up its big endian fixtures and add them to
the rasterio integration test. On main the uint16 predictor 2 fixture
fails tile pixel data matches; the predictor 3 one passes either way,
which guards against swapping its byte planes.
@james-willis

Copy link
Copy Markdown
Author

The big endian fixtures are only in integration-rasterio.test.ts, not integration.test.ts, because geotiff.js (3.0.5) mis-decodes big endian files that use a predictor, so it can't serve as the reference for them:

  • Predictor 2: applyPredictor accumulates over a Uint16Array view (platform byte order) before samples are read with the file's byte order, so it adds byte-swapped values. For uint16_1band_lzw_block128_predictor2_big_endian, the second pixel is stored as FA 00 then FC 18 (64000, then −1000); summed as little endian and read back big endian that gives 63001 instead of 63000, and the error grows along the row (62002, …).
  • Predictor 3: decodeRowFloatingPoint reassembles each float in platform order, which is right, but geotiff.js then reads it with the file's big endian byte order, so every value comes out byte-swapped.

Big endian without a predictor is fine in geotiff.js. rasterio (GDAL) decodes both fixtures correctly, and this PR matches it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Big endian TIFFs decode to wrong pixel values (toTypedArray assumes platform byte order)

2 participants