Skip to content

Skip missing/non-finite coordinates when encoding, with a warning - #56

Merged
dcooley merged 1 commit into
SymbolixAU:masterfrom
UoA-eResearch:claude/nan-polyline-encoding-o4cy4a
Sep 20, 2026
Merged

dcooley merged 1 commit into
SymbolixAU:masterfrom
UoA-eResearch:claude/nan-polyline-encoding-o4cy4a

Conversation

@neon-ninja

Copy link
Copy Markdown

Problem

encode() passed NaN coordinates through an undefined double -> int cast in encode_polyline() (src/googlePolylines.cpp). On x86 the cast yields INT_MIN, so a NaN pair emitted invalid polyline characters (bytes outside the valid 63–126 range, e.g. 0f ce 76 d7) and — because the format is delta-based — corrupted every subsequent point too. decode() on such a string threw a raw C++ exception (basic_string::at: __n >= this->size()). R's NA_real_ is a NaN at the C level, so NA triggered the same behaviour. Relatedly, empty sf POINTs (whose coordinates are NaN) encoded as "??", which silently decodes to the real coordinate (0, 0).

Changes

  • encode_polyline() now skips coordinate pairs where lon or lat is not finite (NA/NaN/Inf); every entry point (encode(), encodeCoordinates(), the sf/sfc path) warns with a count of pairs that could not be encoded.
  • encode(byrow = TRUE) returns NA for such rows instead of skipping (and decode() already maps NA strings back to NA coordinates).
  • Empty sf POINTs now encode to "" (with the warning) instead of "??"/(0, 0); the test pinning the old behaviour was updated.
  • RcppExports regenerated with Rcpp::compileAttributes(); roxygen docs updated (including the previously undocumented precision argument of decode()) and man pages regenerated.
  • New testthat coverage for NA/NaN/Inf inputs, byrow, and empty points; NEWS entry; version bumped to 0.8.8.9000.

Testing

Full testthat suite (with NOT_CRAN=true): 161 pass, 0 fail. R CMD check (no vignettes/manual) shows nothing beyond pre-existing environment notes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UYymBnTDZbXyuus9FY97pd


Generated by Claude Code

Coordinates containing NA, NaN or Inf values previously went through an
undefined double -> int cast in encode_polyline(), corrupting the whole
polyline with out-of-range characters (bytes below 63 and above 127)
that decode() could not even parse. Empty sf POINTs (whose coordinates
are NaN) silently encoded as "??", which decodes to the real coordinate
(0, 0).

Such coordinate pairs are now skipped during encoding and a warning
reports how many were dropped. With encode(byrow = TRUE), rows with
missing coordinates return NA instead (which decode() already maps back
to NA coordinates).

Also documents the previously undocumented `precision` argument of
decode().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UYymBnTDZbXyuus9FY97pd
@dcooley

dcooley commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator

My own notes:

From the tests:

df_nan
     lat      lon
1 38.500 -120.200
2    NaN      NaN
3 43.252 -126.453

Before this update this would round-trip as

decode(encode(df_nan))
[[1]]
     lat      lon
1 38.500 -120.200
2  0.000    0.000
3 43.252 -126.453

After this update this round-trips as

Warning message:
In rcpp_encode_polyline(obj[[lon]], obj[[lat]]) :
  1 coordinate pair(s) with missing or non-finite (NA/NaN/Inf) values could not be encoded

decode(encode(df_nan))
[[1]]
     lat      lon
1 38.500 -120.200
3 43.252 -126.453

The arugment being that a 'missing' row with a warning is better than a forced row of (0, 0) coordinates


Similarly:

df_na
     lat     lon
1     NA -120.20
2 40.700 -120.95
3 43.252     Inf

now round-trips to

Warning message:
In rcpp_encode_polyline(obj[[lon]], obj[[lat]]) :
  2 coordinate pair(s) with missing or non-finite (NA/NaN/Inf) values could not be encoded

[[1]]
   lat     lon
1 40.7 -120.95

@dcooley
dcooley merged commit 91d258d into SymbolixAU:master Sep 20, 2026
0 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants