Repository navigation
Embed ICU data in the linux static render (fixes regex filters and label wrapping) - #4
Merged
Merged
Conversation
Regex filters and label wrapping were broken in every linux static release (v0.1.0, v0.300.0): Alpine builds ICU with archive packaging, so its libicudata.a is a 1300-byte stub and the data lives in a separate icudt76l.dat a relocated binary can't find. - Any .match()/.replace() filter failed to load: Boost.Regex's ICU traits throw 'Could not initialize ICU resources', rethrown by mapnik's expression.cpp. - wrap-width labels logged 'could not create BreakIterator: U_MISSING_RESOURCE_ERROR' (mapnik text_layout.cpp) and didn't wrap. - Pointing ICU_DATA at a host's data can't help: the binary links ICU 76 and needs icudt76l. macOS was unaffected: Homebrew's libicudata.a carries the full data. Fix: embed icu-data-full's .dat into render via .incbin (cmake/icu-data.S.in, OSMFLAT_ICU_DATA_FILE) and register it with udata_setCommonData() before anything touches ICU. That copy wins over ICU_DATA and any host ICU. Full data rather than icu-data-en (~3 MB) keeps all locales and the dictionary line breakers (Thai, Lao, Khmer, Burmese, CJK), matching macOS. Cost: ~30 MB binary, ~12 MB tarball. It slipped through because every test ran inside the build container, where Alpine's ICU data dir was on hand. The new render_icu_data test (regex filter + .replace() + wrapped label) runs with ICU_DATA pointed at nothing and fails on output that only gets logged. Confirmed it fails without the fix and passes with it. The release smoke test now runs it on a bare alpine image with no ICU data. Checked against the published v0.300.0 archive: it fails, printing the ICU error. (Written with an explicit if, not '! grep': set -e ignores negated commands, so that form could never fail.) Verified on aarch64: bare alpine, and ubuntu 24.04 with ICU 74 and ICU_DATA=/usr/share/icu/74.2 -- all regex/wrap styles render, and output is byte-identical to macOS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merged
boydjohnson
added a commit
that referenced
this pull request
Sep 18, 2026
Patch release for the linux static render: v0.300.0's linux archives lack ICU data, so regex filters fail to load and wrap-width labels don't wrap. Fixed in #4 by embedding the data. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Regex filters and label wrapping have been broken in every linux static release (v0.1.0, v0.300.0). macOS is unaffected.
The bug
Alpine builds ICU with archive packaging: its
libicudata.ais a 1,300-byte stub, and the real data is a separateicudt76l.datthat a relocated binary can't find. ICU's code was linked; its data wasn't. Two symptoms with that one cause:Could not initialize ICU resources for expression: ...src/expression.cpp:53.match()/.replace()filter makes the style fail to loadcould not create BreakIterator: U_MISSING_RESOURCE_ERRORsrc/text/text_layout.cpp:251wrap-widthlabels are logged about, not wrappedICU_DATA=/usr/share/icu/74.2can't help: the binary links ICU 76, so it needsicudt76l, not a host's 74.macOS was fine because Homebrew's
libicudata.acarries the full 31.85 MB of data. (That's also why my earlier "regex works" claim was wrong — I had tested only on macOS.)The fix
Embed
icu-data-full's.datintorenderwith.incbin(cmake/icu-data.S.in, newOSMFLAT_ICU_DATA_FILEoption) and register it withudata_setCommonData()at the top ofmain, before anything touches ICU. That's ICU's supported mechanism for stub-data builds. The embedded copy wins overICU_DATAand any host ICU, so the host's ICU version doesn't matter — the same approach as the embeddedproj.db.Full data vs. English-only:
icu-data-enis ~3 MB but drops non-English locales and the dictionary-based line breakers for Thai, Lao, Khmer, Burmese and CJK. Full data matches what macOS gets and costs ~30 MB of binary, ~12 MB of tarball:Why tests missed it, and the new ones
Every test ran inside the build container, where Alpine's ICU data dir was sitting on disk and quietly supplied the data.
render_icu_data(new ctest): a style with a case-insensitive.match()filter, a.replace(), and awrap-widthlabel (test/style-icu.xml). It runs withICU_DATApointed at nothing, so a data dir in the build environment can't mask a binary that lacks its own. It also fails on output mapnik only logs. Fails without the fix, passes with it. Passes on macOS too.if—! grep ...can never fail aset -escript, becauseset -eignores negated commands.)Verified (aarch64)
alpine:3.22, andubuntu:24.04with ICU 74 andICU_DATA=/usr/share/icu/74.2— regex filters, case-insensitive regex,.replace(), and wrapped labels all render.10e35b70…, wrap14e747b4…)..replace('-', ' - ')rewrites the names, andwrap-widthbreaks "Baarle - Nassau" across two lines.Release
The v0.300.0 linux archives are affected, so this probably wants a 0.300.1 patch release. This PR doesn't bump the version — say if you'd like it included.
🤖 Generated with Claude Code