Repository navigation
Static, self-contained render with mapnik built in (linux/musl + macOS) - #2
Merged
Merged
Conversation
OSMFLAT_STATIC_RENDER=ON fetches mapnik v4.2.2, patches osmflat into its compile-time static-plugin table, and links render against the static libmapnik. OSMFLAT_FULLY_STATIC=ON adds -static, so on musl the binary needs nothing at runtime -- no libmapnik, no osmflat.input, no fonts. Dropping PROJ avoids shipping proj.db, but a PROJ-less mapnik knows only epsg:4326 and classifies it as geographic, which scales scale_denominator by ~111319x against what PROJ reports for the equivalent '+proj=longlat +datum=WGS84 +no_defs' -- silently breaking every style's MaxScaleDenominator gating. The patch teaches projection to classify +proj=longlat the way PROJ does, so both builds agree (verified: 1.92857 for the same style/bbox/size on both). scripts/build-static-render.sh builds, tests, and packages it in Alpine; static-render.yml runs that script for x86_64 and aarch64. Verified on aarch64: 16.5M stripped binary, ctest passes, the unpacked tarball renders on a bare alpine image, and fonts/ beside the binary is picked up (8173-byte labelled PNG vs 2785 without). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mapnik is ~175 TUs and dominates the build (~16 min cold on x86_64). Put everything reusable under $BUILD_DIR -- mapnik's fetched source and objects, the cargo registry (CARGO_HOME), and downloaded apk packages -- so one actions/cache path covers it all. Cache keys are unique per run (GitHub caches are immutable) with restore-keys falling back to the newest cache for that arch; changed build plumbing or a new mapnik tag starts a fresh lineage, and ninja decides what to actually redo. Locally: 468M tree, and a re-run with it in place takes 1m11s (one relink) against ~5 min cold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A v* tag builds the static render for x86_64 and aarch64, smoke-tests each unpacked archive on a bare alpine image, and publishes both with a SHA256SUMS. workflow_dispatch builds the same matrix without publishing. Unlike CI, release builds skip the cache: a published binary should come from a clean build of the tagged tree. The tag must match the project VERSION, so archive names can't drift from the release. Archives now carry mapnik's LGPL-2.1 text plus a NOTICE naming every statically linked library and pointing at the sources and build script, which is what the relinking requirement asks for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The repo had no license file at all. Adds both texts, a README License section, and ships them in the release archive. The static archives are the one wrinkle: Apache-2.0 is considered incompatible with LGPL-2.1, so a redistributed binary with mapnik linked in is the MIT arm of the dual license combined with LGPL mapnik. The NOTICE says so. Building the ordinary osmflat.input module against a system mapnik is dynamic linking and raises none of this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Apache-2.0 is considered incompatible with LGPL-2.1 for a combined work, and the static render links LGPL mapnik. Dual licensing worked (a redistributor would take the MIT arm), but MIT alone leaves nothing to explain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four pushes in quick succession each started a full static-render build (~16 min cold on x86_64), all racing to build commits already replaced. A concurrency group per workflow+ref keeps one run per PR branch. main and tags share the group name but never cancel, so a release build can't be killed by a later push. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The build runs as root in the container (apk needs it), and cargo leaves files the runner user can't read, so the post-job cache save died on 'tar: ... Permission denied' -- the first cached run saved nothing. chown the tree back before the cache step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
macOS can't link libSystem statically, so this bundles mapnik and every third-party library and leaves only macOS' own dylibs/frameworks dynamic; the build script fails if otool -L shows anything outside /usr/lib or /System. Three macOS-specific problems: - Homebrew's libharfbuzz.a is built against graphite2, which ships no static archive, so harfbuzz is built from source with just freetype and CoreText (which also drops glib/pcre2/intl). - icu4c, zlib and bzip2 are keg-only, so their prefixes are passed explicitly or nothing finds the archives. - Homebrew's Boost::regex names icu libs bare, which wouldn't resolve and would prefer dylibs; mapnik's own workaround drops them. Packaging moves to scripts/package-render.sh, shared by both platforms. Verified locally on arm64: links only /usr/lib + /System, ctest passes, the unpacked archive renders with its bundled fonts, and output is byte-identical to both the dynamic macOS build and the linux/musl static build (md5 d48d0556925e17e872d0ac25229319d2). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds mapnik from source with the osmflat datasource compiled in as a built-in plugin, and links
renderagainst it. The result loads nothing at runtime — nolibmapnik, noosmflat.input, no system fonts.The dynamic build is untouched and remains the default.
/usr/lib+/SystemonlyAll three render byte-identical output to the dynamic build (md5
d48d0556925e17e872d0ac25229319d2on thestyle-relationsfixture).How osmflat gets into a static libmapnik
Mapnik's static-plugin table is compile-time (
src/datasource_cache_static.cpp+src/CMakeLists.txt), socmake/static-mapnik/patch-mapnik.cmakeedits the fetched tree: it addsplugins/input/osmflat(sources referenced from this repo, no copies), links it intolibmapnik, and registers it in the static datasource map. Each edit anchors on an exact upstream string, fails loudly if mapnik moved it, and is skipped when already applied.The scale_denominator trap
Dropping PROJ keeps a 9 MB
proj.dbout of the binary, but a PROJ-less mapnik knows onlyepsg:4326and classifies it as geographic — makingscale_denominator~111319x larger than what PROJ reports for the equivalent+proj=longlat +datum=WGS84 +no_defs. Every style here is tuned against the degree-based numbers (0.1≈ neighborhood), so that would have silently disabled everyMaxScaleDenominatorgate while still rendering something plausible. (First attempt did exactly this:214688where the dynamic build says1.92857.)The patch instead makes a PROJ-less
projectionclassify+proj=longlatexactly as PROJ does. Other proj4 strings still throw.Platform wrinkles
musl: harfbuzz's CMake config names the shared library (
attempted static link of dynamic object), so the pkg-config fallback is forced; cargo asks for-lgcc_s, which musl ships only shared, so Corrosion's link interface getsgcc_eh.macOS: Homebrew's
libharfbuzz.ais built against graphite2, which has no static archive — so harfbuzz is built from source with just freetype and CoreText (dropping glib/pcre2/intl too). icu4c/zlib/bzip2 are keg-only, so their prefixes are passed explicitly. Homebrew'sBoost::regexnames icu libs bare, which wouldn't resolve and would prefer dylibs; mapnik's own workaround drops them. The build script fails ifotool -Lshows anything outside/usr/libor/System.CI
scripts/build-static-render.sh(Alpine, viadocker run— identical to a localpodman run) andscripts/build-static-render-macos.shbuild, test and package;scripts/package-render.shis shared.static-render.ymlruns all three platforms on every PR and caches the build tree — mapnik is ~16 min cold on x86_64, ~1 min warm.release.ymlbuilds the same three archives from a clean tree (no cache) on av*tag, requires the tag to matchproject(... VERSION), smoke-tests each unpacked linux archive on a bare alpine image, and publishes with aSHA256SUMS.workflow_dispatchbuilds without publishing.Licensing
The repo had no license file; it's now MIT. Apache-2.0 was considered and dropped because it's incompatible with LGPL-2.1 for a combined work, and the static binary links LGPL mapnik. Each archive ships
LICENSE-MIT, mapnik'sLICENSE.mapnik, and aNOTICEnaming every statically linked library plus how to rebuild and relink.Not addressed: x86_64 macOS (the matrix could gain it; only arm64 is built), and a warm-cache CI measurement — the key naming changed in the last commit, so the first restore happens on the next push.
🤖 Generated with Claude Code