Add erofs support to apko.

[smoser]

Implement apko writing of erofs images according to the spec at
https://github.com/erofs/erofs-image-spec (spec.md on main; section numbers
in the code and docs refer to it). The spec is still in its draft phase — no
tagged release, and it says media types, annotation keys and the chunk-index
layout may change before the first stable one — so EROFS support here is
experimental.

The docs/erofs.md file in the pr explains the use and erofs image layout.

Scope: this PR adds the single-image writer (--format=erofs, one layer) and
apko erofs ls. Multi-layer splitting and apko erofs mount/umount were
descoped after review — see #2408 — and land next, in that order. Combining
layering with format: erofs is rejected during config validation rather than
silently producing a single layer.

I've opened a draft PR showing the value of erofs in images.
See chainguard-dev/melange#2605 for example use of erofs support in apko by melange.

Related:

• Compression is separate, at apko erofs with compression support via mkfs.erofs #2406.
• Follow-up work is tracked in EROFS follow-ups: mount/umount hardening, layer splitting, go-erofs bump #2408.
• Found one bug in go-erofs Fix setuid/setgid/sticky bits dropped on read and in Mkdir erofs/go-erofs#41

Review round 2 (2026-08-18)

Addressing @mattmoor's review at 46ac139. Descoped, per his suggestion, the two
areas where the findings were concentrated; the rest is fixed here.

Fixed in this PR:

• Whiteouts now use the encoding the spec mandates. Stack detected
  deletions by filename (.wh.NAME, .wh..wh..opq), which §8.1 item 9 forbids
  in EROFS images outright; §3.6 requires a char device with rdev 0 plus
  trusted.overlay.opaque="y". The comment justifying the old encoding claimed
  it matched go-erofs Writer.Merge, but Merge consumes .wh. entries from
  its source and never writes them into an image — so no producer emits them and
  the reader was honoring an encoding that cannot conformantly exist. Whiteouts
  are interpreted only when the stack has two or more layers, since §7 step 6
  lets a lone layer be mounted directly, applying no overlay semantics.
• ReadOCILayers accepts the role shapes the spec allows. Role-absent
  non-final layers and overlay-lower final layers are both conformant (§2.3,
  §3.8 rule 1, §7 step 1) and were both rejected. device and overlay-data are
  now refused as unsupported compositions rather than as malformed images.
• The build time is always stamped. WithBuildTime was passed only for a
  non-zero timestamp, and go-erofs stamps time.Now().Unix() when it is absent —
  so a library caller passing a zero time.Time got a different digest on every
  build. It cannot be passed verbatim (a zero time.Time has a negative Unix
  seconds value that wraps through uint64), so zero and pre-epoch clamp to
  epoch 0.
• os.features reaches the index platform descriptor. §5.4 is a MUST in two
  places and only the config half was implemented, so consumers filtering on the
  index — before fetching any config — never saw the signal.
• The default output is named .erofs, not apko-<arch>.tar.gz.
• CI installs erofs-utils, so the fsck.erofs cross-checks actually run.
  Without it every one degraded to a t.Log and TestWriteErofs_FsckErofs
  skipped, leaving go-erofs reading its own output as the only always-on
  validation.
• Hardlink materialization and the go-erofs v0.3.1 caveats are written down: the
  pinned version predates Fix setuid/setgid/sticky bits dropped on read and in Mkdir erofs/go-erofs#41, so FileInfo.Mode() misreports
  setuid/setgid/sticky and callers must read *erofs.Stat.Mode off Sys().
• Dropped erofsLayer.LayerPath(), which had no callers. The exported
  state/driver surface went away with the mount descope, which also means the
  package no longer has a build-tag-dependent API — everything left is pure Go
  and builds everywhere.
• Spec citations corrected: the "PR implement entrypoint #1" pointer (merged 2026-06-01) now names
  spec.md, and four places that presented apko's producer convention as a spec
  rule now say which is which.

Moved to #2408, with the code preserved on the feat/apko-erofs-full branch of
smoser/apko so nothing is reconstructed from scratch:

• Package routing never matching on the real tarfs path, and dir-only subtrees
  being dropped from every layer — plus the tarfs-backed test that would have
  caught the first, and the temp-file/fd leak on error paths.
• State-file trust in Unmount, the rerun instruction that cannot work, the
  writable-by-default mount that discards upper, and a Driver seam so the
  orchestration is testable at all.

Not done here: a size measurement on a hardlink-heavy image. I had no such image
on hand, so docs/erofs.md states the arithmetic — one full copy per link,
rounded to the block size — rather than a number I did not measure.

[smoser]

fyi @raharper, @rchincha

[smoser]

I think this is ready for some eyes to review and some testing of it. See the description and linked PRs that i put together to show its usefulness.

[raharper]

Looks really solid and complete with extensive testing. a couple comments/questions I'm interested in see feedback.

[smoser]

I pulled the compression out. it is now separate at #2406.
When this lands, we can look to re-add compression as shown that way or via erofs/go-erofs#45

[mattmoor]

I took a deep pass over this at head 46ac139. The single-image writer path (--format=erofs producing one layer) held up well under scrutiny: close-before-hash ordering, xattr and special-bit handling, and byte-determinism all check out, and the fixes from the earlier review round are all present at head. apko erofs ls is also in good shape.

I found two significant bugs in the multi-layer split path (package routing never matches on the real tarfs path, and dir-only subtrees are dropped from every layer) and two in the umount path (state-file trust, and a rerun instruction that cannot work). Details inline. Given where those are concentrated, one option is to descope: land the single-image format plus ls now, and follow up with the layering split and mount/umount hardening.

Cross-cutting notes that don't anchor to one line:

• melange coordination: melange#2605 currently pins a fork of apko and passes erofs+zstd[,level=N] format strings that this head rejects after the #2406 split; its compressed mode is dead until #2406 merges and melange re-pins. Worth a note on that PR.
• Exported surface: StatePath/WriteState/LoadState/RemoveState/StateSchemaVersion (state.go) and Driver/NewDriver/ResolveMode (driver_linux.go) have no callers outside the package, and the linux-only exports give the package a build-tag-dependent API. Consider unexporting before this ships as a library surface. Similarly erofsLayer.LayerPath() has no callers.
• Test seam for the mount plane: Mount/Unmount construct their Driver internally, so the 305 lines of orchestration in mount_linux.go (cleanup LIFO, state lifecycle, unmount policy) can't be unit-tested with a fake driver. driver_linux_test.go covers only the argv builders; nothing anywhere executes the orchestration.

Things I checked that came back clean, so others can skip them: determinism of the single-image path (go-erofs sorts xattr keys before serialization; no UUIDs or randomness; SOURCE_DATE_EPOCH plumbed through), OCI blob path traversal in erofsmount (ggcr's Hash parsing forecloses it), regressions to tar-path users from the shared-file changes (the rwosfs char-device seeding fix only changes behavior for DirFS over trees that already contain real device nodes, which no in-tree flow constructs), and the go-erofs supply chain (official erofs org, sumdb-verified tag, containerd pins the identical v0.3.1).

[smoser]

Thanks for the depth here, @mattmoor — the two multi-layer findings were both
real and neither was reachable by the existing tests, which is the worst
combination. Took the descope you suggested: this PR is now the single-image
writer plus apko erofs ls, with layering and mount/umount tracked in #2408 and
landing in that order. The pre-descope tree is preserved on
feat/apko-erofs-full in my fork, so nothing gets reconstructed from memory.

Per-comment replies are inline. The three cross-cutting notes:

Exported surface. All of StatePath/WriteState/LoadState/RemoveState/
StateSchemaVersion and Driver/NewDriver/ResolveMode left with the mount
descope, and erofsLayer.LayerPath() is dropped. A knock-on benefit worth
naming: with the Linux-only files gone the package has no build-tag-dependent API
at all — everything left is pure Go and builds everywhere, which was the part of
that note I liked least about the old shape. Ls also lost the Options bag it
only ever read Arch out of and now takes a plain arch string, so Mode and
Options can come back package-private when mount does. Noted in #2408.

Test seam for the mount plane. Agreed, and it's a prerequisite in #2408 rather
than a nice-to-have: a Driver injection point, so the cleanup LIFO, state
lifecycle and unmount policy are exercised by something. Right now nothing
anywhere runs them, which is how two bugs sat in 305 lines undisturbed.

melange coordination. Not done yet — I'd rather not comment on melange#2605
until this lands and I know what it's re-pinning to. It's recorded in #2408 under
the compression item so it doesn't get lost: melange passes
erofs+zstd[,level=N] strings that current apko rejects after the #2406 split, so
its compressed mode is dead until #2406 merges and melange re-pins off my fork.

Also worth flagging two things you didn't ask about but that changed:

• Your WithBuildTime suggestion couldn't be taken literally. Passing a zero
  time.Time through uint64 wraps (Unix() is −62135596800), so zero and
  pre-epoch clamp to epoch 0 instead. And my first test for it was vacuous —
  go-erofs's default has one-second granularity, so back-to-back builds match
  regardless. It now pins zero against an explicit epoch build.
• The .wh. comment's claim that the encoding "matches go-erofs Writer.Merge"
  was simply wrong, which removed the only argument for keeping it. Merge
  consumes .wh. from its source and never writes such names into an image, so I
  deleted the tar handling outright instead of keeping it as a documented extra.

Verified before pushing: go test ./... green, golangci-lint run -n clean, and
a real wolfi-base build with --format=erofs produces os.features: ["erofs"]
on both the index platform descriptor and the config — the §5.4 half that was
missing — with apko erofs ls reading it back with sticky bits and device numbers
intact.

[mattmoor]

Approving the descoped scope. All in-scope fixes from the review verified at 32214b1: always-stamped build time with the pre-epoch clamp, os.features on both the config and the index platform descriptor, spec-legal role shapes accepted, the §3.6 whiteout encoding in Stack pinned against real go-erofs layers, the .erofs default name, the documented hardlink and go-erofs v0.3.1 caveats, and erofs-utils running the fsck cross-checks in CI (which caught the --xattrs portability issue on their first run, a good sign the check earns its keep). The descope is clean: no dangling references to the mount plane or the splitter anywhere, and #2408 accurately tracks the deferred findings.

Two non-blocking notes left on reopened threads: the index digest change for tar builds atop base images whose config carries os.features deserves an explicit ack or a pinning test, and the ls merged view has two pre-existing kernel-overlayfs divergences worth a docs sentence or a #2408 item.