fix(codegen): carry copy-output provenance in a .hephgen beside the file

[raphaelvigee]

The problem

A codegen = "copy" output is a build artifact that lives in the source tree, so heph has to tell it apart from raw source — otherwise a glob() double-sources the generated content, or a target folds its own output back into its input hash.

That answer was a user.heph.codegen xattr on each generated file, and has_codegen_xattr treats every failure as "not stamped". So losing the xattr doesn't fail the build — it silently turns a build output back into a build input.

Losing it is easy: git checkout, tar (so any actions/cache round trip), rsync without -X, cp without -p, an editor's atomic save, and any filesystem without user xattr support — which stamp_codegen_xattr had to hard-fail on, making copy silently non-portable.

The defect is that the record's lifetime was decoupled from the file's.

The change

Put the record in the same directory as the files it describes. Each directory receiving copy outputs gets a .hephgen:

heph-codegen 1
bar.pb.go	//pkg:gen	h=9a1c0f4e…
foo.pb.go	//pkg:gen	h=8f2a77b1…	prev=1c04dd90…
gen	//pkg:gendir	->../../.heph3/cache/blob/…

It moves under cp -r, mv, tar and docker COPY, and dies with git clean -xd — always in step with what it describes. The write-back maintains it from the target's own declared outputs, so nothing walks the graph and nothing has to be run by hand.

Column 1 is fixed as the entry name in every future version, so an older binary reading a newer file degrades to names-only rather than failing open.

Race-free by construction, not by timing

• Register before publish. The entry is written before the bytes are, so a net-new file is registered before it exists (an entry naming nothing is inert), and a rewrite keeps the outgoing hash as prev= until the rename lands — every instant of the swap has the on-disk content matching some accepted hash. Pruning what a target no longer emits waits until the end of the write-back, when what it emits is fully known.
• The directory mtime does the invalidation. The registry is published by rename and parsed into the mtime-validated DirListing, so it can't go stale relative to its own directory and there's no second coherence path to get wrong. Concurrent write-backs into one directory serialize on an flock in the heph home (a lock beside the registry would die with the inode the rename replaces).

The hash earns its place

A record says "heph wrote this name"; the hash says "and these are still the bytes it wrote". Content heph didn't write reads as source again whatever the registry says — so a branch switch that checks a real file in over a generated path, or a hand-edited generated file, isn't silently dropped from the build. The xattr never had that.

It's free: CachedWalker::file_hash already computes and caches exactly this hash, and it's only consulted when there's a record to check.

Two fixes that fall out

• Registering a path changes what a later glob sources, so a registry change folds into the write-back's wrote and drives the fixpoint recompute. The xattr never did — meaning the run that repaired a stripped stamp skipped it.
• Generated files are published by rename rather than written in place, so a consumer never reads one half-written. A rewrite therefore resets the file's mode to the umask default plus the artifact's exec bit instead of preserving the old mode — deterministic, and (content, exec-bit) is what the fs hash covers anyway.

Decisions worth flagging

• .git/info/exclude, on by default. .hephgen is untracked, and requiring heph tool gen-gitignore to keep git status clean would put a "remember to run it" tax right next to correctness. So on first creating a registry, heph appends **/.hephgen there once, idempotently — nothing committed, no diff, nothing to review. It resolves the gitdir without the git binary, following a linked worktree's gitdir: → commondir to the shared directory. codegen: { gitExclude: false } opts out. This is the one part that writes under .git/.
• The xattr stays, for one release. Read as a per-file fallback (a directory can hold both a registered file and a legacy-stamped one whose target hasn't re-run) and still written, but best-effort — the hard failure on an xattr-less filesystem is gone, and a mixed fleet stays readable both ways. Cost: the getxattr per unregistered file survives, so the read-path win waits for a migration sweep and the removal a release later.
• Orphans aren't reclaimed yet. A record whose owning target was deleted still hides its file — the behaviour the xattr had, and the safe direction, since an unreconciled orphan must never be promoted back into a compiled input. Detection and reclamation land with the end-of-run reconciliation in a follow-up (docs/CODEGEN_PROVENANCE.md §6).

Compatibility

• DIR_LISTING_VERSION 1 → 2: a row written before this carries no registry, and serving one would report every generated file in the directory as source. One-time re-read.
• New codegen: config section with one optional key; absent means the previous behaviour plus the exclude line.
• New on-disk format, versioned, with a stated forward-compat rule.
• No break in either fleet direction: old binary on a new tree reads the xattr, new binary on an old tree falls back to it.

Testing

• crates/walk: format round-trip incl. prev= and symlink records, future-version degradation, lenient parsing, retain_owned, and that hashout_bytes agrees with file_hashout on the same file (if those ever diverge, every generated file reads as foreign content).
• crates/engine: register-before-publish ordering, mid-write-back non-pruning, a second target's rows preserved, no rewrite in the steady state, .git/info/exclude idempotence + worktree resolution.
• End to end: a copy output stays excluded from a glob after its xattr is stripped; an in_place target won't clobber a copy-owned file with no xattr present; foreign content at a registered path flips back to source; a deleted registry is restored by the next run of its owner; the registry file is never globbed as source.
• lint clean; full tst green.

Docs

docs/CODEGEN_PROVENANCE.md carries the design, the alternative that lost (one sqlite table per repo — it reintroduces the same decoupling in a narrower, correlated form), and what's deferred. example/fmt/ updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ALT9b7iKzgtHxsFWGD2EEU