Add reusable Apple build with explicit native provenance

.agents/skills/journal-build/SKILL.md

@@ -0,0 +1,70 @@
+---
+name: journal-build
+description: Build and verify Journal Apple apps for macOS, iOS Simulator and iPhoneOS using the repository entry point and explicit native provenance.
+---
+
+# Journal Build
+
+Run from the repository root with `bash tool/build_journal_apple.sh`; use `--help` for current options. It shares the existing Dune, Swift and icon helpers. Build never installs, touches a graph, updates dependencies/pins or deletes caches. Coordinate CPU/actool and borrowed inputs with any active build owner.
+
+## Select inputs and target
+
+All targets are arm64 with deployment target >=26.0. Choose an external `--cache-dir` and local LUI `platform/apple` package. Production LUI must be clean at its locked Git pin, including untracked files.
+
+| Target | SDK | Mode / prerequisites |
+| --- | --- | --- |
+| `macos` | `macosx` | Native Dune object; explicit prepared `--switch` matching locks/pins. Adhoc signing by default. |
+| `ios-simulator` | `iphonesimulator` | Explicit `--link-only` stub for Swift/link/package checks, or the experimental mode below. |
+| `ios-device` | `iphoneos` | Explicit `--experimental-host-restamp --native-manifest FILE`, existing certificate SHA1/profile/entitlements. |
+
+Default iOS runtime builds and raw `JOURNAL_OCAML_OBJECT`/extra-object injection are refused. Existing `lui_probe_host.py` callers keep `--native-object`: the helper selects isolated `--probe-native-object` or explicit `--link-only` when absent, with its own host-local cache. Probe import requires external staged Swift/plist, a distinct `-probe` bundle ID, a matching macOS/Simulator object and runtime/bridge symbols; it hashes the object before/after building. It never restamps, accepts device builds or claims production source/ABI proof. Warm-start Simulator inputs must already match Simulator.
+
+Host ARM64 OCaml/runtime restamping is an experimental compatibility route, **not verified full iOS cross-compilation or ABI acceptance**. Real target C/Swift and signing checks do not establish full OCaml ABI or startup acceptance.
+
+Simulator signing is adhoc. macOS development signing needs `--sign-identity` plus `JOURNAL_MACOS_TEAM_ID`; device mode requires the existing signing inputs shown below.
+
+```sh
+bash tool/build_journal_apple.sh macos --switch YOUR_SWITCH \
+  --lui-package /path/to/lui/platform/apple --cache-dir ../journal-build-cache --preflight
+bash tool/build_journal_apple.sh ios-simulator --link-only \
+  --lui-package /path/to/lui/platform/apple --cache-dir ../journal-build-cache
+bash tool/build_journal_apple.sh ios-device --experimental-host-restamp \
+  --native-manifest ../native-input.json --lui-package /path/to/lui/platform/apple \
+  --cache-dir ../journal-build-cache --sign-identity CERTIFICATE_SHA1 \
+  --provisioning-profile /path/to/device.mobileprovision \
+  --sign-entitlements /path/to/device.entitlements --build-number 1.2.3
+```
+
+`--preflight` checks without compilation; remove it for the macOS build. Unset ambient compiler/OCaml/SDK lookup overrides; SDK selection is explicit. Pins come from active switch state; raw source metadata may omit `url`, while any recorded installed source must still agree. CI local pins require their matching resolution record and clean resolved commit. Provision dependencies separately with `tool/prepare_ci_dependencies.py`, which fetches and pins; use only an authorized switch. The builder never repairs it implicitly. Offline moving refs have no remote freshness claim.
+
+## Experimental object provenance
+
+The envelope references original producer records by SHA-256; paths are relative to the envelope, while producer records retain their own explicit paths:
+
+```json
+{
+  "schema_version": 1,
+  "source": {"path": "preflight.json", "sha256": "SHA256"},
+  "link": {"path": "native-command-manifest.json", "sha256": "SHA256"},
+  "xmlm_force_link": {"path": "xmlm-force-link-proof.json", "sha256": "SHA256"},
+  "validated_build": {"path": "successful-build/manifest.json", "sha256": "SHA256"}
+}
+```
+
+`source` binds native source hashes, producer commit, LUI/Signal/DataScript pins and complete-object hash. `link` binds compiler/dependency/archive identity, including mldoc/XMLm pins. `xmlm_force_link` proves a single XMLm unit retained its original cmx/object bytes. Recorded commands are data, never command hooks.
+
+`validated_build` is optional: use a trusted successful experimental manifest from this entry point binding the same records, native source, compiler and object. Without it, the original producer files and XMLm proof must still verify in full. With it, consumed archives/compiler/source paths become historical hash proof and may later change or disappear. This preserves object provenance; it does not prove those paths can still rebuild it. Keep original records/object/receipt read-only; never rewrite proofs or invent a receipt to bypass mismatch.
+
+The producer commit must be an ancestor of the builder revision. The full native source set, Dune/opam/link configuration and pins must still match exactly; Build/tool/docs commits may advance HEAD, changed native bytes need new evidence. Current object/platform/symbols, runtime headers used by fresh target C, records/receipt, SDK and Swift/signing inputs remain live checks. Reports distinguish builder `source_sha` and `native_source_sha`.
+
+## Cache and failure handling
+
+Identity includes source, actual dependency ABI inputs, toolchain/SDK, flags and target. Identical inputs reuse incremental state; changed inputs select another namespace. Every run leaves external phase logs and `run-*/manifest.json`. The cache has a nonblocking lock; native Dune builds also hold the switch read lock. A busy lock belongs to its active owner: coordinate or wait. Preserve existing output/rollback apps and choose a new `--app-dir`; nothing is cleared automatically.
+
+For failure, read `failed_phase`, `error` and that phase's log. Correct target/environment or select matching prepared inputs. Source/configuration/pin/current-input mismatch requires matching provenance; a historical archive change alone can use the validated receipt contract above. Missing mldoc/XMLm retention requires linker-rule review; protected Dune/spec/lock edits still follow the repository's authorization rules. Legacy object-injecting Dune aliases remain refused; probe helpers use the isolated mode above.
+
+## Accept and hand off
+
+Only `status: built` confirms an artifact. Check recorded revisions/dirty hashes, target/SDK/flags, bundle/executable hashes, version and signing identity. Resources/icons precede signing; device profile/certificate/team/entitlements and strict signature are checked. Current GMP archives are rehashed before native macOS acceptance. Experimental builds additionally check Swift object platforms, fresh target C link-map providers and runtime/mldoc/XMLm retention. Listing a `.cmxa` alone does not establish retention.
+
+`preflight-passed` and `link-only` are not runtime acceptance. Preserve `cross_abi_validated=false` for host-restamped objects and state which compilation was reused. Carry the manifest with the App; installation and phone/graph operations are separate from build. For builder changes, use `PYTHONDONTWRITEBYTECODE=1 python3 tool/test_journal_build.py` and the existing icon test as appropriate.

AGENTS.md

@@ -9,3 +9,7 @@
 - If development is blocked because the `.mli` definitions under `spec/` are unclear or unreasonable, stop development immediately and report the specific spec issue, suggested changes, and rationale.
 - Do not modify any ocaml files in bonsai_flutter repo
 - Prefer native bonsai-ui/SwiftUI components and their built-in layout, scrolling, and animation behavior. Do not recreate these behaviors using per-row geometry tracking, shared observable geometry state, or repeated collection scans during scrolling. If native APIs cannot satisfy a requirement, explain the limitation and obtain explicit approval before implementing custom UI coordination.
+
+## Build
+
+Use `bash tool/build_journal_apple.sh --help` as the Apple build entry point. Read [journal-build](.agents/skills/journal-build/SKILL.md) for target selection, prerequisites, cache/provenance, failure handling and acceptance limits.

docs/agent-guide/implemented/process/2026-10-15-reusable-apple-build.md

@@ -0,0 +1,48 @@
+# Reusable Apple Build
+
+## Problem
+
+Recent Journal builds repeatedly reconstructed dependency, target, cache and signing checks in external task scripts. The existing Apple entry point defaults to a stub, suppresses signing failures and does not record artifact provenance.
+
+## Decision
+
+Extend `tool/build_journal_apple.sh` through a Python companion using the existing Swift package and icon packager. Build the production macOS native object through Dune in an owned external cache after read-only switch/lock/pin checks. Strict default iOS builds still require explicit mode selection. The user has additionally authorized `--experimental-host-restamp --native-manifest FILE`: import only a hash-verified producer host object matching current native sources, validate its single-unit XMLm force-link proof, compile DNS/crypto C freshly for the selected target, and reuse the common Swift/package/sign pipeline. The input manifest references the existing source, link and XMLm proof JSON files by path/hash; it contains no executable command hooks. Device signing inputs are explicit. Record the host runtime ABI limitation; build never installs or changes pins.
+
+After one successful import, an optional hash-bound `validated_build` receipt proves the original producer records and consumed input hashes for the immutable complete object. Historical archives are no longer live build inputs: their later modification/removal cannot alone invalidate that object. Current object/native source/configuration/pins, runtime C headers, original evidence/receipt, SDK identity, Swift and signing inputs remain strict. The producer commit must be an ancestor of the builder revision; exact native bytes/configuration still match across Build/tool/docs commits, and reports distinguish these revisions. The original producer records are never rewritten. The build importer owns this regression; no application reducer can reproduce a filesystem/provenance contract failure, so tests exercise the importer with a tiny real Mach-O producer and temporary external Git/source files.
+
+## Alternatives considered
+
+### Alternative
+
+Do not copy historical device deployment scripts: their host ARM64 restamp route is not a verified iOS OCaml cross toolchain. Do not add a separate competing build entry point or change protected Dune/spec/lock files.
+
+## Acceptance criteria
+
+- CLI failure checks prevent accidental stub production builds, unsupported targets, unsafe output replacement and concurrent cache use.
+- Read-only preflight identifies actual installed dependency mismatches without changing the switch.
+- Explicit link-only assembly verifies target, resources, version and signature, with manifest and phase logs outside Git.
+- Input changes select a new cache without clearing previous caches; repeated identical builds reuse their namespace.
+- Explicit experimental builds validate producer source/configuration/dependency identities, retain runtime/mldoc/XMLm, compile target C, check Swift object platforms, and verify the packaged device signature without claiming full OCaml iOS ABI acceptance.
+
+## Risks
+
+- Current local switch lacks mldoc and XMLm pins; native Dune builds may be blocked rather than repaired implicitly. The explicit imported-native route reuses producer dependency provenance instead of constructing another switch.
+- Host runtime restamping remains experimental, not full target-native OCaml compilation or ABI validation. Production iOS C and Swift targets must be independently checked; raw object injection remains blocked. Link-only is not runnable acceptance.
+
+## Consequences
+
+- The existing shell entry point delegates to one standard-library Python implementation. The documented entry point replaces legacy object-injecting aliases for supported builds; protected Dune/spec/lock files are unchanged.
+- At the user's request, AGENTS retains only the build entry and skill link. Target/prerequisite, provenance/cache, failure and acceptance guidance lives once in `.agents/skills/journal-build/SKILL.md`, following skill-creator guidance and the repository's skill format. This documentation-only reorganization preserves the verified builder bytes and needs no repeat production build.
+- Twenty boundary tests pass on Python 3.9 and 3.13, including changed/missing consumed archives, a later builder-only commit and negative object/source/C/header/evidence/receipt/SDK cases. Two actual Simulator link-only assemblies passed target, resource/icon and strict signing checks, and reused one cache namespace. Concurrent entry was rejected as busy. Icon packaging passed for all three targets.
+- Independent exact-head review reproduced a cancellation race: a leader could exit on SIGTERM while its descendant ignored it, allowing the cache lock to release early. Cancellation now kills the remaining process group even after the leader exits. A real descendant holding a FIFO write endpoint verifies termination before lock release; it failed before the fix and passes afterward.
+- The final independent review also reproduced untracked Swift source being included by SwiftPM while the LUI pin guard ignored it. Production LUI checks now include all untracked files; a preflight regression rejects the added Swift source before native import. The successful production builds used a checkout without this trigger.
+- An actual experimental iPhoneOS build produced a signed production App: runtime/mldoc/XMLm retained, 55 Swift objects verified as iOS ARM64, DNS/crypto link-map providers verified as freshly compiled iPhoneOS C, and profile/signature/entitlements read back successfully. Subsequent review tightened source deletion/configuration, compiler and profile guards. A final retry correctly stopped before compilation when another authorized task changed 14 historical producer archive inputs; it did not accept the old provenance or interfere with that cache. The successful App remains preserved. Independent journal-code-review found no remaining substantive issue after fixes.
+- The later archive mismatch exposed an overly strict importer lifecycle rule: the fixed complete object remained valid while another owner normally updated its consumed archives. Receipt-bound historical provenance now distinguishes that state from changed current inputs. After the repair, a final production iPhoneOS build passed through this entry point: fixed OCaml complete object reused, target C and all Swift/LUI objects compiled in a new cache namespace, icons/resources packaged, signature/entitlements and input stability verified. No OCaml recompilation or phone operation occurred.
+- Full native macOS validation is blocked by the current switch's missing mldoc, nonlocked XMLm and incomplete cached Git source objects. No dependency repair, full OCaml iOS ABI acceptance, phone operation or runtime acceptance is claimed.
+- GitHub review exposed three further build-owner gaps: raw opam metadata was incorrectly used as the active pin, legacy probe helpers still injected a rejected environment object, and GMP lacked final rehashing. Active switch pins now drive lock/CI resolution (including local Git pins), optional installed source metadata remains a conflict guard, and the recorded GMP archive is rehashed before acceptance. Existing probe callers retain `--native-object` through a shared isolated probe mode with explicit target/runtime/hash checks and no production provenance/ABI claim; it cannot build device or production bundles. The four added boundary tests use a real offline temporary opam switch, tiny real Mach-O objects and the actual helper/CLI. These filesystem/CLI failures have no application reducer owner; no product or OCaml test is added.
+- Independent review of that pin repair found local URL refs and fixed lock commits needed separate checks. A valid CI SHA must match a fixed locked SHA, a local URL fragment must resolve to the same commit, and the clean checkout must match that identity. Three real offline negative cases were RED before the additional checks; moving refs keep the stated offline resolution limitation.
+- GMP acceptance also checks Dune's actual linked archive copy against the preflight digest, covering a pkg-config selection change while the original archive stays unchanged. The existing GMP boundary regression includes that RED→GREEN case; no additional test layer or protected Dune change is needed.
+
+## Questions
+
+- The delegated user instruction supplies the target/safety scope and explicitly requests implementation, verification and independent review. No unresolved product choice requires another approval. Protected Dune/spec/lock changes remain outside scope.