Skip to content

fix(receipts): preserve macOS identity across remounts with downgrade-safe migration - #3439

Open
Tyce Herrman (TyceHerrman) wants to merge 7 commits into
microsoft:mainfrom
TyceHerrman:fix/stable-macos-receipt-identity
Open

Tyce Herrman (TyceHerrman) wants to merge 7 commits into
microsoft:mainfrom
TyceHerrman:fix/stable-macos-receipt-identity

Conversation

@TyceHerrman

@TyceHerrman Tyce Herrman (TyceHerrman) commented Nov 2, 2026 •

Copy link
Copy Markdown

Description

APM currently binds receipts to a directory's device number and inode. A device-number change across reboot/remount rejects the same macOS directory. This PR uses the persistent Darwin volume UUID, inode, and creation time for new receipts while retaining strict directory-replacement and ownership checks.

Version-2 receipts live in a separate namespace, leaving version-1 receipts available to older APM. A successful unchanged live install migrates an exactly validated legacy receipt without rewriting skills; dry runs do not migrate. Returning from an older APM admits only changed per-file legacy evidence whose current ownership, target, and bytes validate, so unrelated older writes cannot revive a stale baseline for user-edited files.

The architecture guard and mutation traps now require both legacy and stable identity validation, separate namespaces, and the legacy-write authority/hash checks. The symlinked-path regression asserts that the simulated device-number change actually reaches the session.

Fresh installations now also publish a missing v1 compatibility receipt from validated, committed v2 write evidence. Publication is exclusive/create-only and never overwrites an existing or stale legacy receipt. The v1 projection retains the old device/inode binding and additional stable identity metadata; its initial checkpoint is recorded before publication so unchanged installs remain byte-identical. This addresses the fresh-first downgrade gap reported in independent validation.

Issue and approved scope

Issue: #3421

Does this PR complete the issue, or what remains?

Refs #3421. This is the macOS counterpart of the reported failure mechanism, not a complete fix for the issue's Linux/btrfs case. Linux and other platforms retain the existing strict device/inode identity rule. Stale legacy receipts whose old identity no longer validates still require explicit recovery.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

Tested locally with dependencies from uv.lock on macOS arm64.

Added regression coverage for unchanged migration, symlinked roots, recreated directories, stale/unverified evidence, user edits, and old/new executable compatibility.

Local independent-branch validation: 77 receipt/pipeline/compatibility tests and 43 architecture mutation/frozen-conformance tests passed. Built this branch with the locked PyInstaller and reran all four compatibility cases using that frozen executable against the actual unpatched 0.35.0 executable. The compatibility tests use APM_LEGACY_BINARY and skip when it is not supplied. Device-number changes are simulated

Follow-up validation for 5c19275: 492 focused cases passed across the targeted run and a rerun of four existing alias-diagnostic assertions with wider output. Those four literal warning assertions also failed on the previous PR executable at the narrower width; the tests were not changed. Coverage includes eight compatibility cases using the newly frozen executable and the actual unpatched APM 0.35.0 executable: four legacy-first sequences and four fresh-first sequences, in project/user scope with edited/unedited skills. All four new fresh-first regressions fail against the previous PR executable.

The new tests also cover exclusive-publication races, failed checkpoint writes, pre-projection v2 recovery, stable-identity fallback, unchanged receipt bytes, read-only commands, and changed-claim restrictions. Full-scope Ruff, duplication, auth, architecture, and static guards passed. CodeRabbit raised 0 issues on the follow-up patch.

The full unit-suite run was interrupted by an environment network-policy denial for api.mcp.github.com; no full-suite pass is claimed. No actual reboot/remount or Linux/btrfs/Windows runtime validation was performed. The older executable still cannot be assumed to work after its own device-bound legacy receipt becomes stale.

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour.

The new format is private installation-local materialization evidence, not the public lockfile schema

@OHWONSIK

Copy link
Copy Markdown

Independent validation: fresh-install downgrade coverage is still needed

I evaluated this exact head, ae379c539c29aec128644caf0b1a01472770bae3, together with the independent cleanup fix in #3440 (6e5af56c97533d798a31a2c2eb594e271c0ac764), on top of release v0.35.0 (17a2fc4dfff238c7d93e55cce6ef42ffa1db928e) on macOS arm64. These are local combined-candidate results, not hosted CI results or evidence that either PR alone passed the combined checks.

Additional reproducible compatibility gap

The existing compatibility test starts with an installation made by the legacy executable. The reverse starting point also matters:

  1. Use a disposable HOME and a new consumer with a local APM package containing .apm/skills/demo/SKILL.md and an accompanying asset.md. Select agent-skills explicitly.
  2. Install with the candidate first, without ever running legacy APM in this HOME/project. At this head, this creates v2 evidence but no v1 receipt.
  3. Change the local package from 1.0.0 to 1.1.0 and change both source files, so the next install must perform real writes rather than a no-op.
  4. Run the actual, unpatched v0.35.0 executable against the same installation.

In both project and user scope, legacy APM reported no trustworthy local deployment baseline, then stale cleanup removed the refused deployed files and the command returned success. This reproduced with the two PR patches combined: #3440 cannot protect an invocation of the older executable.

The fresh-first regression requires these postconditions: the deployed directory and every previously installed file still exist; if legacy install succeeds, its files match the new source; if it refuses, the previous bytes and lockfile remain intact. This is separate from the four existing legacy-first → fixed → legacy → fixed cases.

Local experiment, for maintainer consideration

In a disposable candidate only, I tested publishing a missing v1 compatibility projection from a validated v2 receipt at the successful live install's committed-claims boundary. The experiment:

  • uses an exclusive/create-only write and never overwrites an existing v1 receipt;
  • retains the stable identity as additional metadata, checked by the newer reader if v2 is absent;
  • retains the legacy device/inode binding for the old reader rather than weakening its checks;
  • does not create the compatibility receipt during dry-run/audit;
  • does not automatically rebind an already-stale legacy receipt.

There was a second regression to avoid: creating that projection without recording its initial checkpoint makes the next unchanged install treat it as a new legacy write and rewrite the v2 receipt. Recording the expected projection snapshot during v2 persistence, while v1 is absent, preserved repeat-install bytes and the existing no-write/changed-claim restrictions in the local tests.

With those additional local changes, the source-level receipt/missing-receipt/version-compatibility/fresh-first group passed 85 tests. I also built a PyInstaller executable and used the actual original v0.35.0 executable: the four legacy-first compatibility cases and two fresh-first downgrade cases passed. Those six successes include the local additions; they are not a claim about this PR head alone.

The initial broad unit run, before the final checkpoint adjustment, was 25,581 passed / 1 failed / 63 skipped. The sole failure was test_install_dry_run_warns_once_per_invocation: a literal warning assertion was split by Rich wrapping under a long temporary path. It also failed on unmodified v0.35.0 under the same conditions and passed with COLUMNS=240. I have not rerun that entire broad suite on the final local tree.

Remaining acceptance boundary

No actual reboot/remount or Linux/btrfs/Windows runtime validation was performed. Simulated device changes do not establish real remount stability. A compatibility projection also does not promise that the old executable will work after its own device-bound v1 receipt becomes stale.

Could the fresh-first downgrade cases and repeat-byte checkpoint behavior be covered before calling the migration downgrade-safe? If fresh-first downgrade is intentionally unsupported, that boundary and the safe recovery procedure should be explicit. For adoption, the intended path is a validated official release, rather than maintaining a downstream production fork.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The lifecycle test fails on fresh installs because its allowlist omits compatibility receipts, while ledger retirement and publication can discard valid legacy updates or fail on stale records.

3 open issues
What changed in this PR

This PR gives macOS materialization receipts a remount-stable directory identity while preserving compatibility with older APM versions. It addresses the macOS counterpart of #3421; Linux/btrfs behavior remains unchanged.

Changes:

  • Adds Darwin volume UUID, inode, and creation-time identity checks.
  • Introduces separate v2 receipts, validated legacy migration, and create-only v1 compatibility publication.
  • Extends lifecycle, downgrade, and architecture regression coverage.
File Description
tests/​unit/​test_deployment_materialization_receipts.py Covers stable identity, migration, compatibility publication, and edit protection.
tests/​unit/​install/​test_pipeline_ownership_snapshot.py Targets v2 receipt persistence failures.
tests/​integration/​test_skill_materialization_receipt_lifecycle.py Updates namespace and read-only receipt checks.
tests/​integration/​test_required_lifecycle_state_machine.py Validates v2 platform-specific identity fields.
tests/​integration/​test_receipt_version_compatibility.py Adds old/new executable compatibility scenarios.
tests/​integration/​test_generated_lifecycle_state_machine.py Moves the unexpected receipt-write probe to v2.
tests/​integration/​test_architecture_materialization_receipts.py Adds mutation checks for migration and publication safeguards.
src/​apm_cli/​utils/​filesystem_identity.py Reads persistent Darwin volume UUIDs through the native API.
src/​apm_cli/​core/​deployment_ledger.py Implements v2 identity, legacy reconciliation, and compatibility receipts.
scripts/​architecture_linter/​checks/​install_base_integrator_and_contraction.py Guards identity validation and compatibility evidence handling.
.apm/​architecture/​owners/​install-deployment.json Registers the filesystem identity helper under deployment ownership.

Review effort: Balanced

Comment thread src/apm_cli/core/deployment_ledger.py Resolved
Comment thread src/apm_cli/core/deployment_ledger.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Maintainer validation on macOS is still needed to confirm that the native Darwin identity binding remains stable across reboot/remount and rejects recreated directories before it can safely govern overwrite authority.

No open issues 🎉

3 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved ledger reconciliation issues preserve stale baselines after legacy deletions or legacy-path updates, risking overwrites of recreated files and misclassification of subsequent updates.

2 open issues

Review effort: Balanced

Comment thread src/apm_cli/core/deployment_ledger.py Resolved
Comment thread src/apm_cli/core/deployment_ledger.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because legacy and v2 ledger interactions can preserve stale overwrite authority across uninstall and downgrade flows, putting recreated files and user edits at risk.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because subtle cross-version ledger interactions can preserve stale overwrite authority, putting user-edited or recreated files at risk.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Confirmed deletions leave stale v1 receipts intact, allowing a downgraded executable to overwrite recreated user files; validated v1 retirement propagation and a regression test are needed.

1 open issue

Review effort: Balanced

Comment thread src/apm_cli/core/deployment_ledger.py Resolved
@TyceHerrman
Tyce Herrman (TyceHerrman) requested a balanced review from Copilot November 2, 2026 18:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because legacy deletion can leave stale v2 authorization records after bundle membership or target-location changes, potentially allowing recreated files to be overwritten.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@TyceHerrman

Copy link
Copy Markdown
Author

Follow-up to Copilot review #3439 (review), addressed in 4a560df.

I reproduced the bundle-membership gap in all four project/user and default/legacy-path combinations: a later candidate adds a v2-only member, old APM uninstalls it, historical claims and former bytes are restored, and the candidate incorrectly overwrites the recreated member.

The repair refreshes validated v1 coverage from committed, witnessed writes, including newly added members and target changes. The receipt is written before its checkpoint advances so failed publication cannot make unchanged old evidence appear to be a new older-writer update. Observed legacy deletions also revoke uncovered v2 authority; unrelated covered survivors remain usable. No authority is inferred from restored lock claims or adopted file bytes.

Validation: 102 receipt/ownership checks, 29 real-version compatibility cases, and three targeted architecture routing/mutation checks passed, plus Ruff lint/format and diff checks. All four original v2-only end-to-end reproductions also preserved the recreated file and lockfile in a disposable combined candidate with #3440's safety gate. The relocated-target probe stopped safely in old APM, preserving files and ownership, rather than reproducing an overwrite.

The old-release refusal/cleanup limitation and actual reboot/remount validation remain separate boundaries. No full-suite or blanket downgrade-safety claim is made. The latest review has no inline thread; all existing review threads are already resolved. Requesting a fresh Balanced review on the updated head.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Validation only simulated device-number changes, leaving the core guarantee of stable native macOS volume and directory identity across an actual reboot/remount unverified.

No open issues 🎉

Review effort: Balanced

@sergio-sisternes-epam

Copy link
Copy Markdown
Collaborator

Thank you for contributing this pull request. The linked issue #3421 is open but has not been accepted by a maintainer. APM starts with an issue, not an implementation (https://github.com/microsoft/apm/blob/main/CONTRIBUTING.md). Please wait for a maintainer to review and accept the scope on #3421 before proceeding. This PR is labelled status/deferred until then.


Generated by autopilot-pr-triage-worker. This comment is AI-generated and may contain errors.

@sergio-sisternes-epam Sergio Sisternes (sergio-sisternes-epam) added area/distribution Installers (curl/PowerShell/Brew/Scoop), self-update, devcontainer, codespaces. status/deferred Not invited for implementation now; no release commitment. triage/recommended Automated advice completed; not human scope approval. type/bug Something does not work as documented. labels Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/distribution Installers (curl/PowerShell/Brew/Scoop), self-update, devcontainer, codespaces. status/deferred Not invited for implementation now; no release commitment. triage/recommended Automated advice completed; not human scope approval. type/bug Something does not work as documented.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants