Skip to content

[CoreAI] Prepare source bundles - #23390

Draft
metascroy wants to merge 1 commit into
coreai-v2/03a-bookmarksfrom
coreai-v2/03b-source-bundles
Draft

metascroy wants to merge 1 commit into
coreai-v2/03a-bookmarksfrom
coreai-v2/03b-source-bundles

Conversation

@metascroy

@metascroy metascroy commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Materializes the selected Core AI source bundle under the assets root so the SDK can specialize it (runtime/coreai_assets.{h,mm}).

  • Read inline bundle files through NamedDataMap and write them unchanged, keyed by the export-time bundle digest (no runtime payload hashing).
  • Reuse checks the exact file set, sizes, entry types and directory structure. Incomplete or malformed existing sources fail without being replaced, since an SDK model may still be using them; same-size content changes are not detected.
  • Cold publication stages into .staging-<uuid> and publishes with renameatx_np(RENAME_EXCL), all relative to one open root descriptor. If another loader publishes first, its bundle is validated and used.
  • SDK acquisition is not included here.
  • Tests: the source suite in coreai_host_test, plus the filesystem snapshot helpers used from here on. ExclusivePublicationValidatesWinnerAndRemovesLosingStaging covers the concurrent RENAME_EXCL race.
  • README: the "Asset storage" materialization, publication and completeness text.

No changes outside backends/apple/coreai.

Stack: 4 of 9, based on #23389. Review only this PR's commit. Next: #23391.

Test plan: built locally for macOS 27.0; ctest -N lists coreai_host_test, which the Core AI workflow runs on the macOS 27 runner. lintrunner is clean.

@pytorch-bot

pytorch-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23390

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 84780b8 with merge base 0b3d26d (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 3, 2026
Summary:
Materialize the selected Core AI source bundle on disk under the assets root,
so the SDK can specialize it.

- Source preparation (`runtime/coreai_assets.{h,mm}`): read inline bundle
  files through `NamedDataMap` and write them unchanged under the assets root,
  keyed by the export-time bundle digest (no runtime payload hashing). Reuse
  checks the exact file set, sizes, entry types and directory structure.
  Incomplete or malformed existing sources fail without being replaced, since
  an SDK model may still be using them. Same-size content changes are not
  detected.
- Cold publication stages into `.staging-<uuid>` and publishes with
  `renameatx_np(RENAME_EXCL)`, with inspection, staging and publication all
  relative to one open root descriptor. If another loader publishes first, its
  bundle is validated and used.
- Tests: adds the filesystem snapshot helpers used from here on.
- SDK acquisition is not included here.
- README: the "Asset storage" materialization, publication and completeness
  text.

No changes outside `backends/apple/coreai`.

Test Plan:
```
# Local: configure and build only
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=27.0 -DEXECUTORCH_BUILD_COREAI=ON \
  -DEXECUTORCH_BUILD_TESTS=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON
cmake --build build --target backends/apple/coreai/all
# CI (macOS 27 runner, .github/workflows/coreai.yml)
ctest --test-dir build/backends/apple/coreai --output-on-failure --no-tests=error
```
CTest runs `coreai_host_test`, which adds the source suite.
ExclusivePublicationValidatesWinnerAndRemovesLosingStaging covers the
concurrent `RENAME_EXCL` race.
@metascroy
metascroy force-pushed the coreai-v2/03b-source-bundles branch from 9d457ae to 84780b8 Compare October 5, 2026 03:14
@metascroy
metascroy force-pushed the coreai-v2/03a-bookmarks branch from f74a984 to 68a8b17 Compare October 5, 2026 03:14

This branch was successfully deployed

1 active deployment
cadence — 84780b83 Deployed Oct 5, 2026 by metascroy via hifi-op-test / hifi4 #31502
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant