Skip to content

[CoreAI] Add the Core AI SDK Swift and Objective-C bridge - #23391

Draft
metascroy wants to merge 1 commit into
coreai-v2/03b-source-bundlesfrom
coreai-v2/04-sdk-bridge
Draft

metascroy wants to merge 1 commit into
coreai-v2/03b-source-bundlesfrom
coreai-v2/04-sdk-bridge

Conversation

@metascroy

@metascroy metascroy commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Adds the private bridge between ExecuTorch's Objective-C++ code and the Swift CoreAI framework, and packages it for CMake consumers. No ExecuTorch delegate yet; a later PR wires this into CoreAIBackend.

  • runtime/ETCoreAIBridge.h: Objective-C protocols for model acquisition, prepared models, sessions and tensor transfer, plus availability, device architecture and loader factory entry points.
  • runtime/ETCoreAIModel.swift (module CoreAIBridge):
    • Restore models from bookmarks, specialize source bundles with the persistent default SDK cache, bind named functions with ordered inputs/outputs, and evict cache entries.
    • Asynchronous SDK calls run in detached tasks and report through completion blocks. Only an SDK nil result is a cache miss; thrown errors keep their underlying cause.
    • Inputs are borrowed from caller storage through Core AI raw views; outputs use one memcpy for dense row-major arrays and strided copies otherwise.
  • runtime/ETCoreAITensor.mm, runtime/coreai_bridge.mm: tensor wrappers and C entry points, built as the private coreai_bridge_obj object library so the delegate can absorb them and host tests can substitute a fake bridge.
  • Build (backends/apple/coreai/CMakeLists.txt): requires CMake 3.31 and Ninja/Xcode, keeps ExecuTorch's numeric preprocessor definitions away from swiftc, applies an OS 27 minimum only to Core AI targets, and builds coreai_swift. cmake/PublishSwiftHeader.cmake publishes the generated header for Xcode.
  • Tests: runtime/test/SwiftBridgeTests.swift is built with the bridge sources into the coreai_swift_bridge_test XCTest bundle (Ninja, macOS) and run by CTest under xcrun xctest. The invalid-URL case uses a non-file coreai-test: URL.
  • README: "SDK Bridge" and the build requirements in "Building".
  • Changes outside backends/apple/coreai are CMake and CI only, and only take effect when EXECUTORCH_BUILD_COREAI=ON:
    • tools/cmake/ExecuTorchCoreAIDependencies.cmake (new): resolves Foundation, CoreAI and Swift runtime link paths from the consumer's SDK; fails configuration for SDKs older than 27.
    • CMakeLists.txt: install that file into the package config.
    • tools/cmake/preset/default.cmake: note the SDK 27 requirement.
    • .github/workflows/coreai.yml: also trigger on changes to ExecuTorchCoreAIDependencies.cmake.

Stack: 5 of 9, based on #23390. Review only this PR's commit. Next: #23392.

Test plan: built locally for macOS 27.0; ctest -N lists coreai_host_test and coreai_swift_bridge_test, which the Core AI workflow runs on the macOS 27 runner. The XCTest bundle links CoreAI, so it only runs there (first CI run of it). lintrunner is clean.

@pytorch-bot

pytorch-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23391

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 7 Pending, 1 Unclassified Failure

As of commit 756e906 with merge base 0b3d26d (image):

NEW FAILURE - The following job has failed:

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 3, 2026
Summary:
Add the private bridge between ExecuTorch's Objective-C++ code and the Swift
`CoreAI` framework, and package it for CMake consumers. No ExecuTorch delegate
is added yet; a later commit wires this bridge into `CoreAIBackend`.

- `runtime/ETCoreAIBridge.h`: Objective-C protocols for model acquisition,
  prepared models, sessions and tensor transfer, plus availability, device
  architecture and loader factory entry points.
- `runtime/ETCoreAIModel.swift` (module `CoreAIBridge`): restore models from
  bookmarks, specialize source bundles with the persistent default SDK cache,
  bind named functions with ordered inputs/outputs and evict cache entries.
  Asynchronous SDK calls run in detached tasks and report through completion
  blocks; function binding completes synchronously. Only an SDK nil result is
  a cache miss; thrown errors keep their underlying cause.
- Inputs are borrowed from caller storage through Core AI raw views; only
  shape metadata is copied. Outputs use one `memcpy` for dense row-major
  arrays and fall back to strided copies otherwise.
- `runtime/ETCoreAITensor.mm`, `runtime/coreai_bridge.mm`: tensor wrappers and
  the C entry points. They build as the private `coreai_bridge_obj` object
  library so the delegate can absorb them and host tests can substitute a
  fake bridge.
- Build: `backends/apple/coreai/CMakeLists.txt` requires CMake 3.31 and the
  Ninja or Xcode generator, keeps ExecuTorch's numeric preprocessor
  definitions away from swiftc, applies an OS 27 minimum to Core AI targets
  only, and builds `coreai_swift`. `cmake/PublishSwiftHeader.cmake` publishes
  the generated Objective-C header for Xcode.
- Tests: `runtime/test/SwiftBridgeTests.swift` builds with the bridge sources
  into the `coreai_swift_bridge_test` XCTest bundle (Ninja, macOS), which
  CTest runs under `xcrun xctest`.
- README: "SDK Bridge" and the build requirements in "Building".

Changes outside `backends/apple/coreai` are CMake and CI only and only take
effect when `EXECUTORCH_BUILD_COREAI=ON`:
- `tools/cmake/ExecuTorchCoreAIDependencies.cmake` (new): resolves Foundation,
  CoreAI and Swift runtime link paths from the consumer's SDK, so installed
  packages do not embed the producer's Xcode paths. It fails configuration
  for SDKs older than 27.
- `CMakeLists.txt`: install `ExecuTorchCoreAIDependencies.cmake` into the
  package config.
- `tools/cmake/preset/default.cmake`: note the SDK 27 requirement in the
  option description.
- `.github/workflows/coreai.yml`: also trigger on changes to
  `ExecuTorchCoreAIDependencies.cmake`.

Test Plan:
```
# Local: configure and build only
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=27.0 -DEXECUTORCH_BUILD_COREAI=ON \
  -DEXECUTORCH_BUILD_TESTS=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON
cmake --build build --target backends/apple/coreai/all
# CI (macOS 27 runner, .github/workflows/coreai.yml)
ctest --test-dir build/backends/apple/coreai --output-on-failure --no-tests=error
```
CTest runs `coreai_host_test` and `coreai_swift_bridge_test`. The XCTest bundle
links CoreAI, so it only runs on the macOS 27 runner; locally it is built but
not run.
@metascroy
metascroy force-pushed the coreai-v2/03b-source-bundles branch from 9d457ae to 84780b8 Compare October 5, 2026 03:14
@metascroy
metascroy force-pushed the coreai-v2/04-sdk-bridge branch from dae7df4 to 756e906 Compare October 5, 2026 03:14

This branch was successfully deployed

1 active deployment
cadence — 756e9064 Deployed Oct 5, 2026 by metascroy via hifi-op-test / hifi4 #31506
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant