You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 558473d
Browse filesBrowse the repository at this point in the historyBrowse files
The wheel ships the runtime, kernels, delegate, thread pool and profiler
as separate shared
libraries, but nothing outside Python can use them, because the
installed CMake package names
none of them. A C++ application would have to hard-code paths into
the wheel's private layout.
The headers have the same gap. The wheel installs only the subset a
custom-operator build needs,
which leaves out `extension/module`, the entry point the documentation
tells C++ callers to use. So
the wheel ships the libraries to run a model and no way to call them.
Name each shipped library as a CMake component, so `find_package`
locates them, and ship the
headers a caller needs. A component is just a name a consumer can ask
for, and CMake reports a
missing one while configuring rather than at link time.
```cmake
find_package(executorch 1.5 REQUIRED COMPONENTS kernels_optimized)
target_link_libraries(my_app PRIVATE executorch::runtime
executorch::kernels_optimized)
```
| component | library it resolves to |
| --- | --- |
| `executorch::runtime` | `libexecutorch.so` |
| `executorch::kernels_optimized` | `libexecutorch_kernels_optimized.so`
|
| `executorch::backend_xnnpack` | `libexecutorch_backend_xnnpack.so` |
| `executorch::threadpool` | `libexecutorch_threadpool.so` |
| `executorch::etdump` | `libexecutorch_etdump.so` |
Each component records where the wheel keeps its libraries, so an
application built against it
finds them without the caller setting a library search path.
The version file answers on CMake's SameMajorVersion rule: a request is
accepted when it is no newer
than the installed version and names the same major, and rejected
otherwise, including a request one
major below. ExecuTorch promises nothing across a major boundary, so
answering a request written and
tested against an older major would hand the consumer a package it has
never seen and move the
failure to compile or run time instead of to `find_package`. The other
package generators in this
tree already use that rule.
Headers include the module and tensor entry points, the CPU kernel
helpers, the allocator and data
loader concrete classes Module's constructors take, the profiler entry
points, and the
FlatTensorDataMap and MergedDataMap types plus the .ptd file header a
caller writing a .ptd needs.
These headers are part of the published artifact, so the wheel workflows
build on a change to one.
Without that a header change reaches a nightly wheel with no wheel build
having run on it. They are
listed as whole trees rather than as the directories the copy list
names, so growing that list
cannot quietly leave a header outside the filter. Measured over the last
147 commits on main: the
whole trees would newly build a wheel on four of them, the exact
directories on one.
CMake 3.28 or newer gets these targets. Older versions do not, because
they write the `$ORIGIN`
marker (the "look next to me" token in a library search path)
incorrectly:
```
3.24.3, 3.27.9 Makefiles double the dollar sign, Ninja drops the name
3.28.4, 3.31.8 both write the token correctly
```
That would produce a target that runs where it was built and fails once
the application is copied
elsewhere, so no target is defined below 3.28. Those versions get plain
variables instead:
`EXECUTORCH_LIBRARIES` with the runtime and every shipped library by
path, plus
`EXECUTORCH_INCLUDE_DIRS`, `EXECUTORCH_COMPILE_DEFINITIONS` and
`EXECUTORCH_CXX_STANDARD`. All four
are needed, because an imported target carries the definitions and the
C++ standard along with the
library and a plain path carries neither. Linking the libraries alone
stops at
`#error "You need C++17 to compile ExecuTorch"`.
`ET_USE_THREADPOOL` is added to `EXECUTORCH_COMPILE_DEFINITIONS` on the
pre-3.28 route when the
thread pool library ships. Without it the runtime header supplies a
local inline serial fallback
for `parallel_for`, so a consumer following the documented recipe linked
the thread pool library
and still ran serial code with no diagnostic.
Built the wheel, installed it into a clean environment, and built a C++
application against the
installed wheel alone:
- the application links the runtime, runs a model, and matches eager
PyTorch, and still runs after
being copied away from the wheel.
- asking for a component the wheel does not ship fails while
configuring, naming the component.
- a version request is honoured, including ranges, and a request naming
a different major is
rejected in both directions. Measured with cmake 3.31.8 against
fabricated 1.5.0 and 2.0.0
packages: 1.5.0 accepts no request at all, `1.0`, `1.5`, `1.5.0`,
`1.0...<2.0` and `1.0...2.0`,
and rejects `0.1` and `2.0`; 2.0.0 accepts `2.0` and rejects `1.5` and
`2.1`.
- shipped headers can be included on their own, and one entry point per
shipped component also
links against the shipped libraries. A small number are exempt because
they need something outside
the package: a Windows shim, a test framework, or a header that says in
its own text not to
include it directly. The exempt list is compiled too, so an entry that
starts working is reported
rather than left in place.
- the thread pool probe compiles with `ET_USE_THREADPOOL`, on both the
modern-CMake route (from the
runtime target) and the pre-3.28 route (from
`EXECUTORCH_COMPILE_DEFINITIONS`). Without it the
header supplies a local inline definition and the probe linked
identically whether or not the
library was on the link line, so it could not detect the component being
dropped. Measured both
ways.
- an application's runtime search path is recorded as `DT_RUNPATH`, not
the older `DT_RPATH`. That
matters because `DT_RPATH` is searched ahead of `LD_LIBRARY_PATH` and is
inherited by
dependencies, so a consumer could not point a locally built or
instrumented runtime at their
application. Verified by shadowing the runtime through `LD_LIBRARY_PATH`
and watching the loader
pick it up, which `DT_RPATH` ignores.
- on real CMake 3.24 and 3.27, an application configures, builds and
runs through the variables.
Measured what each one contributes, with the consumer pinned to C++14 so
its own standard does not
hide the package's requirement: linking `EXECUTORCH_LIBRARIES` alone
fails on a missing header,
adding the include directories and definitions then fails on the C++
standard, and applying
`EXECUTORCH_CXX_STANDARD` builds and loads a model. The kernels also
need scoped retention there,
because a registration-only library exports nothing the application
references and the linker
drops it, which showed up as "Missing operator" at run time rather than
as a link error. The
smoke test now runs the same shape automatically when
`EXECUTORCH_PRE_328_CMAKE` points at an
older cmake binary, so a future change on the fallback path fails a
check rather than only
showing up on the first user with older cmake.
- `find_package` succeeds when the interpreter on PATH is not the one
the wheel was built for. The
extension's own file name carries its suffix, so asking a different
interpreter for it reported a
complete install as not found.
Ran on Linux x86_64 and aarch64. The macOS wheel keeps the fused
extension and ships no separate
libraries, so these checks do not apply there and its smoke test does
not run them.
Copy file name to clipboardExpand all lines: docs/source/getting-started.md
+17Lines changed: 17 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -171,6 +171,23 @@ For more information on iOS integration, including an API reference, logging set
171
171
ExecuTorch provides C++ APIs, which can be used to target embedded or mobile devices. The C++ APIs provide a greater level of control compared to other language bindings, allowing for advanced memory management, data loading, and platform integration.
172
172
173
173
#### Installation
174
+
On Linux and macOS the quickest route is the pip package, which ships the runtime as prebuilt
175
+
libraries with headers and a CMake package, so there is nothing to build:
See [Using the prebuilt libraries from the pip package](using-executorch-cpp.md#using-the-prebuilt-libraries-from-the-pip-package)
187
+
for a complete walkthrough, and [Running on a GPU with the CUDA package](using-executorch-cpp.md#running-on-a-gpu-with-the-cuda-package)
188
+
for GPU support.
189
+
190
+
For a platform the package does not cover, or to change build options, build from source instead.
174
191
CMake is the preferred build system for the ExecuTorch C++ runtime. To use with CMake, clone the ExecuTorch repository as a subdirectory of your project, and use CMake's `add_subdirectory("executorch")` to include the dependency. The `executorch` target, as well as kernel and backend targets will be made available to link against. The runtime can also be built standalone to support diverse toolchains. See [Using ExecuTorch with C++](using-executorch-cpp.md) and [Building from Source](using-executorch-building-from-source.md) for a detailed description of build integration, targets, and cross compilation.
Leaving out `EXECUTORCH_COMPILE_DEFINITIONS` fails with a missing
237
+
`torch/headeronly/macros/cmake_macros.h`, because the vendored headers look for a file that only
238
+
exists inside a PyTorch build.
239
+
240
+
`INSTALL_RPATH` matters once you run `cmake --install`. CMake gives your program a search path while
241
+
it sits in the build directory and removes that path when installing, so an installed program cannot
242
+
find the libraries unless you record where they live.
243
+
244
+
You should not need `LD_LIBRARY_PATH`. The shipped libraries record where their neighbours live, so
245
+
they find each other once the program links against the installed package.
246
+
247
+
### Building from source
248
+
249
+
43
250
ExecuTorch uses CMake as the primary build system. Inclusion of the module and tensor APIs are controlled by the `EXECUTORCH_BUILD_EXTENSION_MODULE` and `EXECUTORCH_BUILD_EXTENSION_TENSOR` CMake options. As these APIs may not be supported on embedded systems, they are disabled by default when building from source. The low-level API surface is always included. To link, add the `executorch` target as a CMake dependency, along with `executorch_backends`, `executorch_extensions`, and `extension_kernels`, to link all configured backends, extensions, and kernels.
0 commit comments