diff --git a/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_index.md b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_index.md new file mode 100644 index 0000000000..6280ab3af9 --- /dev/null +++ b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_index.md @@ -0,0 +1,71 @@ +--- +title: Deploy containerized workloads to Arm-based Linux targets with Topo + +description: Learn how to use Topo to detect device capabilities, select a compatible template, and deploy containerized workloads to Arm-based Linux targets over SSH. + +minutes_to_complete: 30 + +who_is_this_for: This is an introductory topic for embedded, edge, and cloud software developers who want to easily deploy containerized workloads to Arm-based Linux targets with Topo. + +learning_objectives: + - Understand what Topo is, and why you would use it to deploy containerixed workloads to Arm Linux targets + - Use Topo to perform a health-check on the target, generate a target description to capture Arm processor capabilities + - Clone a compatible Topo template for your hardware and deploy the workload + - (Optional) Deploy containerized workloads across heterogeneous devices (Cortex-A + Cortex-M) with Topo and remoteproc-runtime + - (Optional) Use Command-Line-Interface (CLI) Agents with Topo + +prerequisites: + - A host machine (x86 or Arm) with Linux, macOS, or Windows + - An Arm-based Linux target you can access over SSH, for example AWS Graviton, Raspberry Pi, DGX Spark, i.MX 93 + - Docker installed on host and target. If needed, use [Install Docker](/install-guides/docker/) + - lscpu installed on target (typically pre-installed with Linux) + - SSH key-based authentication configured between host and target - if using password-based authentication, Topo can help you setup key-based authentication + - Basic familiarity with containers and CLI tools + +author: Matt Cossins + +### Tags +skilllevels: Introductory +subjects: Containers and Virtualization +armips: + - Neoverse + - Cortex-A + - Cortex-M +tools_software_languages: + - Topo + - Docker + - SSH + - remoteproc-runtime + - remoteproc + - CLI +operatingsystems: + - Linux + - macOS + - Windows + +further_reading: + - resource: + title: Topo repository + link: https://github.com/arm/topo + type: documentation + - resource: + title: Topo template format + link: https://github.com/arm/topo-template-format + type: documentation + - resource: + title: Topo releases + link: https://github.com/arm/topo/releases/latest + type: website + - resource: + title: remoteproc-runtime + link: https://github.com/arm/remoteproc-runtime + type: documentation + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_next-steps.md b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_next-steps.md new file mode 100644 index 0000000000..727b395ddd --- /dev/null +++ b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # The weight controls the order of the pages. _index.md always has weight 1. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/assess-compatibility.md b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/assess-compatibility.md new file mode 100644 index 0000000000..23f3a90479 --- /dev/null +++ b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/assess-compatibility.md @@ -0,0 +1,218 @@ +--- +title: Use Topo to assess target compatibility +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Run Topo health checks + +### Prepare host environment + +Confirm that the required dependencies are available on the host by running this command in your host terminal: + +```bash +topo health +``` + +The output should appear similar to the following: + +```bash +Host +---- +SSH: ✅ (ssh) +Container Engine: ✅ (docker) + +Target +------ +ℹ️ provide --target or set TOPO_TARGET to check target health +``` + +If Docker is missing, please use [Install Docker](https://learn.arm.com/install-guides/docker/). + +If SSH is missing, please use [Install SSH](https://learn.arm.com/install-guides/ssh/). + +## Prepare target environment + +Now that the host device is prepared, we will setup the target. On the host device, connect to your target with SSH. + +```bash +ssh user@my-target +``` + +Once connected to the target, use the following commands to verify both Docker and `lscpu` are installed: + +```bash +docker --version +lscpu +``` + +The output should apear similar to the following: + +```output +Docker version xx.x.x +Architecture: aarch64 +CPU(s): ... +``` + +### Prepare Topo for target + +We will now run a health check against your target. Run the following command from the terminal of your host device. + +If you are using your host device simultaneously as your target, use `topo health --target localhost`. + +```bash +topo health --target user@my-target +``` + +The output should appear similar to the example from a heterogeneous SoC below, but will differ depending on your hardware: + +```output +Host +---- +SSH: ✅ (ssh) +Container Engine: ✅ (docker) + +Target +------ +Connectivity: ✅ +Container Engine: ✅ (docker) +Remoteproc Runtime: ✅ (remoteproc-runtime) +Remoteproc Shim: ✅ (containerd-shim-remoteproc-v1) +Hardware Info: ✅ (lscpu) +Subsystem Driver (remoteproc): ✅ (m33, m0) +``` + +A Topo health check confirms connectivity between the host and target, as well as the verifying the presence of dependencies such as docker. + +You should resolve any `❌` errors before moving on. Warnings (⚠️) can indicate optional capabilities that may be needed in certain projects. `ℹ️` provides other information. A `✅` confirms the presence of dependencies and no warnings or errors. + +If you are using password-based SSH, you will likely see the `❌` error below: + +```output +Connectivity: ❌ (key-based SSH authentication is not setup) + → run `topo setup-keys --target user@my-target` or manually setup SSH keys for the target +``` + +This is because Topo requires key-based SSH. You can use the command specified above, and Topo will setup the key-based SSH for you. Ensure that if prompted to set a passphrase, you leave it empty. Afterwards, run `topo health` again to confirm it has correctly setup the key-based authentication. + +## Optional: install remoteproc-runtime on heterogeneous devices + +If using a Cortex-A + Cortex-M device, such as the i.MX 93, you may see a `⚠️` warning if `remoteproc-runtime` is not installed on the target. + +[`remoteproc`](https://docs.kernel.org/staging/remoteproc.html) is a Linux kernel framework for managing remote / auxiliary processors in a heterogeneous SoC. It allows the main CPU (e.g., Cortex-A) to load firmware on to the auxiliary processors (e.g., Cortex-M), start and stop them, and to communicate with them (e.g. using [`rpmsg`](https://docs.kernel.org/staging/rpmsg.html)). + +[`remoteproc-runtime`](https://github.com/arm/remoteproc-runtime) builds on this by adding a container-style (OCI - Open Container Initiative) interface. This lets you package and manage firmware like container images using standard tools (e.g. Docker or containerd), even though the code runs as firmware on the Cortex-M. [OCI](https://opencontainers.org/) defines open standards for container image formats and runtimes, ensuring compatibility across container tools. + +You can use Topo to install `remoteproc-runtime`. Run the following command from the host device: + +```bash +topo install remoteproc-runtime --target user@my-target +``` + +Run the health command again to verify installation. Topo uses `remoteproc-runtime` under the hood when deploying to heterogeneous devices. + +## Generate a target description + +In this step, you ask Topo to probe your target and create a machine-readable YAML description of the hardware. + +On your host device, run: + +```bash +topo describe --target user@my-target +``` + +This writes a `target-description.yaml` file in your current directory. + +The file captures details such as CPU architecture features, which Topo uses to select compatible templates. + +Open the file and have a look. An example snippet from an AWS Graviton instance is shown below, showing the main processor and its features, an absence of any remote / auxiliary processors, and the total memory: + +```output +host: + - model: Neoverse-V1 + cores: 4 + features: + - fp + - asimd + - evtstrm + - aes + ... +remoteprocs: [] +totalmemory_kb: 16044280 +``` + +## List templates compatible with your target + +Now that Topo understand the capabilities of your target device, it can advise on the compatibility of templates. + +Use the following command on your host device to to list templates according to the target description: + +```bash +topo templates --target-description target-description.yaml +``` + +You can also query templates directly by specifying the target: + +```bash +topo templates --target user@my-target +``` + +An example output for an AWS Graviton instance is shown below: + +```output +✅ topo-welcome | https://github.com/Arm-Examples/topo-welcome.git | main + A minimal "Hello, World" web app for validating a Topo setup and deployment. + It runs a single service that exposes a web page on the target, + with the greeting text customizable via the GREETING_NAME parameter. + +❌ topo-lightbulb-moment | https://github.com/Arm-Examples/topo-lightbulb-moment.git | main + Features: remoteproc-runtime + Reads a switch over GPIO pins on an M class cpu, reports switch state over + Remoteproc Message, then a web application on the A class reads this and + displays a lightbulb in either the on or off state. The lightbulb state is + described by an LLM in any user-specified style. + +✅ topo-cpu-ai-chat | https://github.com/Arm-Examples/topo-cpu-ai-chat.git | main + Features: SVE, NEON + Complete LLM chat application optimized for Arm CPU inference. + + This project demonstrates running large language models on CPU + using llama.cpp compiled with Arm baseline optimizations and + accelerated using NEON SIMD and SVE (when supported and enabled). + + The stack includes: + - llama.cpp server with Arm NEON optimizations (SVE optional) + - Quantized Qwen2.5-1.5B-Instruct model bundled in the image (~1.12 GB) + - Simple web-based chat interface + - No GPU required - pure CPU inference + + Perfect for demos and testing! The bundled Qwen2.5-1.5B model allows the + project to run immediately without downloading additional models. + + Ideal for testing LLM workloads on Arm hardware without GPU dependencies, + showcasing how far you can push NEON acceleration. Rebuild with SVE enabled + when wider vectors are available. + +✅ topo-simd-visual-benchmark | https://github.com/Arm-Examples/topo-simd-visual-benchmark.git | main + Features: NEON, SVE + Visual demonstration of SIMD performance benefits on Arm processors. + Compare scalar (no SIMD), NEON (128-bit), and SVE (scalable vector) + implementations running identical image processing workloads side-by-side. + + This demo shows real hardware acceleration through three C++ services + compiled with different architecture flags, processing the same box blur + algorithm on images. Performance differences are measured in real-time + and displayed in an interactive web dashboard. + + Perfect for demonstrating to non-technical audiences the concrete benefits + of SIMD optimizations, with visual results and quantified speedups. +``` + +In the above example, `topo-lightbulb-moment` is marked as incompatible, since it requires an SoC with both a Cortex-A and a Cortex-M. The Graviton instance used contains Arm Neoverse cores only. All other templates are marked as compatible. You may see different results depending on the target hardware you use. + +## What you've learned and what's next + +You have performed a health check on your target device and generated a description of its hardware features. Topo has informed you which templates are compatible with your target. In the next step, you will choose and deploy a template containerized workload. + diff --git a/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/deploy-workload.md b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/deploy-workload.md new file mode 100644 index 0000000000..ed3b92a6ad --- /dev/null +++ b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/deploy-workload.md @@ -0,0 +1,115 @@ +--- +title: Deploy a compatible containerized workload with Topo +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Choose a starter template and clone it + +Choose one of the pre-existing templates that is compatible with your target hardware. This learning path will showcase deploying an LLM chatbot, but the same steps apply for the other templates. + +To use a template, we clone it from git on our host device by running the following command: + +```bash +topo clone https://github.com/Arm-Examples/topo-cpu-ai-chat.git +``` + +If a template asks for build arguments, Topo prompts you interactively. + +This creates a project directory using the template. The directory will contain template source files and `compose.yaml`. + +You may find it interesting to examine the `compose.yaml` file. An example for the LLM chatbot application is provided below: + +```output +services: + llama-server: + platform: linux/arm64 + build: + context: ./llama-inference + args: + ENABLE_SVE: OFF + HF_MODEL: bartowski/Qwen_Qwen3.5-0.8B-GGUF + HF_MODEL_FILE: "" + ports: + - "8080:8080" + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8080/health"] + interval: 10s + timeout: 5s + retries: 3 + start_period: 60s + chat-ui: + platform: linux/arm64 + build: + context: ./simple-chat + args: + ENABLE_SVE: OFF + depends_on: + llama-server: + condition: service_healthy + ports: + - "3000:3000" +x-topo: + name: "Topo CPU AI Chat" + description: "Complete LLM chat application optimized for Arm CPU inference.\n\nThis project demonstrates running large language models on CPU\nusing llama.cpp compiled with Arm baseline optimizations and \naccelerated using NEON SIMD and SVE (when supported and enabled).\n\nThe stack includes:\n- llama.cpp server with Arm NEON optimizations (SVE optional)\n- Quantized Qwen3.5-0.8B model bundled in the image\n- Simple web-based chat interface\n- No GPU required - pure CPU inference\n\nPerfect for demos and testing! The bundled Qwen3.5-0.8B model allows the\nproject to run immediately without downloading additional models.\n\nIdeal for testing LLM workloads on Arm hardware without GPU dependencies,\nshowcasing how far you can push NEON acceleration. Rebuild with SVE enabled\nwhen wider vectors are available.\n" + features: + - "SVE" + - "NEON" + args: + HF_MODEL: + description: "Hugging Face model repo ID containing a supported single-file GGUF model" + default: "bartowski/Qwen_Qwen3.5-0.8B-GGUF" + example: "unsloth/SmolLM2-135M-Instruct-GGUF" + HF_MODEL_FILE: + description: "Exact supported GGUF filename to download; sharded and mmproj files are rejected" + default: "" + example: "Qwen_Qwen3.5-0.8B-Q4_0.gguf" + ENABLE_SVE: + description: "Enables building with SVE instructions (OFF/ON)" + default: "OFF" + example: "ON" +``` + +Changes can be made to the `compose.yaml` files to adjust arguments after the fact - for example, SVE can be turned ON or OFF for the LLM chatbot, and the LLM can be changed to use a different model. + +## Deploy the app on the target + +On your host device, enter the project directory created by the `topo clone` command. In the case of the LLM chatbot, this directory is `topo-cpu-ai-chat`: + +```bash +cd topo-cpu-ai-chat/ +``` + +Then use `topo deploy` to automatically build the container images on the host, transfer the images to the target via SSH, and start the application on the target: + +``` +topo deploy --target user@my-target +``` + +Once deployed, you can view the webpage at `http://:3000`. The port used depends on the template application chosen - you can see this in the `compose.yaml`. + +The LLM chatbot application will appear as below: + +![LLM Chatbot](llm_chatbot.png) + +Depending on the permissions you have setup with your target device, you may not be able to use the IP address directly. In this case, you may need to forward to a local port and view at `http://localhost:` instead: + +```bash +ssh -L :localhost: user@my-target +``` + +To stop a deployed Topo application on the target, run `topo stop` on the host: + +```bash +topo stop --target user@my-target +``` + +## (Optional) Repeat this learning path, using a CLI Agent + +Topo is packaged as a single executable with a `README.md` file. It is straightforward for agents to leverage. + +Choose your preferred agent. If you do not have a CLI Agent pre-configured, you can follow our guides to [Install Codex](https://learn.arm.com/install-guides/codex-cli/), [Install Claude Code](https://learn.arm.com/install-guides/claude-code/), or [Install Gemini](https://learn.arm.com/install-guides/gemini/). + +Once your agent is ready, ensure you have setup your host and target to have the required dependencies. Then ask your agent to leverage Topo to deploy an application to your target device. diff --git a/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/llm_chatbot.png b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/llm_chatbot.png new file mode 100644 index 0000000000..5c4461f49f Binary files /dev/null and b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/llm_chatbot.png differ diff --git a/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/overview-and-setup.md b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/overview-and-setup.md new file mode 100644 index 0000000000..2a4a1f076b --- /dev/null +++ b/content/learning-paths/cross-platform/deploy-containerized-workloads-with-topo/overview-and-setup.md @@ -0,0 +1,55 @@ +--- +title: Overview - What is the Topo tool? +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## What is Topo? + +[Topo](https://github.com/arm/topo) is an open-source command-line tool, developed by Arm. Topo connects from a host device (e.g., your development laptop) to an Arm-based Linux target (e.g., a Raspberry Pi) over SSH, detects hardware capabilities, and allows the user to deploy compatible containerized workloads with ease. Topo builds container images on the host, transfers them to the target, and starts the services on the target. Building and deploying on the target is also possible. Topo also showcases several sample templates defining workloads you can deploy, including a "Hello World" webpage, an LLM Chatbot, and a SIMD Benchmark comparing Scalar, Neon, and the Scalable Vector Extension (SVE). + +[Topo templates](https://github.com/arm/topo-template-format) are based on the [Compose Specification](https://github.com/compose-spec/compose-spec), extended with `x-topo` metadata that describes requirements such as CPU features and build arguments. The Compose Specification is a standard, YAML-based format for describing how to run multi-container applications. Instead of manually starting containers one-by-one, you define everything in a single file that includes which services to run, what images to use, how they connect, and what configuration they need. + +In this learning path, you can use a target device of your choice, whether that be Raspberry Pi, AWS Graviton instance, DGX Spark, NXP i.MX 93, or similar. The only stipulation is that the device must be Arm-based, running Linux, and accessible over SSH. Your host can also function as your target simultaneously, but only if your host is an Arm-based Linux device. + +The optional heterogeneous deployment with `remoteproc-runtime` requires a heterogeneous Cortex-A + Cortex-M SoC, such as the i.MX 93. + +## Why use Topo? + +Topo streamlines deploying containerized workloads to Arm-based Linux devices. +If you have an heterogeneous device (e.g. Cortex-A + Cortex-M SoC such as the i.MX 93), Topo enables you to use it fully, deploying both firmware and application as containerized workloads through standard container tooling. + +Once a template is defined, use of Topo removes the need to deal with low-level setup and compatibility issues manually. Topo first assesses the system to identify processor features such as SVE or Neon on CPU, then advises which templates are appropriate for the device. Finally, it automates the end to end deployment, dealing with all the low-level instructions. + +Topo can also be leveraged by AI Agents to further streamline and automate. + +## Install Topo + +In this step, you install Topo on the host, along with other dependencies. + +On your host device, run the following commands to install Topo: + +{{< tabpane code=true >}} + {{< tab header="Linux / MacOS">}} +curl -fsSL https://raw.githubusercontent.com/arm/topo/refs/heads/main/scripts/install.sh | sh + {{< /tab >}} + {{< tab header="Windows" >}} +irm https://raw.githubusercontent.com/arm/topo/refs/heads/main/scripts/install.ps1 | iex + {{< /tab >}} +{{< /tabpane >}} + +Alternatively, find the [latest release of Topo](https://github.com/arm/topo/releases), download the binary for your specific platform (x86/Arm, Linux/MacOS/Windows), and extract it. Topo is provided as a single executable file and `README.md`. Place Topo on your `PATH`: + +Run the following command in the terminal on your host device to confirm the installation: + +```bash +topo --help +``` + +You should see an output confirming Topo is present, and describing the available commands. + +## What you've completed and what's next + +You have now installed Topo on your host. In the next step, you will utilize Topo to prepare your host machine and to probe your target device for features, assessing its compability with template containerized workloads.