Mixed hardware scenario

.mise/config.rust.toml

@@ -105,8 +105,8 @@ shell = "{{ vars.task_shell }}"
 [tasks.build-no-std]
 description = "Build the no_std crates for a bare-metal target"
 run = """
-features=et-ws-service/picoserve-hub,et-ws-server/picoserve-server,et-ws-wasi-runner/board
-set -- -p et-org -p edge-toolkit -p et-ws-service -p et-ws-server -p et-ws-wasi-runner
+features=et-modules-service/board,et-storage-service/board,et-ws-service/board,et-ws-wasi-runner/board
+set -- -p et-org -p edge-toolkit -p et-modules-service -p et-storage-service -p et-ws-service -p et-ws-wasi-runner
 cargo build --lib --target thumbv8m.main-none-eabihf --no-default-features --features "$features" "$@"
 """
 shell = "{{ vars.task_shell }}"
@@ -115,8 +115,8 @@ shell = "{{ vars.task_shell }}"
 [tasks.test-no-std]
 description = "Test the no_std crates built without std"
 run = """
-features=et-ws-service/picoserve-hub,et-ws-server/picoserve-server
-set -- -p edge-toolkit -p et-ws-service -p et-ws-server
+features=et-modules-service/board,et-storage-service/board,et-ws-service/board
+set -- -p edge-toolkit -p et-modules-service -p et-storage-service -p et-ws-service
 cargo nextest run --config-file config/nextest.toml --no-default-features --features "$features" "$@"
 """
 shell = "{{ vars.task_shell }}"

.mise/config.toml

@@ -1343,7 +1343,7 @@ description = "Run conftest OPA/Rego policies over the TOML config + lock files"
 # combined TOML input.
 run = """
 ns_list="cross cargo mise mise_lock pyproject checksums advisories"
-named="Cargo.lock config/upstream-cache/data.toml config/deny.toml config/pyrefly.toml"
+named="Cargo.lock config/cargo-features.toml config/upstream-cache/data.toml config/deny.toml config/pyrefly.toml"
 # Both lists rely on word-splitting when they expand.
 # $ns expands to repeated `--namespace <n>` pairs; $named is a space-separated path list.
 # shellcheck disable=SC2086

CHANGELOG.md

@@ -19,24 +19,31 @@ Changelog 1.1.0.
   frames through the registry and hands back what to send where, leaving the transport, ids and clock to the
   embedder. Its features `std` (the actix-web hub), `nats` (the backplane) and `otel` (spans and metrics) are
   separate. Without `nats`, `WsConfig` has no `backplane`.
-- `et-ws-service`'s `picoserve-hub` feature, on by default, adds `picoserve_hub::Hub`: a `no_std` hub serving
+- `et-ws-service`'s `board` feature, on by default, adds `picoserve_hub::Hub`: a `no_std` hub serving
   `GET /ws` over HTTP and WebSocket through picoserve on any socket the embedder accepts, such as an `embassy-net`
   `TcpSocket`. It is a local hub, with no backplane, admin API, idle timeout or registry file. A crate awaiting
   `Hub::serve` needs `#![recursion_limit = "256"]`.
 - `picoserve_hub::Hub::local` joins an agent the embedder runs itself to the hub, with no socket between them: a
   `LocalLink` routes what it sends as a connection's frames are, queues what others send it, and disconnects the agent
   when dropped. It is how a board's own runner reaches the hub beside it. Its `send_text` and `send_binary` refuse a
   frame larger than the hub reads from a socket with `FrameTooLarge`.
-- `et-ws-server` builds as `no_std` + `alloc` without its new `std` feature. Its `picoserve-server` feature, on by
-  default, adds `picoserve_server::serve`: the hub's `/ws` with `/health` and `/favicon.ico` beside it, over picoserve,
-  and the modules a `picoserve_modules::Modules` builds into the binary under `/modules` and `/`, as the `std` server
-  serves them from disk. A module listed as served elsewhere is redirected to a base URL instead.
-- The `et-ws-server-firmware` firmware runs et-ws-server on a Raspberry Pi Pico 2 W, an ESP32-C6 or a Pimoroni Pico
+- `et-modules-service` and `et-storage-service` build as `no_std` + `alloc` without their new default `std` feature,
+  and each serves through picoserve under its new `board` feature. `et_modules_service::picoserve::ModuleRoutes`
+  serves the modules a `Modules` builds into the binary under `/modules` and `/`, as the `std` service serves them
+  from disk, redirecting a module listed as served elsewhere to a base URL. `et_storage_service::picoserve`'s
+  `StorageRoutes`, mounted at its `path()`, serves agent storage at `/storage/{agent_id}/{filename}`, held to the `std`
+  service's ownership rule against an `AgentRegistry` and kept in the embedder's `Storage`, each file at most
+  `MAX_STORED` bytes. Both are used with `default-features = false` from the workspace, so a consumer of the `std`
+  service now names the `std` feature.
+- The `et-ws-server-firmware` firmware runs the hub on a Raspberry Pi Pico 2 W, an ESP32-C6 or a Pimoroni Pico
   Plus 2 W, over the board's Wi-Fi. `MISE_ENV=rust mise run build-firmware-pico2w` fetches the Pico's radio blobs and
   builds it for the Pico, `build-firmware-esp32c6` for the ESP32-C6, and `build-firmware-pico-plus2w` for the Pico Plus
   2 W, each for the network `WIFI_NETWORK` and `WIFI_PASSWORD` name, embedding the modules `MODULES_PATHS` names and
   serving `MODULES_ROOT` at `/`. The Pico Plus 2 W makes its 8 MiB of PSRAM the heap, and also runs each wasi runner
   `RUNNER_GUESTS` names beside the hub, its guest compiled into the firmware and its websocket a link into the hub.
+  Every board serves agent storage from memory, and the guests' storage buckets are that same storage, so an agent on
+  another machine reads what a guest wrote. Built with `HUB_ADDRESS` naming a hub elsewhere, `address:port`, the
+  firmware serves nothing and runs its wasi runners against that hub instead, over the board's own sockets.
 - `et-cli generate-deployment --output-type firmware-pico2` writes a `mise.firmware-pico2.toml`, `firmware-esp32c6` a
   `mise.firmware-esp32c6.toml`, and `firmware-pico-plus2` a `mise.firmware-pico-plus2.toml`, whose `firmware-flash`
   task builds that firmware with the scenario's modules in it and loads it onto a board, with picotool or espflash.
@@ -46,14 +53,19 @@ Changelog 1.1.0.
   published artifacts is refused on every board. Its `firmware-host-test` task serves the same modules from the host,
   and runs the same runners, as the firmware's host build does, and fetches every module it lists, with no board
   needed.
+- A scenario's machine names the hardware it is with `device:`. A board (`pico2`, `esp32c6` or `pico-plus2`) as the
+  first machine runs the hub as that board's firmware, serving every module of the scenario, with the runners its
+  firmware hosts beside it, while each other machine runs its share as the format's tasks and reaches the board's hub
+  at its `address`. A board on any other machine runs only the runners placed on it, built as firmware that serves
+  nothing and reaches the server's hub at `HUB_ADDRESS`. The `mise.toml` gains `firmware-build-<machine>` and
+  `firmware-flash-<machine>` tasks for each board, each built in a target directory of its own, a
+  `firmware-host-test-<machine>` for a board running the hub, and no collector when the server is a board. Only
+  `mise`, `dekit` and `process-compose` deliver such a scenario.
 - `et-firmware-modules` generates the source a firmware's build script writes to build ws-modules into its image,
   from the same `MODULES_*` variables, so every board's firmware embeds modules the same way. Its `board` writes what
   the board a target picks bakes in -- the Wi-Fi network, and the Pico 2 W's radio blobs and memory layout -- and
   returns the directives that link it. Its `guests` compiles each runner's guest for the board, refusing one that
   imports what a board does not host, or guests needing more memory than the board holds for them.
-- `et-ws-wasi-runner` builds as `no_std` + `alloc` without its new default `std` feature, and its `board` module
-  runs a guest on a microcontroller: compiled ahead of time with `engine::precompile` for the board's Pulley target,
-  and hosted through the firmware's `board::Platform` and `board::Transport`.
 - Native REST requests retry transient HTTP 408/429/500/502/503/504 responses as well as transport failures, using a
   bounded backoff policy for replayable idempotent operations. Permission errors, non-idempotent methods and streaming
   request bodies are not retried. Storage-call timings include the retry/backoff interval; retry attempts are logged.
@@ -65,7 +77,10 @@ Changelog 1.1.0.
 - `et-ws-wasi-runner` builds as `no_std` + `alloc` without its new default `std` feature, and with its new `board`
   feature its `board` module runs a guest on a microcontroller: compiled ahead of time with `engine::precompile` (the
   `precompile` feature) for the board's Pulley target, and hosted through the firmware's `board::Platform` and
-  `board::Transport`. Every guest on a board shares its storage in `board::Buckets`, and draws its memories and
+  `board::Transport`. The transport is the hub as the guest reaches it, its storage included: a board running the hub
+  answers from `board::Buckets`, and `board::net::NetTransport` reaches a hub on another machine over the board's own
+  sockets, its websocket framed by `edge-ws` and its storage over HTTP, while a task of the board's runs
+  `board::net::drive`; `board::net_tokio` gives it tokio's sockets on a host. Every guest draws its memories and
   tables from one `board::MemoryBudget`, which refuses growth past what the board holds for them.
 - An opt-in `et-recovery-supervisor` service observes application progress and coordinates bounded Kubernetes recovery.
 - The Python SDK provides recovery lifecycle messages and integrity-checked checkpoint storage helpers.
@@ -122,9 +137,8 @@ Changelog 1.1.0.
 
 ### Changed
 
-- **Breaking:** `et-ws-server`'s actix server, its binary and its environment config are behind the new default `std`
-  feature. A build with `default-features = false` and none of the features that imply `std` -- `modules`, `otlp`,
-  `qr`, `storage`, `tls`, `websockify` -- now gets the `no_std` library and no binary; add `std` to keep the server.
+- **Breaking:** `et_ws_server::routes::HealthResponse`, the answer to `GET /health`, is now
+  `et_ws_service::health::HealthResponse`, since a board's hub gives the same answer; import it from there.
 - **Breaking:** `et-ws-wasi-runner`'s host runner, its binary and `run_module` are behind the new default `std`
   feature, which `nn`, `webgpu` and `coverage` imply. A build with `default-features = false` and none of them now
   gets the `no_std` board runner and no binary; add `std` to keep the host runner. `server_message_to_wit` moved from

CHECKS.md

@@ -1,11 +1,11 @@
 # Checks
 
-- **Generated:** 2026-11-02
+- **Generated:** 2026-11-03
 - **Checks:** 82
 - **Tools:** 48
 - **Shell-script checks:** 10
 - **Checks with a repair pass:** 22
-- **Rules:** 361 custom local rules, or non-default strict linter settings
+- **Rules:** 363 custom local rules, or non-default strict linter settings
 - **Meta rules:** 20 of those rules are rules about rules
 
 Every check is listed across every `MISE_ENV`, filed under the tool it runs. Each entry names the env whose config
@@ -566,8 +566,8 @@ Rules, from `Cargo.toml`:
 - `conftest-check-policy-tests` (`default` env; run by `conftest-check`) -- Run the Rego unit tests beside the conftest
   policies (conftest verify). Config: `config/conftest/policy`.
 - `conftest-check-toml` (`default` env; run by `conftest-check`) -- Run conftest OPA/Rego policies over the TOML
-  config + lock files. Config: `config/conftest/policy`, `config/deny.toml`, `config/pyrefly.toml`,
-  `config/upstream-cache/data.toml`.
+  config + lock files. Config: `config/cargo-features.toml`, `config/conftest/policy`, `config/deny.toml`,
+  `config/pyrefly.toml`, `config/upstream-cache/data.toml`.
 - `conftest-check-yaml` (`default` env; run by `conftest-check`) -- Run conftest OPA/Rego policies over the GitHub
   Actions YAML, and the backslash ban over all YAML. Config: `config/conftest/policy`.
 
@@ -602,6 +602,8 @@ Rules, from `config/conftest/policy`:
   - crates.io rejects an upload whose manifest carries no description, so every publishable crate has one.
   - An empty `features = []` on a dependency is pointless noise -- drop it.
   - A member crate's feature must not share its name with one of its dependencies.
+  - A member crate's feature must be one config/cargo-features.toml lists.
+  - config/cargo-features.toml lists no feature that no crate declares any longer.
   - No manifest may depend on a `cargo_*` / `cargo-*` crate (`cargo_metadata`, `cargo_toml`, ...).
   - Crates log through `tracing`, so none may depend on the `log` facade directly.
   - tokio's `full` feature is banned on every dependency spec, the workspace table included.

Cargo.toml

@@ -90,6 +90,7 @@ cloudevents-sdk = "0.9"
 command-error = "0.8"
 const-hex = "1.19"
 cortex-m-rt = "0.7"
+critical-section = "1"
 croner = "4"
 cyw43 = "0.7"
 cyw43-pio = "0.10"
@@ -105,11 +106,14 @@ deno_resolver = "0.91"
 deno_runtime = { version = "0.268", features = ["transpile", "hmr"] }
 document-features = "0.2"
 edge-toolkit = { path = "libs/edge-toolkit", version = "0.4.0", default-features = false }
+edge-ws = { version = "0.8", default-features = false, features = ["io"] }
 embassy-executor = "0.10"
 embassy-net = "0.9"
 embassy-rp = "0.10"
+embassy-sync = "0.8"
 embassy-time = "0.5"
 embedded-alloc = { version = "0.7", default-features = false, features = ["llff"] }
+embedded-io-async = "0.7"
 esp-alloc = "0.11"
 esp-backtrace = "0.20"
 esp-bootloader-esp-idf = "0.6"
@@ -122,7 +126,7 @@ et-actix-web-config = { path = "libs/actix-web-config", version = "0.1.0" }
 et-cli = { path = "utilities/cli", version = "0.2.0" }
 et-firmware-modules = { path = "services/firmware-modules", version = "0.1.0" }
 et-hub-backplane = { path = "services/hub-backplane", version = "0.2.0" }
-et-modules-service = { path = "services/modules", version = "0.3.0" }
+et-modules-service = { path = "services/modules", version = "0.3.0", default-features = false }
 et-onnx = { path = "utilities/onnx", version = "0.2.0" }
 et-org = { path = "libs/org", version = "0.2.0" }
 et-otlp = { path = "libs/et-otlp", version = "0.3.0" }

README.md

@@ -443,7 +443,9 @@ MISE_ENV=firmware-pico2 mise run firmware-flash
 A board has no filesystem, so each module from this repository, or from the scenario's `module_paths:`, is built
 into the firmware from its built package, and the modules have to be built first. The npm packages they use, such as
 onnxruntime-web, are bigger than a board's flash: those are redirected to jsDelivr instead. A scenario with more than
-one machine or published artifacts is refused.
+one machine or published artifacts is refused; one spanning a board and other machines names the board as a machine's
+`device:` instead. The board serves agent storage from memory, files of up to 64 KiB, and its own runners' storage is
+that same memory, so an agent elsewhere reads what a runner on the board stored. It is lost when the board restarts.
 
 The board's firmware decides which runners it can run. Only `firmware-pico-plus2`, for the Pimoroni Pico Plus 2 W,
 runs any: its 8 MiB of PSRAM holds the memory a guest needs, so it runs each `wasi` runner beside the hub, its guest
@@ -467,6 +469,30 @@ machines:
     agents: ["math1-twin"]
 ```
 
+### Machines on different hardware
+
+A machine's `device:` names the hardware it is, and that hardware decides what the machine may run. Left out, the
+machine runs its share as the format's own tasks, with mise and this repository's toolchain on it. Naming a board
+(`pico2`, `esp32c6` or `pico-plus2`) makes the machine that board's firmware, which runs only the runners its firmware
+hosts. As the first machine, the server, the board also runs the hub, serving every module of the scenario so the
+runners elsewhere can fetch theirs, and its agents' storage from memory. On any other machine it serves nothing, and
+its runners reach the server's hub at the server's `address`, so a board there needs a runner placed on it. Only
+`mise`, `dekit` and `process-compose` deliver a scenario naming a board, since the others ship container images:
+
+```yaml
+machines:
+  - name: "hub-board"
+    address: "et-hub-board.local"
+    device: "pico2"
+  - name: "runner-board"
+    address: "et-runner-board.local"
+    device: "pico-plus2"
+    agents: ["math1-trigger"]
+  - name: "edge"
+    address: "10.0.0.2"
+    agents: ["pyo3-math1-twin"]
+```
+
 ### Several hubs
 
 `services.hub.machines:` runs a hub on each machine it lists, the server among them. The hubs join into one through