Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .mise/config.coverage.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ version = "0.9.1"
# only be asserted where that path runs. A crate belongs to exactly one list: naming it in both would let a
# gap in one pipeline be answered by the other.
native_cov_libs = """
{{ "actix-web-config core-guest edge-toolkit et-otlp math1 path sanitise test-helpers " ~
{{ "actix-web-config core-guest edge-toolkit et-otlp gpu-probe lidar math1 path sanitise test-helpers " ~
"test-nats test-otlp test-recovery ws-runner-common" -}}
"""
wasm_cov_libs = "wasi-guest web"
Expand Down
6 changes: 3 additions & 3 deletions .mise/config.maint.toml
Original file line number Diff line number Diff line change
Expand Up @@ -848,9 +848,9 @@ crates=(
et-otlp et-web et-rest-client et-ws-runner-common et-ws-wasm-agent
et-wasi-guest et-ws-comm1 et-ws-data1 et-ws-math1 et-ws-math1-sender
et-ws-wasi-comm1 et-ws-wasi-data1 et-ws-wasi-math1 et-ws-wasi-math1-sender et-modules-service
et-storage-service et-websockify-service et-ws-service et-ws-test-server et-ws-pyo3-runner
et-actix-web-config et-ws-server et-ws-wasi-runner et-ws-web-runner et-cli
et-onnx
et-storage-service et-websockify-service et-ws-service et-lidar et-ws-test-server
et-ws-pyo3-runner et-actix-web-config et-ws-server et-ws-wasi-runner et-ws-web-runner
et-cli et-onnx et-ws-wasi-lidar1
)
per_batch=5
# `set --` then `$#` counts the array without the `${` + `#` pair, which mise's Tera pass reads as a comment.
Expand Down
5 changes: 5 additions & 0 deletions .mise/config.python.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@
"pipx:openapi-python-client" = "0.29.1"
"pipx:pyrefly" = "latest"
"pipx:pytest" = "latest"
# The libraries the pyo3 modules reach a robot's devices through.
# pyserial for serial ports, OpenCV for cameras and sounddevice for microphones, which the runner's embedded
# interpreter imports from mise's site-packages. OpenCV and sounddevice have no command, which a pipx tool needs, so
# they install beside pyserial's.
"pipx:pyserial" = { version = "latest", uvx_args = "--with opencv-python-headless --with sounddevice" }
# PyTorch, for the et-ws-pyo3-runner `torch_inference` test.
# Python-only (not in the always-loaded config) -- it's a large optional dependency, and the test skips itself
# when torch isn't among the mise-installed packages on sys.path. Skipped on macos/x64: PyTorch dropped macOS
Expand Down
14 changes: 13 additions & 1 deletion .mise/config.rust.toml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ run = """
target=thumbv8m.main-none-eabihf
features=et-modules-service/board,et-storage-service/board,et-ws-service/board
set -- -p et-org -p edge-toolkit -p et-modules-service -p et-storage-service -p et-ws-service
set -- "$@" -p et-core-guest -p et-math1
set -- "$@" -p et-core-guest -p et-lidar -p et-math1
cargo build --lib --target "$target" --no-default-features --features "$features" "$@"
cargo build --lib --target "$target" --no-default-features --features board,core-modules -p et-ws-wasi-runner
"""
Expand Down Expand Up @@ -250,6 +250,18 @@ coreutils cp "target/wasm32-wasip2/release/$wasm" "modules/wasi/wasi-math1/pkg/$
"""
shell = "{{ vars.task_shell }}"

[tasks.build-ws-wasi-lidar1-module]
depends = ["build-et-cli"]
description = "Build the Rust WASI lidar1 module as a WASI Preview 2 component"
run = """
{{ vars.wasm_cov }} cargo build --release -p et-ws-wasi-lidar1 --target wasm32-wasip2{{ vars.wasi_cov_feat }}
coreutils mkdir -p modules/wasi/wasi-lidar1/pkg
wasm=et_ws_wasi_lidar1.wasm
coreutils cp "target/wasm32-wasip2/release/$wasm" "modules/wasi/wasi-lidar1/pkg/$wasm"
{{ vars.et_cli }} module-package-json --module-dir modules/wasi/wasi-lidar1
"""
shell = "{{ vars.task_shell }}"

[tasks.build-ws-core-math1-module]
depends = ["build-et-cli"]
description = "Build the Rust core math1 module as a Wasm 1.0 core module, for SpaceWasm"
Expand Down
2 changes: 1 addition & 1 deletion .mise/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2855,7 +2855,7 @@ shell = "{{ vars.task_shell }}"

[tasks."gen:wasmtime-bindings"]
depends = ["gen:ws-spec"]
description = "Run et-int-gen: emit the ws-wasi-runner host bindings for the `runner` world"
description = "Run et-int-gen: emit the ws-wasi-runner host bindings for the `runner` and `serial-host` worlds"
# Output lands in the consuming crate, not generated/.
# `cargo package` only includes files under a crate's own directory, so bindings that live anywhere else
# leave et-ws-wasi-runner unable to build from its own tarball -- which is what retired the
Expand Down
8 changes: 8 additions & 0 deletions .mise/mise.python.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 24 additions & 13 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -548,9 +548,10 @@ Languages:
- **Kotlin -> WASM (WasmGC)**: kotlin-data1, kotlin-math1 -- compiled by the Kotlin Gradle plugin's `wasmJs` target;
each module is a WasmGC binary (browser GC manages the Kotlin heap), so it needs a WasmGC-capable engine
- **Python (Pyodide)**: pydata1, pyeye1, pyface1, pymath1
- **Python (native CPython)**: pyo3-math1 -- runs in `et-ws-pyo3-runner` rather than the browser, so it is plain
`.py` source committed as-is rather than a wheel or a wasm binary. The runner fetches the single file its
`main` field names and compiles it in process; the same file also runs from `PYO3_PYTHONPATH` unchanged.
- **Python (native CPython)**: pyo3-imu1, pyo3-lidar1, pyo3-math1, pyo3-media1 -- run in `et-ws-pyo3-runner` rather
than the browser, so each is plain `.py` source committed as-is rather than a wheel or a wasm binary. The runner
fetches the single file its `main` field names and compiles it in process; the same file also runs from
`PYO3_PYTHONPATH` unchanged.
- **C# (.NET WASM)**: dotnet-data1, dotnet-math1
- **Java (TeaVM -> JS)**: java-data1, java-math1 -- both built by the single root `pom.xml` (one compilation, one
teavm-maven-plugin execution per module)
Expand Down Expand Up @@ -1545,23 +1546,33 @@ new deps there, not in individual crate `[dependencies]`.
Before adding, changing or dropping an entry in `config/deny.toml`'s `[advisories] ignore` list, read the comments
above that list and follow them -- they say which date an entry takes and when one may go.

## Avoid GPL and LGPL dependencies
## Avoid GPL dependencies, and link an LGPL C or C++ library only dynamically

Every license `config/deny.toml`'s `[licenses] allow` list currently accepts is permissive (Apache-2.0, MIT,
BSD, Zlib, ISC, MPL-2.0, etc.) -- none copyleft. Keep it that way: do not add a new direct or transitive
dependency whose license is GPL (any version) or LGPL (any version), and do not add GPL/LGPL to the
`config/deny.toml` allow list, even when the license is individually OSI-approved (LGPL is). Being OSI-approved is
not sufficient on its own here -- copyleft obligations are a different category of concern from a permissive
license, and accepting one is a project-policy decision, not a mechanical license check.
Every license `config/deny.toml`'s `[licenses] allow` list currently accepts is permissive (Apache-2.0, MIT, BSD,
Zlib, ISC, MPL-2.0, etc.) -- none copyleft. Keep it that way: do not add a new direct or transitive dependency whose
license is GPL (any version), and do not add GPL or LGPL to the `config/deny.toml` allow list, even when the license
is individually OSI-approved. Being OSI-approved is not sufficient on its own here -- copyleft obligations are a
different category of concern from a permissive license, and accepting one is a project-policy decision, not a
mechanical license check.

The one exception is a C or C++ library under the LGPL (any version), such as libusb or libudev, which a `-sys` crate
may link **dynamically**: loaded at run time from the system's shared library, so whoever runs our binary can swap in
their own build of it, which is what the LGPL asks. It must never be compiled into one of our binaries. Keep off any
`-sys` feature that builds the library from bundled source (`vendored`, `static`, `bundled`), and read the build
script for a fallback that does the same when the system library is missing -- `libusb1-sys` compiles its bundled
libusb whenever `pkg-config` cannot find one. No license check sees any of this, because the `-sys` crate carries
its own permissive license (`libusb1-sys` is MIT), so it is settled in review. A Rust crate under the LGPL is not
covered: cargo links Rust dependencies statically.

Concrete precedent: `viuer` (added for the storage service's tty image-preview feature) pulled in
`ansi_colours` under LGPL-3.0-or-later as an unconditional dependency for its RGB-to-256-color quantization
path. Rather than allow-listing LGPL, the feature was reimplemented as a small hand-rolled ANSI truecolor
renderer in `services/storage/src/tty_image.rs`, built on the `image` crate the workspace already depended on
via `deno_image`/`ws-web-runner` -- truecolor output needs no palette quantization at all, so there was
nothing left for the LGPL dependency to do. If a future dependency's only viable path involves a GPL/LGPL
component, treat that the same way: look for a permissively-licensed alternative or a narrower reimplementation
first, and only bring the question to the user if neither is workable -- don't add the exception yourself.
nothing left for the LGPL dependency to do. If a future dependency's only viable path involves a GPL component,
or an LGPL one that would be linked statically, treat that the same way: look for a permissively-licensed
alternative or a narrower reimplementation first, and only bring the question to the user if neither is workable --
don't add the exception yourself.

## Clippy lints

Expand Down
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,30 @@ Changelog 1.1.0.

### Added

- The `graphics-info` module tells GPUs apart by their arithmetic. It puts sixteen expressions to the WebGPU device,
each as a compute shader of its own so that a device refusing one still answers the rest, and its `info_detected`
event gains a `gpu_numeric_probe` object: how many probes were answered, how many were not correctly rounded, which
of two legal answers the device gave where there are two, a fingerprint of every result, and the known GPU that
answers the same way. The probe set, its shaders and correctly rounded answers are the new `et-gpu-probe` crate,
which drives no GPU itself. The module's `gpu.renderer` now falls back to WebGL's renderer string where a browser
blanks the WebGPU adapter's.
- A WASI module can read a lidar. `et-ws-wasi-runner`'s new `lidar` feature, on by default, hosts
`et:ws-wasi/serial`, lending a guest the serial ports its `SERIAL_PORTS_<NAME>` variables name: a device, or
`tcp://<host>:<port>` for one a bridge such as ser2net serves. The interface's own `serial-host` world, which the
`module` world now includes, keeps it out of `runner`. A guest that imports it fails to instantiate on a board or on
a runner built without the feature. The new `wasi-lidar1` module reads the COIN-D6 lidar on Hiwonder's MentorPi
robot through the port lent as `lidar`, storing each revolution as `lidar-scan.json` in its agent's bucket. The new
`et-lidar` crate decodes the COIN-D6's serial stream, `no_std` for a board, and encodes its packets for a simulated
lidar such as the one `et_ws_test_server::lidar` serves.
- Three pyo3 modules read Hiwonder's MentorPi robot, each from a worker thread its `init` starts, storing the latest
of what it reads in its agent's bucket. `pyo3-lidar1` stores the same `lidar-scan.json` as `wasi-lidar1`;
`pyo3-imu1` reads the IMU on the robot's controller board, from the port `SERIAL_PORTS_CONTROLLER` names, as
`imu-reading.json`; and `pyo3-media1` captures the camera that `MEDIA_CAMERA` names as `camera-frame.jpg` and the
microphone that `MEDIA_MICROPHONE` names as `microphone-clip.wav`. The `python` mise env installs what they import
as `pipx:pyserial`, with OpenCV and sounddevice beside it.
- `edge-toolkit`'s new `edge_toolkit.serial` opens the port a `SERIAL_PORTS_<NAME>` variable names, a device through
pyserial (the new `serial` extra) or a TCP bridge. A pyo3 module imports it from this repository's source by naming
`libs/python-edge-toolkit` on `PYO3_PYTHONPATH`.
- `et-ws-wasi-runner` runs Wasm 1.0 core modules beside WASI components, on NASA's SpaceWasm interpreter, with its
new `core-modules` feature, on by default. It picks the engine from the fetched module's header, so a scenario's
`runner: wasi` covers both. A core module reaches the hub through the `et` imports, which the new `et-core-guest`
Expand Down Expand Up @@ -152,6 +176,9 @@ Changelog 1.1.0.

### Changed

- **Breaking:** `et_ws_wasi_runner::run_module` takes the serial ports it lends a component as a fifth argument, a
`BTreeMap` from each port's name to its device. Pass an empty map to lend none, as a build without the `lidar`
feature does whatever it is given.
- **Breaking:** The modules moved from `services/ws-modules/` to `modules/<type>/`, the type being the runtime that
loads them: `web`, `wasi`, `core` or `pyo3`, as `edge_toolkit::config::MODULE_TYPE_DIRS` lists them.
`default_modules_folders` returns those four in place of `services/ws-modules`, and a generated deployment stages
Expand Down
6 changes: 5 additions & 1 deletion CHECKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- **Tools:** 49
- **Shell-script checks:** 10
- **Checks with a repair pass:** 22
- **Rules:** 379 custom local rules, or non-default strict linter settings
- **Rules:** 381 custom local rules, or non-default strict linter settings
- **Meta rules:** 22 of those rules are rules about rules

Every check is listed across every `MISE_ENV`, filed under the tool it runs. Each entry names the env whose config
Expand Down Expand Up @@ -760,6 +760,7 @@ Rules, from `config/conftest/policy`:
- `pyproject` (`config/conftest/policy/pyproject/pyproject.rego`):
- A `path = ".."` uv source points at the parent directory, which isn't a package -- almost always a mistake.
- A pyproject's uv_build requirement must pin the uv version mise installs.
- A pyproject may not link this repository's GitHub issues in [project.urls], under `Issues` or any other label.
- `tera_comment` (`config/conftest/policy/tera_comment/tera_comment.rego`):
- A mise config value must not contain `{#`, which Tera reads as the start of a comment.

Expand Down Expand Up @@ -1199,6 +1200,9 @@ Rules, from `config/semgrep`:
path is relative to wherever a command happens to run rather than to anything the deployment fixes. Fetch what the
deployment needs from its releases, and let each tool's own default stand for "this directory", so the deployment
builds and runs from its directory alone.
- `pyo3-assigned-hooks-in-all` -- A runner hook bound by assignment rather than `def` is a global the module itself
never reads, so CodeQL reports it as unused. et-ws-pyo3-runner reads it by name, so export it: list every hook the
module binds this way in an `__all__` placed after the module docstring and any `from __future__` import.
- `python-silent-broad-except` -- Silently discarding every ordinary exception can conceal programming errors and lost
work. Catch the expected exception types and explicitly report, recover from, or propagate the failure. Check-only: a
safe recovery policy cannot be inferred from the handler alone.
Expand Down
84 changes: 83 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading