From 8ef8253fbaf5597f66490042f09ea730b0c62f95 Mon Sep 17 00:00:00 2001 From: wudgexyz Date: Tue, 13 Oct 2026 23:39:52 +0300 Subject: [PATCH 1/2] Rename to cerberus_*, ship static per-arch binaries, install from releases The agent's U-Boot keys become cerberus_* and its variables CERBERUS_*. Cameras installed under the old names keep working with nothing to do: on start the agent copies its own keys (device key, claim code, join material) from streq_* to cerberus_*, checks each copy, and leaves the old ones in place so an older init script or a rolled-back binary still finds a working camera. S99cerberus reads cerberus_ and falls back to streq_; STREQ_* variables are still accepted. Forgetting an enrolment, `clean` and uninstall.sh remove both names, so an old copy can never bring a revoked node back. Once nothing is left to copy, a start writes nothing to flash. Releases now carry one static musl binary per CPU architecture - arm (ARMv5TE soft-float, so every ARM camera from ARMv5 to ARMv7), mipsel (Ingenic), aarch64, x86_64 - built with Bootlin toolchains against mbedTLS 3.6.7, both pinned by sha256. Each binary is run under QEMU and its X25519 key checked against `wg pubkey`. A v* tag publishes them with install.sh, uninstall.sh, S99cerberus and SHA256SUMS; names carry no version, so the installer can fetch releases/latest/download/. install.sh (formerly bootstrap.sh) picks the build by CPU instead of by chip name, verifies it against SHA256SUMS, and now waits for the enrolment itself before printing the claim code: the code exists before the camera ever reaches the cloud, so E21 never fired when it did not. uninstall.sh only rewrites keys that exist instead of 21 blind flash writes. The suite covers the migration against a fake U-Boot environment, the installer end to end from a release laid out on disk, and E21. --- .github/workflows/ci.yml | 58 +++--- .gitignore | 1 + S99cerberus | 50 +++--- bootstrap.sh | 80 --------- ci/assemble.sh | 24 +++ ci/build.sh | 266 ++++++++++----------------- install.sh | 105 +++++++++++ main.c | 376 +++++++++++++++++++++++++-------------- tests/suite.sh | 233 ++++++++++++++++++------ uninstall.sh | 20 ++- 10 files changed, 716 insertions(+), 497 deletions(-) delete mode 100755 bootstrap.sh create mode 100755 ci/assemble.sh create mode 100755 install.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9cdb249..cba91e7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,16 +19,16 @@ jobs: # a key that is not set is simply empty. steps: - uses: actions/checkout@v4 - - run: shellcheck -S warning -e SC2155 S99cerberus bootstrap.sh uninstall.sh ci/build.sh tests/run.sh tests/suite.sh + - run: shellcheck -S warning -e SC2155 S99cerberus install.sh uninstall.sh ci/build.sh ci/assemble.sh tests/run.sh tests/suite.sh test: name: tests (${{ matrix.name }}) runs-on: ubuntu-latest - # Both mbedTLS generations are in the field: 3.x on current distros and in Alpine (musl, which is - # what OpenIPC is), 2.28 LTS in Buildroot-era firmware — and Debian bookworm carries exactly that. - # The suite builds with -Werror, enrols against a fake coordinator over real TLS, and brings up a - # real WireGuard tunnel to a hub in its own netns; REQUIRE_WG turns a runner without kernel - # WireGuard into a failure instead of a silently skipped section. ubuntu-latest has it. + # Both mbedTLS generations are in the field: 3.x on current distros and in Alpine, 2.28 LTS in + # Buildroot-era firmware — and Debian bookworm carries exactly that. The suite builds with -Werror, + # enrols against a fake coordinator over real TLS, walks the streq_* -> cerberus_* migration and the + # installer, and brings up a real WireGuard tunnel to a hub in its own netns; REQUIRE_WG turns a + # runner without kernel WireGuard into a failure instead of a silently skipped section. strategy: fail-fast: false matrix: @@ -49,61 +49,63 @@ jobs: build: name: build (${{ matrix.arch }}) runs-on: ubuntu-latest - # The real target is arm; a compile-only x64 job would keep reporting green while the binary the - # firmware needs does not exist. musl-armv7 is the artefact that actually goes into OpenIPC - # firmware and builds in an emulated armv7 Alpine, so it is the one target here that needs binfmt. - # The ELF-architecture assertion lives in the script, next to the build that produced the binary. + # One static musl binary per CPU architecture, which is what a camera's installer downloads. The + # script asserts the ELF machine and static linking, and then runs the binary under QEMU and + # checks its X25519 key against `wg pubkey` — a binary for the right machine can still die on its + # first instruction, and only running it shows that. strategy: fail-fast: false matrix: - arch: [musl-armv7, armhf, arm64, amd64] + arch: [arm, mipsel, aarch64, x86_64] steps: - uses: actions/checkout@v4 with: # git describe needs the tags: the version compiled into the binary comes from them. fetch-depth: 0 - - name: Enable arm emulation - if: matrix.arch == 'musl-armv7' - uses: docker/setup-qemu-action@v3 + - name: Toolchain and mbedTLS cache + uses: actions/cache@v4 with: - platforms: arm + path: .cache + key: toolchain-${{ matrix.arch }}-${{ hashFiles('ci/build.sh') }} - - name: Build and package + - name: Build run: ci/build.sh ${{ matrix.arch }} - uses: actions/upload-artifact@v4 with: name: cerberus-${{ matrix.arch }} - path: dist/*.tar.gz* + path: dist/cerberus-${{ matrix.arch }} if-no-files-found: error release: name: release - # A v* tag publishes the tarballs that were just built and tested — the same artefacts, not a - # rebuild — as a GitHub release. + # A v* tag publishes the binaries that were just built and tested — the same files, not a rebuild — + # together with the installer, the uninstaller, the init script and SHA256SUMS over all of them. + # Asset names carry no version: cameras fetch releases/latest/download/. if: startsWith(github.ref, 'refs/tags/v') needs: [lint, test, build] runs-on: ubuntu-latest permissions: contents: write steps: + - uses: actions/checkout@v4 + - uses: actions/download-artifact@v4 with: pattern: cerberus-* path: dist merge-multiple: true - - name: Every tarball carries the tag's version + - name: The binaries were built as this tag run: | - want="${GITHUB_REF_NAME#v}" - ls dist - for f in dist/*.tar.gz; do - case "$(basename "$f")" in - "cerberus-$want-linux-"*) ;; - *) echo "$f was not built as $want"; exit 1 ;; - esac - done + chmod +x dist/cerberus-* + got=$(dist/cerberus-x86_64 --help 2>&1 | head -1 | cut -d' ' -f2) + echo "binary says $got, tag says ${GITHUB_REF_NAME#v}" + [ "$got" = "${GITHUB_REF_NAME#v}" ] + + - name: Lay out the release + run: ci/assemble.sh - name: Publish env: diff --git a/.gitignore b/.gitignore index 456377a..d3a3557 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,4 @@ /cerberus /dist/ *.o +/.cache/ diff --git a/S99cerberus b/S99cerberus index 6ca4e9a..5fe9fa1 100755 --- a/S99cerberus +++ b/S99cerberus @@ -1,43 +1,45 @@ #!/bin/sh # OpenIPC uses BusyBox init, not systemd. Config lives in the U-Boot environment, which is also -# where cerberus persists the join material it gets back from the coordinator. +# where cerberus keeps the join material it gets back from the coordinator, so a camera is +# provisioned with `fw_setenv` and a restart, without touching the read-only rootfs. DAEMON=/usr/sbin/cerberus PIDFILE=/var/run/cerberus.pid -# Read the deployment config out of the U-Boot env (streq_api, streq_enrollment_token, ...) so a -# camera is provisioned with `fw_setenv` and a reboot, without touching the read-only rootfs. -export STREQ_API="$(fw_printenv -n streq_api 2>/dev/null)" -export STREQ_ENROLLMENT_TOKEN="$(fw_printenv -n streq_enrollment_token 2>/dev/null)" -export STREQ_SUBNET="$(fw_printenv -n streq_subnet 2>/dev/null)" +# cerberus_, or streq_ — what the keys were called before the agent was renamed. A +# missing key is harmless: an empty variable means "the default" to the agent. +uenv() { + fw_printenv -n "cerberus_$1" 2>/dev/null || fw_printenv -n "streq_$1" 2>/dev/null +} -# Пустая переменная для агента означает "как по умолчанию" (env_or), поэтому отсутствующий ключ в -# u-boot env безвреден и все, что ниже, необязательно. -# -# streq_ca — не роскошь: прошивка может не нести бандл доверенных CA вовсе, и тогда энролл падает на -# проверке сертификата координатора, а поправить это, не перепрошивая камеру, больше нечем. Класть -# туда путь к корневому сертификату (ISRG Root X1 для Let's Encrypt, или свой CA). -export STREQ_CA="$(fw_printenv -n streq_ca 2>/dev/null)" -# Имя, под которым камера появится в панели; иначе там будет один id. -export STREQ_HOSTNAME="$(fw_printenv -n streq_hostname 2>/dev/null)" -# Адрес для пробы внутри туннеля (по умолчанию выводится из streq_subnet) и её период. -export STREQ_PING_TARGET="$(fw_printenv -n streq_ping_target 2>/dev/null)" -export STREQ_PING_INTERVAL="$(fw_printenv -n streq_ping_interval 2>/dev/null)" -STREQ_IFACE=$(fw_printenv -n streq_iface 2>/dev/null) -export STREQ_IFACE=${STREQ_IFACE:-wg0} +export CERBERUS_API="$(uenv api)" +export CERBERUS_ENROLLMENT_TOKEN="$(uenv enrollment_token)" +export CERBERUS_HOSTNAME="$(uenv hostname)" +# A firmware may carry no CA bundle at all; then enrolment fails on the coordinator's certificate +# and nothing short of a reflash fixes it. Point cerberus_ca at the root certificate instead. +export CERBERUS_CA="$(uenv ca)" +export CERBERUS_KIND="$(uenv kind)" +export CERBERUS_SUBNET="$(uenv subnet)" +export CERBERUS_HEARTBEAT="$(uenv heartbeat)" +export CERBERUS_PING_TARGET="$(uenv ping_target)" +export CERBERUS_PING_INTERVAL="$(uenv ping_interval)" +export CERBERUS_PING_TIMEOUT="$(uenv ping_timeout)" +export CERBERUS_PING_FAILS="$(uenv ping_fails)" +CERBERUS_IFACE=$(uenv iface) +export CERBERUS_IFACE=${CERBERUS_IFACE:-wg0} case "$1" in start) - [ -n "$STREQ_API" ] || { echo "streq: not provisioned (fw_setenv streq_api ...)"; exit 0; } + [ -n "$CERBERUS_API" ] || { echo "cerberus: not provisioned (fw_setenv cerberus_api ...)"; exit 0; } echo "Starting cerberus..." start-stop-daemon -S -q -m -b -p $PIDFILE -x $DAEMON -- run ;; stop) echo "Stopping cerberus..." start-stop-daemon -K -q -p $PIDFILE - # Интерфейса может не быть — агент мог не дойти до его подъёма. Это не повод завершиться с - # ошибкой: для init "stop" неподнятого сервиса обязан быть успехом. - ip link del dev "$STREQ_IFACE" 2>/dev/null || true + # The agent may never have got as far as the interface; for init, stopping a service that + # is not up is still a success. + ip link del dev "$CERBERUS_IFACE" 2>/dev/null || true ;; restart|reload) "$0" stop diff --git a/bootstrap.sh b/bootstrap.sh deleted file mode 100755 index 29dcb1f..0000000 --- a/bootstrap.sh +++ /dev/null @@ -1,80 +0,0 @@ -#!/bin/sh -set -eu - -DIST="${STREQ_DIST:-https://cloud.openipc.ru/cerberus}" -API="${STREQ_API:-https://cloud-api.openipc.ru/v1}" -DOCS="${STREQ_DOCS:-https://cloud.openipc.ru/docs/openipc-camera/#codes}" -TOKEN="${1:-}" - -say() { echo "cerberus: $*"; } -die() { echo "cerberus: E$1" >&2; echo "cerberus: $DOCS" >&2; exit "$1"; } - -[ "$(id -u)" = 0 ] || die 10 - -command -v fw_printenv >/dev/null 2>&1 || die 11 -chip=$(fw_printenv -n soc 2>/dev/null | tr 'A-Z' 'a-z' | tr -d ' \t\r') || die 12 -[ -n "$chip" ] || die 12 -say "chip $chip" - -tmp="/tmp/cerberus.$$" -rc="/tmp/S99cerberus.$$" -trap 'rm -f "$tmp" "$tmp.sha" "$rc"' EXIT INT TERM - -if ! curl -fsL -o "$tmp" "$DIST/cerberus_$chip"; then - curl -fsL -m 5 -o /dev/null -X POST -H 'Content-Type: application/json' -d "{\"chip\":\"$chip\"}" "$API/builds/missing" 2>/dev/null || true - die 13 -fi -curl -fsL -o "$rc" "$DIST/S99cerberus" || die 14 - -if curl -fsL -o "$tmp.sha" "$DIST/cerberus_$chip.sha256" 2>/dev/null; then - want=$(cut -d' ' -f1 <"$tmp.sha") - have=$(sha256sum "$tmp" | cut -d' ' -f1) - [ "$want" = "$have" ] || die 15 - say "sha256 $have" -fi - -if [ -x /etc/init.d/S99cerberus ]; then - /etc/init.d/S99cerberus stop 2>/dev/null || true -fi - -chmod 0755 "$tmp" "$rc" || die 16 -mv "$tmp" /usr/sbin/cerberus || die 16 -mv "$rc" /etc/init.d/S99cerberus || die 16 -mkdir -p /etc/streq || die 16 -chmod 0700 /etc/streq || die 16 -say "installed /usr/sbin/cerberus" - -if [ -z "$TOKEN" ]; then - say "E20" - say "$DOCS" - echo " fw_setenv streq_api '$API'" - echo " fw_setenv streq_enrollment_token 'tok_...'" - echo " /etc/init.d/S99cerberus restart" - exit 0 -fi - -fw_setenv streq_api "$API" || die 17 -fw_setenv streq_enrollment_token "$TOKEN" || die 17 - -/etc/init.d/S99cerberus restart >/dev/null 2>&1 || die 18 -say "started" - -claim="" -i=0 -while [ "$i" -lt 40 ]; do - claim=$(/usr/sbin/cerberus status 2>/dev/null | awk '$1 == "claim" && $2 == "code" { print $3 }') - if [ -n "$claim" ] && [ "$claim" != "-" ]; then - break - fi - claim="" - i=$((i + 1)) - sleep 1 -done - -if [ -n "$claim" ]; then - say "claim $claim" -else - say "E21" - say "$DOCS" - echo " cerberus status" -fi diff --git a/ci/assemble.sh b/ci/assemble.sh new file mode 100755 index 0000000..d22b717 --- /dev/null +++ b/ci/assemble.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# Lays a release out next to the binaries ci/build.sh left in $OUT: the installer and the uninstaller, +# the init script, the license, and SHA256SUMS over everything a camera downloads. The installer reads +# the sums from the same release, so a binary and the sum it is checked against always travel together. +# +# Usage: ci/assemble.sh (OUT=dist by default) +set -euo pipefail + +cd "$(dirname "$0")/.." +OUT=${OUT:-dist} + +ls "$OUT"/cerberus-* >/dev/null 2>&1 || { echo "no $OUT/cerberus-* — run ci/build.sh first"; exit 1; } + +cp install.sh uninstall.sh S99cerberus LICENSE.md "$OUT/" +chmod 0755 "$OUT"/cerberus-* "$OUT/install.sh" "$OUT/uninstall.sh" "$OUT/S99cerberus" + +( + cd "$OUT" + # Plain names, no "*" binary marker: the installer matches on the bare name. + for f in cerberus-* install.sh uninstall.sh S99cerberus; do + printf '%s %s\n' "$(sha256sum "$f" | cut -d' ' -f1)" "$f" + done >SHA256SUMS + cat SHA256SUMS +) diff --git a/ci/build.sh b/ci/build.sh index 9763d2a..2028b20 100755 --- a/ci/build.sh +++ b/ci/build.sh @@ -1,57 +1,50 @@ #!/usr/bin/env bash -# Package the on-camera agent for real hardware. +# Builds the release binaries: one static musl binary per CPU architecture. # -# The camera is not an edge box: no room for a runtime, and the rootfs is a read-only squashfs. So the -# agent is one C binary, mbedTLS its only dependency, using the kernel WireGuard the firmware ships. +# Static musl is the whole point. A camera's firmware may be musl or uClibc or glibc, may carry +# mbedTLS 2.28, 3.x or none at all — a static binary does not care, so one build per architecture +# runs on every camera of that architecture, whatever SoC and firmware it has: # -# Cross-compiles inside a Debian container with multiarch, so the only thing a build host needs is -# Docker — no toolchain, no emulation, no arm hardware. +# arm ARMv5TE, soft-float EABI — runs on every ARM camera from ARMv5 (no VFP) to ARMv7 +# (HiSilicon, Goke, SigmaStar, Fullhan, ...). Floating point never matters here: the +# crypto is integer, and soft-float code runs on a CPU with an FPU just as well. +# mipsel MIPS32 little-endian — Ingenic T10/T20/T21/T31/T40 and other MIPS cameras. +# aarch64 64-bit ARM. +# x86_64 VMs and test boxes. # -# Usage: ci/build.sh # arm64 -# ci/build.sh arm64 armhf amd64 # explicit set (Debian arch names) -# ci/build.sh musl-armv7 # OpenIPC (ARM): static musl, no dependencies +# Everything happens in a Debian container, so the host needs only Docker: the Bootlin musl +# toolchain and the mbedTLS source are downloaded once into .cache/ and checked against pinned +# sha256 sums. Each binary is then *run* under QEMU in the same container — `--help`, and a keypair +# compared with what `wg pubkey` derives — because a binary for the right ELF machine can still die +# on the first instruction its CPU does not have, and that is invisible to `file`. +# +# Usage: ci/build.sh # all four +# ci/build.sh arm mipsel # a subset # VERSION=1.2.0 OUT=dist ci/build.sh # # VERSION defaults to the nearest v* tag, without the v. It is compiled into the binary — it is what -# the agent reports to the coordinator — so a tarball and the agent inside it never disagree. With no -# tag in reach the binary keeps the version written in main.c and the tarball is named -dev. -# -# NOTE ON LIBC. arm64/armhf/amd64 here are glibc, which is what a generic arm SBC runs, and they are -# dynamically linked on purpose: statically linked glibc still needs matching shared NSS libraries at -# runtime for getaddrinfo — exactly the call that resolves the hub endpoint. They are also the -# mbedTLS 2.28 builds (Debian bookworm), the generation Buildroot-era firmware carries. -# -# OpenIPC is musl, so a glibc binary does not run there at all: the loader it names does not exist and -# the kernel answers with a bare "not found". That is what `musl-armv7` is for. It builds inside an -# armv7 Alpine container (binfmt/QEMU — Docker Desktop has it; on a plain Linux host run -# `docker run --privileged --rm tonistiigi/binfmt --install arm` once) and links statically, because -# static is the only thing that makes a binary independent of which mbedTLS a given firmware carries. -# musl static has none of glibc's NSS problem. -# -# It does not cover every OpenIPC device: Ingenic T20/T31 are MIPS, and ARMv5 SoCs have no VFP, so -# neither runs an ARMv7 build. Those need the firmware's own Buildroot toolchain: +# the agent reports to the coordinator. With no tag in reach the binary keeps the version in main.c. # -# make CC=arm-openipc-linux-musleabi-gcc MBEDTLS= STATIC=1 +# Output: $OUT/cerberus-. Names carry no version on purpose: the installer fetches +# releases/latest/download/cerberus-, and that address must not change between releases. set -euo pipefail cd "$(dirname "$0")/.." ARCHES=("$@") -[ ${#ARCHES[@]} -gt 0 ] || ARCHES=(arm64) +[ ${#ARCHES[@]} -gt 0 ] || ARCHES=(arm mipsel aarch64 x86_64) OUT=${OUT:-dist} +CACHE=${CACHE:-.cache} IMAGE=${CERBERUS_BUILD_IMAGE:-debian:bookworm-slim} -# --platform picks the architecture; the single-architecture repository keeps that choice from leaking. -# `--platform linux/arm/v7 alpine:3.20` overwrites the local alpine:3.20 tag with an armv7 image, and -# every later `docker run alpine:3.20` on that machine then runs emulated — tests/run.sh included, -# where kernel WireGuard fails and it reads as a bug in the agent. -CERBERUS_ARM_IMAGE=${CERBERUS_ARM_IMAGE:-arm32v7/alpine:3.20} + +TOOLCHAIN_RELEASE=stable-2025.08-1 +MBEDTLS_VERSION=3.6.7 +MBEDTLS_SHA256=a7e8bcbec0e6f761b4af24f25677626b35f762f68eef79c08677a363212d11f6 VERSION=${VERSION:-$(git describe --tags --match 'v[0-9]*' --dirty 2>/dev/null || true)} VERSION=${VERSION#v} -SRC_VERSION=$(sed -n 's/^#define CERBERUS_VERSION "\(.*\)"$/\1/p' main.c) -NAME_VERSION=${VERSION:-$SRC_VERSION-dev} -mkdir -p "$OUT" +mkdir -p "$OUT" "$CACHE" # Docker on Windows wants a native path for -v, and MSYS would rewrite the container-side one. host_path() { @@ -60,162 +53,87 @@ host_path() { *) (cd "$1" && pwd) ;; esac } -SRC=$(host_path .) for arch in "${ARCHES[@]}"; do case "$arch" in - arm64) triple=aarch64-linux-gnu; want='ARM aarch64' ;; - armhf) triple=arm-linux-gnueabihf; want='ARM, EABI' ;; - amd64) triple=x86_64-linux-gnu; want='x86-64' ;; - musl-armv7) triple=armv7-alpine-linux-musleabihf; want='ARM, EABI' ;; - *) echo "unknown arch '$arch' (want arm64 | armhf | amd64 | musl-armv7)"; exit 2 ;; + arm) + tc=armv5-eabi; tc_sha=8cdb4ad70c6b5a66427fa3315fe3ddde1c19c90232674dec59045e04d2a36cf1 + qemu="qemu-arm-static"; want='ARM, EABI5' ;; + mipsel) + tc=mips32el; tc_sha=12a5e2f50b94b5c619730914db380ce9679798fb3c7460a40d480f77bb50628f + qemu="qemu-mipsel-static"; want='LSB executable, MIPS, MIPS32' ;; + aarch64) + tc=aarch64; tc_sha=defba831ffa1175236f137069333e21ed46d4d19feb5080a90cf248b6fc2cb08 + qemu="qemu-aarch64-static"; want='ARM aarch64' ;; + x86_64) + tc=x86-64; tc_sha=09fca3aa89540f1b01b5f4210d488cbeb00f522044c53e9989b1dd8a38076912 + qemu=; want='x86-64' ;; + *) echo "unknown arch '$arch' (want arm | mipsel | aarch64 | x86_64)"; exit 2 ;; esac - echo "==> building linux-$arch ($triple), version ${VERSION:-$SRC_VERSION (from main.c)}" - stage="$OUT/cerberus-$NAME_VERSION-linux-$arch" - rm -rf "$stage" - mkdir -p "$stage" - - if [ "$arch" = musl-armv7 ]; then - # Native build inside an emulated armv7 Alpine: no cross toolchain to find, and the libc is - # the one OpenIPC actually runs. Slow (QEMU) but this is one C file. - MSYS_NO_PATHCONV=1 docker run -i --rm \ - --platform linux/arm/v7 \ - -v "$SRC:/src:ro" \ - -v "$(host_path "$stage"):/out" \ - -e WANT="$want" -e VERSION="$VERSION" \ - "$CERBERUS_ARM_IMAGE" sh -s <<'MUSLBUILD' || { -set -e -apk add --no-cache gcc musl-dev make mbedtls-dev mbedtls-static file >/dev/null -mkdir /build && cp /src/main.c /src/Makefile /build/ && cd /build -make STATIC=1 VERSION="$VERSION" EXTRA_CFLAGS=-Werror - -file cerberus -file cerberus | grep -q "$WANT" -# Static is the point: a firmware that carries no mbedTLS, or a different one, must not matter. -file cerberus | grep -q "statically linked" - -cp cerberus /out/ -MUSLBUILD - echo "musl-armv7 build failed. If docker could not run linux/arm/v7, install binfmt:" - echo " docker run --privileged --rm tonistiigi/binfmt --install arm" - exit 1 - } - else + echo "==> cerberus-$arch (Bootlin $tc musl $TOOLCHAIN_RELEASE, mbedTLS $MBEDTLS_VERSION, version ${VERSION:-from main.c})" # -i matters: without it the container gets no stdin, `sh -s` reads EOF, and the build silently # does nothing while exiting 0. MSYS_NO_PATHCONV=1 docker run -i --rm \ - -v "$SRC:/src:ro" \ - -v "$(host_path "$stage"):/out" \ - -e ARCH="$arch" -e TRIPLE="$triple" -e WANT="$want" -e VERSION="$VERSION" \ - "$IMAGE" sh -s <<'BUILD' -set -e + -v "$(host_path .):/src:ro" \ + -v "$(host_path "$CACHE"):/cache" \ + -v "$(host_path "$OUT"):/out" \ + -e ARCH="$arch" -e TC="$tc" -e TC_SHA="$tc_sha" -e TC_RELEASE="$TOOLCHAIN_RELEASE" \ + -e MBEDTLS_VERSION="$MBEDTLS_VERSION" -e MBEDTLS_SHA256="$MBEDTLS_SHA256" \ + -e QEMU="$qemu" -e WANT="$want" -e VERSION="$VERSION" \ + "$IMAGE" bash -s <<'BUILD' +set -euo pipefail export DEBIAN_FRONTEND=noninteractive - -# amd64 is the container's own architecture: no multiarch, no cross toolchain, plain gcc. -if [ "$ARCH" = amd64 ]; then - apt-get update -qq >/dev/null - apt-get install -y -qq gcc libc6-dev libmbedtls-dev make file >/dev/null - CC=gcc - STRIP=strip -else - dpkg --add-architecture "$ARCH" - apt-get update -qq >/dev/null - apt-get install -y -qq "crossbuild-essential-$ARCH" "libmbedtls-dev:$ARCH" make file >/dev/null - CC="$TRIPLE-gcc" - STRIP="$TRIPLE-strip" -fi - -mkdir /build && cp /src/main.c /src/Makefile /build/ && cd /build -make CC="$CC" STRIP="$STRIP" VERSION="$VERSION" EXTRA_CFLAGS=-Werror - -# A binary built for the wrong architecture is the failure mode worth catching here, and it is -# invisible in a build log. -file cerberus -file cerberus | grep -q "$WANT" - -# Its one shared dependency has to actually be mbedTLS and nothing more exotic. -"${TRIPLE}-objdump" -p cerberus 2>/dev/null | grep NEEDED || objdump -p cerberus | grep NEEDED - -cp cerberus /out/ -BUILD +apt-get update -qq >/dev/null +apt-get install -y -qq --no-install-recommends ca-certificates curl xz-utils bzip2 make file \ + qemu-user-static wireguard-tools >/dev/null + +fetch() { # fetch + if [ ! -f "/cache/$3" ] || ! echo "$2 /cache/$3" | sha256sum -c --quiet - 2>/dev/null; then + curl -fsSL --retry 3 -o "/cache/$3.part" "$1" + echo "$2 /cache/$3.part" | sha256sum -c --quiet - + mv "/cache/$3.part" "/cache/$3" fi +} - cp S99cerberus "$stage/S99cerberus" - chmod 755 "$stage/S99cerberus" - # The licence travels with every copy, binaries included: its Notices rule asks for exactly that. - cp LICENSE.md "$stage/LICENSE.md" - - cat >"$stage/install.sh" <<'EOF' -#!/bin/sh -set -e -[ "$(id -u)" = 0 ] || { echo "cerberus: E10" >&2; exit 10; } - -cp cerberus /usr/sbin/cerberus -cp S99cerberus /etc/init.d/S99cerberus -chmod 0755 /usr/sbin/cerberus /etc/init.d/S99cerberus -mkdir -p /etc/streq && chmod 0700 /etc/streq - -echo "done. Provisioning is two fw_setenv calls and a reboot:" -echo " fw_setenv streq_api https://api.example.com/v1" -echo " fw_setenv streq_enrollment_token tok_..." -echo " reboot" -echo -echo "After start the claim code is shown by: cerberus status" -EOF - chmod 755 "$stage/install.sh" - - if [ "$arch" = musl-armv7 ]; then - LIBC_NOTE="**Static musl build for ARMv7 — for OpenIPC firmware.** No dependencies at all: -mbedTLS is inside the binary, so whichever mbedTLS the firmware carries does not matter. It does not -fit MIPS devices (Ingenic T20/T31) or ARMv5 without VFP — those need the firmware's own toolchain." - else - LIBC_NOTE='**This is a glibc build for a generic arm SBC.** It will not start on OpenIPC -firmware at all — that is musl: a camera needs the `musl-armv7` build.' - fi +tarball="$TC--musl--$TC_RELEASE.tar.xz" +fetch "https://toolchains.bootlin.com/downloads/releases/toolchains/$TC/tarballs/$tarball" "$TC_SHA" "$tarball" +fetch "https://github.com/Mbed-TLS/mbedtls/releases/download/mbedtls-$MBEDTLS_VERSION/mbedtls-$MBEDTLS_VERSION.tar.bz2" \ + "$MBEDTLS_SHA256" "mbedtls-$MBEDTLS_VERSION.tar.bz2" - cat >"$stage/README.md" </dev/null +cp /build/mbedtls/library/*.a /build/prefix/lib/ +cp -r /build/mbedtls/include /build/prefix/ -\`\`\`sh -./install.sh -fw_setenv streq_api https://api.example.com/v1 -fw_setenv streq_enrollment_token tok_... -reboot -\`\`\` +cp /src/main.c /src/Makefile /build/ +cd /build +make CC="$CC" STRIP="${CROSS}strip" MBEDTLS=/build/prefix STATIC=1 VERSION="$VERSION" EXTRA_CFLAGS=-Werror -$LIBC_NOTE +# The wrong ELF machine, or a dynamic binary, is the failure worth catching here — and it is +# invisible in a build log. +file cerberus +file cerberus | grep -q "$WANT" +file cerberus | grep -q "statically linked" -Full guide, variables and troubleshooting: https://cloud.openipc.ru/docs/openipc-camera/ -Source: https://github.com/OpenIPC/cerberus +# ...and so is a binary that is the right machine and still cannot run. Run it. +run() { if [ -n "$QEMU" ]; then "$QEMU" ./cerberus "$@"; else ./cerberus "$@"; fi; } +run --help 2>&1 | head -1 +CERBERUS_STORE=file:/tmp/smoke.env run keygen >/tmp/keys 2>/dev/null +[ "$(sed -n 1p /tmp/keys | wg pubkey)" = "$(sed -n 2p /tmp/keys)" ] \ + || { echo "the X25519 key this binary derives is not the one WireGuard derives"; exit 1; } +echo "keygen on $ARCH matches wg pubkey" -Licensed under the Prosperity Public License 3.0.0 — see LICENSE.md: free for noncommercial use, -a thirty-day trial for commercial use. -EOF +cp cerberus "/out/cerberus-$ARCH" +BUILD - # Packed inside a container, from a copy that is not on a bind mount: a Windows host has no - # execute bit to give, and a tarball whose binary unpacks as 0644 fails at `./cerberus` — the - # first thing anyone types after unpacking. - MSYS_NO_PATHCONV=1 docker run -i --rm \ - -v "$(host_path "$OUT"):/out" \ - -e STAGE="$(basename "$stage")" \ - "$IMAGE" sh -s <<'PACK' -set -e -cd /tmp -cp -r "/out/$STAGE" . -cd "$STAGE" -chmod 0755 cerberus install.sh S99cerberus -chmod 0644 README.md LICENSE.md -cd /tmp -tar -czf "/out/$STAGE.tar.gz" "$STAGE" -PACK - rm -rf "$stage" - (cd "$OUT" && sha256sum "cerberus-$NAME_VERSION-linux-$arch.tar.gz" \ - >"cerberus-$NAME_VERSION-linux-$arch.tar.gz.sha256") - echo " $OUT/cerberus-$NAME_VERSION-linux-$arch.tar.gz ($(du -h "$OUT/cerberus-$NAME_VERSION-linux-$arch.tar.gz" | cut -f1))" + echo " $OUT/cerberus-$arch ($(wc -c <"$OUT/cerberus-$arch") bytes)" done diff --git a/install.sh b/install.sh new file mode 100755 index 0000000..f263676 --- /dev/null +++ b/install.sh @@ -0,0 +1,105 @@ +#!/bin/sh +set -eu + +DIST="${CERBERUS_DIST:-${STREQ_DIST:-https://github.com/OpenIPC/cerberus/releases/latest/download}}" +API="${CERBERUS_API:-${STREQ_API:-https://cloud-api.openipc.ru/v1}}" +DOCS="${CERBERUS_DOCS:-${STREQ_DOCS:-https://cloud.openipc.ru/docs/openipc-camera/#codes}}" +TOKEN="${1:-}" + +say() { echo "cerberus: $*"; } +die() { echo "cerberus: E$1" >&2; echo "cerberus: $DOCS" >&2; exit "$1"; } + +[ "$(id -u)" = 0 ] || die 10 + +command -v fw_printenv >/dev/null 2>&1 || die 11 +chip=$(fw_printenv -n soc 2>/dev/null | tr 'A-Z' 'a-z' | tr -d ' \t\r') || chip="" +machine=$(uname -m) + +case "$machine" in + aarch64|arm64) arch=aarch64 ;; + arm*) arch=arm ;; + mips*) + if [ "$(dd if=/bin/sh bs=1 skip=5 count=1 2>/dev/null | od -An -tx1 | tr -d ' ')" = 01 ]; then + arch=mipsel + else + arch="" + fi + ;; + x86_64|amd64) arch=x86_64 ;; + *) arch="" ;; +esac + +say "chip ${chip:-?}, cpu $machine" + +if [ -z "$arch" ]; then + curl -fsL -m 5 -o /dev/null -X POST -H 'Content-Type: application/json' \ + -d "{\"chip\":\"${chip:-$machine}\"}" "$API/builds/missing" 2>/dev/null || true + die 13 +fi + +tmp="/tmp/cerberus.$$" +rc="/tmp/S99cerberus.$$" +trap 'rm -f "$tmp" "$tmp.sums" "$rc"' EXIT INT TERM + +curl -fsL -o "$tmp" "$DIST/cerberus-$arch" || die 14 +curl -fsL -o "$rc" "$DIST/S99cerberus" || die 14 + +if command -v sha256sum >/dev/null 2>&1; then + curl -fsL -o "$tmp.sums" "$DIST/SHA256SUMS" || die 15 + for pair in "$tmp:cerberus-$arch" "$rc:S99cerberus"; do + file=${pair%%:*} + name=${pair#*:} + want=$(awk -v n="$name" '$2 == n || $2 == "*" n { print $1 }' "$tmp.sums") + have=$(sha256sum "$file" | cut -d' ' -f1) + [ -n "$want" ] && [ "$want" = "$have" ] || die 15 + done + say "cerberus-$arch sha256 ok" +fi + +if [ -x /etc/init.d/S99cerberus ]; then + /etc/init.d/S99cerberus stop 2>/dev/null || true +fi + +chmod 0755 "$tmp" "$rc" || die 16 +mv "$tmp" /usr/sbin/cerberus || die 16 +mv "$rc" /etc/init.d/S99cerberus || die 16 +mkdir -p /etc/cerberus || die 16 +chmod 0700 /etc/cerberus || die 16 +say "installed /usr/sbin/cerberus" + +if [ -z "$TOKEN" ]; then + say "E20" + say "$DOCS" + echo " fw_setenv cerberus_api '$API'" + echo " fw_setenv cerberus_enrollment_token 'tok_...'" + echo " /etc/init.d/S99cerberus restart" + exit 0 +fi + +fw_setenv cerberus_api "$API" || die 17 +fw_setenv cerberus_enrollment_token "$TOKEN" || die 17 + +/etc/init.d/S99cerberus restart >/dev/null 2>&1 || die 18 +say "started" + +claim="" +i=0 +while [ "$i" -lt 40 ]; do + st=$(CERBERUS_PING_INTERVAL=0 /usr/sbin/cerberus status 2>/dev/null || true) + node=$(echo "$st" | awk '$1 == "node" { print $2 }') + claim=$(echo "$st" | awk '$1 == "claim" && $2 == "code" { print $3 }') + if [ -n "$node" ] && [ "$node" != "(not" ] && [ -n "$claim" ] && [ "$claim" != "-" ]; then + break + fi + claim="" + i=$((i + 1)) + sleep 1 +done + +if [ -n "$claim" ]; then + say "claim $claim" +else + say "E21" + say "$DOCS" + echo " cerberus status" +fi diff --git a/main.c b/main.c index 964bfb4..5d92766 100644 --- a/main.c +++ b/main.c @@ -56,9 +56,9 @@ /* Both mbedTLS generations are in the field: 3.x on current distros, 2.28 LTS in Buildroot (and so * in most OpenIPC-era firmware). The only call that differs is the one-shot SHA-256. */ #if MBEDTLS_VERSION_MAJOR < 3 -#define streq_sha256(in, len, out) mbedtls_sha256_ret((in), (len), (out), 0) +#define sha256_once(in, len, out) mbedtls_sha256_ret((in), (len), (out), 0) #else -#define streq_sha256(in, len, out) mbedtls_sha256((in), (len), (out), 0) +#define sha256_once(in, len, out) mbedtls_sha256((in), (len), (out), 0) #endif /* Release builds pass the tag in (`make VERSION=...`); a plain `make` reports the source version. */ @@ -118,7 +118,7 @@ static void die(const char *fmt, ...) /* ----------------------------------------------------------------- config */ -/* Bound from STREQ_* env, same convention as the .NET agent (AgentOptions). */ +/* Bound from CERBERUS_* env, same convention as the .NET agent (AgentOptions). */ struct cfg { const char *api; /* coordinator base URL incl. /v1 */ const char *token; /* one-shot enrollment token */ @@ -144,48 +144,57 @@ struct cfg { static struct cfg g_cfg; -static const char *env_or(const char *name, const char *fallback) +/* CERBERUS_, then the name this agent had before it was renamed — CERBERUS_ — so a script or an + * init file written for the old name keeps working. */ +static const char *cfg_env(const char *name, const char *fallback) { - const char *v = getenv(name); + char key[64]; + const char *v; + + snprintf(key, sizeof key, "CERBERUS_%s", name); + v = getenv(key); + if (v && *v) return v; + snprintf(key, sizeof key, "STREQ_%s", name); + v = getenv(key); return (v && *v) ? v : fallback; } static void cfg_load(void) { - g_cfg.api = env_or("STREQ_API", "https://api.localhost/v1"); - g_cfg.token = env_or("STREQ_ENROLLMENT_TOKEN", ""); - g_cfg.kind = env_or("STREQ_KIND", "Camera"); - g_cfg.hostname = env_or("STREQ_HOSTNAME", NULL); - g_cfg.iface = env_or("STREQ_IFACE", "wg0"); - g_cfg.subnet = env_or("STREQ_SUBNET", "10.80.0.0/16"); - g_cfg.ca = env_or("STREQ_CA", "/etc/ssl/certs/ca-certificates.crt"); - g_cfg.store = env_or("STREQ_STORE", "auto"); - g_cfg.keyfile = env_or("STREQ_KEYFILE", "/etc/streq/wg.key"); - g_cfg.insecure = atoi(env_or("STREQ_INSECURE", "0")); - g_quiet = atoi(env_or("STREQ_QUIET", "0")); - g_cfg.dry_run = atoi(env_or("STREQ_DRY_RUN", "0")); - g_cfg.watch_interval = atoi(env_or("STREQ_WATCH_INTERVAL", "30")); - g_cfg.stale_after = atoi(env_or("STREQ_STALE_AFTER", "180")); + g_cfg.api = cfg_env("API", "https://api.localhost/v1"); + g_cfg.token = cfg_env("ENROLLMENT_TOKEN", ""); + g_cfg.kind = cfg_env("KIND", "Camera"); + g_cfg.hostname = cfg_env("HOSTNAME", NULL); + g_cfg.iface = cfg_env("IFACE", "wg0"); + g_cfg.subnet = cfg_env("SUBNET", "10.80.0.0/16"); + g_cfg.ca = cfg_env("CA", "/etc/ssl/certs/ca-certificates.crt"); + g_cfg.store = cfg_env("STORE", "auto"); + g_cfg.keyfile = cfg_env("KEYFILE", "/etc/streq/wg.key"); + g_cfg.insecure = atoi(cfg_env("INSECURE", "0")); + g_quiet = atoi(cfg_env("QUIET", "0")); + g_cfg.dry_run = atoi(cfg_env("DRY_RUN", "0")); + g_cfg.watch_interval = atoi(cfg_env("WATCH_INTERVAL", "30")); + g_cfg.stale_after = atoi(cfg_env("STALE_AFTER", "180")); /* The coordinator's contract: 20s by default, never faster than 5. */ - g_cfg.heartbeat = atoi(env_or("STREQ_HEARTBEAT", "20")); + g_cfg.heartbeat = atoi(cfg_env("HEARTBEAT", "20")); if (g_cfg.heartbeat > 0 && g_cfg.heartbeat < 5) g_cfg.heartbeat = 5; if (g_cfg.watch_interval < 1) g_cfg.watch_interval = 1; /* The in-tunnel probe. Slower than the handshake check on purpose: it costs a packet round trip * over the overlay, and what it is looking for (a path that is up on paper and dead in fact) does * not appear between two ticks. */ - g_cfg.ping_target = env_or("STREQ_PING_TARGET", ""); - g_cfg.ping_interval = atoi(env_or("STREQ_PING_INTERVAL", "60")); - g_cfg.ping_timeout = atoi(env_or("STREQ_PING_TIMEOUT", "3")); - g_cfg.ping_fails = atoi(env_or("STREQ_PING_FAILS", "3")); + g_cfg.ping_target = cfg_env("PING_TARGET", ""); + g_cfg.ping_interval = atoi(cfg_env("PING_INTERVAL", "60")); + g_cfg.ping_timeout = atoi(cfg_env("PING_TIMEOUT", "3")); + g_cfg.ping_fails = atoi(cfg_env("PING_FAILS", "3")); if (g_cfg.ping_interval > 0 && g_cfg.ping_interval < 10) g_cfg.ping_interval = 10; if (g_cfg.ping_timeout < 1) g_cfg.ping_timeout = 1; if (g_cfg.ping_fails < 1) g_cfg.ping_fails = 1; /* Ждать координатор, а не умирать. См. enroll_with_retry. */ - g_cfg.retry_min = atoi(env_or("STREQ_RETRY_MIN", "5")); - g_cfg.retry_max = atoi(env_or("STREQ_RETRY_MAX", "300")); + g_cfg.retry_min = atoi(cfg_env("RETRY_MIN", "5")); + g_cfg.retry_max = atoi(cfg_env("RETRY_MAX", "300")); if (g_cfg.retry_min < 1) g_cfg.retry_min = 1; if (g_cfg.retry_max < g_cfg.retry_min) g_cfg.retry_max = g_cfg.retry_min; } @@ -225,7 +234,7 @@ static void sha256_hex(const char *in, char out[65]) { unsigned char d[32]; int i; - if (streq_sha256((const unsigned char *)in, strlen(in), d) != 0) + if (sha256_once((const unsigned char *)in, strlen(in), d) != 0) die("sha256 failed"); for (i = 0; i < 32; i++) sprintf(out + i * 2, "%02x", d[i]); @@ -380,10 +389,26 @@ static int store_is_uboot(void) return have("fw_setenv"); /* auto */ } +#define STORE_FILE_DEFAULT "/etc/cerberus/agent.env" +#define STORE_FILE_LEGACY "/etc/streq/agent.env" + +static void mkparent(const char *path); + +/* The default file moved with the rename; one still in the old place is moved over, once. */ static const char *store_file(void) { + static const char *path; + if (!strncmp(g_cfg.store, "file:", 5)) return g_cfg.store + 5; - return "/etc/streq/agent.env"; + if (path) return path; + + path = STORE_FILE_DEFAULT; + if (access(STORE_FILE_DEFAULT, F_OK) != 0 && access(STORE_FILE_LEGACY, F_OK) == 0) { + mkparent(STORE_FILE_DEFAULT); + if (rename(STORE_FILE_LEGACY, STORE_FILE_DEFAULT) != 0) + path = STORE_FILE_LEGACY; + } + return path; } static void mkparent(const char *path) @@ -430,7 +455,8 @@ static int store_get(const char *key, char *out, size_t out_sz) return 0; } -static void store_set(const char *key, const char *value) +/* Sets `key`, or removes it when `value` is NULL. */ +static void store_put(const char *key, const char *value) { const char *path; char tmp[512], line[1024]; @@ -438,8 +464,8 @@ static void store_set(const char *key, const char *value) FILE *in, *out; if (store_is_uboot()) { - char *argv[] = { "fw_setenv", (char *)key, (char *)value, NULL }; - if (run(argv, NULL, 0) != 0) + char *argv[] = { "fw_setenv", (char *)key, (char *)value, NULL }; /* no value = unset */ + if (run(argv, NULL, 0) != 0 && value) die("fw_setenv %s failed — is /etc/fw_env.config right?", key); return; } @@ -447,11 +473,12 @@ static void store_set(const char *key, const char *value) /* Rewrite the file with this key replaced. Small and rare enough that a full rewrite beats * carrying an in-memory model of it. */ path = store_file(); + in = fopen(path, "r"); + if (!in && !value) return; mkparent(path); snprintf(tmp, sizeof tmp, "%s.tmp", path); out = fopen(tmp, "w"); if (!out) die("cannot write %s: %s", tmp, strerror(errno)); - in = fopen(path, "r"); if (in) { while (fgets(line, sizeof line, in)) { if (!strncmp(line, key, klen) && line[klen] == '=') continue; @@ -459,12 +486,106 @@ static void store_set(const char *key, const char *value) } fclose(in); } - fprintf(out, "%s=%s\n", key, value); + if (value) fprintf(out, "%s=%s\n", key, value); fclose(out); if (chmod(tmp, 0600) != 0) warn_("chmod %s: %s", tmp, strerror(errno)); if (rename(tmp, path) != 0) die("cannot replace %s: %s", path, strerror(errno)); } +static void store_set(const char *key, const char *value) { store_put(key, value); } +static void store_unset(const char *key) { store_put(key, NULL); } + +/* + * The agent's keys used to be called streq_*: streq_device_key, streq_node_id, ... + * + * What is moved is only what the agent writes itself — identity and join material. Configuration + * (cerberus_api, cerberus_enrollment_token, ...) is written by people and read by the init script, + * which looks for both names, so there is nothing to move. + * + * The old keys are copied and left where they are, deliberately. An older S99cerberus, or an older + * binary someone rolls back to, still finds a camera that works; deleting them would buy a tidier + * `fw_printenv` and a camera that does not come up. Where an old copy could do harm — a node the + * coordinator has forgotten coming back to life on the next start — both names are removed together + * (store_forget_enrollment, `clean`). + * + * One read of the whole store per start, and no write at all once there is nothing left to move: on + * a camera every write is a flash erase. + */ +static const char *const g_agent_keys[] = { + "device_key", "claim_code", /* first: the identity matters most if this is interrupted */ + "node_id", "overlay_ip", "hub_endpoint", "hub_pubkey", "allowed_ips", "network_subnet", "hub_ip", + NULL +}; + +/* `key=value` out of a dump of the whole store; an empty value counts as absent, as in store_get. */ +static int dump_get(const char *dump, const char *key, char *out, size_t out_sz) +{ + size_t klen = strlen(key); + const char *p = dump; + + while (p && *p) { + if (!strncmp(p, key, klen) && p[klen] == '=') { + size_t n = strcspn(p + klen + 1, "\r\n"); + if (n + 1 > out_sz) n = out_sz - 1; + memcpy(out, p + klen + 1, n); + out[n] = 0; + return n > 0; + } + p = strchr(p, '\n'); + if (p) p++; + } + return 0; +} + +static void store_migrate_legacy(void) +{ + static char dump[32768]; + char newk[64], oldk[64], val[512], check[512]; + size_t n = 0; + int whole, i, moved = 0; + + if (store_is_uboot()) { + char *argv[] = { "fw_printenv", NULL }; + if (run(argv, dump, sizeof dump) != 0) return; + n = strlen(dump); + } else { + FILE *f = fopen(store_file(), "r"); + if (!f) return; + n = fread(dump, 1, sizeof dump - 1, f); + fclose(f); + dump[n] = 0; + } + /* An environment bigger than the buffer is read key by key instead of trusted half-read. */ + whole = n + 1 < sizeof dump; + if (whole && !strstr(dump, "streq_")) return; + + for (i = 0; g_agent_keys[i]; i++) { + snprintf(newk, sizeof newk, "cerberus_%s", g_agent_keys[i]); + snprintf(oldk, sizeof oldk, "streq_%s", g_agent_keys[i]); + if (whole ? dump_get(dump, newk, val, sizeof val) : store_get(newk, val, sizeof val)) + continue; + if (!(whole ? dump_get(dump, oldk, val, sizeof val) : store_get(oldk, val, sizeof val))) + continue; + store_set(newk, val); + if (!store_get(newk, check, sizeof check) || strcmp(check, val) != 0) + die("moving %s to %s did not stick — nothing else is touched until it does", oldk, newk); + moved++; + } + if (moved) + note("moved %d key(s) from streq_* to cerberus_*; the streq_* copies stay for older init scripts", + moved); +} + +/* Removes both spellings of an agent-written key, so the migration can never bring an old one back. */ +static void store_unset_both(const char *suffix) +{ + char key[64]; + snprintf(key, sizeof key, "cerberus_%s", suffix); + store_unset(key); + snprintf(key, sizeof key, "streq_%s", suffix); + store_unset(key); +} + /* * Forgets the enrolment, keeping the device's own key and claim code. Called when the coordinator * says it has never heard of this node: it was revoked, or rebuilt from under us. Enrolment is @@ -473,22 +594,15 @@ static void store_set(const char *key, const char *value) */ static void store_forget_enrollment(void) { - static const char *keys[] = { - "streq_node_id", "streq_overlay_ip", "streq_hub_endpoint", - "streq_hub_pubkey", "streq_allowed_ips", "streq_network_subnet", - "streq_hub_ip", NULL + static const char *suffixes[] = { + "node_id", "overlay_ip", "hub_endpoint", "hub_pubkey", "allowed_ips", "network_subnet", + "hub_ip", NULL }; int i; - for (i = 0; keys[i]; i++) { - if (store_is_uboot()) { - char *argv[] = { "fw_setenv", (char *)keys[i], NULL }; /* no value = unset */ - run(argv, NULL, 0); - } else { - store_set(keys[i], ""); - } - } - warn_("enrolment discarded; restart with a fresh STREQ_ENROLLMENT_TOKEN to join again"); + for (i = 0; suffixes[i]; i++) + store_unset_both(suffixes[i]); + warn_("enrolment discarded; restart with a fresh CERBERUS_ENROLLMENT_TOKEN to join again"); } /* ------------------------------------------------------------------- HTTPS */ @@ -597,7 +711,7 @@ static int http_post_json(const char *url, const char *body, char *resp, size_t if (g_cfg.insecure) { /* Dev only. Loud, because an enrollment token in the clear is a device someone else * can enroll. */ - warn_("TLS verification disabled (STREQ_INSECURE=1)"); + warn_("TLS verification disabled (CERBERUS_INSECURE=1)"); mbedtls_ssl_conf_authmode(&c.conf, MBEDTLS_SSL_VERIFY_NONE); } else { struct stat st; @@ -607,7 +721,7 @@ static int http_post_json(const char *url, const char *body, char *resp, size_t ? mbedtls_x509_crt_parse_path(&c.ca, g_cfg.ca) : mbedtls_x509_crt_parse_file(&c.ca, g_cfg.ca); if (loaded < 0) - die("no usable CA trust store at %s (set STREQ_CA)", g_cfg.ca); + die("no usable CA trust store at %s (set CERBERUS_CA)", g_cfg.ca); mbedtls_ssl_conf_authmode(&c.conf, MBEDTLS_SSL_VERIFY_REQUIRED); mbedtls_ssl_conf_ca_chain(&c.conf, &c.ca, NULL); } @@ -701,12 +815,12 @@ static int json_str(const char *json, const char *key, char *out, size_t out_sz) * * Not from a file on the rootfs, and that is the point. On an OpenIPC camera /etc is squashfs (or an * overlay on a partition a firmware update replaces), so a key kept there dies with the next reflash - * — while `streq_node_id` next to it in the U-Boot environment survives. The node would come back + * — while `cerberus_node_id` next to it in the U-Boot environment survives. The node would come back * with an identity the hub has a peer for and no key to prove it, and enrolment is skipped whenever a * node id is stored: not "re-enrollable", *stuck*. Key and node id have to share a lifetime, so they * share a store. * - * An older key sitting in STREQ_KEYFILE is migrated in on sight, once. It is left on disk rather than + * An older key sitting in CERBERUS_KEYFILE is migrated in on sight, once. It is left on disk rather than * deleted: the reflash that motivates all of this will take it anyway, and a delete racing a failed * fw_setenv would be the one unrecoverable outcome here. */ @@ -717,9 +831,9 @@ static void device_key(char priv_b64[64], char pub_b64[64]) size_t olen = 0; FILE *f; - if (store_get("streq_device_key", stored, sizeof stored) && stored[0]) { + if (store_get("cerberus_device_key", stored, sizeof stored) && stored[0]) { if (b64_decode(stored, priv, sizeof priv, &olen) != 0 || olen != 32) - die("streq_device_key is not a base64 WireGuard key"); + die("cerberus_device_key is not a base64 WireGuard key"); b64(priv, sizeof priv, priv_b64, 64); /* re-encode: normalises whatever was stored */ } else if ((f = fopen(g_cfg.keyfile, "r")) != NULL) { int ok = fgets(line, sizeof line, f) != NULL; @@ -728,14 +842,14 @@ static void device_key(char priv_b64[64], char pub_b64[64]) if (!ok || b64_decode(line, priv, sizeof priv, &olen) != 0 || olen != 32) die("%s is not a base64 WireGuard key", g_cfg.keyfile); b64(priv, sizeof priv, priv_b64, 64); - store_set("streq_device_key", priv_b64); + store_set("cerberus_device_key", priv_b64); note("migrated the device key from %s into the store — it now survives a firmware update", g_cfg.keyfile); } else { if (mbedtls_ctr_drbg_random(&g_drbg, priv, sizeof priv) != 0) die("rng failed"); wg_clamp(priv); b64(priv, sizeof priv, priv_b64, 64); - store_set("streq_device_key", priv_b64); + store_set("cerberus_device_key", priv_b64); note("generated a device key into the %s store", store_is_uboot() ? "u-boot env" : store_file()); } @@ -755,7 +869,7 @@ static int key_materialise(char *path, size_t path_sz) int fd; device_key(priv_b64, pub_b64); - snprintf(path, path_sz, "%s", env_or("STREQ_KEY_TMP", "/tmp/.streq-wg.key")); + snprintf(path, path_sz, "%s", cfg_env("KEY_TMP", "/tmp/.cerberus-wg.key")); fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600); if (fd < 0) { @@ -775,9 +889,9 @@ static int key_materialise(char *path, size_t path_sz) static void claim_code_get(char *code, size_t code_sz) { char fresh[13]; - if (store_get("streq_claim_code", code, code_sz) && code[0]) return; + if (store_get("cerberus_claim_code", code, code_sz) && code[0]) return; claim_code_new(fresh); - store_set("streq_claim_code", fresh); + store_set("cerberus_claim_code", fresh); snprintf(code, code_sz, "%s", fresh); } @@ -792,14 +906,14 @@ static int enroll(void) char node[128], ip[64], ep[256], hubkey[64], allowed[80], subnet[64], hubip[64]; int status = 0; - if (store_get("streq_node_id", node, sizeof node)) { - store_get("streq_overlay_ip", ip, sizeof ip); + if (store_get("cerberus_node_id", node, sizeof node)) { + store_get("cerberus_overlay_ip", ip, sizeof ip); note("already enrolled as %s (overlay %s)", node, ip[0] ? ip : "?"); return 0; } if (!*g_cfg.token) - die("not enrolled and no STREQ_ENROLLMENT_TOKEN set"); + die("not enrolled and no CERBERUS_ENROLLMENT_TOKEN set"); device_key(priv_b64, pub_b64); claim_code_get(code, sizeof code); @@ -839,19 +953,19 @@ static int enroll(void) snprintf(allowed, sizeof allowed, "%s/32", ip); /* Подсеть своей сети и адрес хаба. Их может не быть — координатор старше арендаторов; тогда - * работает STREQ_SUBNET, как и раньше. */ + * работает CERBERUS_SUBNET, как и раньше. */ if (!json_str(resp, "networkSubnet", subnet, sizeof subnet)) subnet[0] = 0; if (!json_str(resp, "hubOverlayIp", hubip, sizeof hubip)) hubip[0] = 0; /* Persist before touching the interface: a camera that got a peer on the hub and then lost the * material would re-enroll on the next boot with a token already spent. */ - store_set("streq_node_id", node); - store_set("streq_overlay_ip", ip); - store_set("streq_hub_endpoint", ep); - store_set("streq_hub_pubkey", hubkey); - store_set("streq_allowed_ips", allowed); - if (subnet[0]) store_set("streq_network_subnet", subnet); - if (hubip[0]) store_set("streq_hub_ip", hubip); + store_set("cerberus_node_id", node); + store_set("cerberus_overlay_ip", ip); + store_set("cerberus_hub_endpoint", ep); + store_set("cerberus_hub_pubkey", hubkey); + store_set("cerberus_allowed_ips", allowed); + if (subnet[0]) store_set("cerberus_network_subnet", subnet); + if (hubip[0]) store_set("cerberus_hub_ip", hubip); note("enrolled as %s, overlay %s/%s via hub %s", node, ip, subnet[0] ? subnet : "?", ep); note("claim code: %.4s-%.4s-%.4s — enter it in the panel to bind this camera to an account", @@ -876,16 +990,16 @@ static int prefix_of(const char *cidr) * it, or the camera keeps answering on an address the coordinator has already handed to somebody else. */ /* - * Подсеть, в которой стоит этот узел: из стора, если координатор её назвал, иначе STREQ_SUBNET. + * Подсеть, в которой стоит этот узел: из стора, если координатор её назвал, иначе CERBERUS_SUBNET. * * With one network per instance these are the same string. With tenants they are not: the node's - * own /24 is what it may claim on the interface, and STREQ_SUBNET is a guess made by whoever wrote + * own /24 is what it may claim on the interface, and CERBERUS_SUBNET is a guess made by whoever wrote * the u-boot environment — usually the instance-wide pool, which belongs to everybody. */ static const char *node_subnet(void) { static char subnet[64]; - if (store_get("streq_network_subnet", subnet, sizeof subnet) && subnet[0]) + if (store_get("cerberus_network_subnet", subnet, sizeof subnet) && subnet[0]) return subnet; return g_cfg.subnet; } @@ -894,7 +1008,7 @@ static const char *node_subnet(void) static const char *hub_overlay_ip(void) { static char hubip[64]; - if (store_get("streq_hub_ip", hubip, sizeof hubip) && hubip[0]) + if (store_get("cerberus_hub_ip", hubip, sizeof hubip) && hubip[0]) return hubip; return ""; } @@ -905,9 +1019,9 @@ static int link_apply(int flush) const char *subnet, *hubip; int rc; - if (!store_get("streq_overlay_ip", ip, sizeof ip) || - !store_get("streq_hub_endpoint", ep, sizeof ep) || - !store_get("streq_hub_pubkey", hubkey, sizeof hubkey)) { + if (!store_get("cerberus_overlay_ip", ip, sizeof ip) || + !store_get("cerberus_hub_endpoint", ep, sizeof ep) || + !store_get("cerberus_hub_pubkey", hubkey, sizeof hubkey)) { warn_("no stored join material — run `cerberus enroll` first"); return 1; } @@ -988,7 +1102,7 @@ static long handshake_age(void) /* * Кого пинговать внутри туннеля. Хаб: единственный пир камеры и первый адрес подсети (10.80.0.0/16 → * 10.80.0.1, ровно так его поднимает хаб). Материал вступления его не называет — - * узлу он больше ни для чего не нужен — поэтому адрес выводится из подсети, а STREQ_PING_TARGET + * узлу он больше ни для чего не нужен — поэтому адрес выводится из подсети, а CERBERUS_PING_TARGET * перекрывает вывод там, где хаб стоит не на первом адресе. */ static const char *ping_target(void) @@ -1091,35 +1205,35 @@ static void apply_directive(const char *json) int rotated = 0; if (json_str(json, "hubEndpoint", ep, sizeof ep) && - (!store_get("streq_hub_endpoint", cur, sizeof cur) || strcmp(cur, ep) != 0)) { + (!store_get("cerberus_hub_endpoint", cur, sizeof cur) || strcmp(cur, ep) != 0)) { note("hub endpoint rotated: %s -> %s", cur[0] ? cur : "(none)", ep); - store_set("streq_hub_endpoint", ep); + store_set("cerberus_hub_endpoint", ep); rotated = 1; } if (json_str(json, "hubPublicKey", hubkey, sizeof hubkey) && - (!store_get("streq_hub_pubkey", cur, sizeof cur) || strcmp(cur, hubkey) != 0)) { + (!store_get("cerberus_hub_pubkey", cur, sizeof cur) || strcmp(cur, hubkey) != 0)) { note("hub public key rotated"); - store_set("streq_hub_pubkey", hubkey); + store_set("cerberus_hub_pubkey", hubkey); rotated = 1; } if (json_str(json, "overlayIp", ip, sizeof ip) && - (!store_get("streq_overlay_ip", cur, sizeof cur) || strcmp(cur, ip) != 0)) { + (!store_get("cerberus_overlay_ip", cur, sizeof cur) || strcmp(cur, ip) != 0)) { note("overlay address rotated: %s -> %s", cur[0] ? cur : "(none)", ip); - store_set("streq_overlay_ip", ip); + store_set("cerberus_overlay_ip", ip); rotated = 1; } if (json_str(json, "allowedIps", allowed, sizeof allowed)) - store_set("streq_allowed_ips", allowed); + store_set("cerberus_allowed_ips", allowed); if (json_str(json, "networkSubnet", fresh, sizeof fresh) && fresh[0] && - (!store_get("streq_network_subnet", cur, sizeof cur) || strcmp(cur, fresh) != 0)) { + (!store_get("cerberus_network_subnet", cur, sizeof cur) || strcmp(cur, fresh) != 0)) { note("network subnet rotated: %s -> %s", cur[0] ? cur : "(none)", fresh); - store_set("streq_network_subnet", fresh); + store_set("cerberus_network_subnet", fresh); rotated = 1; } if (json_str(json, "hubOverlayIp", fresh, sizeof fresh) && fresh[0] && - (!store_get("streq_hub_ip", cur, sizeof cur) || strcmp(cur, fresh) != 0)) { + (!store_get("cerberus_hub_ip", cur, sizeof cur) || strcmp(cur, fresh) != 0)) { note("hub overlay address rotated: %s -> %s", cur[0] ? cur : "(none)", fresh); - store_set("streq_hub_ip", fresh); + store_set("cerberus_hub_ip", fresh); rotated = 1; } @@ -1174,10 +1288,10 @@ static int watch(void) catch_signals(); /* повторно — безвредно; watch() бывает и точкой входа отладки */ - if (!store_get("streq_hub_endpoint", ep, sizeof ep) || - !store_get("streq_hub_pubkey", hubkey, sizeof hubkey)) + if (!store_get("cerberus_hub_endpoint", ep, sizeof ep) || + !store_get("cerberus_hub_pubkey", hubkey, sizeof hubkey)) return 1; - store_get("streq_node_id", node, sizeof node); + store_get("cerberus_node_id", node, sizeof node); snprintf(beat_url, sizeof beat_url, "%s/nodes/%s/heartbeat", g_cfg.api, node); tick = g_cfg.watch_interval; @@ -1199,8 +1313,8 @@ static int watch(void) misses++; warn_("no handshake for %lds (miss %d) — re-resolving %s", age, misses, ep); /* Re-read: a directive may have rotated the hub since this loop started. */ - store_get("streq_hub_endpoint", ep, sizeof ep); - store_get("streq_hub_pubkey", hubkey, sizeof hubkey); + store_get("cerberus_hub_endpoint", ep, sizeof ep); + store_get("cerberus_hub_pubkey", hubkey, sizeof hubkey); if (*hub_overlay_ip()) runf("wg set %s peer %s endpoint %s allowed-ips %s,%s/32 persistent-keepalive 25", g_cfg.iface, hubkey, ep, node_subnet(), hub_overlay_ip()); @@ -1307,15 +1421,15 @@ static int cmd_status(void) char v[512]; long age; printf("store %s\n", store_is_uboot() ? "u-boot env (fw_setenv)" : store_file()); - printf("node %s\n", store_get("streq_node_id", v, sizeof v) ? v : "(not enrolled)"); - printf("device key %s\n", store_get("streq_device_key", v, sizeof v) ? "in the store" : "-"); - printf("overlay ip %s\n", store_get("streq_overlay_ip", v, sizeof v) ? v : "-"); - printf("hub endpoint %s\n", store_get("streq_hub_endpoint", v, sizeof v) ? v : "-"); - printf("hub public key %s\n", store_get("streq_hub_pubkey", v, sizeof v) ? v : "-"); + printf("node %s\n", store_get("cerberus_node_id", v, sizeof v) ? v : "(not enrolled)"); + printf("device key %s\n", store_get("cerberus_device_key", v, sizeof v) ? "in the store" : "-"); + printf("overlay ip %s\n", store_get("cerberus_overlay_ip", v, sizeof v) ? v : "-"); + printf("hub endpoint %s\n", store_get("cerberus_hub_endpoint", v, sizeof v) ? v : "-"); + printf("hub public key %s\n", store_get("cerberus_hub_pubkey", v, sizeof v) ? v : "-"); printf("network subnet %s%s\n", node_subnet(), - store_get("streq_network_subnet", v, sizeof v) ? "" : " (из STREQ_SUBNET)"); - printf("hub overlay ip %s\n", *hub_overlay_ip() ? hub_overlay_ip() : "- (внутри своей подсети)"); - printf("claim code %s\n", store_get("streq_claim_code", v, sizeof v) ? v : "-"); + store_get("cerberus_network_subnet", v, sizeof v) ? "" : " (from CERBERUS_SUBNET)"); + printf("hub overlay ip %s\n", *hub_overlay_ip() ? hub_overlay_ip() : "- (inside its own subnet)"); + printf("claim code %s\n", store_get("cerberus_claim_code", v, sizeof v) ? v : "-"); age = handshake_age(); if (age < 0) printf("handshake never\n"); else printf("handshake %lds ago\n", age); @@ -1350,12 +1464,11 @@ static int cmd_keygen(void) */ static int cmd_clean(void) { - static const char *keys[] = { - "streq_node_id", "streq_overlay_ip", "streq_hub_endpoint", "streq_hub_pubkey", - "streq_allowed_ips", "streq_network_subnet", "streq_hub_ip", "streq_claim_code", - "streq_device_key", "streq_api", "streq_enrollment_token", "streq_hostname", - "streq_kind", "streq_iface", "streq_subnet", "streq_ca", "streq_heartbeat", - "streq_ping_target", "streq_ping_interval", "streq_ping_timeout", "streq_ping_fails", + /* Both spellings of each: whatever an older install left under streq_* goes too. */ + static const char *suffixes[] = { + "node_id", "overlay_ip", "hub_endpoint", "hub_pubkey", "allowed_ips", "network_subnet", + "hub_ip", "claim_code", "device_key", "api", "enrollment_token", "hostname", "kind", "iface", + "subnet", "ca", "heartbeat", "ping_target", "ping_interval", "ping_timeout", "ping_fails", NULL }; char self[256]; @@ -1375,14 +1488,8 @@ static int cmd_clean(void) if (unlink(INIT_SCRIPT) != 0 && errno != ENOENT) warn_("cannot remove %s: %s", INIT_SCRIPT, strerror(errno)); - for (i = 0; keys[i]; i++) { - if (store_is_uboot()) { - char *argv[] = { "fw_setenv", (char *)keys[i], NULL }; /* без значения — сброс */ - run(argv, NULL, 0); - } else { - store_set(keys[i], ""); - } - } + for (i = 0; suffixes[i]; i++) + store_unset_both(suffixes[i]); if (!store_is_uboot()) unlink(store_file()); @@ -1402,31 +1509,32 @@ static int cmd_clean(void) static void usage(void) { fputs( - "cerberus " CERBERUS_VERSION " — on-camera Streq agent\n" + "cerberus " CERBERUS_VERSION " — on-camera cloud agent for OpenIPC\n" "\n" - "usage: cerberus [enroll|up|run|status|keygen]\n" + "usage: cerberus [enroll|up|run|status|keygen|clean]\n" " enroll enroll into the coordinator and persist the join material (idempotent)\n" " up program the WireGuard interface from the stored material\n" " run enroll + up + watch the tunnel (the default; what the init script calls)\n" " status print what is stored, how old the handshake is, whether the tunnel answers\n" " keygen print the device keypair (private, public), creating it if needed\n" + " clean remove the agent: interface, init script, stored material, the binary\n" "\n" - "config (env):\n" - " STREQ_API coordinator base URL incl. /v1 [https://api.localhost/v1]\n" - " STREQ_ENROLLMENT_TOKEN one-shot token from the owner\n" - " STREQ_KIND node kind [Camera]\n" - " STREQ_IFACE WireGuard interface [wg0]\n" - " STREQ_SUBNET overlay CIDR routed to the hub [10.80.0.0/16]\n" - " STREQ_CA CA file or dir for TLS [/etc/ssl/certs/ca-certificates.crt]\n" - " STREQ_INSECURE=1 skip TLS verification (dev only)\n" - " STREQ_STORE auto | uboot | file:/path [auto] — holds the device key too\n" - " STREQ_KEYFILE legacy key file, migrated into the store [/etc/streq/wg.key]\n" - " STREQ_HEARTBEAT seconds between heartbeats; 0 disables [20]\n" - " STREQ_PING_TARGET in-tunnel probe target [first address of STREQ_SUBNET]\n" - " STREQ_PING_INTERVAL seconds between probes; 0 disables [60]\n" - " STREQ_PING_TIMEOUT seconds one probe waits [3]\n" - " STREQ_PING_FAILS failed probes before the interface is rebuilt [3]\n" - " STREQ_DRY_RUN=1 print ip/wg commands instead of running them\n", + "config (env; the old STREQ_* names are accepted too):\n" + " CERBERUS_API coordinator base URL incl. /v1 [https://api.localhost/v1]\n" + " CERBERUS_ENROLLMENT_TOKEN one-shot token from the owner\n" + " CERBERUS_KIND node kind [Camera]\n" + " CERBERUS_IFACE WireGuard interface [wg0]\n" + " CERBERUS_SUBNET overlay CIDR routed to the hub [10.80.0.0/16]\n" + " CERBERUS_CA CA file or dir for TLS [/etc/ssl/certs/ca-certificates.crt]\n" + " CERBERUS_INSECURE=1 skip TLS verification (dev only)\n" + " CERBERUS_STORE auto | uboot | file:/path [auto] — holds the device key too\n" + " CERBERUS_KEYFILE legacy key file, migrated into the store [/etc/streq/wg.key]\n" + " CERBERUS_HEARTBEAT seconds between heartbeats; 0 disables [20]\n" + " CERBERUS_PING_TARGET in-tunnel probe target [the hub]\n" + " CERBERUS_PING_INTERVAL seconds between probes; 0 disables [60]\n" + " CERBERUS_PING_TIMEOUT seconds one probe waits [3]\n" + " CERBERUS_PING_FAILS failed probes before the interface is rebuilt [3]\n" + " CERBERUS_DRY_RUN=1 print ip/wg commands instead of running them\n", stderr); } @@ -1442,6 +1550,10 @@ int main(int argc, char **argv) cfg_load(); crypto_init(); + /* Before anything reads the store: `clean` removes both spellings itself. */ + if (strcmp(cmd, "clean") != 0) + store_migrate_legacy(); + if (!strcmp(cmd, "status")) return cmd_status(); if (!strcmp(cmd, "keygen")) return cmd_keygen(); if (!strcmp(cmd, "clean")) return cmd_clean(); diff --git a/tests/suite.sh b/tests/suite.sh index f71dd2d..08bdaf7 100755 --- a/tests/suite.sh +++ b/tests/suite.sh @@ -4,7 +4,7 @@ # without needing a cloud — identity, TLS verification, enrolment, the heartbeat back-channel, # supervision — and then, where the kernel allows, a real WireGuard tunnel to a hub in its own netns. # -# Most of it runs with STREQ_DRY_RUN=1: the agent prints the `ip`/`wg` commands it would run, and the +# Most of it runs with CERBERUS_DRY_RUN=1: the agent prints the `ip`/`wg` commands it would run, and the # suite checks those. The data-plane section at the end runs them for real. # # Env: REQUIRE_WG=1 fail (rather than skip) when kernel WireGuard or netns are unavailable — CI. @@ -31,8 +31,9 @@ dump() { sed 's/^/ | /' "$1"; } # ------------------------------------------------------------------------------- toolchain echo "==> installing the toolchain" if command -v apk >/dev/null 2>&1; then + # openrc only for start-stop-daemon, which S99cerberus calls: BusyBox on a camera has it, Alpine's does not. apk add --no-cache gcc musl-dev make mbedtls-dev mbedtls-static wireguard-tools iproute2 \ - openssl python3 curl >/dev/null 2>&1 || { echo "apk add failed"; exit 1; } + openssl python3 curl openrc >/dev/null 2>&1 || { echo "apk add failed"; exit 1; } LIBC=musl else export DEBIAN_FRONTEND=noninteractive @@ -78,6 +79,37 @@ openssl x509 -req -in srv.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 2 \ openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes -days 2 \ -subj /CN=rogue-ca -keyout rogue.key -out rogue.crt >/dev/null 2>&1 +# A U-Boot environment in a file, behind the same two commands a camera has. `fw_setenv k` with +# no value removes k, as the real one does; every write is counted, because on a camera each one +# is a flash erase. +mkdir -p $T/fw +cat >$T/fw/fw_printenv <<'EOF' +#!/bin/sh +f=${FAKE_UBOOT:?} +touch "$f" +if [ "${1:-}" = -n ]; then + awk -v k="$2" 'index($0, k "=") == 1 { print substr($0, length(k) + 2); found = 1 } END { exit !found }' "$f" \ + || { echo "## Error: \"$2\" not defined" >&2; exit 1; } + exit 0 +fi +cat "$f" +EOF +cat >$T/fw/fw_setenv <<'EOF' +#!/bin/sh +f=${FAKE_UBOOT:?} +touch "$f" +k=$1 +shift +awk -v k="$k" 'index($0, k "=") != 1' "$f" >"$f.tmp" +[ $# -gt 0 ] && printf '%s=%s\n' "$k" "$*" >>"$f.tmp" +mv "$f.tmp" "$f" +echo "$k" >>"$f.writes" +EOF +chmod +x $T/fw/fw_printenv $T/fw/fw_setenv +uboot() { _u=$1; shift; env PATH="$T/fw:$PATH" FAKE_UBOOT="$_u" CERBERUS_STORE=uboot "$@"; } +ub() { FAKE_UBOOT="$1" $T/fw/fw_printenv -n "$2" 2>/dev/null; } +writes() { if [ -f "$1.writes" ]; then wc -l <"$1.writes" | tr -d ' '; else echo 0; fi; } + wg genkey >hub.key HUB_PUB=$(wg pubkey coordinator.log 2>&1 & COORD=$! trap 'kill $COORD 2>/dev/null' EXIT @@ -102,15 +134,15 @@ for _ in 1 2 3 4 5 6 7 8 9 10; do c -o /dev/null "$URL/_test/requests" && break; c -o /dev/null "$URL/_test/requests" || { echo "the fake coordinator never came up"; dump coordinator.log; exit 1; } A=/build/cerberus -export STREQ_API=$URL/v1 -export STREQ_CA=$T/ca.crt -export STREQ_STORE=file:$T/agent.env -export STREQ_KEYFILE=$T/no-legacy-key -export STREQ_KEY_TMP=$T/.wg.key -export STREQ_HOSTNAME=cerberus-test -export STREQ_IFACE=cbr0 -export STREQ_DRY_RUN=1 -export STREQ_PING_INTERVAL=0 +export CERBERUS_API=$URL/v1 +export CERBERUS_CA=$T/ca.crt +export CERBERUS_STORE=file:$T/agent.env +export CERBERUS_KEYFILE=$T/no-legacy-key +export CERBERUS_KEY_TMP=$T/.wg.key +export CERBERUS_HOSTNAME=cerberus-test +export CERBERUS_IFACE=cbr0 +export CERBERUS_DRY_RUN=1 +export CERBERUS_PING_INTERVAL=0 # ------------------------------------------------------------------------- identity section "device identity" @@ -118,28 +150,28 @@ $A keygen >keys 2>keygen.log PRIV=$(sed -n 1p keys) PUB=$(sed -n 2p keys) check "X25519 pubkey matches \`wg pubkey\`" "$(printf '%s\n' "$PRIV" | wg pubkey)" "$PUB" -check "the private key lives in the store" "$(sv streq_device_key agent.env)" "$PRIV" +check "the private key lives in the store" "$(sv cerberus_device_key agent.env)" "$PRIV" check "and nowhere on the rootfs" "$(yes_if test -e $T/no-legacy-key)" "no" check "the store is 0600" "$(stat -c %a agent.env)" "600" check "the key is stable across runs" "$($A keygen 2>/dev/null | sed -n 2p)" "$PUB" wg genkey >legacy.key -STREQ_STORE=file:$T/legacy.env STREQ_KEYFILE=$T/legacy.key $A keygen >legacy.keys 2>legacy.log -check "a legacy key file is migrated in" "$(sv streq_device_key legacy.env)" "$(cat legacy.key)" +CERBERUS_STORE=file:$T/legacy.env CERBERUS_KEYFILE=$T/legacy.key $A keygen >legacy.keys 2>legacy.log +check "a legacy key file is migrated in" "$(sv cerberus_device_key legacy.env)" "$(cat legacy.key)" check "unchanged" "$(sed -n 2p legacy.keys)" "$(wg pubkey broken.env -STREQ_STORE=file:$T/broken.env $A keygen >/dev/null 2>broken.log +printf 'cerberus_device_key=not-a-key\n' >broken.env +CERBERUS_STORE=file:$T/broken.env $A keygen >/dev/null 2>broken.log check "a corrupt stored key is refused" "$(yes_if grep -q 'not a base64 WireGuard key' broken.log)" "yes" # ------------------------------------------------------------------------------- TLS section "TLS (mbedTLS $MBEDTLS_VERSION)" -STREQ_STORE=file:$T/rogue.env STREQ_CA=$T/rogue.crt STREQ_ENROLLMENT_TOKEN=$TOKEN $A enroll >rogue.log 2>&1 +CERBERUS_STORE=file:$T/rogue.env CERBERUS_CA=$T/rogue.crt CERBERUS_ENROLLMENT_TOKEN=$TOKEN $A enroll >rogue.log 2>&1 check "an unrelated CA is rejected" "$(yes_if grep -q 'TLS handshake .* failed' rogue.log)" "yes" -STREQ_STORE=file:$T/sni.env STREQ_API=https://127.0.0.1:8443/v1 STREQ_ENROLLMENT_TOKEN=$TOKEN $A enroll >sni.log 2>&1 +CERBERUS_STORE=file:$T/sni.env CERBERUS_API=https://127.0.0.1:8443/v1 CERBERUS_ENROLLMENT_TOKEN=$TOKEN $A enroll >sni.log 2>&1 check "a certificate for another name is rejected" "$(yes_if grep -q 'TLS handshake .* failed' sni.log)" "yes" check "neither reached the coordinator" "$(q 'print(len(R))')" "0" -STREQ_STORE=file:$T/noca.env STREQ_CA=$T/missing.crt STREQ_ENROLLMENT_TOKEN=$TOKEN $A enroll >noca.log 2>&1 +CERBERUS_STORE=file:$T/noca.env CERBERUS_CA=$T/missing.crt CERBERUS_ENROLLMENT_TOKEN=$TOKEN $A enroll >noca.log 2>&1 check "no trust store is fatal, and says so" "$(yes_if grep -q 'no usable CA trust store' noca.log)" "yes" check "fatal errors point at the docs" "$(yes_if grep -q 'cerberus: https://' noca.log)" "yes" @@ -147,16 +179,16 @@ check "fatal errors point at the docs" "$(yes_if grep -q 'cerberus: https://' no section "enrolment" $A enroll >notoken.log 2>&1 check "no token: refuses, non-zero" "$?" "1" -check "and says why" "$(yes_if grep -q 'no STREQ_ENROLLMENT_TOKEN' notoken.log)" "yes" -STREQ_ENROLLMENT_TOKEN=tok_wrong $A enroll >badtoken.log 2>&1 +check "and says why" "$(yes_if grep -q 'no CERBERUS_ENROLLMENT_TOKEN' notoken.log)" "yes" +CERBERUS_ENROLLMENT_TOKEN=tok_wrong $A enroll >badtoken.log 2>&1 check "a wrong token is reported, non-zero" "$?" "1" check "with the coordinator's status" "$(yes_if grep -q 'enroll rejected: HTTP 401' badtoken.log)" "yes" -STREQ_ENROLLMENT_TOKEN=$TOKEN $A enroll >enroll.log 2>&1 +CERBERUS_ENROLLMENT_TOKEN=$TOKEN $A enroll >enroll.log 2>&1 check "enrolls" "$?" "0" check "over TLS" "$(yes_if grep -q '^cerberus: TLS ok' enroll.log)" "yes" -NODE=$(sv streq_node_id agent.env) -CLAIM=$(sv streq_claim_code agent.env) +NODE=$(sv cerberus_node_id agent.env) +CLAIM=$(sv cerberus_claim_code agent.env) check "node id stored" "$(echo "$NODE" | grep -c '^node_[0-9a-f]\{32\}$')" "1" check "claim code: 12 chars of the unambiguous alphabet" \ "$(yes_if sh -c "echo '$CLAIM' | grep -Eq '^[23456789ABCDEFGHJKMNPQRSTVWXYZ]{12}$'")" "yes" @@ -172,20 +204,20 @@ check "hostname" "$(q "$E; print(B['hostname'])")" "cerberus-test" check "agentVersion" "$(q "$E; print(B['agentVersion'])")" "$SRC_VERSION" check "User-Agent" "$(q "$E; print(E['userAgent'])")" "cerberus/$SRC_VERSION" -check "overlay ip stored" "$(sv streq_overlay_ip agent.env)" "$OVERLAY" -check "hub endpoint stored" "$(sv streq_hub_endpoint agent.env)" "$HUB_EP" -check "hub public key stored" "$(sv streq_hub_pubkey agent.env)" "$HUB_PUB" -check "network subnet stored" "$(sv streq_network_subnet agent.env)" "$SUBNET" -check "hub overlay ip stored" "$(sv streq_hub_ip agent.env)" "$HUB_IP" +check "overlay ip stored" "$(sv cerberus_overlay_ip agent.env)" "$OVERLAY" +check "hub endpoint stored" "$(sv cerberus_hub_endpoint agent.env)" "$HUB_EP" +check "hub public key stored" "$(sv cerberus_hub_pubkey agent.env)" "$HUB_PUB" +check "network subnet stored" "$(sv cerberus_network_subnet agent.env)" "$SUBNET" +check "hub overlay ip stored" "$(sv cerberus_hub_ip agent.env)" "$HUB_IP" check "store still 0600" "$(stat -c %a agent.env)" "600" BEFORE=$(q 'print(len(R))') -STREQ_ENROLLMENT_TOKEN=$TOKEN $A enroll >again.log 2>&1 +CERBERUS_ENROLLMENT_TOKEN=$TOKEN $A enroll >again.log 2>&1 check "a second enroll is a no-op" "$(yes_if grep -q 'already enrolled' again.log)" "yes" check "and spends nothing" "$(q 'print(len(R))')" "$BEFORE" cp agent.env enrolled.env -STREQ_ENROLLMENT_TOKEN=$TOKEN $A status >status.out 2>/dev/null +CERBERUS_ENROLLMENT_TOKEN=$TOKEN $A status >status.out 2>/dev/null check "status names the node" "$(yes_if grep -q "^node *$NODE" status.out)" "yes" check "status shows the claim code" "$(yes_if grep -q "^claim code *$CLAIM" status.out)" "yes" @@ -203,7 +235,7 @@ check "the key file does not outlive the call" "$(yes_if test -e $T/.wg.key)" "n section "heartbeat and the back-channel" runfor() { # runfor [env...] — `cerberus run` for a while, then SIGTERM _t=$1; _log=$2; shift 2 - env STREQ_HEARTBEAT=5 "$@" timeout "$_t" $A run >"$_log" 2>&1 + env CERBERUS_HEARTBEAT=5 "$@" timeout "$_t" $A run >"$_log" 2>&1 } cp enrolled.env agent.env @@ -234,13 +266,13 @@ check "an unknown command is ignored, loudly" "$(yes_if grep -q "unknown command ctl join '{"hubEndpoint":"192.0.2.9:51820"}' runfor 7 rot.log check "a moved hub is noticed" "$(yes_if grep -q "hub endpoint rotated: $HUB_EP -> 192.0.2.9:51820" rot.log)" "yes" -check "persisted" "$(sv streq_hub_endpoint agent.env)" "192.0.2.9:51820" +check "persisted" "$(sv cerberus_hub_endpoint agent.env)" "192.0.2.9:51820" check "and the interface reprogrammed" "$(yes_if grep -q 'flush dev cbr0' rot.log)" "yes" NEWHUB=$(wg genkey | wg pubkey) ctl join "{\"hubEndpoint\":\"$HUB_EP\",\"hubPublicKey\":\"$NEWHUB\",\"overlayIp\":\"10.80.7.9\"}" runfor 7 rekey.log check "a re-keyed hub is noticed" "$(yes_if grep -q 'hub public key rotated' rekey.log)" "yes" -check "persisted" "$(sv streq_hub_pubkey agent.env)" "$NEWHUB" +check "persisted" "$(sv cerberus_hub_pubkey agent.env)" "$NEWHUB" check "a new overlay address replaces the old" "$(yes_if grep -qF 'ip address replace 10.80.7.9/24 dev cbr0' rekey.log)" "yes" ctl join "{\"hubPublicKey\":\"$HUB_PUB\",\"overlayIp\":\"$OVERLAY\"}" cp enrolled.env agent.env @@ -254,13 +286,13 @@ check "and survived" "$(yes_if grep -q 'heartbeat recovered' outage.log)" "yes" # ---------------------------------------------------------------- waiting it out section "an unreachable coordinator" W0=$(date +%s) -STREQ_STORE=file:$T/wait.env STREQ_API=https://127.0.0.1:9/v1 STREQ_ENROLLMENT_TOKEN=$TOKEN \ - STREQ_RETRY_MIN=2 timeout 9 $A run >wait.log 2>&1 +CERBERUS_STORE=file:$T/wait.env CERBERUS_API=https://127.0.0.1:9/v1 CERBERUS_ENROLLMENT_TOKEN=$TOKEN \ + CERBERUS_RETRY_MIN=2 timeout 9 $A run >wait.log 2>&1 check "is waited out, not fatal" "$([ $(( $(date +%s) - W0 )) -ge 8 ] && echo yes || echo no)" "yes" check "with a backoff between tries" "$(yes_if grep -q 'retrying in 4s' wait.log)" "yes" -STREQ_STORE=file:$T/wait2.env STREQ_API=https://127.0.0.1:9/v1 STREQ_ENROLLMENT_TOKEN=$TOKEN \ - STREQ_RETRY_MIN=30 $A run >wait2.log 2>&1 & +CERBERUS_STORE=file:$T/wait2.env CERBERUS_API=https://127.0.0.1:9/v1 CERBERUS_ENROLLMENT_TOKEN=$TOKEN \ + CERBERUS_RETRY_MIN=30 $A run >wait2.log 2>&1 & WPID=$! sleep 3 S0=$(date +%s) @@ -288,7 +320,7 @@ if wg_ok; then ip -n hub route add $SUBNET dev wg-hub cp enrolled.env agent.env - export STREQ_DRY_RUN=0 STREQ_IFACE=wg0 + export CERBERUS_DRY_RUN=0 CERBERUS_IFACE=wg0 $A up >dp-up.log 2>&1 check "wg0 comes up with the overlay address" \ "$(yes_if sh -c "ip -4 -o addr show wg0 | grep -q ' $OVERLAY/24 '")" "yes" @@ -303,13 +335,13 @@ if wg_ok; then check "handshake with the hub" "$hs" "yes" check "ping $HUB_IP through the tunnel" "$(yes_if ping -c3 -W2 $HUB_IP)" "yes" - STREQ_PING_INTERVAL=60 $A status >dp-status.out 2>/dev/null + CERBERUS_PING_INTERVAL=60 $A status >dp-status.out 2>/dev/null check "status reads the handshake back" "$(yes_if grep -Eq '^handshake +[0-9]+s ago' dp-status.out)" "yes" check "and probes the tunnel itself" "$(yes_if grep -q "^tunnel ping ok ($HUB_IP)" dp-status.out)" "yes" # A tunnel that is up on paper and dead in fact: the handshake stays fresh, the route is gone. # Only the in-tunnel probe can see it, and the agent has to repair it by itself. - env STREQ_HEARTBEAT=0 STREQ_PING_INTERVAL=10 STREQ_PING_FAILS=1 timeout 26 $A run >dp-run.log 2>&1 & + env CERBERUS_HEARTBEAT=0 CERBERUS_PING_INTERVAL=10 CERBERUS_PING_FAILS=1 timeout 26 $A run >dp-run.log 2>&1 & RPID=$! sleep 3 ip route del $HUB_IP/32 dev wg0 @@ -320,7 +352,7 @@ if wg_ok; then check "route restored" "$(yes_if sh -c "ip route show $HUB_IP | grep -q 'dev wg0'")" "yes" ip link del wg0 2>/dev/null - export STREQ_DRY_RUN=1 STREQ_IFACE=cbr0 + export CERBERUS_DRY_RUN=1 CERBERUS_IFACE=cbr0 else if [ "${REQUIRE_WG:-}" = 1 ]; then check "kernel WireGuard and netns available" "no" "yes" @@ -329,6 +361,101 @@ else fi fi +# -------------------------------------------------------------- the old names +section "the rename: streq_* keys and STREQ_* variables" +# A camera installed before the rename: its whole identity under the old names, in U-Boot env. +OLD=$T/old-uboot.env +sed -n 's/^cerberus_\(.*\)=\(.*\)$/streq_\1=\2/p' enrolled.env >$OLD +uboot $OLD $A status >mig.out 2>mig.log +check "moves its identity to cerberus_*" "$(ub $OLD cerberus_device_key)" "$PRIV" +check "and its enrolment" "$(ub $OLD cerberus_node_id)" "$NODE" +check "every agent-written key" "$(yes_if grep -q 'moved 9 key(s)' mig.log)" "yes" +check "keeps the streq_* copies for an older init script" "$(ub $OLD streq_node_id)" "$NODE" +check "the camera is the same camera" "$(uboot $OLD $A keygen 2>/dev/null | sed -n 2p)" "$PUB" +check "and knows it is enrolled" "$(yes_if grep -q "^node *$NODE" mig.out)" "yes" +W=$(writes $OLD) +uboot $OLD $A status >/dev/null 2>&1 +uboot $OLD $A status >/dev/null 2>&1 +check "after that, starts write nothing to flash" "$(writes $OLD)" "$W" + +OLDF=$T/old-file.env +sed 's/^cerberus_/streq_/' enrolled.env >$OLDF +CERBERUS_STORE=file:$OLDF $A status >/dev/null 2>&1 +check "a file store is moved the same way" "$(sv cerberus_node_id $OLDF)" "$NODE" + +check "STREQ_* variables are still read" \ + "$( (unset CERBERUS_STORE; STREQ_STORE=file:$T/old-var.env $A keygen >/dev/null 2>&1); yes_if test -s $T/old-var.env)" "yes" +check "CERBERUS_* wins over STREQ_*" \ + "$( (STREQ_STORE=file:$T/loser.env $A keygen >/dev/null 2>&1); yes_if test -e $T/loser.env)" "no" + +# The init script reads the new name, and the old one when the new one is not there. +SUB=$T/s99.env +printf 'streq_subnet=10.99.0.0/16\n' >$SUB +mkdir -p /usr/sbin && cp $A /usr/sbin/cerberus +check "S99cerberus falls back to streq_* keys" \ + "$(env PATH="$T/fw:$PATH" FAKE_UBOOT=$SUB CERBERUS_STORE=file:$T/nothing.env sh /src/S99cerberus status 2>/dev/null | sed -n 's/^network subnet *//p')" \ + "10.99.0.0/16 (from CERBERUS_SUBNET)" +printf 'cerberus_subnet=10.98.0.0/16\n' >>$SUB +check "and prefers cerberus_*" \ + "$(env PATH="$T/fw:$PATH" FAKE_UBOOT=$SUB CERBERUS_STORE=file:$T/nothing.env sh /src/S99cerberus status 2>/dev/null | sed -n 's/^network subnet *//p')" \ + "10.98.0.0/16 (from CERBERUS_SUBNET)" +check "unprovisioned: start says so and exits 0" \ + "$(env PATH="$T/fw:$PATH" FAKE_UBOOT=$T/empty.env sh /src/S99cerberus start 2>&1; echo "rc=$?")" \ + "cerberus: not provisioned (fw_setenv cerberus_api ...) +rc=0" + +# ------------------------------------------------------------------ the installer +section "the installer, from a release laid out on disk" +# What ci/assemble.sh publishes, with this machine's build standing in for the release binary. +REL=$T/release +mkdir -p $REL +ARCH=$(uname -m) +cp $A $REL/cerberus-$ARCH +cp /src/install.sh /src/uninstall.sh /src/S99cerberus $REL/ +(cd $REL && for f in cerberus-$ARCH install.sh uninstall.sh S99cerberus; do + printf '%s %s\n' "$(sha256sum $f | cut -d' ' -f1)" "$f"; done >SHA256SUMS) +rm -f /usr/sbin/cerberus /etc/init.d/S99cerberus +mkdir -p /etc/init.d + +INST=$T/inst.env +printf 'cerberus_ca=%s\n' "$T/ca.crt" >$INST +inst() { env PATH="$T/fw:$PATH" FAKE_UBOOT=$INST CERBERUS_DIST=file://$REL CERBERUS_API=$URL/v1 \ + CERBERUS_STORE= CERBERUS_IFACE= sh /src/install.sh "$@"; } + +inst >inst-notoken.log 2>&1 +check "no token: installs, exits 0" "$?" "0" +check "picks the build for this CPU" "$(cmp -s /usr/sbin/cerberus $REL/cerberus-$ARCH && echo yes)" "yes" +check "verifies it against SHA256SUMS" "$(yes_if grep -q "cerberus-$ARCH sha256 ok" inst-notoken.log)" "yes" +check "installs the init script" "$(yes_if test -x /etc/init.d/S99cerberus)" "yes" +check "and says what to set, by the new names" "$(yes_if grep -q 'fw_setenv cerberus_api' inst-notoken.log)" "yes" + +cp $REL/SHA256SUMS $T/sums.good +sed -i "s/^[0-9a-f]* cerberus-$ARCH\$/$(printf '0%.0s' $(seq 64)) cerberus-$ARCH/" $REL/SHA256SUMS +inst >inst-bad.log 2>&1 +check "a binary that does not match its sum is refused" "$?" "15" +cp $T/sums.good $REL/SHA256SUMS + +# A claim code exists before the camera ever reaches the cloud — its hash is what enrolment sends — +# so seeing one proves nothing. The installer has to wait for the enrolment itself, and say E21 +# when it does not come. +env PATH="$T/fw:$PATH" FAKE_UBOOT=$INST CERBERUS_DIST=file://$REL CERBERUS_API=https://127.0.0.1:9/v1 \ + CERBERUS_STORE= CERBERUS_IFACE= sh /src/install.sh "$TOKEN" >inst-noreach.log 2>&1 +check "an unreachable cloud is reported as E21" "$(yes_if grep -q 'cerberus: E21' inst-noreach.log)" "yes" +check "not as a claim code" "$(yes_if grep -q 'cerberus: claim' inst-noreach.log)" "no" +env PATH="$T/fw:$PATH" FAKE_UBOOT=$INST sh /etc/init.d/S99cerberus stop >/dev/null 2>&1 +printf 'cerberus_ca=%s\n' "$T/ca.crt" >$INST + +inst "$TOKEN" >inst.log 2>&1 +check "with a token: installs, writes it, starts the agent" "$?" "0" +check "the token went to cerberus_enrollment_token" "$(ub $INST cerberus_enrollment_token)" "$TOKEN" +check "the agent enrolled and the installer read its claim code" "$(yes_if grep -Eq '^cerberus: claim [2-9A-Z]{12}$' inst.log)" "yes" +check "into the U-Boot environment" "$(ub $INST cerberus_node_id | grep -c '^node_')" "1" +env PATH="$T/fw:$PATH" FAKE_UBOOT=$INST sh /etc/init.d/S99cerberus stop >/dev/null 2>&1 + +env PATH="$T/fw:$PATH" FAKE_UBOOT=$INST sh /src/uninstall.sh >uninst.log 2>&1 +check "uninstall.sh removes it" "$(yes_if test -e /usr/sbin/cerberus)$(yes_if test -e /etc/init.d/S99cerberus)" "nono" +check "and every key it wrote" "$(grep -c -E '^(cerberus|streq)_' $INST)" "0" + # ------------------------------------------------------------------ revocation section "revocation and removal" cp enrolled.env agent.env @@ -337,15 +464,21 @@ F0=$(date +%s) runfor 20 gone.log check "a forgotten node discards its enrolment" "$(yes_if grep -q 'enrolment discarded' gone.log)" "yes" check "and stops by itself" "$([ $(( $(date +%s) - F0 )) -lt 15 ] && echo yes || echo no)" "yes" -check "node id cleared" "$(sv streq_node_id agent.env)" "" -check "hub endpoint cleared" "$(sv streq_hub_endpoint agent.env)" "" -check "device key kept" "$(sv streq_device_key agent.env)" "$PRIV" -check "claim code kept" "$(sv streq_claim_code agent.env)" "$CLAIM" +check "node id cleared" "$(sv cerberus_node_id agent.env)" "" +check "hub endpoint cleared" "$(sv cerberus_hub_endpoint agent.env)" "" +check "device key kept" "$(sv cerberus_device_key agent.env)" "$PRIV" +check "claim code kept" "$(sv cerberus_claim_code agent.env)" "$CLAIM" + +# The migration must never undo a revocation: forgetting removes the streq_* copies too. +check "U-Boot store: a revoked node is forgotten" "$( (uboot $OLD env CERBERUS_HEARTBEAT=5 timeout 20 $A run >gone-ub.log 2>&1); yes_if grep -q 'enrolment discarded' gone-ub.log)" "yes" +check "under both names" "$(ub $OLD cerberus_node_id)$(ub $OLD streq_node_id)" "" +check "and stays forgotten on the next start" "$(uboot $OLD $A status 2>/dev/null | sed -n 's/^node *//p')" "(not enrolled)" +check "with the device key kept" "$(ub $OLD cerberus_device_key)" "$PRIV" mkdir -p /etc/init.d $T/victim cp $A $T/victim/cerberus && touch /etc/init.d/S99cerberus cp enrolled.env clean.env -STREQ_STORE=file:$T/clean.env $T/victim/cerberus clean >clean.log 2>&1 +CERBERUS_STORE=file:$T/clean.env $T/victim/cerberus clean >clean.log 2>&1 check "clean: removes the init script" "$(yes_if test -e /etc/init.d/S99cerberus)" "no" check "the stored material" "$(yes_if test -e $T/clean.env)" "no" sleep 4 @@ -353,10 +486,10 @@ check "and, a moment later, itself" "$(yes_if test -e $T/victim/cerberus)" "no" # ------------------------------------------------------------------ installers section "shipped scripts" -for s in S99cerberus bootstrap.sh uninstall.sh; do +for s in S99cerberus install.sh uninstall.sh; do check "$s parses" "$(yes_if sh -n /src/$s)" "yes" done -for s in bootstrap.sh uninstall.sh; do +for s in install.sh uninstall.sh; do check "$s carries no comments" "$(tail -n +2 /src/$s | grep -c '^[[:space:]]*#')" "0" done diff --git a/uninstall.sh b/uninstall.sh index ef0e1be..866c08b 100755 --- a/uninstall.sh +++ b/uninstall.sh @@ -1,7 +1,7 @@ #!/bin/sh set -eu -DOCS="${STREQ_DOCS:-https://cloud.openipc.ru/docs/openipc-camera/#codes}" +DOCS="${CERBERUS_DOCS:-${STREQ_DOCS:-https://cloud.openipc.ru/docs/openipc-camera/#codes}}" AGENT=/usr/sbin/cerberus INIT=/etc/init.d/S99cerberus @@ -15,18 +15,20 @@ if [ -x "$INIT" ]; then "$INIT" stop >/dev/null 2>&1 || true fi -iface=$(fw_printenv -n streq_iface 2>/dev/null || true) +iface=$(fw_printenv -n cerberus_iface 2>/dev/null || fw_printenv -n streq_iface 2>/dev/null || true) ip link del dev "${iface:-wg0}" >/dev/null 2>&1 || true -for key in streq_node_id streq_overlay_ip streq_hub_endpoint streq_hub_pubkey \ - streq_allowed_ips streq_network_subnet streq_hub_ip streq_claim_code \ - streq_device_key streq_api streq_enrollment_token streq_hostname \ - streq_kind streq_iface streq_subnet streq_ca streq_heartbeat \ - streq_ping_target streq_ping_interval streq_ping_timeout streq_ping_fails; do - fw_setenv "$key" >/dev/null 2>&1 || die 30 +for name in node_id overlay_ip hub_endpoint hub_pubkey allowed_ips network_subnet hub_ip \ + claim_code device_key api enrollment_token hostname kind iface subnet ca heartbeat \ + ping_target ping_interval ping_timeout ping_fails; do + for prefix in cerberus streq; do + if fw_printenv -n "${prefix}_$name" >/dev/null 2>&1; then + fw_setenv "${prefix}_$name" >/dev/null 2>&1 || die 30 + fi + done done rm -f "$INIT" || die 30 rm -f "$AGENT" || die 30 -rm -rf /etc/streq || die 30 +rm -rf /etc/cerberus /etc/streq || die 30 say "removed" From 8e5da01896014e54c71917efa2d3536e0f6fc504 Mon Sep 17 00:00:00 2001 From: wudgexyz Date: Tue, 13 Oct 2026 23:39:52 +0300 Subject: [PATCH 2/2] README in English, Russian and Chinese, laid out like OpenIPC/decoder Every page carries the same header - logo, a language bar listing all three, badges - and ends with License and Support. The content follows the new release layout: install from releases/latest, one static binary per CPU architecture, the cerberus_* keys and how cameras with streq_* keys are moved. --- README.md | 437 ++++++++++++++++++++++-------------------------- README.ru.md | 429 ++++++++++++++++++++++------------------------- README.zh-CN.md | 256 ++++++++++++++++++++++++++++ 3 files changed, 657 insertions(+), 465 deletions(-) create mode 100644 README.zh-CN.md diff --git a/README.md b/README.md index e3ea1d6..a4e85b3 100644 --- a/README.md +++ b/README.md @@ -1,209 +1,137 @@ -# cerberus +![OpenIPC logo][logo] -[Русская версия](README.ru.md) +# OpenIPC Cerberus -The cloud agent for OpenIPC cameras: a small C daemon that lives **inside the camera firmware** and -puts the camera on a private WireGuard overlay, so it can be reached from anywhere — video, ONVIF, -settings and SSH — with nothing between it and the cloud but the tunnel. +**English** | [Русский](README.ru.md) | [简体中文](README.zh-CN.md) + +[![CI](https://github.com/OpenIPC/cerberus/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenIPC/cerberus/actions/workflows/ci.yml) +[![Release](https://img.shields.io/github/v/release/OpenIPC/cerberus)](https://github.com/OpenIPC/cerberus/releases/latest) +[![License](https://img.shields.io/badge/license-Prosperity%203.0-blue)](LICENSE.md) +[![Platform](https://img.shields.io/badge/platform-OpenIPC-brightgreen)](https://openipc.org) + +The cloud agent for cameras running [OpenIPC](https://openipc.org) firmware: a small C daemon that +lives **inside the camera** and puts it on a private WireGuard network, so the camera can be reached +from anywhere — video, ONVIF, settings and SSH — with nothing between it and the cloud but the +tunnel. It does five things and nothing else: 1. generates the device's X25519 keypair and a claim code — once, on first start; 2. enrols at the coordinator over TLS (`POST /enroll`) and takes the join material; -3. keeps key, claim code and join material in the **U-Boot environment** via `fw_setenv` (a file is - the fallback) — the one place on a camera that survives a firmware reflash; -4. brings up `wg0` from it and supervises the tunnel from both ends: the handshake age from `wg` - **and** an ICMP probe of the hub *inside* the tunnel — a fresh handshake over a dead path looks - alive; -5. reports liveness with `POST /nodes/{id}/heartbeat` and applies what comes back in the answer: - changed join material and one-shot commands. - -**mbedTLS** (TLS, X25519, SHA-256, base64, CTR-DRBG) is the only dependency; everything else is libc. -It drives the kernel WireGuard the firmware already ships, so it is tens of kilobytes rather than a -userspace WireGuard implementation. +3. keeps key, claim code and join material in the **U-Boot environment** via `fw_setenv` — the one + place on a camera that survives a firmware reflash; +4. brings up `wg0` and supervises the tunnel from both ends: the handshake age from `wg` **and** an + ICMP probe of the hub *inside* the tunnel — a fresh handshake over a dead path looks alive; +5. reports liveness with `POST /nodes/{id}/heartbeat` and applies what comes back: changed join + material and one-shot commands. -## Installing on a camera - -The usual way is one line over SSH, with an enrolment token from the cloud panel: - -```sh -curl -fsSL https://cloud.openipc.ru/install.sh | sh -s -- tok_... -``` +**mbedTLS** is the only dependency; everything else is libc. It drives the kernel WireGuard the +firmware already ships, so it is a few hundred kilobytes static, not a userspace WireGuard. -The installer (`bootstrap.sh` here) asks the camera for its processor (`fw_printenv -n soc`), fetches -the matching build, installs `/usr/sbin/cerberus` and `/etc/init.d/S99cerberus`, writes the address -and the token into the U-Boot environment and starts the agent. On failure it prints a short code — -`cerberus: E13` — and the page that decodes it: -. `uninstall.sh` takes all of it off again. +## Install -By hand, from a release tarball: +One line over SSH on the camera, with an enrolment token from the cloud panel (the panel gives you +this command ready-filled): ```sh -tar xzf cerberus--linux-musl-armv7.tar.gz && cd cerberus--linux-musl-armv7 -./install.sh # /usr/sbin/cerberus + /etc/init.d/S99cerberus -fw_setenv streq_api https://cloud-api.openipc.ru/v1 -fw_setenv streq_enrollment_token tok_... -reboot # or: /etc/init.d/S99cerberus restart +curl -fsSL https://github.com/OpenIPC/cerberus/releases/latest/download/install.sh | sh -s -- tok_... ``` -The init script is BusyBox, not systemd: `S99cerberus` reads its config out of the U-Boot env and -starts `cerberus run` with `start-stop-daemon`. Updating is replacing the binary and restarting the -service; key, node id and claim code live in the U-Boot env and survive it. - -A token minted in the panel for one camera binds the camera to whoever minted it. With a batch token -the agent prints a claim code (also shown by `cerberus status`) that is entered in the panel to -claim the camera. Until it is claimed the camera is on the overlay, but nobody can watch or command -it. - -### What the camera needs - -| needed | for | without it | -|---|---|---| -| kernel WireGuard + `wg` | the whole data plane | the agent is useless | -| `ip` | address and link on `wg0` | same | -| `fw_setenv` / `fw_printenv` | storage that survives a reflash | falls back to a file (which does not) | -| shared mbedTLS | TLS, X25519, SHA-256 | use a static build | -| `ping` | the in-tunnel probe | the probe is switched off with a warning; the handshake watch stays | -| `reboot` | the `Reboot` command | the command is logged as impossible | +The installer picks the build for the camera's CPU, checks it against `SHA256SUMS` from the same +release, installs `/usr/sbin/cerberus` and `/etc/init.d/S99cerberus`, writes the address and the token +into the U-Boot environment and starts the agent. No reboot is needed. On failure it prints a short +code — `cerberus: E13` — and the page that decodes it: +. -## Building +To take it all off again: ```sh -make # native, against the system mbedTLS -make STATIC=1 # static -make VERSION=1.2.0 # the version the agent reports (default: the one in main.c) -make install DESTDIR=/tmp/rootfs # /usr/sbin/cerberus + /etc/init.d/S99cerberus +curl -fsSL https://github.com/OpenIPC/cerberus/releases/latest/download/uninstall.sh | sh ``` -It builds against both mbedTLS generations in the field: 3.x and 2.28 LTS — the latter is what -Buildroot, and so most OpenIPC-era firmware, carries. - -Release tarballs are built in containers; the host needs only Docker: +By hand, the same thing: ```sh -ci/build.sh musl-armv7 # OpenIPC on ARMv7: static musl, no dependencies at all -ci/build.sh armhf arm64 amd64 # generic SBCs (glibc, dynamic) - # → dist/cerberus--linux-.tar.gz (+ .sha256) +# cerberus-arm, cerberus-mipsel, ... — see Releases below +curl -fsSL -o /usr/sbin/cerberus https://github.com/OpenIPC/cerberus/releases/latest/download/cerberus-arm +curl -fsSL -o /etc/init.d/S99cerberus https://github.com/OpenIPC/cerberus/releases/latest/download/S99cerberus +chmod +x /usr/sbin/cerberus /etc/init.d/S99cerberus +fw_setenv cerberus_api https://cloud-api.openipc.ru/v1 +fw_setenv cerberus_enrollment_token tok_... +/etc/init.d/S99cerberus restart ``` -**A glibc build does not start on OpenIPC at all.** The firmware is musl: a glibc binary asks for a -loader (`/lib/ld-linux-armhf.so.3`) that is not there, and the shell answers `not found` about a file -that is right in front of you. Hence the separate `musl-armv7` build. It runs inside an emulated -armv7 Alpine (binfmt/QEMU — Docker Desktop has it; on plain Linux run -`docker run --privileged --rm tonistiigi/binfmt --install arm` once), so no cross toolchain is -needed: the libc is the one the firmware runs. It is static, with mbedTLS inside, so it does not -matter which mbedTLS a given firmware carries, or whether it carries one at all. - -Check the camera before picking a build: - -```sh -uname -m # armv7l → musl-armv7 fits; mips/mipsel → it does not -grep -i features /proc/cpuinfo | head -1 # needs vfp: ARMv5 without it cannot run armv7 code -ls /lib/ld-musl-* 2>/dev/null # confirms musl firmware -ls /usr/lib/libmbedtls* 2>/dev/null # firmware has mbedTLS → can link dynamically -``` - -Ingenic T20/T31 are MIPS, and ARMv5 SoCs have no VFP; both need the firmware's own Buildroot -toolchain: - -```sh -make CC=mipsel-openipc-linux-musl-gcc MBEDTLS= STATIC=1 -# or, when the firmware has a matching mbedTLS, without STATIC=1 — tens of KB instead of hundreds -``` - -`STATIC=1` is only sane with musl: static glibc still needs the shared NSS libraries at runtime for -`getaddrinfo` — exactly the call that resolves the hub. +A token minted in the panel for one camera binds the camera to whoever minted it. With a batch token +the agent prints a claim code (also shown by `cerberus status`) that is entered in the panel to claim +the camera. -### Size +## Releases -`-Os`, `--gc-sections`, stripped; measured from the `ci/build.sh` tarballs: +Every release carries one **static** binary per CPU architecture — static with musl and mbedTLS +inside, so it does not matter which libc or which mbedTLS a given firmware has, or whether it has one +at all: -| target | bytes | +| file | runs on | |---|---| -| armv7, musl, static — OpenIPC | 538 080 | -| armv7, glibc, dynamic (armhf) | 26 568 | -| arm64, glibc, dynamic | 68 568 | -| x86-64, glibc, dynamic | 44 120 | - -The agent itself is the dynamic armv7 number. Nearly all of the static build is mbedTLS as Alpine -builds it — server, DTLS, RSA and every cipher suite. Firmware builds mbedTLS with its own config -(TLS client, ECDHE + AEAD, X.509), and a static agent against that is a fraction of the size. - -At runtime it is one process, one thread, about 1.4 MB RSS — mostly mbedTLS buffers during the TLS -handshake. Each heartbeat is a new TLS connection, ~4 KB on the wire, so the default 20 seconds is -~17 MB a day; `STREQ_HEARTBEAT=60` divides that by three on a metered link, at the price of commands -and rotations arriving that much later. - -## Tests - -```sh -make test # Alpine: musl + mbedTLS 3.x -CERBERUS_TEST_IMAGE=debian:bookworm-slim tests/run.sh # glibc + mbedTLS 2.28 -``` - -The suite needs only Docker. It builds the agent with `-Werror` and runs it against a fake -coordinator (`tests/fake_coordinator.py`) over real TLS with its own CA: - -- **identity** — the X25519 key is the one `wg pubkey` derives, it lives in the store and nowhere - else, the store is 0600, a legacy key file is migrated in unchanged; -- **TLS** — an unrelated CA and a certificate for another name are both refused before a byte of - the request leaves; -- **enrolment** — the request carries the public key and `sha256(claim code)`, never the code - itself; the join material is persisted; a second enrol spends nothing; -- **the back-channel** — `Restart` re-execs the agent, `Reboot` reboots, unknown commands are - ignored, a moved or re-keyed hub is picked up and the interface reprogrammed, a coordinator outage - is one warning and is survived, a revoked node discards its enrolment and stops; -- **start-up** — an unreachable coordinator is waited out with backoff, and SIGTERM during the - pause is obeyed at once; -- **the data plane** — a real kernel WireGuard tunnel to a hub in its own network namespace: - handshake, ping through the tunnel, and a stolen route noticed by the in-tunnel probe and repaired - by rebuilding the interface; -- **the shipped scripts** parse, and the installers carry no comments. - -The data-plane part needs kernel WireGuard on the host (any current Linux, Docker Desktop's VM) and -is skipped without it; `REQUIRE_WG=1` makes that a failure instead, which is how CI runs it. - -## CI and releases - -`.github/workflows/ci.yml` runs shellcheck, the suite on both mbedTLS generations, and builds the -four tarballs on every push and pull request. Pushing a `v*` tag publishes those same tarballs, with -their `.sha256` files, as a GitHub release; the version compiled into the binary comes from the tag. - -```sh -git tag v0.2.0 && git push origin v0.2.0 -``` +| `cerberus-arm` | every ARM camera from ARMv5 to ARMv7 — HiSilicon, Goke, SigmaStar, Fullhan, … (soft-float, so ARMv5 without VFP too) | +| `cerberus-mipsel` | MIPS cameras — Ingenic T10, T20, T21, T31, T40, … | +| `cerberus-aarch64` | 64-bit ARM | +| `cerberus-x86_64` | VMs and test boxes | +| `install.sh`, `uninstall.sh` | the installer and the uninstaller | +| `S99cerberus` | the BusyBox init script | +| `SHA256SUMS` | checksums of all of the above; the installer verifies against it | +| `LICENSE.md` | the license | + +Asset names carry no version on purpose: `releases/latest/download/` always points at the +current release. + +| binary | bytes | +|---|---| +| `cerberus-arm` | 570 564 | +| `cerberus-mipsel` | 660 900 | +| `cerberus-aarch64` | 587 320 | +| `cerberus-x86_64` | 575 144 | -## Running +Almost all of that is mbedTLS built with its default configuration; the agent itself is ~30 KB. -Configuration is `STREQ_*` environment variables; `S99cerberus` fills them from the U-Boot env. +## What the camera needs -| variable | default | meaning | +| needed | for | without it | |---|---|---| -| `STREQ_API` | `https://api.localhost/v1` | coordinator base URL, including `/v1` | -| `STREQ_ENROLLMENT_TOKEN` | — | one-shot token from the owner | -| `STREQ_KIND` | `Camera` | node kind | -| `STREQ_HOSTNAME` | — | the name the camera shows up under in the panel | -| `STREQ_IFACE` | `wg0` | WireGuard interface | -| `STREQ_SUBNET` | `10.80.0.0/16` | fallback: the network's subnet and the hub's address arrive with the join material | -| `STREQ_CA` | `/etc/ssl/certs/ca-certificates.crt` | CA file or directory for TLS | -| `STREQ_INSECURE` | `0` | skip TLS verification (debugging only) | -| `STREQ_STORE` | `auto` | `auto` \| `uboot` \| `file:/path` — the device key lives here too | -| `STREQ_KEYFILE` | `/etc/streq/wg.key` | legacy key location; migrated into the store on sight | -| `STREQ_KEY_TMP` | `/tmp/.streq-wg.key` | where the key is materialised for one `wg set` call | -| `STREQ_HEARTBEAT` | `20` | seconds between heartbeats; `0` disables | -| `STREQ_PING_TARGET` | the hub | in-tunnel probe target | -| `STREQ_PING_INTERVAL` | `60` | seconds between probes; `0` disables | -| `STREQ_PING_TIMEOUT` | `3` | seconds one probe waits | -| `STREQ_PING_FAILS` | `3` | consecutive failed probes before the interface is rebuilt | -| `STREQ_STALE_AFTER` | `180` | seconds without a handshake before the peer is re-poked | -| `STREQ_RETRY_MIN` / `STREQ_RETRY_MAX` | `5` / `300` | enrolment retry backoff, seconds | -| `STREQ_DRY_RUN` | `0` | print the `ip`/`wg` commands instead of running them | - -The U-Boot keys `S99cerberus` reads: `streq_api`, `streq_enrollment_token`, `streq_hostname`, -`streq_ca`, `streq_subnet`, `streq_ping_target`, `streq_ping_interval`, `streq_iface` (all but the -first two optional; empty means default). `streq_ca` matters more than it looks: a firmware may -carry no CA bundle at all, and then enrolment fails on the coordinator's certificate with no way to -fix it short of a reflash — point it at the root certificate (ISRG Root X1 for Let's Encrypt). +| kernel WireGuard + `wg` | the whole data plane | the agent is useless | +| `ip` | address and link on `wg0` | same | +| `fw_setenv` / `fw_printenv` | storage that survives a reflash | falls back to a file (which does not) | +| `ping` | the in-tunnel probe | the probe is switched off with a warning; the handshake watch stays | +| `reboot` | the `Reboot` command | the command is logged as impossible | +| `curl` with TLS | the installer | download the files by hand | + +## Configuration + +`S99cerberus` reads the U-Boot environment and hands it to the agent as `CERBERUS_*` variables: + +| U-Boot key | variable | default | meaning | +|---|---|---|---| +| `cerberus_api` | `CERBERUS_API` | — | coordinator base URL, including `/v1` | +| `cerberus_enrollment_token` | `CERBERUS_ENROLLMENT_TOKEN` | — | one-shot token from the owner | +| `cerberus_hostname` | `CERBERUS_HOSTNAME` | — | the name the camera shows up under in the panel | +| `cerberus_ca` | `CERBERUS_CA` | `/etc/ssl/certs/ca-certificates.crt` | CA file or directory for TLS | +| `cerberus_iface` | `CERBERUS_IFACE` | `wg0` | WireGuard interface | +| `cerberus_subnet` | `CERBERUS_SUBNET` | `10.80.0.0/16` | fallback only: the network's subnet arrives with the join material | +| `cerberus_heartbeat` | `CERBERUS_HEARTBEAT` | `20` | seconds between heartbeats; `0` disables | +| `cerberus_ping_target` | `CERBERUS_PING_TARGET` | the hub | in-tunnel probe target | +| `cerberus_ping_interval` | `CERBERUS_PING_INTERVAL` | `60` | seconds between probes; `0` disables | +| `cerberus_ping_timeout` | `CERBERUS_PING_TIMEOUT` | `3` | seconds one probe waits | +| `cerberus_ping_fails` | `CERBERUS_PING_FAILS` | `3` | failed probes in a row before the interface is rebuilt | + +`cerberus_ca` matters more than it looks: a firmware may carry no CA bundle at all, and then +enrolment fails on the coordinator's certificate with no way to fix it short of a reflash — point it +at the root certificate (ISRG Root X1 for Let's Encrypt). + +Variables only the agent reads: `CERBERUS_STORE` (`auto` \| `uboot` \| `file:/path`), +`CERBERUS_STALE_AFTER` (`180`, seconds without a handshake before the peer is re-poked), +`CERBERUS_RETRY_MIN` / `CERBERUS_RETRY_MAX` (`5` / `300`, enrolment backoff), `CERBERUS_INSECURE=1` +(skip TLS verification, debugging only), `CERBERUS_DRY_RUN=1` (print the `ip`/`wg` commands instead +of running them). ``` cerberus run # enroll + up + watch (the default; what S99cerberus starts) @@ -214,43 +142,40 @@ cerberus keygen # print the keypair (stdout is data, diagnostics go to stderr cerberus clean # remove the agent: interface, init script, stored material, the binary ``` -## What lives where +### Formerly `streq_*` -`streq_device_key`, `streq_node_id`, `streq_overlay_ip`, `streq_hub_endpoint`, `streq_hub_pubkey`, -`streq_allowed_ips`, `streq_network_subnet`, `streq_hub_ip`, `streq_claim_code` — all in the U-Boot -environment. +Before this repository the keys were called `streq_*` and the variables `STREQ_*`. Cameras installed +then keep working, with nothing to do: -**The key is there too, deliberately.** A separate 0600 file would be the obvious place, but on a -camera `/etc` is squashfs or an overlay on a partition a reflash overwrites, while `streq_node_id` -next to it in the U-Boot env survives. The node would come back with an identity the hub has a peer -for and no key to prove it — and enrolment is skipped whenever a node id is stored. That is not -"re-enrols", it is *stuck for good*. Key and node id have to live and die together, so they share a -store. `wg set private-key` takes a path, so for exactly one call the key is materialised in `/tmp` -(0600) and deleted straight after; it is never written to the rootfs. +- on start the agent copies its own keys — device key, claim code, join material — from `streq_*` + to `cerberus_*`, checks each copy, and from then on writes only the new names; +- the `streq_*` copies are **left in place**, so an older `S99cerberus`, or an older binary someone + rolls back to, still finds a working camera; +- `S99cerberus` reads `cerberus_` and falls back to `streq_`, so settings written by the + old instructions still apply; `STREQ_*` variables are still accepted; +- forgetting an enrolment (a revoked node), `cerberus clean` and `uninstall.sh` remove both names, so + an old copy can never bring a revoked node back; +- once there is nothing left to copy, a start writes nothing to flash. -`streq_network_subnet` and `streq_hub_ip` arrive with enrolment: every network on a multi-tenant -instance has its own range, and the node takes its address mask and a separate route to the hub -from them. The route matters because WireGuard's `allowed-ips` is cryptokey routing, not the -kernel's routing table: without it the handshake succeeds and the packets go around the tunnel. +## How it works -## The control plane: REST, not the hub +### What lives where -Full nodes hold a SignalR connection to the coordinator and get the network map pushed to them. A C -client for that is not this kind of program, so thin nodes take the degraded path: +Device key, node id, overlay address, hub endpoint and key, allowed IPs, the network's subnet, the +hub's overlay address and the claim code — all in the U-Boot environment. -| hub | what cerberus does | -|---|---| -| `Heartbeat(nodeId)` | `POST /nodes/{id}/heartbeat` every `STREQ_HEARTBEAT` seconds | -| `DirectiveIssued` (push) | the answer to that same heartbeat — the same `NodeDirective` | -| `JoinNetwork(nodeId)` → map | `GET /nodes/{id}/map` exists, but the agent does not poll it | -| `NetworkMapUpdated` / `PathRevoked` | no pushes | +**The key is there too, deliberately.** On a camera `/etc` is squashfs or an overlay on a partition a +reflash overwrites, while the node id in the U-Boot env survives it. A camera that kept its node id +and lost its key would come back with an identity the hub knows and no key to prove it — and +enrolment is skipped whenever a node id is stored. Key and node id have to live and die together. +`wg set private-key` takes a path, so for exactly one call the key is written to `/tmp` (0600) and +deleted straight after; it never touches the rootfs. -A camera does not need the map: it has exactly one peer, the hub, and that arrives with the join -material. A `404` on the heartbeat means the coordinator no longer knows the node (revoked or -rebuilt): the agent drops the enrolment from the store and stops, so the next start enrols afresh -instead of knocking with a dead node id forever. +### The control plane: REST -### The back-channel: the heartbeat answer +Full nodes hold a SignalR connection to the coordinator. A C client for that is not this kind of +program, so the camera uses the REST twins instead: a heartbeat every `CERBERUS_HEARTBEAT` seconds, +and the answer to that heartbeat is its whole back-channel: ```jsonc { @@ -264,48 +189,92 @@ instead of knocking with a dead node id forever. ``` The agent compares `join` with what it has stored and applies **only the difference**: when the hub -moved (a new VPS, a new DNS name) or changed its key, the camera rewrites its store and reprograms -the interface by itself. `Restart` re-execs the agent (there may be no supervisor on the camera); -`Reboot` is `sync` and `reboot`. +moves or changes its key, the camera rewrites its store and reprograms the interface by itself. +`Restart` re-execs the agent; `Reboot` is `sync` and `reboot`. A `404` on the heartbeat means the +coordinator no longer knows the node: the agent drops its enrolment and stops, so the next start +enrols afresh instead of knocking with a dead node id forever. ### Supervising the tunnel -The handshake and the probe answer different questions, and neither follows from the other: -WireGuard re-handshakes about every two minutes and shows a perfectly fresh one while nothing at all -crosses the tunnel — a route someone took, an interface somebody else reused, a hub that completes -handshakes and forwards nothing. The handshake says the peer is there; only a packet that comes back -says the tunnel works. - -The recovery ladder goes up in cost, and the camera has to climb it alone: +A fresh handshake does not mean a working tunnel: WireGuard re-handshakes every two minutes while +nothing crosses it — a route someone took, an interface somebody else reused, a hub that forwards +nothing. Only a packet that comes back says the tunnel works. The recovery ladder goes up in cost, +and the camera climbs it alone: | what is seen | what is done | |---|---| -| handshake older than `STREQ_STALE_AFTER` | the peer is re-poked (which also re-resolves the hub's name) | -| `STREQ_PING_FAILS` probes in a row unanswered | the interface is torn down and rebuilt | +| handshake older than `CERBERUS_STALE_AFTER` | the peer is re-poked (which also re-resolves the hub's name) | +| `CERBERUS_PING_FAILS` probes in a row unanswered | the interface is torn down and rebuilt | | twice that — the rebuild did not help | the agent re-execs itself | -A camera that reboots over one lost packet is worse than one that just stays down, so thresholds -count runs, not events. +`cerberus run` does not die of an unreachable coordinator either: init on a camera starts it once and +never looks again, so enrolment is retried with a doubling pause for as long as it takes. + +Video does not pass through the agent: the relay pulls `rtsp://{overlayIp}:554/stream0` straight from +the camera through the tunnel. There is no peer-to-peer on this agent — the camera stays on the hub +path, which is the path that always works. + +## Building + +```sh +make # native, against the system mbedTLS (3.x or 2.28 LTS) +make STATIC=1 # static +make VERSION=1.2.0 # the version the agent reports (default: the one in main.c) +make install DESTDIR=/tmp/rootfs # /usr/sbin/cerberus + /etc/init.d/S99cerberus +``` + +The release binaries come from `ci/build.sh`, which needs only Docker: it fetches a +[Bootlin](https://toolchains.bootlin.com) musl toolchain and the mbedTLS source (both pinned by +sha256), builds statically, asserts the ELF machine, and then **runs** each binary under QEMU and +checks its X25519 key against `wg pubkey`: -Start-up is separate. `cerberus run` **does not die of an unreachable coordinator**: enrolment is -retried with a doubling pause from `STREQ_RETRY_MIN` to `STREQ_RETRY_MAX`, as long as it takes. Init -on a camera starts the agent once and never looks at it again, so exiting there would mean a camera -lost until the next power cycle — over a network, DNS or NTP that came up a few seconds late. -Configuration errors still exit at once: retrying does not invent a token nobody wrote down. +```sh +ci/build.sh # arm, mipsel, aarch64, x86_64 → dist/cerberus- +ci/build.sh mipsel # one of them +ci/assemble.sh # + installers, init script, SHA256SUMS → a release in dist/ +``` + +Inside a firmware build, with the firmware's own toolchain: + +```sh +make CC=mipsel-openipc-linux-musl-gcc MBEDTLS= +``` -## What it does not do +## Tests -No pushes and no peer-to-peer. `PathRevoked` only travels over the hub; a thin node picks a -revocation up from the next heartbeat. Hole-punching goes through the hub (`ReportCandidates` → -`PunchNow` → `ReportPathState`) and needs a simultaneity polling cannot give, so this agent stays on -the hub path — which is exactly the path that always works. +```sh +make test # Alpine: musl + mbedTLS 3.x +CERBERUS_TEST_IMAGE=debian:bookworm-slim tests/run.sh # glibc + mbedTLS 2.28 +``` -Video does not pass through the agent at all: the relay pulls `rtsp://{overlayIp}:554/stream0` -straight from the camera through the tunnel. +The suite needs only Docker. It builds the agent with `-Werror` and runs it against a fake +coordinator over real TLS: identity, TLS verification, enrolment, the back-channel (commands, a moved +or re-keyed hub, an outage, a revoked node), start-up backoff, the `streq_*` migration against a fake +U-Boot environment, the installer end to end from a release laid out on disk — and a real kernel +WireGuard tunnel to a hub in its own network namespace, including a stolen route the in-tunnel probe +has to notice and repair. The data-plane part needs kernel WireGuard on the host; `REQUIRE_WG=1` +makes its absence a failure, which is how CI runs it. + +## CI and releases + +Every push and pull request runs shellcheck, the suite on both mbedTLS generations, and the four +release builds. A `v*` tag publishes those same binaries, with the installers, the init script and +`SHA256SUMS`, as a GitHub release; the version compiled into the binaries comes from the tag. + +```sh +git tag v0.2.0 && git push origin v0.2.0 +``` ## License -[The Prosperity Public License 3.0.0](LICENSE.md), © OpenIPC. Free for noncommercial use — -personal, hobby, research, and noncommercial organizations — and a thirty-day trial for commercial -use; commercial use beyond the trial is not covered by this license. Every copy, binaries included, -carries the license text with it, which is why the release tarballs ship `LICENSE.md`. +[The Prosperity Public License 3.0.0](LICENSE.md): free for noncommercial use, with a 30-day trial +for commercial use. See [LICENSE.md](LICENSE.md) for the full terms. + +## Support + +- Questions and bug reports: [GitHub Issues](https://github.com/OpenIPC/cerberus/issues) +- Documentation: [openipc.org](https://openipc.org) + +[![Telegram](https://openipc.org/images/telegram_button.svg)](https://openipc.org/our-channels) + +[logo]: https://raw.githubusercontent.com/OpenIPC/.github/refs/heads/main/assets/openipc-logo-black.svg diff --git a/README.ru.md b/README.ru.md index 511e9d3..1f950ba 100644 --- a/README.ru.md +++ b/README.ru.md @@ -1,206 +1,133 @@ -# cerberus +![OpenIPC logo][logo] -[English version](README.md) +# OpenIPC Cerberus -Облачный агент для камер OpenIPC: маленький демон на C, который живёт **внутри прошивки камеры** и -включает камеру в приватную WireGuard-сеть — так, что видео, ONVIF, настройки и SSH доступны откуда -угодно, а между камерой и облаком нет ничего, кроме туннеля. +[English](README.md) | **Русский** | [简体中文](README.zh-CN.md) + +[![CI](https://github.com/OpenIPC/cerberus/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenIPC/cerberus/actions/workflows/ci.yml) +[![Release](https://img.shields.io/github/v/release/OpenIPC/cerberus)](https://github.com/OpenIPC/cerberus/releases/latest) +[![License](https://img.shields.io/badge/license-Prosperity%203.0-blue)](LICENSE.md) +[![Platform](https://img.shields.io/badge/platform-OpenIPC-brightgreen)](https://openipc.org) + +Облачный агент для камер с прошивкой [OpenIPC](https://openipc.org): маленький демон на C, который +живёт **внутри камеры** и включает её в приватную WireGuard-сеть — так, что видео, ONVIF, настройки и +SSH доступны откуда угодно, а между камерой и облаком нет ничего, кроме туннеля. Делает ровно пять вещей: 1. заводит ключевую пару X25519 и claim-код — один раз, на первом старте; 2. энроллится в координатор по TLS (`POST /enroll`) и забирает join-материал; -3. кладёт ключ, claim-код и join-материал в **U-Boot environment** через `fw_setenv` (файл — - фолбэк) — единственное место на камере, которое переживает перепрошивку; -4. поднимает из него `wg0` и следит за туннелем с двух сторон: возраст хендшейка из `wg` **и** - ICMP-проба хаба *внутри* туннеля — свежий хендшейк на мёртвом пути выглядит как живой; +3. кладёт ключ, claim-код и join-материал в **U-Boot environment** через `fw_setenv` — единственное + место на камере, которое переживает перепрошивку; +4. поднимает `wg0` и следит за туннелем с двух сторон: возраст хендшейка из `wg` **и** ICMP-проба хаба + *внутри* туннеля — свежий хендшейк на мёртвом пути выглядит как живой; 5. шлёт живость через `POST /nodes/{id}/heartbeat` и применяет то, что приезжает в ответе: сменившийся join-материал и одноразовые команды. -Единственная зависимость — **mbedTLS** (TLS, X25519, SHA-256, base64, CTR-DRBG), всё остальное — libc. -Агент рулит kernel WireGuard, который в прошивке уже есть, поэтому весит десятки килобайт, а не -целую userspace-реализацию WireGuard. - -## Установка на камеру - -Обычный путь — одна строка по SSH с токеном энролла из панели облака: - -```sh -curl -fsSL https://cloud.openipc.ru/install.sh | sh -s -- tok_... -``` - -Установщик (здесь это `bootstrap.sh`) спрашивает у камеры её процессор (`fw_printenv -n soc`), -скачивает сборку под него, ставит `/usr/sbin/cerberus` и `/etc/init.d/S99cerberus`, пишет адрес и -токен в U-Boot env и запускает агента. Если что-то не так, он печатает короткий код — -`cerberus: E13` — и страницу, где код расшифрован: -. `uninstall.sh` снимает всё обратно. - -Руками, из тарболла релиза: - -```sh -tar xzf cerberus--linux-musl-armv7.tar.gz && cd cerberus--linux-musl-armv7 -./install.sh # /usr/sbin/cerberus + /etc/init.d/S99cerberus -fw_setenv streq_api https://cloud-api.openipc.ru/v1 -fw_setenv streq_enrollment_token tok_... -reboot # или: /etc/init.d/S99cerberus restart -``` - -Init — BusyBox, не systemd: `S99cerberus` читает конфиг из U-Boot env и запускает `cerberus run` -через `start-stop-daemon`. Обновление — заменить бинарь и перезапустить службу; ключ, node id и -claim-код лежат в U-Boot env и обновление переживают. - -Токен, выписанный в панели на одну камеру, привязывает камеру к тому, кто его выписал. С токеном -партии агент печатает claim-код (его же показывает `cerberus status`), который вводят в панели, -чтобы забрать камеру себе. До этого камера в сети, но смотреть её и командовать ей некому. - -### Что должно быть на камере - -| нужно | зачем | если нет | -|---|---|---| -| kernel WireGuard + `wg` | весь data plane | агент бесполезен | -| `ip` | адрес и линк на `wg0` | то же | -| `fw_setenv` / `fw_printenv` | хранилище, переживающее перепрошивку | фолбэк в файл (и он не переживёт) | -| разделяемый mbedTLS | TLS, X25519, SHA-256 | статическая сборка | -| `ping` | проба внутри туннеля | проба выключается с предупреждением, присмотр за хендшейком остаётся | -| `reboot` | команда `Reboot` | команда логируется как невыполнимая | - -## Сборка - -```sh -make # нативно, против системного mbedTLS -make STATIC=1 # статически -make VERSION=1.2.0 # версия, которую агент сообщает (по умолчанию — из main.c) -make install DESTDIR=/tmp/rootfs # /usr/sbin/cerberus + /etc/init.d/S99cerberus -``` +Единственная зависимость — **mbedTLS**, всё остальное — libc. Агент рулит kernel WireGuard, который в +прошивке уже есть, поэтому статически весит несколько сотен килобайт, а не целый userspace-WireGuard. -Собирается против обоих поколений mbedTLS, что встречаются в поле: 3.x и 2.28 LTS — последний лежит -в Buildroot и, значит, в большинстве прошивок эпохи OpenIPC. +## Установка -Тарболлы релиза собираются в контейнерах, хосту нужен только Docker: +Одна строка по SSH на камере, с токеном энролла из панели облака (панель выдаёт эту команду уже +заполненной): ```sh -ci/build.sh musl-armv7 # OpenIPC на ARMv7: статический musl, без зависимостей -ci/build.sh armhf arm64 amd64 # обычные SBC (glibc, динамически) - # → dist/cerberus--linux-.tar.gz (+ .sha256) +curl -fsSL https://github.com/OpenIPC/cerberus/releases/latest/download/install.sh | sh -s -- tok_... ``` -**glibc-сборка на OpenIPC не запускается вообще.** Прошивка на musl: glibc-бинарь просит загрузчик -`/lib/ld-linux-armhf.so.3`, которого нет, и `sh` отвечает `not found` про файл, лежащий перед -глазами. Отсюда отдельная сборка `musl-armv7`. Она идёт внутри эмулированного armv7 Alpine -(binfmt/QEMU — в Docker Desktop есть; на голом Linux один раз -`docker run --privileged --rm tonistiigi/binfmt --install arm`), поэтому кросс-тулчейн не нужен: -libc ровно та, что в прошивке. Сборка статическая, mbedTLS внутри — неважно, какой mbedTLS несёт -прошивка и несёт ли вообще. +Установщик выбирает сборку под процессор камеры, сверяет её с `SHA256SUMS` из того же релиза, ставит +`/usr/sbin/cerberus` и `/etc/init.d/S99cerberus`, пишет адрес и токен в U-Boot env и запускает агента. +Перезагрузка не нужна. Если что-то не так, он печатает короткий код — `cerberus: E13` — и страницу, где +код расшифрован: . -Посмотрите на камеру, прежде чем выбирать сборку: +Снять всё обратно: ```sh -uname -m # armv7l → musl-armv7 годится; mips/mipsel → нет -grep -i features /proc/cpuinfo | head -1 # нужен vfp: ARMv5 без него armv7-код не запустит -ls /lib/ld-musl-* 2>/dev/null # подтверждает, что прошивка на musl -ls /usr/lib/libmbedtls* 2>/dev/null # есть свой mbedTLS → можно линковать динамически +curl -fsSL https://github.com/OpenIPC/cerberus/releases/latest/download/uninstall.sh | sh ``` -Ingenic T20/T31 — это MIPS, а у ARMv5 нет VFP; обоим нужен тулчейн самой прошивки (Buildroot): +То же самое руками: ```sh -make CC=mipsel-openipc-linux-musl-gcc MBEDTLS= STATIC=1 -# или, если mbedTLS в прошивке есть и версия совпадает, без STATIC=1 — десятки КБ вместо сотен +# cerberus-arm, cerberus-mipsel, ... — см. «Релизы» ниже +curl -fsSL -o /usr/sbin/cerberus https://github.com/OpenIPC/cerberus/releases/latest/download/cerberus-arm +curl -fsSL -o /etc/init.d/S99cerberus https://github.com/OpenIPC/cerberus/releases/latest/download/S99cerberus +chmod +x /usr/sbin/cerberus /etc/init.d/S99cerberus +fw_setenv cerberus_api https://cloud-api.openipc.ru/v1 +fw_setenv cerberus_enrollment_token tok_... +/etc/init.d/S99cerberus restart ``` -`STATIC=1` разумен только с musl: статический glibc всё равно тянет в рантайме разделяемые -NSS-библиотеки для `getaddrinfo` — ровно того вызова, которым резолвится хаб. +Токен, выписанный в панели на одну камеру, привязывает камеру к тому, кто его выписал. С токеном +партии агент печатает claim-код (его же показывает `cerberus status`), который вводят в панели, чтобы +забрать камеру себе. -### Сколько весит +## Релизы -`-Os`, `--gc-sections`, strip; мерено по тарболлам из `ci/build.sh`: +В каждом релизе — по одному **статическому** бинарю на архитектуру процессора: musl и mbedTLS внутри, +поэтому неважно, какая libc и какой mbedTLS в конкретной прошивке и есть ли он там вообще: -| цель | байт | +| файл | на чём работает | |---|---| -| armv7, musl, статически — OpenIPC | 538 080 | -| armv7, glibc, динамически (armhf) | 26 568 | -| arm64, glibc, динамически | 68 568 | -| x86-64, glibc, динамически | 44 120 | - -Сам агент — это динамическая armv7-цифра. Почти всё в статической сборке — mbedTLS в том виде, в -каком его собирает Alpine: сервер, DTLS, RSA, все наборы шифров. Прошивка собирает mbedTLS со своим -конфигом (TLS-клиент, ECDHE + AEAD, X.509), и статический агент против него в разы меньше. - -В работе это один процесс, один поток, около 1,4 МБ RSS — в основном буферы mbedTLS на время -TLS-хендшейка. Каждый heartbeat — новое TLS-соединение, ~4 КБ по сети, так что при 20 секундах по -умолчанию это ~17 МБ в сутки; на мобильном тарифе `STREQ_HEARTBEAT=60` делит счёт на три — ценой того, -что команды и ротация доедут на столько же позже. - -## Тесты - -```sh -make test # Alpine: musl + mbedTLS 3.x -CERBERUS_TEST_IMAGE=debian:bookworm-slim tests/run.sh # glibc + mbedTLS 2.28 -``` - -Нужен только Docker. Набор собирает агента с `-Werror` и гоняет его против поддельного координатора -(`tests/fake_coordinator.py`) по настоящему TLS со своим CA: - -- **личность** — ключ X25519 тот же, что выводит `wg pubkey`, лежит в сторе и больше нигде, стор - 0600, ключ из старого файла переносится без изменений; -- **TLS** — чужой CA и сертификат на другое имя отвергаются раньше, чем уйдёт хоть байт запроса; -- **энролл** — в запросе публичный ключ и `sha256(claim-код)`, но никогда не сам код; join-материал - сохраняется; повторный энролл ничего не тратит; -- **обратный канал** — `Restart` перезапускает агента, `Reboot` перезагружает, неизвестные команды - игнорируются, переехавший или сменивший ключ хаб подхватывается с перепрограммированием - интерфейса, недоступность координатора — одно предупреждение и пережита, отозванный узел - выбрасывает энролл и останавливается; -- **старт** — недоступный координатор пережидается с отступом, SIGTERM в паузе выполняется сразу; -- **data plane** — настоящий kernel-WireGuard туннель до хаба в отдельном network namespace: - хендшейк, ping по туннелю, и украденный маршрут, который видит проба внутри туннеля и чинит - пересборка интерфейса; -- **поставляемые скрипты** разбираются шеллом, а в установщиках нет комментариев. - -Для data plane на хосте нужен kernel WireGuard (любой свежий Linux, VM Docker Desktop); без него -секция пропускается, а `REQUIRE_WG=1` превращает пропуск в провал — так запускает CI. - -## CI и релизы - -`.github/workflows/ci.yml` на каждый push и pull request гоняет shellcheck, тесты на обоих поколениях -mbedTLS и собирает четыре тарболла. Тег `v*` публикует эти же тарболлы с `.sha256` как GitHub -release; версия, вкомпилированная в бинарь, берётся из тега. - -```sh -git tag v0.2.0 && git push origin v0.2.0 -``` +| `cerberus-arm` | все ARM-камеры от ARMv5 до ARMv7 — HiSilicon, Goke, SigmaStar, Fullhan, … (soft-float, поэтому и ARMv5 без VFP) | +| `cerberus-mipsel` | MIPS-камеры — Ingenic T10, T20, T21, T31, T40, … | +| `cerberus-aarch64` | 64-битный ARM | +| `cerberus-x86_64` | виртуалки и тестовые стенды | +| `install.sh`, `uninstall.sh` | установщик и деинсталлятор | +| `S99cerberus` | init-скрипт BusyBox | +| `SHA256SUMS` | контрольные суммы всего перечисленного; установщик сверяется с ними | +| `LICENSE.md` | лицензия | + +Версии в именах файлов нет специально: `releases/latest/download/<имя>` всегда ведёт на текущий +релиз. + +| бинарь | байт | +|---|---| +| `cerberus-arm` | 570 564 | +| `cerberus-mipsel` | 660 900 | +| `cerberus-aarch64` | 587 320 | +| `cerberus-x86_64` | 575 144 | -## Запуск +Почти всё это — mbedTLS с конфигурацией по умолчанию; сам агент — около 30 КБ. -Конфиг — переменные окружения `STREQ_*`; `S99cerberus` заполняет их из U-Boot env. +## Что должно быть на камере -| переменная | по умолчанию | смысл | +| нужно | зачем | если нет | |---|---|---| -| `STREQ_API` | `https://api.localhost/v1` | база координатора, включая `/v1` | -| `STREQ_ENROLLMENT_TOKEN` | — | одноразовый токен от владельца | -| `STREQ_KIND` | `Camera` | тип узла | -| `STREQ_HOSTNAME` | — | имя, под которым камера появится в панели | -| `STREQ_IFACE` | `wg0` | интерфейс WireGuard | -| `STREQ_SUBNET` | `10.80.0.0/16` | фолбэк: подсеть сети и адрес хаба приезжают в join-материале | -| `STREQ_CA` | `/etc/ssl/certs/ca-certificates.crt` | файл или каталог доверенных CA | -| `STREQ_INSECURE` | `0` | отключить проверку TLS (только для отладки) | -| `STREQ_STORE` | `auto` | `auto` \| `uboot` \| `file:/path` — здесь же лежит и ключ | -| `STREQ_KEYFILE` | `/etc/streq/wg.key` | старое место ключа; при первом же виде переносится в стор | -| `STREQ_KEY_TMP` | `/tmp/.streq-wg.key` | куда ключ кладётся на время одного `wg set` | -| `STREQ_HEARTBEAT` | `20` | секунд между heartbeat; `0` — выключить | -| `STREQ_PING_TARGET` | хаб | кого пинговать внутри туннеля | -| `STREQ_PING_INTERVAL` | `60` | секунд между пробами; `0` — выключить | -| `STREQ_PING_TIMEOUT` | `3` | сколько секунд ждать ответа на одну пробу | -| `STREQ_PING_FAILS` | `3` | неотвеченных проб подряд до пересборки интерфейса | -| `STREQ_STALE_AFTER` | `180` | секунд без хендшейка до перетыкания пира | -| `STREQ_RETRY_MIN` / `STREQ_RETRY_MAX` | `5` / `300` | отступ повторов энролла, секунды | -| `STREQ_DRY_RUN` | `0` | печатать команды `ip`/`wg`, а не выполнять их | - -Ключи U-Boot env, которые читает `S99cerberus`: `streq_api`, `streq_enrollment_token`, -`streq_hostname`, `streq_ca`, `streq_subnet`, `streq_ping_target`, `streq_ping_interval`, -`streq_iface` (все, кроме первых двух, необязательные; пусто — по умолчанию). `streq_ca` важнее, чем -кажется: в прошивке может не быть бандла CA вовсе, и тогда энролл падает на сертификате координатора, -а поправить это без перепрошивки нечем — укажите путь к корневому сертификату (ISRG Root X1 для -Let's Encrypt). +| kernel WireGuard + `wg` | весь data plane | агент бесполезен | +| `ip` | адрес и линк на `wg0` | то же | +| `fw_setenv` / `fw_printenv` | хранилище, переживающее перепрошивку | фолбэк в файл (и он не переживёт) | +| `ping` | проба внутри туннеля | проба выключается с предупреждением, присмотр за хендшейком остаётся | +| `reboot` | команда `Reboot` | команда логируется как невыполнимая | +| `curl` с TLS | установщик | скачать файлы руками | + +## Настройка + +`S99cerberus` читает U-Boot env и передаёт агенту переменными `CERBERUS_*`: + +| ключ U-Boot | переменная | по умолчанию | смысл | +|---|---|---|---| +| `cerberus_api` | `CERBERUS_API` | — | база координатора, включая `/v1` | +| `cerberus_enrollment_token` | `CERBERUS_ENROLLMENT_TOKEN` | — | одноразовый токен от владельца | +| `cerberus_hostname` | `CERBERUS_HOSTNAME` | — | имя, под которым камера появится в панели | +| `cerberus_ca` | `CERBERUS_CA` | `/etc/ssl/certs/ca-certificates.crt` | файл или каталог доверенных CA | +| `cerberus_iface` | `CERBERUS_IFACE` | `wg0` | интерфейс WireGuard | +| `cerberus_subnet` | `CERBERUS_SUBNET` | `10.80.0.0/16` | только фолбэк: подсеть сети приезжает в join-материале | +| `cerberus_heartbeat` | `CERBERUS_HEARTBEAT` | `20` | секунд между heartbeat; `0` — выключить | +| `cerberus_ping_target` | `CERBERUS_PING_TARGET` | хаб | кого пинговать внутри туннеля | +| `cerberus_ping_interval` | `CERBERUS_PING_INTERVAL` | `60` | секунд между пробами; `0` — выключить | +| `cerberus_ping_timeout` | `CERBERUS_PING_TIMEOUT` | `3` | сколько секунд ждать ответа на пробу | +| `cerberus_ping_fails` | `CERBERUS_PING_FAILS` | `3` | неотвеченных проб подряд до пересборки интерфейса | + +`cerberus_ca` важнее, чем кажется: в прошивке может не быть бандла CA вовсе, и тогда энролл падает на +сертификате координатора, а поправить это без перепрошивки нечем — укажите путь к корневому +сертификату (ISRG Root X1 для Let's Encrypt). + +Переменные, которые читает только агент: `CERBERUS_STORE` (`auto` \| `uboot` \| `file:/path`), +`CERBERUS_STALE_AFTER` (`180`, секунд без хендшейка до перетыкания пира), `CERBERUS_RETRY_MIN` / +`CERBERUS_RETRY_MAX` (`5` / `300`, отступ повторов энролла), `CERBERUS_INSECURE=1` (без проверки TLS, +только для отладки), `CERBERUS_DRY_RUN=1` (печатать команды `ip`/`wg`, а не выполнять). ``` cerberus run # enroll + up + watch (по умолчанию; это запускает S99cerberus) @@ -211,42 +138,40 @@ cerberus keygen # напечатать пару ключей (stdout — да cerberus clean # убрать агента: интерфейс, автозапуск, сохранённое и сам бинарь ``` -## Что где лежит +### Раньше — `streq_*` -`streq_device_key`, `streq_node_id`, `streq_overlay_ip`, `streq_hub_endpoint`, `streq_hub_pubkey`, -`streq_allowed_ips`, `streq_network_subnet`, `streq_hub_ip`, `streq_claim_code` — всё в U-Boot env. +До этого репозитория ключи назывались `streq_*`, а переменные — `STREQ_*`. Камеры, поставленные +тогда, продолжают работать, делать ничего не нужно: -**Ключ там же, и это осознанно.** Прямолинейнее держать его отдельным файлом 0600, но на камере -`/etc` — это squashfs или оверлей на разделе, который перепрошивка перезаписывает, а `streq_node_id` -рядом в U-Boot env её переживает. Узел вернулся бы с личностью, на которую у хаба есть пир, и без -ключа, которым её доказывают, — а энролл пропускается всегда, когда node id сохранён. Это не -«перерегистрируется», это **насмерть**. Ключ и node id обязаны жить и умирать вместе, поэтому лежат -в одном сторе. `wg set private-key` хочет путь, поэтому на время ровно одного вызова ключ -материализуется в `/tmp` (0600) и сразу удаляется; на rootfs он не пишется вообще. +- на старте агент копирует свои ключи — ключ устройства, claim-код, join-материал — из `streq_*` в + `cerberus_*`, проверяет каждую копию и дальше пишет только новые имена; +- копии `streq_*` **остаются на месте**, поэтому старый `S99cerberus` или старый бинарь, на который + кто-то откатился, по-прежнему находят рабочую камеру; +- `S99cerberus` читает `cerberus_<имя>`, а при его отсутствии — `streq_<имя>`, так что настройки по + старым инструкциям действуют; переменные `STREQ_*` тоже принимаются; +- сброс энролла (отозванный узел), `cerberus clean` и `uninstall.sh` стирают оба имени, поэтому + старая копия не может воскресить отозванный узел; +- когда копировать больше нечего, старт не пишет во флеш ничего. -`streq_network_subnet` и `streq_hub_ip` приходят с энроллом: на инстансе с арендаторами у каждой -сети свой диапазон, и узел берёт из них маску своего адреса и отдельный маршрут до хаба. Маршрут -нужен потому, что `allowed-ips` у WireGuard — это криптомаршрутизация, а не таблица ядра: без него -хендшейк проходит, а пакеты уходят мимо туннеля. +## Как это устроено -## Control plane: REST, а не хаб +### Что где лежит -Полноценные узлы держат SignalR-соединение с координатором и получают пуши карты сети. Клиент такого -на C — не тот жанр, поэтому у тонких узлов деградированный путь: +Ключ устройства, node id, overlay-адрес, адрес и ключ хаба, allowed IPs, подсеть сети, overlay-адрес +хаба и claim-код — всё в U-Boot env. -| хаб | что делает cerberus | -|---|---| -| `Heartbeat(nodeId)` | `POST /nodes/{id}/heartbeat` раз в `STREQ_HEARTBEAT` секунд | -| `DirectiveIssued` (пуш) | ответ на тот же heartbeat — та же `NodeDirective` | -| `JoinNetwork(nodeId)` → карта | `GET /nodes/{id}/map` есть, но агент его не опрашивает | -| `NetworkMapUpdated` / `PathRevoked` | пушей нет | - -Карта камере не нужна: пир у неё ровно один — хаб, и он приезжает в join-материале. `404` на -heartbeat значит, что координатор узел больше не знает (отозван или пересобран): агент выбрасывает -энролл из стора и останавливается, чтобы следующий старт зашёл заново, а не долбился мёртвым node id -до конца времён. +**Ключ там же, и это осознанно.** На камере `/etc` — это squashfs или оверлей на разделе, который +перепрошивка перезаписывает, а node id в U-Boot env её переживает. Камера, сохранившая node id и +потерявшая ключ, вернулась бы с личностью, которую хаб знает, и без ключа, которым её доказывают, — а +энролл пропускается всегда, когда node id сохранён. Ключ и node id обязаны жить и умирать вместе. +`wg set private-key` хочет путь, поэтому на время ровно одного вызова ключ пишется в `/tmp` (0600) и +сразу удаляется; на rootfs он не попадает никогда. -### Обратный канал: ответ на heartbeat +### Control plane: REST + +Полноценные узлы держат SignalR-соединение с координатором. Клиент такого на C — не тот жанр, поэтому +камера ходит в REST-двойники: heartbeat раз в `CERBERUS_HEARTBEAT` секунд, и ответ на него — весь её +обратный канал: ```jsonc { @@ -259,50 +184,92 @@ heartbeat значит, что координатор узел больше не } ``` -`join` агент сверяет с сохранённым и применяет **только расхождение**: хаб переехал (новый VPS, -новый DNS) или сменил ключ — камера перезаписывает стор и перепрограммирует интерфейс сама. -`Restart` — это re-exec самого агента (супервизора на камере может и не быть), `Reboot` — `sync` и -`reboot`. +`join` агент сверяет с сохранённым и применяет **только расхождение**: хаб переехал или сменил ключ — +камера перезаписывает стор и перепрограммирует интерфейс сама. `Restart` — это re-exec агента, +`Reboot` — `sync` и `reboot`. `404` на heartbeat значит, что координатор узел больше не знает: агент +выбрасывает энролл и останавливается, чтобы следующий старт зашёл заново, а не долбился мёртвым node id +до конца времён. ### Присмотр за туннелем -Хендшейк и проба отвечают на разные вопросы, и ни один не следует из другого: WireGuard -переустанавливает хендшейк примерно раз в две минуты и покажет совершенно свежий, пока по туннелю не -ходит ничего — забранный маршрут, интерфейс, который занял кто-то другой, хаб, который отвечает на -хендшейк и не форвардит. Хендшейк говорит «пир на месте», и только вернувшийся пакет говорит -«туннель работает». - -Лестница восстановления — по возрастанию цены, и камера обязана пройти её сама: +Свежий хендшейк не значит рабочий туннель: WireGuard переустанавливает его раз в две минуты, даже +когда по туннелю не ходит ничего — забранный маршрут, интерфейс, который занял кто-то другой, хаб, +который ничего не форвардит. Только вернувшийся пакет говорит «туннель работает». Лестница +восстановления — по возрастанию цены, и камера проходит её сама: | что видно | что делается | |---|---| -| хендшейку больше `STREQ_STALE_AFTER` | перетыкается пир (это же и перерезолвит имя хаба) | -| `STREQ_PING_FAILS` проб подряд без ответа | интерфейс сносится и собирается заново | +| хендшейку больше `CERBERUS_STALE_AFTER` | перетыкается пир (это же и перерезолвит имя хаба) | +| `CERBERUS_PING_FAILS` проб подряд без ответа | интерфейс сносится и собирается заново | | вдвое больше — пересборка не помогла | агент перезапускает сам себя | -Камера, которая перезагружается от одного потерянного пакета, хуже камеры, которая просто лежит, — -поэтому пороги считают серии, а не события. +От недоступного координатора `cerberus run` тоже не умирает: init на камере запускает его один раз и +больше не смотрит, поэтому энролл повторяется с удваивающейся паузой сколько понадобится. + +Видео через агента не идёт: релей тянет `rtsp://{overlayIp}:554/stream0` прямо с камеры по туннелю. +P2P этот агент не делает — камера остаётся на пути через хаб, который работает всегда. + +## Сборка + +```sh +make # нативно, против системного mbedTLS (3.x или 2.28 LTS) +make STATIC=1 # статически +make VERSION=1.2.0 # версия, которую агент сообщает (по умолчанию — из main.c) +make install DESTDIR=/tmp/rootfs # /usr/sbin/cerberus + /etc/init.d/S99cerberus +``` + +Бинари релиза собирает `ci/build.sh`, которому нужен только Docker: он скачивает musl-тулчейн +[Bootlin](https://toolchains.bootlin.com) и исходники mbedTLS (оба сверяются по sha256), собирает +статически, проверяет архитектуру ELF, а затем **запускает** каждый бинарь под QEMU и сверяет его +ключ X25519 с `wg pubkey`: + +```sh +ci/build.sh # arm, mipsel, aarch64, x86_64 → dist/cerberus- +ci/build.sh mipsel # одну из них +ci/assemble.sh # + установщики, init-скрипт, SHA256SUMS → релиз в dist/ +``` + +Внутри сборки прошивки, её собственным тулчейном: + +```sh +make CC=mipsel-openipc-linux-musl-gcc MBEDTLS= +``` + +## Тесты + +```sh +make test # Alpine: musl + mbedTLS 3.x +CERBERUS_TEST_IMAGE=debian:bookworm-slim tests/run.sh # glibc + mbedTLS 2.28 +``` -Отдельно — старт. `cerberus run` **не умирает от недоступного координатора**: энролл повторяется с -удваивающейся паузой от `STREQ_RETRY_MIN` до `STREQ_RETRY_MAX`, сколько понадобится. Init на камере -запускает агента ровно один раз и больше на него не смотрит, так что выход здесь означал бы камеру, -потерянную до следующего выключения питания — из-за сети, DNS или NTP, опоздавших на несколько -секунд. Ошибки конфигурации по-прежнему завершают сразу: повторы не выдумают токен, которого не -прописали. +Нужен только Docker. Набор собирает агента с `-Werror` и гоняет его против поддельного координатора +по настоящему TLS: личность, проверка TLS, энролл, обратный канал (команды, переехавший или сменивший +ключ хаб, недоступность координатора, отозванный узел), отступ на старте, миграция `streq_*` на +поддельном U-Boot env, установщик целиком из релиза, разложенного на диске, — и настоящий +kernel-WireGuard туннель до хаба в отдельном network namespace, включая украденный маршрут, который +проба внутри туннеля должна заметить и починить. Для data plane на хосте нужен kernel WireGuard; +`REQUIRE_WG=1` делает его отсутствие провалом — так запускает CI. -## Чего он не делает +## CI и релизы -Пушей и P2P. `PathRevoked` ходит только через хаб — тонкий узел узнает об отзыве со следующим -heartbeat. Hole-punch идёт через хаб (`ReportCandidates` → `PunchNow` → `ReportPathState`) и требует -одновременности, которой опрос не даёт, поэтому этот агент остаётся на пути через хаб — ровно на том -пути, который работает всегда. +На каждый push и pull request — shellcheck, тесты на обоих поколениях mbedTLS и четыре сборки релиза. +Тег `v*` публикует эти же бинари вместе с установщиками, init-скриптом и `SHA256SUMS` как GitHub +release; версия, вкомпилированная в бинари, берётся из тега. -Видео через агента не идёт вовсе: релей тянет `rtsp://{overlayIp}:554/stream0` прямо с камеры по -туннелю. +```sh +git tag v0.2.0 && git push origin v0.2.0 +``` ## Лицензия -[The Prosperity Public License 3.0.0](LICENSE.md), © OpenIPC. Бесплатно для некоммерческого -использования — личного, любительского, исследовательского и некоммерческими организациями — и -тридцать дней пробы для коммерческого; коммерческое использование сверх пробы эта лицензия не покрывает. Текст лицензии -обязан идти с каждой копией, бинарной в том числе, поэтому `LICENSE.md` лежит и в тарболлах релиза. +[The Prosperity Public License 3.0.0](LICENSE.md): бесплатно для некоммерческого использования, +30 дней пробы для коммерческого. Полные условия — в [LICENSE.md](LICENSE.md). + +## Поддержка + +- Вопросы и баг-репорты: [GitHub Issues](https://github.com/OpenIPC/cerberus/issues) +- Документация: [openipc.org](https://openipc.org) + +[![Telegram](https://openipc.org/images/telegram_button.svg)](https://openipc.org/our-channels) + +[logo]: https://raw.githubusercontent.com/OpenIPC/.github/refs/heads/main/assets/openipc-logo-black.svg diff --git a/README.zh-CN.md b/README.zh-CN.md new file mode 100644 index 0000000..9f83c76 --- /dev/null +++ b/README.zh-CN.md @@ -0,0 +1,256 @@ +![OpenIPC logo][logo] + +# OpenIPC Cerberus + +[English](README.md) | [Русский](README.ru.md) | **简体中文** + +[![CI](https://github.com/OpenIPC/cerberus/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenIPC/cerberus/actions/workflows/ci.yml) +[![Release](https://img.shields.io/github/v/release/OpenIPC/cerberus)](https://github.com/OpenIPC/cerberus/releases/latest) +[![License](https://img.shields.io/badge/license-Prosperity%203.0-blue)](LICENSE.md) +[![Platform](https://img.shields.io/badge/platform-OpenIPC-brightgreen)](https://openipc.org) + +适用于运行 [OpenIPC](https://openipc.org) 固件摄像机的云代理：一个用 C 编写的小型守护进程，**运行在摄像机内部**， +将摄像机接入私有 WireGuard 网络。这样无论身在何处，都能访问摄像机的视频、ONVIF、设置和 SSH， +摄像机与云之间除了隧道别无他物。 + +它只做五件事： + +1. 首次启动时生成设备的 X25519 密钥对和认领码（claim code），仅此一次； +2. 通过 TLS 向协调器注册（`POST /enroll`），获取入网材料（join material）； +3. 通过 `fw_setenv` 将密钥、认领码和入网材料保存在 **U-Boot 环境变量**中——这是摄像机上唯一能在刷写固件后保留下来的地方； +4. 启动 `wg0` 并从两方面监控隧道：来自 `wg` 的握手时间，**以及**在隧道*内部*对 hub 的 ICMP 探测—— + 死链路上的新鲜握手看起来也像是活的； +5. 通过 `POST /nodes/{id}/heartbeat` 上报存活状态，并执行响应中返回的内容：变更后的入网材料和一次性命令。 + +**mbedTLS** 是唯一的依赖，其余全部是 libc。它驱动固件自带的内核 WireGuard，因此静态编译后只有几百 KB， +而不是一个用户态 WireGuard 实现。 + +## 安装 + +在摄像机上通过 SSH 执行一行命令，并附上云控制面板中的注册令牌（面板会直接给出已填好的命令）： + +```sh +curl -fsSL https://github.com/OpenIPC/cerberus/releases/latest/download/install.sh | sh -s -- tok_... +``` + +安装脚本会根据摄像机的 CPU 选择对应的构建，使用同一发布中的 `SHA256SUMS` 进行校验，安装 +`/usr/sbin/cerberus` 和 `/etc/init.d/S99cerberus`，将地址和令牌写入 U-Boot 环境变量并启动代理，无需重启。 +出错时会打印一个简短的代码，例如 `cerberus: E13`，以及解释该代码的页面： +。 + +完全卸载： + +```sh +curl -fsSL https://github.com/OpenIPC/cerberus/releases/latest/download/uninstall.sh | sh +``` + +手动安装，效果相同： + +```sh +# cerberus-arm、cerberus-mipsel 等——见下文“发布” +curl -fsSL -o /usr/sbin/cerberus https://github.com/OpenIPC/cerberus/releases/latest/download/cerberus-arm +curl -fsSL -o /etc/init.d/S99cerberus https://github.com/OpenIPC/cerberus/releases/latest/download/S99cerberus +chmod +x /usr/sbin/cerberus /etc/init.d/S99cerberus +fw_setenv cerberus_api https://cloud-api.openipc.ru/v1 +fw_setenv cerberus_enrollment_token tok_... +/etc/init.d/S99cerberus restart +``` + +在面板中为单台摄像机签发的令牌会把摄像机绑定到签发者名下。使用批量令牌时，代理会打印认领码 +（`cerberus status` 也会显示），在面板中输入该认领码即可认领摄像机。 + +## 发布 + +每个发布都为每种 CPU 架构提供一个**静态**二进制文件：musl 和 mbedTLS 已编译在内，因此无论固件使用哪种 libc、 +带有哪个版本的 mbedTLS，甚至没有 mbedTLS，都没有关系： + +| 文件 | 适用于 | +|---|---| +| `cerberus-arm` | 从 ARMv5 到 ARMv7 的所有 ARM 摄像机——海思、国科、星宸、富瀚……（软浮点，因此也支持没有 VFP 的 ARMv5） | +| `cerberus-mipsel` | MIPS 摄像机——君正 T10、T20、T21、T31、T40…… | +| `cerberus-aarch64` | 64 位 ARM | +| `cerberus-x86_64` | 虚拟机和测试环境 | +| `install.sh`、`uninstall.sh` | 安装和卸载脚本 | +| `S99cerberus` | BusyBox 启动脚本 | +| `SHA256SUMS` | 以上所有文件的校验和，安装脚本据此校验 | +| `LICENSE.md` | 许可证 | + +文件名中故意不带版本号：`releases/latest/download/<文件名>` 始终指向最新发布。 + +| 二进制文件 | 字节 | +|---|---| +| `cerberus-arm` | 570 564 | +| `cerberus-mipsel` | 660 900 | +| `cerberus-aarch64` | 587 320 | +| `cerberus-x86_64` | 575 144 | + +其中绝大部分是默认配置的 mbedTLS，代理本身只有约 30 KB。 + +## 摄像机需要具备 + +| 需要 | 用途 | 缺失时 | +|---|---|---| +| 内核 WireGuard + `wg` | 整个数据平面 | 代理无法工作 | +| `ip` | `wg0` 上的地址和链路 | 同上 | +| `fw_setenv` / `fw_printenv` | 刷写固件后仍保留的存储 | 退回到文件存储（刷写后不会保留） | +| `ping` | 隧道内探测 | 探测关闭并给出警告，握手监控仍然有效 | +| `reboot` | `Reboot` 命令 | 命令记录为无法执行 | +| 支持 TLS 的 `curl` | 安装脚本 | 手动下载文件 | + +## 配置 + +`S99cerberus` 读取 U-Boot 环境变量，并以 `CERBERUS_*` 变量的形式传给代理： + +| U-Boot 键 | 变量 | 默认值 | 含义 | +|---|---|---|---| +| `cerberus_api` | `CERBERUS_API` | — | 协调器地址，包含 `/v1` | +| `cerberus_enrollment_token` | `CERBERUS_ENROLLMENT_TOKEN` | — | 所有者签发的一次性令牌 | +| `cerberus_hostname` | `CERBERUS_HOSTNAME` | — | 摄像机在面板中显示的名称 | +| `cerberus_ca` | `CERBERUS_CA` | `/etc/ssl/certs/ca-certificates.crt` | TLS 使用的 CA 文件或目录 | +| `cerberus_iface` | `CERBERUS_IFACE` | `wg0` | WireGuard 接口 | +| `cerberus_subnet` | `CERBERUS_SUBNET` | `10.80.0.0/16` | 仅作后备：网络子网随入网材料下发 | +| `cerberus_heartbeat` | `CERBERUS_HEARTBEAT` | `20` | 心跳间隔（秒），`0` 表示关闭 | +| `cerberus_ping_target` | `CERBERUS_PING_TARGET` | hub | 隧道内探测的目标 | +| `cerberus_ping_interval` | `CERBERUS_PING_INTERVAL` | `60` | 探测间隔（秒），`0` 表示关闭 | +| `cerberus_ping_timeout` | `CERBERUS_PING_TIMEOUT` | `3` | 单次探测等待的秒数 | +| `cerberus_ping_fails` | `CERBERUS_PING_FAILS` | `3` | 连续多少次探测无响应后重建接口 | + +`cerberus_ca` 比看上去更重要：固件可能根本不带 CA 证书包，此时注册会因协调器证书校验失败而中断， +除了重新刷写固件之外无法修复——请将其指向根证书（Let's Encrypt 使用 ISRG Root X1）。 + +仅由代理读取的变量：`CERBERUS_STORE`（`auto` \| `uboot` \| `file:/path`）、`CERBERUS_STALE_AFTER` +（`180`，多少秒没有握手后重新设置对端）、`CERBERUS_RETRY_MIN` / `CERBERUS_RETRY_MAX`（`5` / `300`， +注册重试的退避时间）、`CERBERUS_INSECURE=1`（跳过 TLS 校验，仅用于调试）、`CERBERUS_DRY_RUN=1` +（只打印 `ip`/`wg` 命令而不执行）。 + +``` +cerberus run # enroll + up + watch（默认；S99cerberus 启动的就是它） +cerberus enroll # 仅注册，幂等 +cerberus up # 用已保存的材料启动接口 +cerberus status # 已保存的内容、握手时间、隧道是否有响应 +cerberus keygen # 打印密钥对（stdout 为数据，诊断信息输出到 stderr） +cerberus clean # 移除代理：接口、启动脚本、已保存的材料以及二进制文件本身 +``` + +### 以前的 `streq_*` + +在本仓库之前，这些键名为 `streq_*`，变量名为 `STREQ_*`。当时安装的摄像机无需任何操作即可继续工作： + +- 启动时，代理会把自己的键——设备密钥、认领码、入网材料——从 `streq_*` 复制到 `cerberus_*`， + 逐一校验，此后只写入新名称； +- `streq_*` 副本**保留不动**，因此旧版 `S99cerberus` 或回退到的旧版二进制文件仍能找到一台可用的摄像机； +- `S99cerberus` 读取 `cerberus_<名称>`，不存在时回退到 `streq_<名称>`，因此按旧说明写入的设置依然有效； + `STREQ_*` 变量同样被接受； +- 注销（节点被撤销）、`cerberus clean` 和 `uninstall.sh` 会同时删除两种名称，因此旧副本永远无法让已撤销的节点复活； +- 当没有需要复制的内容时，启动过程不会向闪存写入任何东西。 + +## 工作原理 + +### 数据存放位置 + +设备密钥、节点 ID、overlay 地址、hub 地址和密钥、allowed IPs、网络子网、hub 的 overlay 地址以及认领码—— +全部保存在 U-Boot 环境变量中。 + +**密钥也放在那里，这是有意为之。** 在摄像机上，`/etc` 是 squashfs 或刷写固件时会被覆盖的分区上的 overlay， +而 U-Boot 环境变量中的节点 ID 能在刷写后保留。如果摄像机保留了节点 ID 却丢失了密钥，它会带着一个 hub +认识的身份回来，却没有能证明这一身份的密钥——而只要保存了节点 ID，就会跳过注册。密钥和节点 ID 必须同生共死。 +`wg set private-key` 需要的是文件路径，因此密钥只在一次调用期间写入 `/tmp`（0600），随即删除，从不写入 rootfs。 + +### 控制平面：REST + +完整节点与协调器之间保持 SignalR 连接。用 C 实现这样的客户端并不适合这个程序，因此摄像机改用对应的 REST +接口：每隔 `CERBERUS_HEARTBEAT` 秒发送一次心跳，而心跳的响应就是它全部的反向通道： + +```jsonc +{ + "command": "None", // None | Restart | Reboot —— 一次性命令，只下发一次 + "join": { // 当前的入网材料，总是完整的，而不是增量 + "kind": "Wireguard", "overlayIp": "10.80.0.7", + "hubEndpoint": "hub.example:51820", "hubPublicKey": "base64…", "allowedIps": "10.80.0.7/32" + }, + "mapVersion": 42 +} +``` + +代理将 `join` 与已保存的内容比较，**只应用差异**：当 hub 迁移或更换密钥时，摄像机会自行更新存储并重新配置接口。 +`Restart` 会重新执行代理本身；`Reboot` 即 `sync` 加 `reboot`。心跳返回 `404` 表示协调器已不再认识该节点： +代理会丢弃注册信息并停止，以便下次启动时重新注册，而不是永远用一个失效的节点 ID 反复请求。 + +### 隧道监控 + +握手新鲜并不代表隧道可用：即使隧道中没有任何流量，WireGuard 也会每两分钟重新握手一次——路由被抢占、 +接口被他人占用、hub 不转发任何数据。只有返回的数据包才能说明隧道可用。恢复措施按代价从低到高排列， +摄像机会自行逐级执行： + +| 观察到的情况 | 采取的措施 | +|---|---| +| 握手时间超过 `CERBERUS_STALE_AFTER` | 重新设置对端（同时重新解析 hub 的域名） | +| 连续 `CERBERUS_PING_FAILS` 次探测无响应 | 拆除并重建接口 | +| 达到两倍次数——重建无效 | 代理重新执行自身 | + +协调器不可达时 `cerberus run` 也不会退出：摄像机上的 init 只启动它一次，此后不再关注，因此注册会以倍增的间隔 +不断重试，直到成功。 + +视频不经过代理：中继服务器通过隧道直接从摄像机拉取 `rtsp://{overlayIp}:554/stream0`。该代理不做点对点连接—— +摄像机始终走经由 hub 的路径，而这条路径总是可用的。 + +## 构建 + +```sh +make # 本机构建，链接系统的 mbedTLS（3.x 或 2.28 LTS） +make STATIC=1 # 静态构建 +make VERSION=1.2.0 # 代理上报的版本（默认取自 main.c） +make install DESTDIR=/tmp/rootfs # /usr/sbin/cerberus + /etc/init.d/S99cerberus +``` + +发布用的二进制文件由 `ci/build.sh` 构建，只需要 Docker：它会下载 [Bootlin](https://toolchains.bootlin.com) +的 musl 工具链和 mbedTLS 源码（均按 sha256 校验），进行静态构建，检查 ELF 架构，然后在 QEMU 下**实际运行** +每个二进制文件，并将其 X25519 密钥与 `wg pubkey` 的结果进行比对： + +```sh +ci/build.sh # arm、mipsel、aarch64、x86_64 → dist/cerberus- +ci/build.sh mipsel # 其中之一 +ci/assemble.sh # + 安装脚本、启动脚本、SHA256SUMS → dist/ 中的完整发布 +``` + +在固件构建环境中，使用固件自带的工具链： + +```sh +make CC=mipsel-openipc-linux-musl-gcc MBEDTLS= +``` + +## 测试 + +```sh +make test # Alpine：musl + mbedTLS 3.x +CERBERUS_TEST_IMAGE=debian:bookworm-slim tests/run.sh # glibc + mbedTLS 2.28 +``` + +测试只需要 Docker。测试套件使用 `-Werror` 构建代理，并通过真实的 TLS 让它与一个模拟协调器交互：身份、TLS 校验、 +注册、反向通道（命令、迁移或更换密钥的 hub、协调器不可用、节点被撤销）、启动时的退避、在模拟 U-Boot 环境上的 +`streq_*` 迁移、从磁盘上的发布目录端到端运行安装脚本——以及一条连接到独立网络命名空间中 hub 的真实内核 WireGuard +隧道，包括隧道内探测必须发现并修复的被抢占路由。数据平面部分需要宿主机支持内核 WireGuard；`REQUIRE_WG=1` +会将其缺失视为失败，CI 就是这样运行的。 + +## CI 与发布 + +每次 push 和 pull request 都会运行 shellcheck、两代 mbedTLS 上的测试套件以及四个发布构建。推送 `v*` 标签后， +这些二进制文件会连同安装脚本、启动脚本和 `SHA256SUMS` 一起作为 GitHub release 发布；编译进二进制文件的版本号取自标签。 + +```sh +git tag v0.2.0 && git push origin v0.2.0 +``` + +## 许可证 + +[The Prosperity Public License 3.0.0](LICENSE.md)：非商业用途免费，商业用途可试用 30 天。完整条款见 +[LICENSE.md](LICENSE.md)。 + +## 支持 + +- 问题与缺陷反馈：[GitHub Issues](https://github.com/OpenIPC/cerberus/issues) +- 文档：[openipc.org](https://openipc.org) + +[![Telegram](https://openipc.org/images/telegram_button.svg)](https://openipc.org/our-channels) + +[logo]: https://raw.githubusercontent.com/OpenIPC/.github/refs/heads/main/assets/openipc-logo-black.svg