Skip to content

operations.md

Latest commit

 

History

History
450 lines (355 loc) · 20.1 KB

operations.md

File metadata and controls

450 lines (355 loc) · 20.1 KB
Raw

Operations

New sites run new 0.x under systemd, or the same release package in Docker. 2.x and 3.x receive no further updates. Switch using Install and update FTW; it covers every old and new layout. A new setup is available now; guided migration of old data is not ready. The control loop is local; Energyplan ships with Core and Core falls back safely when it is unavailable.

Install

Install and update FTW is the full guide and explains how to choose a published new 0.x beta. Use the installer from the same tag you install, on a fresh 64-bit Raspberry Pi OS, Debian or Ubuntu host.

Follow the complete native install block. It stops on errors and uses a temporary download. Do not run it on the old SD card.

Do not use GitHub releases/latest; it stays on 2.x for old boxes. --fresh-host confirms that no FTW site exists, including a stopped Docker site in a custom directory, and the installer refuses a site it recognises. It checks the package's SHA-256, creates the native layout and starts ftw.service. Open http://<host>:8080/setup on the LAN, then check health and live device readings. If the install stops partway, inspect journalctl -u ftw, then rerun with --resume --tag and the same tag. It keeps any data already saved.

Docker runs the same release package; see Docker.

Existing Docker, Home Assistant and older native sites can switch with a separate new setup; preserve old data and follow the one-Core checks in the guide. No native 0.x package for macOS has shipped; the old macOS Docker installer is retired.

Everyday commands

Native (see native-beta.md):

ftw status                                  # version, releases, last update, disk, health
ftw update [--channel beta|stable] [--retry]
ftw rollback                                # previous release, same data
ftw backup --output-dir /media/usb/ftw      # verified full backup, copied off the disk
ftw support [--output FILE]                 # redacted support file
sudo systemctl restart ftw
journalctl -u ftw -n 200 --no-pager         # logs; -f follows

Docker 0.x, in ~/ftw-local: docker compose ps, docker compose logs --tail 200 and docker compose exec ftw ftw status. Update or go back by changing FTW_VERSION in .env and running docker compose up -d --build.

Older Docker installs (1.x–3.x), in ~/ftw or /opt/ftw on the old Raspberry Pi image: docker compose ps, docker compose logs -f ftw ftw-updater and docker compose restart ftw. They keep their Update Center. Do not use it, the retired paired-upgrade procedure or a moving Docker alias to cross release lines.

The host operating system is maintained separately; see self-update.md.

Persistent state

The data directory is /var/lib/ftw on a native install, ~/ftw-local/data in Docker 0.x and data/ in an older Docker install. It holds:

  • config.yaml — operator configuration;
  • state.db plus SQLite WAL files — settings, state and learned models;
  • history.db — history as SQLite buckets (10 s for 7 days, 1 min for 90 days, 1 h for 5 years) and the energy ledger;
  • cache.db — prices and forecasts; past prices cost the savings history and come back only from a full backup;
  • key files such as nova.key — the box's identity; keep them with the data;
  • drivers/ — custom and managed drivers;
  • backups/ — full backups, unless state.backup_dir points elsewhere.

Core no longer writes a Parquet cold/ directory. A box converted from an older version may still have one.

Do not store mutable state inside a container. In Docker the data directory must be writable by uid 100/gid 101. A native install runs as the ftw user.

Make a verified full backup with ftw backup --output-dir <another disk>. It captures the persistent data, verifies file hashes and SQLite, and records the installed component versions. In Docker 0.x, back up the data directory while FTW is stopped. See backup-and-restore.md.

A native update takes no local rollback point: ftw rollback returns to the previous release with the current data. Older Docker installs keep bounded pre-update rollback points of the settings database and configuration. They remain on the same disk and cannot recover a failed SD card.

Core exposes a read-only storage check at GET /api/storage/inventory. It reports allocated, live and free SQLite pages for state.db and cache.db, their WAL/SHM sizes, and free space on the volume that contains the data directory. It does not expose paths, scan a storage engine's internal files, change retention, checkpoint WAL, run VACUUM, or delete data. GET /api/backups lists full backups and their sizes.

Configuration

config.example.yaml and go/internal/config define the current schema. Edits are validated before application. A rejected hot reload leaves the previous live configuration intact.

Driver set and most control values reload live. Listener addresses, state paths and some integration transports are startup bindings; restart after changing them. When unsure, inspect the restart classification in go/internal/config/restart_required.go.

LAN and API access

FTW accepts state-changing requests without credentials only when they are addressed through a local name or address: loopback, private/link-local IP, .local, .localhost, or .home.arpa. The setup wizard follows the same rule, and the actual client address must also be local. Browser writes must also be same-origin; FTW checks Origin, Host, and Sec-Fetch-Site and does not advertise CORS. The box refuses to be framed: every response includes Content-Security-Policy: frame-ancestors 'none' and X-Frame-Options: DENY. Local non-browser clients such as curl and Home Assistant may omit browser fetch headers, but JSON bodies must use Content-Type: application/json. Active reads that start discovery, begin an authorization flow, or force an external update check pass through the same boundary. So do reads of config, logs, support dump and report, driver source and identity, integration status, notification history and rules, system and storage info, local repository and snapshot paths, research dump, the app-link device list, driver health, EV charger detail, planner diagnose, time series, and the fleet-ping payload. Live dashboard reads such as status, energy, prices, plan, loadpoints and history stay open, even if someone publishes the port. Still do not publish the box directly to the internet.

Mutation requests addressed through any other hostname or a public IP fail closed. To expose that API intentionally, generate a random token of at least 32 characters and give it to Core as the environment variable FTW_API_TOKEN:

  • Native: run sudo systemctl edit ftw, add Environment=FTW_API_TOKEN=<secret> under [Service], then sudo systemctl restart ftw. The drop-in stays when install.sh --refresh replaces the unit. Local users can read it; on a shared machine use EnvironmentFile= with a root-only file instead.
  • Docker 0.x: add the entry below to a compose.override.yaml beside compose.yaml, keep the value in .env with mode 0600, and run docker compose up -d.
  • Older Docker installs already pass it from the project .env; run docker compose up -d ftw after setting it.
services:
  ftw:
    environment:
      FTW_API_TOKEN: ${FTW_API_TOKEN}

Send the token as a Bearer credential:

curl -X POST -H "Authorization: Bearer <same-random-secret>" \
  https://ftw.example.net/api/restart

On the Home Assistant app the same call shuts Core down and re-execs the binary in-process. Supervisor does not restart a stopped app unless Watchdog is on, so exiting would leave the app stopped.

The same Bearer is accepted on the LAN when api.lan_auth is on. FTW_API_TOKEN and the house password are independent secrets: the token is not hashed as a password guess, and a mismatch does not lock Settings.

The built-in browser UI does not store API tokens. For a public/FQDN browser deployment, put FTW behind an operator-managed HTTPS reverse proxy with login or session authentication and have that trusted proxy inject the Bearer header upstream.

Recovery cannot be disabled by a bad token: connect through localhost, the host's private IP, or its .local name, correct/remove FTW_API_TOKEN, and restart Core. Tokens shorter than 32 characters are ignored and remote mutations remain locked.

FTW_API_TOKEN is an operator-managed mechanism, not the credential for remote access; remote access is the FTW app. See architecture.md.

api.lan_auth is off by default. Turn it on from loopback inside the process: http://127.0.0.1:8080 Settings → System on the machine, or curl to 127.0.0.1. When on, protected LAN routes need the house password or FTW_API_TOKEN. curl sends Authorization: Bearer <house-password> or Authorization: Bearer <FTW_API_TOKEN>. The browser login form sets a session cookie (ftw_lan, 12 hours). Loopback (127.0.0.1 / ::1) never asks. Live status stays readable without the password; a viewer caller is minted for those reads. The two secrets do not share a lockout: retrying the API token cannot lock the household out of Settings. A first enable from another LAN address is refused, so a visitor cannot set the password. POST /api/config cannot flip the flag.

On an older Docker install under Docker Desktop (docker-compose.macos.yml), a host curl to localhost arrives as the bridge gateway, not loopback, so Settings and host curl get 403. Desktop SNAT uses that same peer for every published-port client, including LAN visitors, so it must not count as loopback. Enable from inside the container:

docker compose -f docker-compose.macos.yml exec ftw \
  wget -qO- --header='Content-Type: application/json' \
  --post-data='{"enabled":true,"password":"ETT-LANGT-LOSEN"}' \
  http://127.0.0.1:8080/api/auth/password

An owner pairing QR is minted only from loopback or with the house password. Promoting a paired phone to owner uses the same gate. The first pairing on an empty box is an owner whatever the code said, so that mint also stays at the box. A viewer invite still works from the LAN once an owner exists.

The FTW app and Home Assistant MQTT are unchanged.

Recovery: curl to 127.0.0.1, or set api.lan_auth: false in config.yaml and restart Core.

"remote access to protected API routes is disabled"

The full message is remote access to protected API routes is disabled; configure FTW_API_TOKEN or use a local address. The dashboard still loads — the message appears when a protected request (saving settings, a scan) is refused. It means the request failed the locality rules above, and "local" is judged on the request, not on which network the browser sits on:

  1. The address in the browser's address bar must be a private or loopback IP, localhost, or a .local/.localhost/.home.arpa name.
  2. The address the connection arrives from must be loopback, private, or link-local.

Being on the same LAN as the box satisfies neither by itself. The common ways to trip the boundary from the couch next to the Pi:

  • A plain hostname without a dot — http://myhost:8080. Since v2.2.1 a no-dot name is deliberately not local (a DNS-rebinding guard), so an address that worked before an update stops working. Use the .local name or the IP instead.
  • A router-issued name with a dot in it — pi.lan, ftw.fritz.box, box.home. Only the suffixes listed above count as local names.
  • Tailscale — both 100.x.y.z addresses (CGNAT space) and *.ts.net names count as remote.
  • IPv6 — when the name resolves to a global IPv6 address, the browser connects from a global address and fails the second rule, even with a .local name in the address bar.
  • A reverse proxy or Home Assistant ingress reached through a public URL — the request arrives carrying the public hostname.

The reliable fix for a browser is to open the UI through the box's private IPv4 address, for example http://192.168.1.123:8080. Setting FTW_API_TOKEN does not change what the built-in UI sends — the token is only for API clients that attach the Authorization: Bearer header, as described above — and a ?token=... query parameter in the URL does nothing.

Logs and health

curl -fsS http://localhost:8080/api/health answers on every install; the commands above show logs. Warnings about stale site-meter telemetry are safety actions, not cosmetic noise. Dispatch remains idle until fresh data returns.

Troubleshooting

Driver offline or tick count stopped

Check the driver-specific error and transport connectivity. Core should already have marked it offline and sent its default mode. Restart only after capturing the useful log context. If telemetry resumes, watchdog recovery is automatic.

Batteries do not follow the plan

Check, in order:

  1. site-meter freshness and sign;
  2. selected mode and current plan freshness;
  3. asset SoC/capability;
  4. fuse/per-phase saturation;
  5. driver health and command errors;
  6. optimizer status and fallback messages.

A safety clamp or fallback should be visible in status/logs. Do not raise a limit until the physical installation and configuration agree.

Optimizer unavailable

Inspect Core logs and /api/components for the Energyplan worker status. Core uses its Go fallback when needed; recovery needs no data reset.

MQTT device missing

Verify the broker address from the same network namespace as core, then inspect broker and driver logs. Device credentials and topic mappings belong to the driver configuration.

A device .local name does not resolve

With allow_unverified_local enabled, FTW resolves .local device names itself because a CGO_ENABLED=0 Go binary never consults NSS. Without the option, FTW leaves the name to Go's system DNS path. FTW's own lookup has two backends, and the log line for a successful lookup says which:

resolved host over mDNS host=zap.local addr=192.168.1.42 via=avahi

via=avahi — the host's avahi-daemon answered over its socket. This is preferred when available because avahi already holds a record cache. When NSS and the socket mount are configured, getent hosts zap.local asks the same daemon, though cache timing can still differ.

via=multicast — FTW queried the LAN directly. This is what happens when the avahi socket is absent, and it needs no host software at all.

Failures log mDNS resolution failed, and the message names both backends when both were tried.

Either way multicast has to reach the LAN. A native install is on the host network, and both Docker compose files for Linux use network_mode: host. Under the old docker-compose.macos.yml the container is bridged and multicast does not reach the LAN, so configure devices by IP there. The direct path sends on every active, non-loopback multicast interface. It supports IPv4 and IPv6; a link-local IPv6 answer is used only with its interface zone, and an unscoped link-local answer is discarded.

mDNS has no built-in authentication. Treat a .local name as a LAN trust boundary, reserve names used by control drivers, and use TLS certificate pins where the driver supports them. Network allowlists still check the configured host name and port before resolution; they do not prove that an mDNS responder is the intended device.

That risk is not unique to names, which is why the opt-in below gates the resolver rather than the connection. A raw IP address is no more an identity on a LAN than a name is — it can be claimed by ARP, and DHCP can hand it to a different device with no attacker involved at all. The durable check is the identity a device reports once connected: make and serial, or its MAC.

So by default FTW does not use its own mDNS answer for a .local name: the name goes to the system resolver, and on platforms that answer .local themselves — Home Assistant does, through Supervisor's DNS service — it simply works. Opt in per driver to let FTW resolve the name itself, over Avahi or the LAN:

capabilities:
  allow_unverified_local: true
  modbus:
    host: inverter.local
    port: 502

For the Home Assistant bridge, set homeassistant.allow_unverified_local: true instead. It applies to every transport — HTTP, WebSocket, MQTT, Modbus and raw TCP — and is per driver, so a name allowlist never becomes a server identity for another driver. Literal IP addresses and ordinary DNS names are untouched.

Without it, a .local dial is not refused; it is handed to the system resolver, and the log records why FTW's own answer was not used. That matters on a host where nothing else resolves .local — a native or Docker install whose resolv.conf points at the router — because there the name will simply not resolve until you opt in.

HTTP and WebSocket requests to .local hosts always bypass configured proxies, so credentials and control payloads stay on the LAN path. Ordinary DNS names still use the configured proxy. A TLS pin does not bypass the mDNS gate yet.

Letting FTW use avahi

A native install uses /run/avahi-daemon/socket when the host runs avahi-daemon. In Docker, host networking shares ports, not Unix sockets, so avahi has to be bind-mounted in. The older docker-compose.yml carries the line commented out:

    volumes:
      - /run/avahi-daemon:/run/avahi-daemon:ro

Mount the directory, not the socket file inside it. If the host path is missing Docker creates it, and an empty directory is harmless — whereas a directory created where the socket belongs stops avahi-daemon from ever starting. Restarting avahi detaches the mount, so restart FTW after you do.

This is an optimisation, not a requirement: device connectivity is unchanged without it. In the older image it also lets getent hosts zap.local, curl and wget check the name from inside the container.

Under the Home Assistant add-on none of this applies: Supervisor mounts only a fixed set of named paths, so the socket cannot be provided and FTW always queries the LAN directly. The add-on runs with host_network: true, which is what makes that work.

Configuration rejected

Read the validation error, compare with config.example.yaml, fix the file and save again. Do not delete state.db to resolve a YAML error.

Port already in use

sudo ss -ltnp | grep ':8080'

Stop the conflicting service or change the configured API port, then restart.

Native deployment

The installer creates:

/opt/ftw/ftw-launcher             starts the current release slot
/opt/ftw/slots.json               current and previous release, trial state
/opt/ftw/releases/<tag>/          Core, ftw-backup, web, bundled drivers, Energyplan
/var/lib/ftw/                     config.yaml, state.db, history.db, drivers/, backups/
/etc/systemd/system/ftw.service   deploy/ftw-native.service, alias forty-two-watts.service
/usr/local/bin/ftw                operator command

deploy/ftw-native.service runs the launcher as the ftw user. ftw update changes only the release slots; the launcher, the ftw command and the unit move with install.sh --refresh (native-beta.md).

Release recovery

No 0.x stable has shipped; run a beta only on an agreed test site. A beta and its promoted stable build identify the same commit.

  • Native: ftw rollback returns to the previous release when it reads the same data. If Core does not start and ftw cannot reach it, roll back offline:

    sudo -u ftw /opt/ftw/ftw-launcher -root /opt/ftw rollback
sudo systemctl restart ftw

  • Docker 0.x: set the previous FTW_VERSION in .env and run docker compose up -d --build. There is no automatic fallback.

  • Across a state-schema change, restore a full backup made before the update; see backup-and-restore.md.

  • Older Docker installs (1.x–3.x) roll back within their own line through their Update Center. Never use it to reach 0.x.