Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changeset/no-device-support-packages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
"ftw": patch
---

FTW no longer reads Device Support driver packages. Drivers come from the
release and from FTW's own signed driver channel, as before. A site that still
lists a Device Support package source keeps starting: the source is removed
from its settings with one warning, and a package that was active is switched
off at startup so the release's own driver runs. A driver `control` opt-in has
no effect any more; it is removed with a warning instead of stopping that
driver from starting. The driver diagnostics report now names a driver
installed from the channel as `managed`.
116 changes: 2 additions & 114 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -217,118 +217,6 @@ jobs:
run: bash scripts/check-driver-versions.sh "${{ needs.changes.outputs.base_sha }}"
-head "${{ needs.changes.outputs.head_sha }}"

device-support-contract:
name: Device Support driver contract
needs: changes
if: needs.changes.outputs.core == 'true'
runs-on: ubuntu-latest
env:
DEVICE_SUPPORT_TOKEN: ${{ secrets.SOURCEFUL_CI_REPO_TOKEN }}
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v7
if: env.DEVICE_SUPPORT_TOKEN != ''
with:
python-version: '3.12'
- id: baseline
name: Read pinned Device Support baseline
run: |
baseline=go/internal/driverrepo/testdata/device-support-baseline.json
for key in repository commit driver version; do
echo "${key}=$(jq -r ".${key}" "${baseline}")" >> "${GITHUB_OUTPUT}"
done
- uses: actions/setup-go@v7
with:
go-version: '1.26'
cache-dependency-path: go/go.sum

- name: Verify pinned Python-signed fixture and FTW Lua target
working-directory: go
env:
FTW_DEVICE_SUPPORT_INDEX: ${{ github.workspace }}/go/internal/driverrepo/testdata/device-support-v1/index.envelope.json
FTW_DEVICE_SUPPORT_PACKAGE: ${{ github.workspace }}/go/internal/driverrepo/testdata/device-support-v1/manifest.envelope.json
FTW_DEVICE_SUPPORT_ARTIFACT_DIR: ${{ github.workspace }}/go/internal/driverrepo/testdata/device-support-v1
run: |
public_key="$(tr -d '\n' < internal/driverrepo/testdata/device-support-v1/public.raw.b64)"
FTW_DEVICE_SUPPORT_PUBLIC_KEY="${public_key}" \
go test -count=1 ./internal/driverrepo -run '^TestDeviceSupportPythonContract$'

- name: Check out canonical Device Support package source
if: env.DEVICE_SUPPORT_TOKEN != ''
uses: actions/checkout@v7
with:
repository: ${{ steps.baseline.outputs.repository }}
ref: ${{ steps.baseline.outputs.commit }}
path: .device-support
token: ${{ env.DEVICE_SUPPORT_TOKEN }}

- name: Build and sign the canonical SDM630 package and index from source
if: env.DEVICE_SUPPORT_TOKEN != ''
env:
DRIVER: ${{ steps.baseline.outputs.driver }}
VERSION: ${{ steps.baseline.outputs.version }}
SOURCE_COMMIT: ${{ steps.baseline.outputs.commit }}
run: |
set -euo pipefail
output="${RUNNER_TEMP}/device-support-package"
mkdir -p "${output}"
uv run --python 3.12 --with cryptography python - <<'PY'
import base64
from pathlib import Path
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey

output = Path("${{ runner.temp }}") / "device-support-package"
key = Ed25519PrivateKey.generate()
(output / "private.pem").write_bytes(key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.PKCS8,
serialization.NoEncryption(),
))
(output / "public.pem").write_bytes(key.public_key().public_bytes(
serialization.Encoding.PEM,
serialization.PublicFormat.SubjectPublicKeyInfo,
))
(output / "public.raw.b64").write_text(base64.b64encode(
key.public_key().public_bytes(
serialization.Encoding.Raw,
serialization.PublicFormat.Raw,
)
).decode())
PY
source_date_epoch="$(git -C .device-support show -s --format=%ct "${SOURCE_COMMIT}")"
uv run --python 3.12 --with cryptography --with jsonschema --with referencing \
python .device-support/tools/driver_package.py package \
--source ".device-support/packages/v1/${DRIVER}/package-source.json" \
--repo-root .device-support \
--output-dir "${output}/artifacts" \
--base-url "https://packages.example/${DRIVER}/${VERSION}" \
--source-commit "${SOURCE_COMMIT}" \
--source-date-epoch "${source_date_epoch}" \
--key "${output}/private.pem" \
--key-id sourceful-test-1
uv run --python 3.12 --with cryptography --with jsonschema --with referencing \
python .device-support/tools/driver_package.py index \
--package-envelope "${output}/artifacts/manifest.envelope.json" \
--package-url "https://packages.example/${DRIVER}/${VERSION}/manifest.envelope.json" \
--channel beta \
--source-date-epoch "${source_date_epoch}" \
--public-key "${output}/public.pem" \
--key "${output}/private.pem" \
--key-id sourceful-test-1 \
--output "${output}/index.envelope.json"
- name: Verify source-built Python signatures, package binding and FTW Lua target in Go
if: env.DEVICE_SUPPORT_TOKEN != ''
working-directory: go
env:
FTW_DEVICE_SUPPORT_INDEX: ${{ runner.temp }}/device-support-package/index.envelope.json
FTW_DEVICE_SUPPORT_PACKAGE: ${{ runner.temp }}/device-support-package/artifacts/manifest.envelope.json
FTW_DEVICE_SUPPORT_ARTIFACT_DIR: ${{ runner.temp }}/device-support-package/artifacts
run: |
public_key="$(tr -d '\n' < "${{ runner.temp }}/device-support-package/public.raw.b64")"
FTW_DEVICE_SUPPORT_PUBLIC_KEY="${public_key}" \
go test -count=1 ./internal/driverrepo -run '^TestDeviceSupportPythonContract$'

compose:
name: module boundaries
needs: changes
Expand Down Expand Up @@ -448,13 +336,13 @@ jobs:
name: go test + vet
if: always()
needs:
[changes, core, web, drivers, device-support-contract, compose, e2e, contract]
[changes, core, web, drivers, compose, e2e, contract]
runs-on: ubuntu-latest
env:
RESULTS: >-
${{ needs.changes.result }} ${{ needs.core.result }}
${{ needs.web.result }}
${{ needs.drivers.result }} ${{ needs.device-support-contract.result }}
${{ needs.drivers.result }}
${{ needs.compose.result }}
${{ needs.e2e.result }} ${{ needs.contract.result }}
steps:
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,10 @@ fetched from that repository at the commit pinned in
make drivers
```

The files still ship. FTW's offline recovery set exists because startup is
deliberately local — a gateway boots and runs without the network, so a remote
refresh must never block it — so the image, the release tarballs and the tests
all read `drivers/`. They are simply fetched rather than committed, which is
The files still ship: they are the release's own drivers and what normally
runs. Startup is deliberately local — a gateway boots and runs without the
network, so a remote refresh must never block it — and the image, the release
tarballs and the tests all read `drivers/`. They are simply fetched rather than committed, which is
why a driver cannot be edited here at all. There is no file to open a pull
request against; fix it upstream and move the pin. CI fails if one is
committed.
Expand Down
16 changes: 8 additions & 8 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ by themselves add runtime behaviour or new protocol capabilities.
| Module | Source | Runtime | Responsibility |
|---|---|---|---|
| Core | [`go/cmd/ftw`](../go/cmd/ftw), [`go/internal`](../go/internal), [`web`](../web) | One Go binary | Configuration, telemetry, state, API/UI, safety, control and fallback planning |
| Drivers | Editable source in [`srcfl/device-drivers`](https://github.com/srcfl/device-drivers); bundled recovery in `drivers/*.lua`; host in [`go/internal/drivers`](../go/internal/drivers) | One sandboxed Lua VM per configured device | Vendor protocol, sign conversion and device commands |
| Drivers | Editable source in [`srcfl/device-drivers`](https://github.com/srcfl/device-drivers); the release's own copies in `drivers/*.lua`; host in [`go/internal/drivers`](../go/internal/drivers) | One sandboxed Lua VM per configured device | Vendor protocol, sign conversion and device commands |
| Optimizer | [`optimizer`](../optimizer), contracts in [`go/internal/mpc`](../go/internal/mpc) and [`go/internal/energyforecast`](../go/internal/energyforecast) | Compiled Energyplan worker | Solve the long-horizon plan and supply primary PV and household-load forecasts |

Core can run without the optimizer. Hardware cannot be accessed without a
Expand Down Expand Up @@ -194,9 +194,7 @@ is refused.
The public `srcfl/device-drivers` repo owns editable driver source, versions,
contracts, tests and FTW's signed release channel. FTW downloads only an
explicitly selected, content-addressed Lua asset after it verifies the signed
manifest. It never runs raw code from the repository branch. Device Support
may later consume an exact public commit for other products or a higher support
level.
manifest. It never runs raw code from the repository branch.

Each Lua artifact still contains its own `DRIVER` metadata and implements the
FTW lifecycle. [`go/internal/drivers/lua.go`](../go/internal/drivers/lua.go) is
Expand All @@ -212,9 +210,11 @@ Drivers are the only hardware-specific layer. They must:
- avoid policy decisions that belong in core;
- remain independently testable and hot-editable.

Bundled drivers provide the offline recovery set. A signed distribution index
is discovery only; FTW independently verifies the selected package and
artifact, while activation remains explicit and atomic. See
Bundled drivers are the release's own drivers and normally run. An owner's
selected signed version runs instead while it is at least as new as the
release's copy, or when it was chosen over a newer one. The signed manifest is
discovery only; FTW verifies the selected artifact against it, and activation
remains explicit and atomic. See
[writing-a-driver.md](writing-a-driver.md) and
[device-repository.md](device-repository.md).

Expand Down Expand Up @@ -639,7 +639,7 @@ There are two channels:
- `stable`: promotion of the exact commit already published and tested as beta.

Core ships as one release package, `ftw-linux-<arch>.tar.gz`, with the
launcher, the `ftw` command, web files, recovery drivers and Energyplan. On
launcher, the `ftw` command, web files, the release's drivers and Energyplan. On
a native install, systemd starts `ftw-launcher`, which runs Core from release
slots under `/opt/ftw`. `ftw update` downloads the next package into a slot;
the launcher runs it as a trial and commits it only once it becomes ready,
Expand Down
13 changes: 3 additions & 10 deletions docs/device-repository.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,13 @@ an expert try one driver ahead of a release. Its release workflow builds an FTW
artifact for each catalog driver from a reviewed `main` commit, signs one
manifest and publishes the files through GitHub Releases.

Device Support may later consume an exact public commit for another product or
a higher support level. That path does not own a second editable driver copy
and does not replace FTW's default channel.

## Resolution and recovery

A configured driver resolves in this order:

1. operator-owned local override;
2. explicitly activated managed artifact;
3. bundled recovery driver.
3. the release's own driver.

Drivers ship with the release. A managed artifact the owner selected runs
while it is at least as new as the release's own copy at the same path, or
Expand Down Expand Up @@ -129,8 +125,5 @@ makes that Lua artifact write-inert. These checks do not claim hardware test
coverage; the public catalog and support status hold that evidence.

Remote Lua never runs from a URL. Local unsigned drivers need an explicit
operator file and never claim signed or managed status. Bundled drivers remain
the offline recovery set.

FTW still understands `sourceful.driver-index/v1` for later signed Device
Support packages. That format is optional and is not the default source.
operator file and never claim signed or managed status. Bundled drivers are the
release's own drivers and work offline.
16 changes: 9 additions & 7 deletions docs/writing-a-driver.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,10 @@ A device that answers Modbus before its registers mean anything can call

## Where FTW loads a driver from

FTW resolves a driver file as local, then managed signed, then bundled.
Settings and fleet inventory mark the first case `local / unsigned`.
A local file wins. Otherwise the owner's selected signed version runs while it
is at least as new as the release's copy, or when it was chosen over a newer
one. Otherwise the release's own driver runs. Settings and fleet inventory mark
the first case `local / unsigned`.

Operator-only drivers belong in the persistent user-driver directory, not
inside a container layer:
Expand All @@ -146,12 +148,12 @@ inside a container layer:
- systemd: `/var/lib/ftw/drivers`;
- another native run: pass `-user-drivers <dir>`.

Local code works offline and never needs GitHub or Device Support. It gets no
auto-update or promotion and cannot claim signed package control. The normal
host capabilities and lifecycle still apply.
Local code works offline and never needs GitHub. It gets no auto-update or
promotion and never claims signed status. The normal host capabilities and
lifecycle still apply.

The bundled set under `drivers/` is FTW's offline recovery snapshot, generated
from the commit pinned in
The bundled set under `drivers/` is the release's own drivers, generated from
the commit pinned in
[`drivers/BUNDLED_SOURCE.json`](../drivers/BUNDLED_SOURCE.json) and fetched by
`make drivers`. It is not editable here: the files are gitignored and CI fails
if one is committed. Fix a driver upstream and move the pin. Managed drivers
Expand Down
35 changes: 9 additions & 26 deletions go/cmd/ftw/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,9 @@ func main() {
for _, w := range cfg.LoadWarnings {
slog.Error(w)
}
for _, notice := range cfg.Retired {
slog.Warn(notice)
}

// ---- Open persistent state (SQLite) ----
statePath := "state.db"
Expand Down Expand Up @@ -526,10 +529,10 @@ func main() {
slog.Error("initialize config database", "err", err)
os.Exit(1)
}
if dropped, err := config.DropRetiredSettings(st, *configPath, cfg); err != nil {
slog.Warn("could not remove retired Ask why settings", "err", err)
} else if dropped {
slog.Info("Ask why has been removed; its settings and API key were deleted")
if removed, err := config.DropRetiredSettings(st, *configPath, cfg); err != nil {
slog.Warn("could not remove retired settings", "err", err)
} else if len(removed) > 0 {
slog.Info("deleted the stored settings of removed features", "removed", strings.Join(removed, "; "))
}

if cfg.State != nil && cfg.State.ColdRetentionDays != 0 {
Expand Down Expand Up @@ -3325,34 +3328,14 @@ func inventoryRepositoryArtifacts(manager *driverrepo.Manager) []driverinventory
active := manager.Status().Active
out := make([]driverinventory.RepositoryArtifact, 0, len(active))
for _, installed := range active {
item := driverinventory.RepositoryArtifact{
out = append(out, driverinventory.RepositoryArtifact{
LogicalPath: installed.LogicalPath,
InstalledPath: installed.InstalledPath,
DriverID: installed.DriverID,
Version: installed.Version,
Comment on lines +3331 to 3335

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve managed provenance in driver inventory

For every active signed-channel driver, this now creates a RepositoryArtifact without PackageID or PackageChannel. driverinventory.inspectDriver only labels an artifact managed when both fields identify a package/channel; otherwise it emits source: "legacy_repository" (go/internal/driverinventory/inventory.go:192-198). Consequently, after retiring the package format, current FTW-channel installations are reported to Nova/fleet inventory as legacy repositories and lose their channel provenance. Update the inventory classification to recognize the remaining signed manifest format rather than depending on removed package fields.

Useful? React with 👍 / 👎.

SHA256: installed.SHA256,
RepositoryID: installed.RepoID,
}
versions, err := manager.AvailableVersions(installed.DriverID)
if err == nil {
for _, candidate := range versions {
driver := candidate.Driver
if candidate.RepositoryID != installed.RepoID || driver.Version != installed.Version || !strings.EqualFold(driver.SHA256, installed.SHA256) {
continue
}
item.PackageID = driver.PackageID
item.PackageChannel = driver.Channel
if driver.PackageID != "" {
if driver.Metadata.ReadOnly {
item.ControlClass = "read_only"
} else {
item.ControlClass = "control"
}
}
break
}
}
out = append(out, item)
})
}
return out
}
Expand Down
2 changes: 1 addition & 1 deletion go/internal/api/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -1864,7 +1864,7 @@ func (s *Server) handleDriversCatalog(w http.ResponseWriter, r *http.Request) {
if s.deps.DriverRepository != nil {
managedDir = s.deps.DriverRepository.EffectiveDir()
}
// Local override > activated managed artifact > bundled recovery snapshot.
// Local override > owner's selected signed driver > the release's own driver.
entries, err := drivers.LoadCatalogSources(
drivers.CatalogSource{Dir: s.deps.UserDriverDir, Source: "local"},
drivers.CatalogSource{Dir: managedDir, Source: "managed"},
Expand Down
Loading
Loading