From a9223fc8ade1b98076da190e5a54f327b17d772c Mon Sep 17 00:00:00 2001 From: "Sean P. Kelly" Date: Mon, 24 Nov 2025 23:20:13 +0000 Subject: [PATCH] updater: add update metadata documentation Add documentation for update metadata format (manifest.json schema, migrations, release waves). Signed-off-by: Sean P. Kelly --- sources/updater/README.md | 31 +++++++++ sources/updater/update_metadata/README.md | 81 +++++++++++++++++++++++ sources/updater/updog/README.md | 44 +++++++++++- 3 files changed, 153 insertions(+), 3 deletions(-) create mode 100644 sources/updater/update_metadata/README.md diff --git a/sources/updater/README.md b/sources/updater/README.md index d9e4205a8..cf4edfdc0 100644 --- a/sources/updater/README.md +++ b/sources/updater/README.md @@ -1,6 +1,8 @@ # Bottlerocket update infrastructure This document describes the Bottlerocket update system and its components, namely; +**Keywords:** updates, TUF, tough, updog, signpost, update API, metadata, repository, manifest, targets, root.json, timestamp, snapshot, waves, rollout, versions, upgrade, downgrade, migrations + - [tough](#tuf-and-tough): implementation of "The Update Framework" (TUF) - [Bottlerocket update API](#update-api): allows for checking and starting system updates from TUF repo - [apiclient](../api/apiclient/README.md): automates interactions with the update API @@ -32,6 +34,35 @@ For Bottlerocket this includes a 'manifest.json' file and any update images or m Update metadata and files can be found by requesting and verifying these metadata files in order, and then requesting the manifest.json target which describes all available updates. Any file listed in the manifest is also a TUF 'target' listed in targets.json and can only be downloaded via the TUF repository, preventing the client from downloading untrusted data. +``` + ┌─────────────┐ + │ API Server │ + │ (renders) │ + └──────┬──────┘ + ▼ + ┌─────────────┐ + │ /etc/updog │──┐ + │ .toml │ │ metadata_base_url + └─────────────┘ │ targets_base_url + ▼ + ┌──────────┐ + │ TUF │ + │ Repo │ + └──────────┘ + │ + ┌────────────┼────────────┐ + │ │ │ + ▼ ▼ ▼ + root.json targets.json manifest.json + │ │ │ + └────────────┴────────────┘ + │ + ▼ (lists files & hashes) + ┌───────┴───────┐ + ▼ ▼ + update images migrations +``` + ## What's Updog [Updog](updog/) is a low-level update client, used behind the scenes by the update API, that interacts with a 'The Update Framework' (TUF) repository to download and write updates to a Bottlerocket partition. Updog will parse the manifest.json file from the TUF repository and will update to a new image if the following criteria are satisfied: diff --git a/sources/updater/update_metadata/README.md b/sources/updater/update_metadata/README.md new file mode 100644 index 000000000..8cfefe332 --- /dev/null +++ b/sources/updater/update_metadata/README.md @@ -0,0 +1,81 @@ +# Update Metadata Format + +This crate defines the data structures for Bottlerocket's update manifest and metadata. + +**Keywords:** update metadata, manifest.json, TUF targets, updates, migrations, waves, rollout, version, images, boot partition, root filesystem, dm-verity, schema, data structures + +## Manifest Schema + +The `manifest.json` file contains all available updates and is published as a TUF target in the update repository. + +### Top-Level Structure + +- `updates[]` - Array of Update objects describing available versions +- `migrations{}` - Map of version pairs to migration file lists + +### Update Object + +Each update in the manifest describes a specific Bottlerocket version: + +- `variant` - Bottlerocket variant (e.g., "aws-k8s-1.31", "metal-k8s-1.31") +- `arch` - Architecture ("x86_64" or "aarch64") +- `version` - Update version (semver format) +- `max_version` - Maximum version this update can safely upgrade to +- `waves{}` - Release wave schedule mapping seed values to start timestamps +- `images` - Image file paths (see below) + +### Images Object + +Paths to update files, relative to the TUF repository's `targets_base_url`: + +- `boot` - Boot partition image (typically `.ext4.lz4` compressed) +- `root` - Root filesystem image (typically `.ext4.lz4` compressed) +- `hash` - dm-verity hash tree (typically `.verity.lz4` compressed) + +All image files must be listed in the TUF repository's `targets.json` with their cryptographic hashes. + +### Example + +```json +{ + "updates": [ + { + "variant": "aws-k8s-1.31", + "arch": "x86_64", + "version": "1.20.0", + "max_version": "1.20.0", + "waves": { + "0": "2024-01-15T00:00:00Z", + "512": "2024-01-15T12:00:00Z", + "1024": "2024-01-16T00:00:00Z", + "2048": "2024-01-16T12:00:00Z" + }, + "images": { + "boot": "bottlerocket-aws-k8s-1.31-x86_64-v1.20.0-boot.ext4.lz4", + "root": "bottlerocket-aws-k8s-1.31-x86_64-v1.20.0-root.ext4.lz4", + "hash": "bottlerocket-aws-k8s-1.31-x86_64-v1.20.0-root.verity.lz4" + } + } + ], + "migrations": { + "(1.19.0, 1.20.0)": ["migrate_v1.19.0_v1.20.0"], + "(1.20.0, 1.19.0)": ["migrate_v1.19.0_v1.20.0"] + } +} +``` + +## Migrations + +The `migrations` map specifies migration binaries required when moving between versions. Each key is a tuple of `(from_version, to_version)`, and the value is a list of migration file names. + +Migrations are bidirectional - the same migration binary typically handles both upgrade and downgrade paths between two versions. + +## Release Waves + +Update waves allow staged rollouts of new versions. The `waves` map associates seed ranges with start times: + +- Hosts calculate their position in the wave based on their `settings.updates.seed` value (0-2048) +- Updates become available to a host only after its wave's start time has passed +- This prevents all hosts from updating simultaneously + +See [waves/README.md](../waves/README.md) for more details on wave scheduling. diff --git a/sources/updater/updog/README.md b/sources/updater/updog/README.md index 5a64d2e5d..988416c60 100644 --- a/sources/updater/updog/README.md +++ b/sources/updater/updog/README.md @@ -2,6 +2,8 @@ not much what's up with you +**Keywords:** updog, updates, TUF client, check-update, apply updates, download, waves, version, upgrade, rollout, update command + ## no really The Updog client provides an interface to a TUF repository and prepares for, downloads, and applies updates to the Bottlerocket instance. Updog can be called manually, but will more commonly be called automatically by some cluster orchestrator. For usage run `updog --help`. @@ -48,7 +50,43 @@ Starting update to 0.1.4 Update applied: aws-k8s-1.15 0.1.4 ``` -## Proxy Support +## Configuration + +Updog reads its configuration from `/etc/updog.toml`. This file is typically rendered by the Bottlerocket API server from system settings. + +### Configuration Format + +```toml +# TUF repository metadata location (required) +metadata_base_url = "https://updates.bottlerocket.aws/2020-07-07/aws-k8s-1.31/x86_64/" + +# TUF repository targets location (required) +targets_base_url = "https://updates.bottlerocket.aws/targets/" + +# Update wave seed value (required, 0-2048) +# Determines when this host receives updates in a staged rollout +seed = 1234 + +# Version selection policy (required) +# Options: "latest", "1.2.3" (specific version), "^1.2" (semver range) +version_lock = "latest" + +# Skip wave scheduling and update immediately (required) +ignore_waves = false + +# HTTPS proxy for update downloads (optional) +https_proxy = "http://proxy.example.com:3128" + +# Hosts to exclude from proxy (optional) +no_proxy = ["localhost", "169.254.169.254"] +``` + +### Configuration Fields -The `network.https-proxy` and `network.no-proxy` settings are taken from updog's config file. -These will override the environment variables `HTTPS_PROXY` and `NO_PROXY`. +- **metadata_base_url** - Base URL for TUF metadata files (root.json, timestamp.json, snapshot.json, targets.json) +- **targets_base_url** - Base URL for update images and migration files. Image paths in manifest.json are relative to this URL +- **seed** - Integer from 0-2048 used to calculate this host's position in update waves +- **version_lock** - Controls which updates are considered: "latest" for newest available, specific version string, or semver range +- **ignore_waves** - If true, updates are applied immediately without waiting for wave schedule +- **https_proxy** - Optional HTTP proxy URL for downloading updates (overrides HTTPS_PROXY environment variable) +- **no_proxy** - Optional list of hosts to exclude from proxy (overrides NO_PROXY environment variable)