20260924 - Repair node state before install and write .env atomically

[Purple10101]

Phase 1 of ClickUp OTA deployments miss part of the fleet on every release (123zgec4tmx). Draft until the Josh Test Node result from the v0.4.7.0-dev test build is in the ticket.

Why

Two kinds of node drift have failed retina-node installs and then failed their rollbacks the same way, leaving the radar stack stopped. The docker-compose Update Module only finds either after it has stopped the running stack.

• NUL-filled manifests/.env (ret9573ecda, v0.4.5.0 and v0.4.6.0). The config-merger rewrote .env in place with shutil.copy and no fsync, seconds after every boot, and a power cut then left it zeroed. Compose refuses to load the project, including the run config-merger that would regenerate it, so the node cannot recover by itself.
• A stray container on a project container_name (Josh Test Node, v0.4.6.0 twice): a hand-run blah2:specfold held blah2.

What

• scripts/mender-state-scripts/ArtifactInstall_Enter_10_retina_preflight, shipped in the artifact, runs before the module. It sets a corrupt .env aside and parks non-project containers on project names (restart no, stopped, renamed). It repairs and never aborts: an aborted install sends the module's rollback down its "rollback after a commit" branch, which moves the live current/ aside. Nothing is deleted, it always exits 0, and each repair is logged to stderr and /data/mender-docker-compose/preflight.log (capped at 200 lines).
• config-merger: write_file_atomic() (temp file, fsync, rename, fsync the directory) for both compose .env copies, tar1090.env and retina-tracker.yaml.
• CI now also runs scripts/test/.

Expected result once released

ret9573ecda installs at its next online window with a set aside ... NUL bytes line in its deployment log. A node with a stray container installs with a parked container line. Every clean node logs nothing from the preflight and installs exactly as before.

Risks

Runs as root on every node before install. A false positive on .env renames a good file and tar1090 starts with default location settings until the config-merger rewrites it on the next start. Parking could stop a container someone wanted: it is stopped and renamed, not removed.

Evidence so far

• 69 tests pass: 57 config-merger (4 new), 11 preflight tests running the real script under dash against a stub docker, and 1 asserting every state script is executable. That last one fails against 991b0aa, which was committed as 100644 from a core.fileMode=false checkout and fixed in defca52.
• The script run on Josh Test Node against real docker with an injected NUL .env and stray container: both repaired, project container untouched, second run silent, live stack unchanged.
• Still to prove with v0.4.7.0-dev (branch 20260924-ota-preflight-dev, which adds only the self-referential CONFIG_MERGER_V pin and is not for main): the script inside a real Mender install, whether its stderr reaches the deployment log, and the manifests.tar payload path.

🤖 Generated with Claude Code

[Purple10101]

Josh Test Node result, v0.4.7.0-dev (2026-09-24)

Faults injected before deploying: a 208-byte NUL current/manifests/.env, and a stray retina-spectrum container (not in the project, restart always). Single-device deployment fe9c6b26:

• success in 4 min 25 s
• preflight ran at 09:23:10, before the module's Installing docker-compose artifact, and mender-updated collected both lines into the deployment log: set aside .../.env (contains NUL bytes) and parked container retina-spectrum ... as retina-spectrum-parked-...
• .env regenerated as a new file (new inode, 0 NUL bytes, content identical to the pre-test file) by retina-config-merger:v0.4.7.0-dev
• radar up afterwards: blah2 v0.6.0, 0 restarts, /api/map 200

Artifact header checked before deploying: the script is present, -rwxr-xr-x, byte-identical to this branch.

Found alongside, not caused by this PR: after any retina-node update without a following reboot, tar1090 runs with RECEIVER_LAT=0 and adsb.lol off, because the module starts the stack from new/manifests, which has no .env. Normally hidden by the owl-os reboot that follows a release (Fairforest B, 2026-09-23). Recorded in the ticket, to be fixed separately.

Full record: ClickUp 123zgec4tmx, Verification.

[Purple10101]

tar1090 reload (46e8f0a), Josh Test Node result, v0.4.7.0-dev2

ArtifactCommit_Leave_10_retina_env_reload runs up -d --no-deps tar1090 in current/manifests after the install is committed.

Deployment 92efa170 on a clean node, success. Docker events show the defect and the fix in one install:

• 09:43:50 the module creates tar1090 06c6d8b132f4 from new/manifests (no .env, so defaults)
• 09:43:58 the reload script runs
• 09:44:03 compose destroys 06c6d8b132f4 and starts 80f2a0de733c, because the config changed; blah2 (started 09:43:52) is untouched
• afterwards tar1090 has RECEIVER_LAT=42.241528, ADSBLOL_ENABLED=true, healthy

The same command run while tar1090 was already correct changed no container (Container tar1090 Running, all IDs identical). The preflight stayed silent on this clean node. 73 tests pass.