Skip to content

20260924 - Write config.yml and user.yml atomically too - #43

Merged
Purple10101 merged 1 commit into
mainfrom
20260924-atomic-config-yml
Sep 24, 2026
Merged

Purple10101 merged 1 commit into
mainfrom
20260924-atomic-config-yml

Conversation

@Purple10101

@Purple10101 Purple10101 commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to #42, ClickUp OTA deployments miss part of the fleet on every release (123zgec4tmx), Design item 2a.

Why

#42 routed the compose .env copies, tar1090.env and retina-tracker.yaml through write_file_atomic(), but not blah2's config.yml (plain open(..., 'w')) or either write of user.yml (temp file and rename, no fsync).

Field evidence from ret9573ecda, read on 2026-09-24: at 2026-08-27 23:23:18 one power cut zeroed every file the config-merger wrote in that run.

File Written by State
config.yml plain open('w') 1643/1643 bytes NUL
tar1090.env temp + rename, no fsync 294/294 NUL
retina-tracker.yaml temp + rename, no fsync 24/24 NUL
manifests/.env shutil.copy in place 294/294 NUL
user.yml not written in that run intact

So temp-and-rename without fsync is not enough on these SD cards. With config.yml zeroed, blah2 aborts in its YAML parser (exit 134) and that node has produced no radar data since.

What

  • config.yml, the node_id sync of user.yml, and first-run creation of user.yml all go through write_file_atomic() (temp file, fsync, os.replace, fsync the directory). shutil is no longer used.
  • test_every_write_goes_through_write_file_atomic parses the script and fails on any write-mode open(), copy or rename outside the helper. test_config_yml_is_replaced_not_rewritten checks the file is replaced, not rewritten in place. Both fail against main.

Expected result once released

A power cut during a config-merger run leaves each file old or new, never zeroed. Together with #42's preflight, a node like ret9573ecda recovers at its next install.

Risk

An owner's user.yml is never reset by this. It is only created when it does not exist (if not os.path.exists(user_config_path)), which is a node's first boot, and the node_id sync loads the existing file, changes only network.node_id and writes the rest back as it was. On a release user.yml is only read, as before.

The one behaviour change is on that first boot: the old code had a branch for "another process created user.yml first". With os.replace, a second writer in that race replaces the file with the same default content, so nothing is lost. Only one container runs the config-merger today.

Evidence

75 tests pass (2 new); ruff clean.

🤖 Generated with Claude Code

#42 routed the compose .env copies, tar1090.env and retina-tracker.yaml
through write_file_atomic(), but not blah2's config.yml or either write of
user.yml. On ret9573ecda one power cut at 2026-08-27 23:23:18 zeroed every
file the config-merger wrote in that run, including the two written by
temp file and rename without fsync. With config.yml zeroed, blah2 aborts in
its YAML parser, and that node has produced no radar data since.

Every write now goes through the helper, and a test parses the script and
fails on any write-mode open(), copy or rename outside it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Purple10101
Purple10101 merged commit 5da7018 into main Sep 24, 2026
1 check passed
@Purple10101
Purple10101 deleted the 20260924-atomic-config-yml branch September 24, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant