20260924 - Write config.yml and user.yml atomically too - #43
Merged
Merged
Conversation
#42 routed the compose .env copies, tar1090.env and retina-tracker.yaml through write_file_atomic(), but not blah2's config.yml or either write of user.yml. On ret9573ecda one power cut at 2026-08-27 23:23:18 zeroed every file the config-merger wrote in that run, including the two written by temp file and rename without fsync. With config.yml zeroed, blah2 aborts in its YAML parser, and that node has produced no radar data since. Every write now goes through the helper, and a test parses the script and fails on any write-mode open(), copy or rename outside it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #42, ClickUp OTA deployments miss part of the fleet on every release (123zgec4tmx), Design item 2a.
Why
#42 routed the compose
.envcopies,tar1090.envandretina-tracker.yamlthroughwrite_file_atomic(), but not blah2'sconfig.yml(plainopen(..., 'w')) or either write ofuser.yml(temp file and rename, no fsync).Field evidence from ret9573ecda, read on 2026-09-24: at 2026-08-27 23:23:18 one power cut zeroed every file the config-merger wrote in that run.
config.ymlopen('w')tar1090.envretina-tracker.yamlmanifests/.envshutil.copyin placeuser.ymlSo temp-and-rename without fsync is not enough on these SD cards. With
config.ymlzeroed, blah2 aborts in its YAML parser (exit 134) and that node has produced no radar data since.What
config.yml, the node_id sync ofuser.yml, and first-run creation ofuser.ymlall go throughwrite_file_atomic()(temp file, fsync,os.replace, fsync the directory).shutilis no longer used.test_every_write_goes_through_write_file_atomicparses the script and fails on any write-modeopen(),copyorrenameoutside the helper.test_config_yml_is_replaced_not_rewrittenchecks the file is replaced, not rewritten in place. Both fail against main.Expected result once released
A power cut during a config-merger run leaves each file old or new, never zeroed. Together with #42's preflight, a node like ret9573ecda recovers at its next install.
Risk
An owner's
user.ymlis never reset by this. It is only created when it does not exist (if not os.path.exists(user_config_path)), which is a node's first boot, and the node_id sync loads the existing file, changes onlynetwork.node_idand writes the rest back as it was. On a releaseuser.ymlis only read, as before.The one behaviour change is on that first boot: the old code had a branch for "another process created user.yml first". With
os.replace, a second writer in that race replaces the file with the same default content, so nothing is lost. Only one container runs the config-merger today.Evidence
75 tests pass (2 new); ruff clean.
🤖 Generated with Claude Code