Repository navigation
test(rust): stage watch-test writes and rename them into place - #1247
Merged
Merged
Conversation
fs::write truncates before it writes, so the watcher could read an empty file and the first Update carried an empty name.
8 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1246
Status
Test-only change in
packages/rust/tests/sdk.rs.just preflightpasses (0 failing, 3 packaging skips). No e2e receipt or changelog fragment owed.The race
fs::writeopens withO_TRUNCand then writes. The watcher's per-file hook runs on the first inotify event, which can land between the truncate and the write, soread_to_stringreturns""and the firstUpdate(orInsert) carries an emptyname.it_streams_watch_eventshad the same shape onnew_item.txt;async_sdk.rs::it_streams_watch_eventsalready avoided it by staging and renaming.What changed
A
write_watchedhelper insdk.rswrites to a.tmpsibling (outside the**/*.txtglob) andfs::renames it over the target, so the watcher sees one atomic replace with the final bytes. Bothit_streams_watch_eventsandit_streams_watch_update_eventsuse it. No sleeps, timeouts, retries, or loosened assertions. The other writes in the file happen beforewatch()or feed an extractor that ignores content, so they are not exposed.Reproduction
Serial runs never tripped it: 30 sequential runs of the prebuilt
sdktest binary (filtered to the two watch tests) understress --cpu 32 --io 8 --hdd 4passed 30/30 (/tmp/dirsql-1246-repro.sh).Concurrent copies did. Same binary, 10 copies at once per round under
stress --cpu 32 --io 8, on a 32-core box (/tmp/dirsql-1246-rounds.sh <bin> <label> 10 <rounds>):it_streams_watch_update_events:left: Text(""),right: Text("final")atsdk.rs:426A first attempt at 30 concurrent copies produced 8 failures, all
Too many open files (os error 24)from the inotify instance cap (max_user_instances=128, ~100 already in use); those are not the race and were discarded.Manual run:
cargo test -p dirsql --test sdkpasses 37/37 on this branch.