Sweep a killed request's tree from a supervisor-forked sweeper - #63
Merged
Merged
Conversation
The supervisor renamed a killed worker's home aside and left the unlinking to the next worker that answered on that slot. A worker killed at its deadline never reached that sweep, so a slot whose every request was killed stacked one tree per kill until the scratch was full. Fork a `Sweeper` every `sweep_interval` seconds when a slot holds a discarded tree, hold it to the cell's `deadline`, and run one at a time. The worker's own sweep stays as the fast path; `Filesystem.remove_tree` now counts a tree the other sweeper removed first as removed. See `HotCell::Sweeper` and the `sweep_interval` row in docs/DEPLOYMENT.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
flavorjones
commented
Sep 9, 2026
flavorjones
commented
Sep 9, 2026
flavorjones
commented
Sep 9, 2026
flavorjones
commented
Sep 9, 2026
flavorjones
commented
Sep 9, 2026
flavorjones
commented
Sep 9, 2026
`Filesystem.remove_tree` read a path it could not stat as gone, so a tree behind a slot directory a tool made unsearchable was counted as swept. The supervisor listed and sorted every slot directory in its loop to see whether a sweep was due, and a sweeper that died by signal left no line saying so. Only ENOENT means gone. Stream the slot directory and stop at the first discarded entry. Apply the cell's limits in the sweeper, log `sweeper.died` from the reap, and give `slot.unswept` the WARN level docs/LOGS.md already promised. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The sweeper took every worker limit, and RLIMIT_FSIZE killed it on its first log line whenever the log was a regular file already past `file_size`. A slot reported early was then the last one ever swept. Apply `memory` alone; the sweeper writes nothing but log lines. The comment on `Sweeper#run` now says what that limit does and does not bound. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Member
Author
|
🤖 Adversarial review by Codex, three rounds, converged. Fixed (commits eec09ad and c50f6d1)
Declined, with reasons
|
The sweeper carried its own copy of the worker's deadline timer and its own kill, `present?` said nothing about what it answered, and the `sweep_interval` row and changelog entries ran long. Share the timer as `Timed` between `Child` and `Sweep`, latch both with `killed_for`, and kill both through `kill_group`. Rename `present?` to `gone?`. Rewrite the setting's row, trim the changelog, and say in `sweep_test.rb` what in it is real. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The supervisor renamed a killed worker's home aside and left the unlinking to
the next worker that answered on that slot. A worker killed at its deadline
never reached that sweep, so a slot whose every request was killed stacked
one tree per kill until the scratch was full.
Fork a
Sweepereverysweep_intervalseconds when a slot holds a discardedtree, hold it to the cell's
deadline, and run one at a time. The worker'sown sweep stays as the fast path;
Filesystem.remove_treenow counts a treethe other sweeper removed first as removed. See
HotCell::Sweeperand thesweep_intervalrow in docs/DEPLOYMENT.md.🤖 Generated with Claude Code