Conversation
Require a desired shutdown state before retiring an older-revision task when no node can accept it. Keep still-runnable Pending tasks queued, and verify assignment of the same task after node recovery. Fixes moby/moby#53722 Related to moby/moby#36699 Assisted-By: Astra (AI) Signed-off-by: Mazyad Alabduljaleel <mazjaleel@gmail.com>
koriyoshi2041
approved these changes
Sep 20, 2026
koriyoshi2041
left a comment
There was a problem hiding this comment.
Verified the scheduler ownership boundary at f54226a: only older-revision tasks already desired Shutdown or later take the retirement path; runnable and job-completion targets stay on the existing retry path. The regression also preserves the original task ID across node recovery, while TestUnscheduleableTask continues to cover obsolete Pending-task shutdown. Full scheduler tests and the race-enabled scheduler suite pass locally; diff hygiene is clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes moby/moby#53722
Related to moby/moby#36699. Its later node-restart reports may share this cause, but lack the service-revision evidence needed to confirm that. The original issue concerns obsolete tasks desired Shutdown, whose behavior this patch preserves.
- What I did
Fix a valid Pending task being dropped from scheduler retries after a service-only metadata update. If no node can accept the task during a subsequent scheduler pass, the older-revision branch currently skips re-enqueueing it even though it is still desired Running. It can remain Pending after node recovery until a force update creates a replacement.
- How I did it
Require desired state Shutdown or later before taking the older-revision retirement branch in
noSuitableNode. Still-runnable tasks reach the existing re-enqueue path. Add one regression with identical task specifications and different revision numbers, checking queue retention and assignment of the same task ID after Down becomes Ready.- How to test it
Base:
1fd637ba5cc32ff30d1dd2bdb14997bd4f424b46; Go 1.27.1 linux/amd64.go test ./manager/scheduler -run '^TestPendingOldRevisionRecoversWhenNodeReturns$' -count=1fails because the task is absent fromunassignedTasks.go test ./manager/scheduler -count=1andgo test -race ./manager/scheduler -count=1pass. The existingTestUnscheduleableTaskcontinues to verify retirement of an obsolete Pending task.go test -parallel 8 -timeout 20m ./... -count=1passes for the full root module, including integration, jobs, task reaper, and updater packages. A short temporary path is needed for the node tests' Unix sockets in a deeply nested checkout.Runtime comparison: Moby
464cd50c3d9e92877d56940ea160de6fca7bea23, vendoring SwarmKitad0357aedfca72c288abb8395ec5603e1136a9e5; Go 1.27.1,CGO_ENABLED=0, tagsnetgo osusergo static_build nri_no_wasm. Binary SHA-256: stock34747d599445c61b0a79e1e7dc7bd175ce9f3a9885576cded4a6d16d7b46e6c6; fixedd2fd714b5dcddc237192cff9a1da44774e0874209da1b81593dc475cafecf495. The separateswarmdmodule and full Moby suite were not run.- Description for the changelog
Keep valid pending tasks eligible for scheduling after service metadata updates and node recovery.
Created with: Astra (AI), under my supervision and review.