Fix flaky up --wait failure on bucket-creation exit - #269
Merged
Conversation
docker compose up --wait fails whenever a container exits (even with code 0) unless some other service depends on it via condition: service_completed_successfully (docker/compose#10596, open since 2023). Neither sublime_create_buckets nor sublime_create_azure_blob_containers has such a dependent, so their normal successful exit races the rest of the stack's health checks and intermittently fails up --wait for no real reason. This is what tripped a go-mantis CI shard: container sublime_create_buckets-1 exited (0), then "Process completed with exit code 1." sublime_storage_ready is a long-running no-op that depends on both one-shot containers completing, giving compose the dependent it needs to treat their exit as expected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
There was a problem hiding this comment.
Pull request overview
This PR addresses a known docker compose up --wait flake where one-shot “init” containers exiting successfully can still cause --wait to fail unless another service depends on them via condition: service_completed_successfully.
Changes:
- Add a long-running sentinel service (
sublime_storage_ready) that depends onsublime_create_bucketsandsublime_create_azure_blob_containersviacondition: service_completed_successfully. - Document the rationale and link the upstream Compose issue to explain the workaround.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
sleep infinity works with the pinned alpine:3.20 BusyBox (verified), but tail -f /dev/null is the more universally portable no-op-forever idiom, per PR review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
thunderkatz
marked this pull request as ready for review
September 8, 2026 17:25
cameron-dunn-sublime
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docker compose up --waittreats a container exiting — even with code 0 — as a failure unless some other service depends on it viacondition: service_completed_successfully(docker/compose#10596, open since 2023, no upstream fix landed). Neithersublime_create_bucketsnorsublime_create_azure_blob_containershas such a dependent:sublime_nipper/sublime_screenshot_servicelistsublime_create_bucketsunder plaindepends_on(which only waits for it to start, not finish), and nothing depends onsublime_create_azure_blob_containersat all. So their normal, successful exit races the rest of the stack's healthchecks and can failup --waitfor no real reason.This is what tripped a go-mantis CI shard:
(https://github.com/sublime-security/go-mantis/actions/runs/34249933616/job/102141472992)
sublime_storage_readyis a long-running no-op (sleep infinity) that depends on both one-shot containers viacondition: service_completed_successfully, giving compose the dependent it needs to recognize their exit as expected — the documented workaround for this issue.Is this related to #268?
No — orthogonal. #268 fixed a real but different bug:
sublime_create_bucketsracing MinIO startup with a blindsleep 15and swallowing errors via an unconditionalexit 0. That's about whether bucket creation succeeds. This PR is about--wait's handling of a successful exit, which is gated purely by thedepends_ongraph — something #268 didn't touch.Concretely: the container still runs to completion and exits either way, before or after #268. In the failing run,
sublime_create_bucketsdidn't exit until ~14s in, coinciding withsublime_hydra's custom healthcheck (which takes a while to pass) — but the pre-#268 entrypoint's fixedsleep 15would have landed the exit in essentially the same window, so #268 doesn't look like it shifted the exposure meaningfully. It's plausible but not verified that #268 changed timing at the margins; either way it doesn't touch the actual root cause (the missingservice_completed_successfullydependent).Test plan
docker compose config --quieton the resulting file — valid.sublimes3,sublime_azurite,sublime_create_buckets,sublime_create_azure_blob_containers,sublime_storage_ready): with the sentinel,docker compose up -d --waitexits 0 after both one-shot containers exit.main-*CI shards over the next several days for absence of recurrence (this is an intermittent race — a single green run doesn't prove it).Note: local testing used a newer
docker compose(v5.5.1) than the CI runner, which reported the one-shot containers asHealthyrather thanExitedeven on the unmodifieddevcompose file — so the exact upstream trigger condition may be version-dependent, and I couldn't reproduce the CI failure signature locally byte-for-byte. The fix (adding aservice_completed_successfullydependent) is the documented, version-agnostic workaround for this class of bug regardless.🤖 Generated with Claude Code