fix(registry): reopen when another caller wins creation - #241
Open
KaiqiJinWow wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RolloutRegistry::open_or_createcan fail during concurrent initialization. Two callers may both observeDatasetNotFound; if one creates the registry before the other attempts creation, the second caller receivesDatasetAlreadyExistsand fails startup despite a registry being available. This is what we observed in nightly build.For example, two server replicas can start against the same new object-store location:
In this PR, we handle this outcome by reopening the existing registry with the same storage options. Other create and load errors continue to propagate.
Testing
Added a deterministic regression test that simulates this interleaving: both callers observe an absent registry, one creates it and inserts a row, and the other attempts creation through the recovery path. The test checks that the reopened dataset has the expected version and retains the row.