Repository navigation
TTMETAD: make WALKERS_MPI work with more than one rank per walker (#1443) - #1486
Merged
Merged
Conversation
multi_sim_comm is valid only on the first rank of each replica (plumed#1443). With WALKERS_MPI and domain decomposition, TTMETAD read the walker index and count from it and communicated over it on every rank, so the other ranks of each walker believed they were walker 0 of 1 and the run hung. - Take the walker count and index from getNumberOfReplicas() and getReplicaIndex(). - Use multi_sim_comm on the first rank of each walker only, then broadcast over comm: the walkers directory, the sketch count and the gathered hills. - Broadcast the sketch seed over comm, so that all ranks of a walker build the same sketch when DETERMINISTIC is not set (it came from time(nullptr) on each rank). - Write the sketch file from one rank of the replica only. The regtest runs 2 walkers with 2 ranks each under driver --debug-dd; COLVAR, HILLS and the forces match the run with one rank per walker. Without the fix it hangs.
This was referenced Oct 10, 2026
3 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes #1488.
Follow-up to #1443 for
TTMETADwithWALKERS_MPI.multi_sim_commis valid only on the first rank of each replica, butTTMETADread the walker count and index from it, and communicated over it, on every rank. With more than one MPI rank per walker (domain decomposition), the other ranks of each walker believed they were walker 0 of 1, and the run hung.getNumberOfReplicas()/getReplicaIndex()(Replica count and index on every rank of a replica (#1443, part C) #1481).multi_sim_common the first rank of each walker only, then are broadcast overcomm.comm. WithoutDETERMINISTICit came fromtime(nullptr)on each rank, so the ranks of one walker could build different sketches.Single-walker runs and runs with one rank per walker behave as before: the existing
ttsketch/rt-ttmetadandrt-ttmetad-2pass unchanged.Target release
I would like my code to appear in release master (v2.11)
Type of contribution
Copyright
Tests
New
ttsketch/rt-ttmetad-walkers-dd: 2 walkers with 2 ranks each underdriver --debug-dd yes. Walker 1 reads the same frames in reverse order, and two sketches are built.COLVAR,HILLSand all four force dumps are byte-identical to the run with one rank per walker. Without the fix the run hangs. Tested locally with ITensor 3.2.0 built as in.ci/install.itensor.