Skip to content

kenref: support more than one rank per replica (#1443) - #1490

Open
aalhossary wants to merge 2 commits into
plumed:masterfrom
aalhossary:issue1443-kenref-dd
Open

aalhossary wants to merge 2 commits into
plumed:masterfrom
aalhossary:issue1443-kenref-dd

Conversation

@aalhossary

@aalhossary aalhossary commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor
Description

Follow-up to #1443 for KENREF, in the same way as #1486 for TTMETAD. KENREF refused to run when a replica spans several ranks; this makes it work.

calculate() read the replica count and index from multi_sim_comm on every step and on every rank. A rank that is not the first of its replica only has the MPI_COMM_SELF stand-in, which reports "replica 0 of 1", so it would have applied a one-replica restraint to the atoms it owns without any error. That is why the module refused this case until now.

  • The replica count and index come from getNumberOfReplicas() / getReplicaIndex() (Replica count and index on every rank of a replica (#1443, part C) #1481), once, at setup.
  • Only the first rank of each replica gathers and scatters over multi_sim_comm. The derivatives are then broadcast over comm. The broadcast is required: every rank applies forces to the atoms it owns, so a rank without derivatives would apply none.
  • The energy is printed once per job, not once per rank.

Merge order. This needs the matching KEnRef change, Smith-Group/KEnRef@b4aa301 on release/2.x. The new code uses members that only that KEnRef declares, and a PLUMED and a KEnRef that do not match fail to compile; they never build and give wrong forces. This PR deliberately pins .ci/install.kenref to release/2.x, which contains that commit, so CI builds the KEnRef it needs. Between approval and merge (or in a follow-up), I will move v2.0.0 to that commit and return the pin to v2.0.0.

Target release

I would like my code to appear in release master (v2.11)

Type of contribution
  • changes to code or doc authored by PLUMED developers, or additions of code in the core or within the default modules
  • changes to a module not authored by you
  • new module contribution or edit of a module authored by you
Copyright
  • the module I added or modified contains a COPYRIGHT file with the correct license information. Code should be released under an open source license. I also used the command cd src && ./header.sh mymodulename in order to make sure the headers of the module are correct.
Tests
  • I added a new regtest or modified an existing regtest to validate my changes.
  • I verified that all regtests are passed successfully on GitHub Actions.

New kenref/rt-kenref-sigma-multi-dd: 2 replicas × 2 ranks with driver --debug-dd yes, using real two-member data. The replicas start from different frames, so their forces differ (by up to 4.56). The COLVAR files and all four force dumps are byte-identical to the same run with one rank per replica. Removing the broadcast makes all four force files fail. rt-kenref-sigma is unchanged and passes.

Also checked locally:

calculate() read the replica count and index from multi_sim_comm on
every step and on every rank. A rank that is not the first rank of its
replica only has the MPI_COMM_SELF stand-in, which reports "replica 0
of 1", so the module refused to run with more than one rank per
replica.

- The replica count and index come from getNumberOfReplicas() /
  getReplicaIndex() (plumed#1481), once, at setup.
- Only the first rank of each replica gathers and scatters over
  multi_sim_comm. The derivatives are then broadcast over comm, because
  every rank applies forces to the atoms it owns.
- The energy is printed once per job, not once per rank.

New regtest rt-kenref-sigma-multi-dd: 2 replicas x 2 ranks with
--debug-dd. Its references are byte-identical to the same run with one
rank per replica.

This needs the matching KEnRef change on Smith-Group/KEnRef release/2.x.
The kenref module now needs the KEnRef change that goes with it, which
is on release/2.x but not yet in the v2.0.0 tag. Before merge, v2.0.0
moves to that commit and this pin goes back to v2.0.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant