Skip to content

tests: accept staged Cray MPICH in Comm_Backends provenance - #188

Merged
bobrobey merged 1 commit into
amd:mainfrom
dannybaths:c21-comm-backends-mpi-provenance
Sep 7, 2026
Merged

tests: accept staged Cray MPICH in Comm_Backends provenance#188
bobrobey merged 1 commit into
amd:mainfrom
dannybaths:c21-comm-backends-mpi-provenance

Conversation

@dannybaths

@dannybaths dannybaths commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds an MPI PROVENANCE: verdict to Pytorch_Comm_Backends_Check and
uses that verdict as the CTest pass criterion.

The existing GPU-AWARE MPI: line is left unchanged as the strict OpenMPI
diagnostic. The new provenance verdict applies Bob's 3 September rule: pass
when PyTorch is using either GPU-aware OpenMPI from the loaded module or a
staged Cray/MPICH-family MPI, and fail when it resolves to OS/system MPI or an
unrecognised MPI.

Why

After PR187, Tcl Environment Modules can be parsed correctly, but AAC7 still
fails the original gate honestly because the site PyTorch is linked to staged
Cray MPICH rather than the loaded OpenMPI module. That is not an OS MPI leak.

The new gate keeps the OpenMPI diagnosis visible while changing the CTest
decision to the intended provenance rule.

Validation

  • AAC7 job 10537: standalone Pytorch_Comm_Backends_Check passed with
    MPI PROVENANCE: OK for staged Cray MPICH.
  • AAC7 job 10539: focused Phase C in-harness run passed with the new CTest
    gate, Regex=[MPI PROVENANCE: OK].
  • AAC6 job 19659: Lmod/OpenMPI no-regression proof passed. The stock output
    before the new provenance line was identical; with this PR's CTest rule, the
    added MPI PROVENANCE: OK line matched the pass expression and the test
    passed.

@bobrobey bobrobey self-assigned this Sep 7, 2026
@bobrobey

bobrobey commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Good to get this to pass with Cray MPICH

@bobrobey bobrobey closed this Sep 7, 2026
@bobrobey bobrobey reopened this Sep 7, 2026
@bobrobey
bobrobey merged commit 7339602 into amd:main Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants