[bartgol]

Adds a HIP variant, gnu13-mpich4-hip (x86_64 only), built through the same single Dockerfile as the CUDA one: a WITH_HIP=yes build arg (plus ROCM_VERSION, default 6.4.2) next to WITH_CUDA. It targets Instinct MI200 (gfx90a).

Design

• ROCm from AMD's RHEL 9 RPM repo (repo.radeon.com/rocm/el9/<ver>/main), not spack: spack's llvm-amdgpu is a multi-hour build. The amdgpu repo is also added, only for libdrm-amdgpu-devel, which hsa-rocr-devel needs and UBI lacks. Packages: rocm-hip-runtime-devel, rocprim-devel, rocthrust-devel, hipcub-devel, rocminfo, rocm-smi-lib. No kernel driver; the host provides it, as with CUDA.
• Env goes into the login profile at the end (like CUDA on the NVIDIA-repo path), so the spack builds do not see /opt/rocm/lib on LD_LIBRARY_PATH. HIPCC_*_FLAGS_APPEND=--gcc-install-dir=... points amdclang at gcc-toolset's libstdc++.
• Torch: ROCm wheel (torch==2.8.0+rocm6.4). No cupy equivalent.
• Arch: gfx90a. The EAMxx Kokkos arch defaults to AMD_GFX90A and E3SM_KOKKOS_HIP_ARCH overrides it. scream_mach_specs.py sets MPICH_CXX=hipcc and counts GPUs via rocminfo.
• aarch64: not supported (the Dockerfile errors out); no ARM host has these GPUs.
• Workflow: the new variant is in the env matrix, so dev, smoke, smoke-dev, promote and buildcache pick it up. smoke gets EXPECT_HIP. build-multiarch.yaml gets an opt-in free_disk input that deletes the runner's preinstalled SDKs only when less than 60 GB is free.
• Smoke test: Torch is a ROCm build, and hipcc compiles and links a gfx90a kernel and is driven through mpicxx.

Not done

• No GPU in CI, so nothing runs on a device (as with the CUDA images).
• CIME: the HIP image's machine offers the CPU gnu compiler only. A gnu-mphipcc compiler (the name E3SM already uses on Frontier, as craygnu-mphipcc) and a build-only e3sm-case job are planned as a stacked follow-up PR.
• The image is left out of e3sm-case and omega-case until then.

🤖 Generated with Claude Code

https://claude.ai/code/session_019UTqLci8P681gR1erTA488

[bartgol]

intel2024-openmpi4-dev (x86_64) failed in ghci/dev/Dockerfile's Node install: curl exit 18 (partial transfer) fetching the tarball from nodejs.org. This PR does not touch ghci/dev/ or the Intel variant, and the same step passed in the other -dev builds, so it looks like a network flake. No fix to port. The run is still in progress (the HIP and CUDA x86_64 builds), so GitHub refuses a re-run for now (403). I will re-run the failed job once when the run finishes.

---

Generated by Claude Code

[Copilot]

🟡 Changes recommended

Native HIP builds lack the required gcc-toolset compiler setting and an explicit HIP architecture, leaving compilation without needed headers and libraries and causing configuration to fail on hosts without a visible AMD GPU.

2 open issues

• Native HIP misses gcc-toolset headers without HIPFLAGS · New
• Native HIP architecture is unset on GPU-less hosts · New

What changed in this PR

Adds an x86_64 gnu13-mpich4-hip image for AMD Instinct MI200 GPUs,
extending the existing GHCI image pipeline with ROCm support.

Changes:

• Installs ROCm and ROCm-enabled Torch through the shared Dockerfile.
• Configures EAMxx standalone HIP builds and adds build-only smoke checks.
• Extends CI matrices, adds optional disk cleanup, and documents HIP usage.

File	Description
README.md	Lists the HIP image and host driver requirements.
ghci/​tests/​smoke.sh	Checks ROCm Torch and HIP compilation through hipcc and MPI.
ghci/​README.md	Documents HIP usage, configuration, and current limitations.
ghci/​env/​Dockerfile	Installs ROCm and configures its environment and metadata.
ghci/​env/​cime/​scream_mach_specs.py	Selects HIP compilation and counts available GPUs.
ghci/​env/​cime/​e3sm-container-eamxx.cmake	Enables Kokkos HIP with a configurable GPU architecture.
.github/​workflows/​ghci.yaml	Adds the HIP variant and smoke-test expectations.
.github/​workflows/​build-multiarch.yaml	Adds opt-in runner disk cleanup for large builds.

_{Review effort: Balanced}