Skip to content

Add Docker/Singularity image, GHCR workflow and docs - #65

Merged
koldunovn merged 3 commits into
mainfrom
feature/docker
Oct 15, 2026
Merged

koldunovn merged 3 commits into
mainfrom
feature/docker

Conversation

@armellereca

Copy link
Copy Markdown
Collaborator
  • Dockerfile (micromamba + environment.yml, pinned nereus, TeX Live), entrypoint, offline cartopy/regionmask data, smoke test

  • GitHub Actions: build + smoke test on PRs, push :latest/:sha-* to GHCR

  • pyproject: add missing runtime deps (gsw, kerchunk, fastparquet, ...), grib extra, explicit package-data for builds without .git

  • provenance.git_info falls back to FEATHER_GIT_* env vars in the image

    Container image (Docker + Singularity)

    One image for every pipeline step (diagnostics, analyze, report, website) plus the tests.

    • Dockerfile: micromamba + environment.yml (nereus from the pinned commit), TeX Live for report, offline cartopy Natural Earth +
      regionmask AR6 data
    • docker/: entrypoint (flags go to feather, anything else runs as a command; writable matplotlib cache under Singularity), offline-data
      fetch, smoke_test.sh, README (Docker, Levante Singularity, SLURM, API keys)
    • .github/workflows/docker.yml: build + smoke test on PRs; on main push ghcr.io/clidyn/feather:latest and :sha-<short>; v* tags →
      :<version>
    • pyproject.toml: adds runtime dependencies that were only in environment.yml (gsw, kerchunk, fastparquet, pandas, cftime, fsspec), a
      grib extra, and explicit package data so templates ship in builds without .git
    • provenance.git_info(): falls back to the FEATHER_GIT_* build args when there is no .git, so run records still name the commit (3 new
      tests)

    Tested on Levante: provenance tests (36 passed), wheel built without .git includes templates, offline-data fetch and offline lookup. The
    image itself is first built by this PR's workflow.

    🤖 Generated with Claude Code

- Dockerfile (micromamba + environment.yml, pinned nereus, TeX Live),
  entrypoint, offline cartopy/regionmask data, smoke test
- GitHub Actions: build + smoke test on PRs, push :latest/:sha-* to GHCR
- pyproject: add missing runtime deps (gsw, kerchunk, fastparquet, ...),
  grib extra, explicit package-data for builds without .git
- provenance.git_info falls back to FEATHER_GIT_* env vars in the image

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@armellereca
armellereca requested a review from koldunovn October 14, 2026 12:13
@armellereca

Copy link
Copy Markdown
Collaborator Author

@koldunovn , After this is merged, main publishes ghcr.io/clidyn/feather:latest. If that publish step fails with a permission error, an organisation admin
needs to allow it under Settings → Actions → General → Workflow permissions, by choosing "Read and write permissions". After that, will do the
Levante test is to pull the image with Singularity and run the smoke test plus one small diagnostic on a compute node.

@koldunovn

Copy link
Copy Markdown
Collaborator

The test stack on Anaconda terms of service. I woul dnot use Anacanda at all, and replace it with conda forge

Reviewer request: the Anaconda 'defaults' channel falls under the
Anaconda Terms of Service. environment.yml now lists conda-forge +
nodefaults, the image installs with --override-channels -c conda-forge,
and the smoke test fails if any conda package comes from another channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@armellereca

Copy link
Copy Markdown
Collaborator Author

Packages are still downloaded from conda.anaconda.org/conda-forge. That is the community conda-forge channel, which Anaconda only hosts;
their terms of service cover the defaults channel (repo.anaconda.com), not this one.

Expect about an hour for this build, since changing environment.yml invalidates the cached environment.

For @koldunovn :

  • Removed the defaults channel: environment.yml is now conda-forge + nodefaults, the image installs with --override-channels -c conda-forge, and the smoke test fails if any conda package comes from a non-conda-forge channel.

@koldunovn

Copy link
Copy Markdown
Collaborator

Still can't finish the docker build

The 2.0.5 solver took ~53 min on environment.yml (the runner was shut
down mid-solve on the conda-forge-only build). A dry run with 2.9.0
solves the same file in 13 s, all 328 packages from conda-forge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@armellereca

Copy link
Copy Markdown
Collaborator Author

the environment is conda-forge only now, and the smoke test checks every package's channel on each build. the PR can be merged and will try to test it on levante.

@koldunovn
koldunovn merged commit f46adfd into main Oct 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants