Migrate conda packaging to rattler-build (v1 recipe format) for GitHub Actions CI - #3031
Conversation
3991bb9 to
ee3ec67
Compare
b9750b9 to
b2a68e2
Compare
|
@antonwolfy @ndgrigorian It looks like I've managed to stabilize GitHub CI, so please take a look at it when you have time. I would also consider getting rid of installing |
|
@antonwolfy @vlad-perevezentsev @ndgrigorian please take a look when you have time |
Co-authored-by: Anton <100830759+antonwolfy@users.noreply.github.com>
antonwolfy
left a comment
There was a problem hiding this comment.
Thank you @ekomarova for contributing and improving the conda packaging
…b Actions CI (#3031) ## Add rattler-build support for GitHub Actions CI ### Summary This PR adds rattler-build as the package-building tool for the Conda package GitHub Actions workflow, alongside a new `rattler_recipe.yaml` (v1 recipe schema) generated from the existing `meta.yaml`. `meta.yaml` is left untouched and fully functional. This PR only changes what runs on GitHub Actions; it does not retire or replace the `conda-build` path. ### What changed - `.github/workflows/conda-package.yml`: build steps now call rattler-build instead of conda-build - `conda-recipe/rattler_recipe.yaml`: new file, the rattler-build (v1 schema) equivalent of `meta.yaml`. A handful of things had to be restructured because v1 format must always be valid plain YAML (no Jinja control-flow blocks) - `environments/build_conda_pkg.yml`: no longer needed for use ### Why these specific changes were needed - CONDA_BLD path — `rattler-build` outputs to `output/<platform>/` relative to the working directory by default, unlike `conda-build`'s $CONDA_PREFIX/conda-bld/ - `Compute version from git describe` step — `conda-build` runs `git describe` internally before rendering to populate GIT_DESCRIBE_TAG/GIT_DESCRIBE_NUMBER. `rattler-build` has no equivalent, so `rattler_recipe.yaml` now reads these via `env.get(...)`, and we compute + export them ourselves beforehand. - RATTLER_CHANNELS array — `channels-list` includes `--override-channels`, which `rattler-build`'s CLI doesn't recognize, so it's stripped before use. Expanded into a bash array (instead of a bare $VAR) so each `-c <channel>` reaches `rattler-build` as a separate argument — also fixes a shellcheck SC2086 warning. - Removed usage of `conda-verify` — it's a conda-build-specific post-build checker; `rattler-build` validates internally, so it's no longer needed. - Removed the two guard blocks and the `pyproject.toml` loop in `rattler_recipe.yaml` — v1 format must always be valid plain YAML, so Jinja control-flow ({% if %}/{% for %}) isn't supported. The version guard checks are dropped (an invalid override still fails to build, just via a less friendly solver error instead of a custom message), and the dependency list previously generated from `pyproject.toml's [build-system]` requires is now static — it needs manual updates if those requirements change. - Added `tests: - script: ...` block — v1 schema requires an explicit test type; the old `test: requires:` block installed test deps without running anything, which isn't valid anymore. Added a minimal import/version smoke test - Replace `miniconda` with `rattler-build` action in the build job — `rattler-build` is a standalone binary with no conda/Python dependency, so building conda packages with it doesn't need a full conda distribution on the runner. The installation takes about 1s d1fe302
Add rattler-build support for GitHub Actions CI
Summary
This PR adds rattler-build as the package-building tool for the Conda package GitHub Actions workflow, alongside a new
rattler_recipe.yaml(v1 recipe schema) generated from the existingmeta.yaml.meta.yamlis left untouched and fully functional. This PR only changes what runs on GitHub Actions; it does not retire or replace theconda-buildpath.What changed
.github/workflows/conda-package.yml: build steps now call rattler-build instead of conda-buildconda-recipe/rattler_recipe.yaml: new file, the rattler-build (v1 schema) equivalent ofmeta.yaml. A handful of things had to be restructured because v1 format must always be valid plain YAML (no Jinja control-flow blocks)environments/build_conda_pkg.yml: no longer needed for useWhy these specific changes were needed
CONDA_BLD path —
rattler-buildoutputs tooutput/<platform>/relative to the working directory by default, unlikeconda-build's $CONDA_PREFIX/conda-bld/Compute version from git describestep —conda-buildrunsgit describeinternally before rendering to populate GIT_DESCRIBE_TAG/GIT_DESCRIBE_NUMBER.rattler-buildhas no equivalent, sorattler_recipe.yamlnow reads these viaenv.get(...), and we compute + export them ourselves beforehand.RATTLER_CHANNELS array —
channels-listincludes--override-channels, whichrattler-build's CLI doesn't recognize, so it's stripped before use. Expanded into a bash array (instead of a bare $VAR) so each-c <channel>reachesrattler-buildas a separate argument — also fixes a shellcheck SC2086 warning.Removed usage of
conda-verify— it's a conda-build-specific post-build checker;rattler-buildvalidates internally, so it's no longer needed.Removed the two guard blocks and the
pyproject.tomlloop inrattler_recipe.yaml— v1 format must always be valid plain YAML, so Jinja control-flow ({% if %}/{% for %}) isn't supported. The version guard checks are dropped (an invalid override still fails to build, just via a less friendly solver error instead of a custom message), and the dependency list previously generated frompyproject.toml's [build-system]requires is now static — it needs manual updates if those requirements change.Added
tests: - script: ...block — v1 schema requires an explicit test type; the oldtest: requires:block installed test deps without running anything, which isn't valid anymore. Added a minimal import/version smoke testReplace
minicondawithrattler-buildaction in the build job —rattler-buildis a standalone binary with no conda/Python dependency, so building conda packages with it doesn't need a full conda distribution on the runner. The installation takes about 1s