Repository navigation
Dockerfile missing runtime deps for newly-merged modules (astroquery, numba, fitsio) #712
Description
Activity
Some packages are still (newly?) missing:
- scipy, used in ngmix
- joblib
- importlib_metadata
- tqdm
- LSSTDESC.Coord
- pyyaml
- astropy_iers_data
- pyerfa
We dug into this empirically against current
ghcr.io/cosmostat/shapepipe:develop(dev target, post-#728), testing each listed package both in read-only mode and in a writable sandbox. All import cleanly:Package Version astroquery 0.4.11 numba 0.65.1 fitsio 1.3.0 scipy 1.17.1 joblib 1.5.3 importlib_metadata (imports cleanly) tqdm 4.67.3 LSSTDESC.Coord 1.3.1 pyyaml 6.0.3 astropy_iers_data 0.2026.4.27.1.3.2 pyerfa 2.0.1.5 All three originally-flagged runner modules (
shapepipe.modules.{mask,ngmix,uncompress_fits}_package.*) also import cleanly. Read-only and writable sandbox give identical results.Pyproject confirms
astroquery/numba/joblib/tqdmare core deps;scipy/pyyaml/astropy_iers_data/pyerfa/LSSTDESC.Coord/importlib_metadatacome in as transitives ofastropy/mccd/modopt/galsim;fitsiois gated in both targets (runtime:--extra jupyter --extra fitsio;dev:--extra dev).@martinkilbinger — could the May 18 report have been against a cached or older image? If you can repro on a fresh
apptainer pull docker://ghcr.io/cosmostat/shapepipe:develop, drop the trace and we'll dig back in. Otherwise we'd like to close.— Claude on behalf of Cail
Resolved on
develop. The image's dependencies are now declared inpyproject.tomland frozen inuv.lock:astroqueryandnumbaare core dependencies, andfitsioships via thefitsioextra (the runtime image installs--extra fitsio; the dev image--extra dev, which includes it). All three of the flagged modules (mask,ngmix,uncompress_fits) import cleanly in the published image, and the test suite now runs inside that image in CI, so this class of missing-dep regression is caught automatically going forward.— Claude on behalf of Cail
Symptom
Several modules on
developimport packages that aren't installed in theghcr.io/cosmostat/shapepipe:developimage:shapepipe.modules.mask_package.mask+mask_runnerastroqueryshapepipe.modules.ngmix_package.ngmix+ngmix_runnernumbashapepipe.modules.uncompress_fits_package.uncompress_fits+uncompress_fits_runnerfitsioEach raises
ModuleNotFoundErrorat import time.Root cause
Likely from a recent merge (PR #702?) that landed module changes requiring new deps without updating the Dockerfile's
pip installlist. The v2.0 PR (#706) does include these in its Dockerfile rewrite, but develop's Dockerfile lags.Fix
Add the missing deps to the
pip installblock inDockerfileon develop. Quick patch; doesn't need to wait for the v2.0 Dockerfile overhaul in #706.Caught by
tests/unit/test_imports.py::test_module_imports[...]for the 6 affected modules (xfailed in #708).