fix(image): unblock bash kernel venv activation and workspace cache

CHANGELOG.md

@@ -23,9 +23,14 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 - Install Calrissian from commit `a95d87dd213d51b11faea7c6def4ced46d466a4a`
   with compatible isolated dependencies and bundled Dask resources.
-- Preinstall Hatch 1.16.2 and its runtime dependencies in an isolated environment,
-  avoiding a first-use download; add offline Hatch and Calrissian smoke checks.
-- Ensure the default workspace is writable by jovyan for tool configuration.
+- Preinstall Hatch 1.16.5 and its runtime dependencies in an isolated environment,
+  avoiding a first-use download and fixing interpreter discovery with virtualenv
+  21; add offline Hatch environment creation/activation and Calrissian smoke checks.
+- Set `VIRTUAL_ENV_DISABLE_PROMPT=1` for the Bash kernel only: when `~/.bashrc` activates a
+  virtual environment, activating another one in a notebook cell restored the pre-kernel
+  prompt and the cell never finished. Terminals keep the `(venv)` prompt.
+- Ensure the default workspace is writable by jovyan for tool configuration, and
+  remove the root-owned build cache that blocked `uv` and pip caches there.
 - Make Mike Farah's pinned `yq` the default command while retaining the Python
   `yq` dependency and `tomlq`/`xq` launchers; smoke-test YAML and TOML processing.
 

Dockerfile

@@ -107,6 +107,11 @@ RUN pip install --no-cache-dir \
     cwltest && \
     python -m bash_kernel.install
 
+# Bash kernels detect a finished cell by their prompt. When ~/.bashrc activates a virtual
+# environment, activating another one in a cell restores the pre-kernel prompt and the cell
+# never finishes; keep venv activation away from the kernel prompt (terminals keep theirs).
+RUN python -c "import json, pathlib; p = pathlib.Path('/usr/local/share/jupyter/kernels/bash/kernel.json'); spec = json.loads(p.read_text()); spec.setdefault('env', {})['VIRTUAL_ENV_DISABLE_PROMPT'] = '1'; p.write_text(json.dumps(spec))"
+
 # Resolve the commit's cwltool/dependency requirements together, independently
 # of the global CWL tools, rather than constraining cwl-utils to the old release.
 RUN python -m venv /opt/calrissian-venv && \
@@ -131,7 +136,7 @@ RUN curl -fsSL \
 # -------------------------------------------------------------------
 # hatch
 # -------------------------------------------------------------------
-ARG HATCH_VERSION=1.16.2
+ARG HATCH_VERSION=1.16.5
 ENV HATCH_CONFIG=/home/jovyan/.config/hatch/config.toml \
     HATCH_DATA_DIR=/home/jovyan/.local/share/hatch \
     HATCH_CACHE_DIR=/home/jovyan/.cache/hatch
@@ -186,8 +191,10 @@ RUN apt-get update && apt-get install -y \
 # tomlq requires the Python yq package, whose console launcher shadows the
 # pinned Mike Farah binary on Conda's PATH. Keep its modules and tomlq/xq
 # launchers; replace only the conflicting command, without changing PATH.
-# Prepare the default workspace for non-root tools after runtime installation.
+# Prepare the default workspace for non-root tools after runtime installation,
+# dropping the root-owned cache that build-isolated pip runs leave in HOME.
 RUN ln -sf /usr/local/bin/yq /opt/conda/bin/yq && \
+    rm -rf /workspace/.cache && \
     install -d -o ${USER} -g users /workspace
 
 # -------------------------------------------------------------------

README.md

@@ -111,7 +111,8 @@ YAML fixture with `yq eval`, reads a TOML fixture with `tomlq`, and runs
 `python -m pip check`. The Python `yq` package remains installed for `tomlq`;
 only its conflicting `yq` launcher is replaced, leaving Conda's PATH intact.
 The focused runtime check also runs Hatch version and local project metadata
-commands without networking, verifies Calrissian's Git commit provenance, Dask
+commands, creates and activates a default environment without networking (skipping
+project installation), verifies Calrissian's Git commit provenance, Dask
 import, CLI flags, and bundled schema/init/dispose scripts, and checks both
 isolated environments' dependencies. It does not execute Dask on Kubernetes.
 To run these checks against a loaded image on either platform:
@@ -246,7 +247,7 @@ Installed via pip (Python 3.12):
 
 ### Python Build & Packaging
 
-- hatch: 1.16.2
+- hatch: 1.16.5
 
   Python project manager and build tool, installed with all runtime dependencies
   in `/opt/hatch-venv`, using the base image's native Python on AMD64 and ARM64.

ci/test-runtime.sh

@@ -6,19 +6,37 @@ test "$(id -u)" = 1001
 test -w "$HATCH_CONFIG"
 test -w "$HATCH_DATA_DIR"
 test -w "$HATCH_CACHE_DIR"
+# Tools such as uv and pip create caches under the default /workspace HOME.
+test -z "$(find /workspace ! -user jovyan -print -quit)"
 test "$(readlink -f "$(command -v hatch)")" = /opt/hatch-venv/bin/hatch
 test "$(readlink -f "$(command -v calrissian)")" = /opt/calrissian-venv/bin/calrissian
 fixture_dir="$(mktemp -d)"
 trap 'rm -rf "$fixture_dir"' EXIT
 
-[[ "$(hatch --version)" == 'Hatch, version 1.16.2' ]]
+[[ "$(hatch --version)" == 'Hatch, version 1.16.5' ]]
 cat > "$fixture_dir/pyproject.toml" <<'TOML'
 [project]
 name = "offline-smoke"
 version = "1.2.3"
+
+[tool.hatch.envs.default]
+skip-install = true
 TOML
 cd "$fixture_dir"
 [[ "$(hatch version)" == '1.2.3' ]]
+# Exercise interpreter discovery, which failed with older Hatch and virtualenv 21.
+# Skip project installation so this regression check can run without networking.
+hatch env create default
+environment_dir="$(hatch env find default)"
+test -x "$environment_dir/bin/python"
+(
+  source "$environment_dir/bin/activate"
+  test "$(command -v python)" = "$environment_dir/bin/python"
+  python -c 'import os, sys; assert sys.prefix == os.environ["VIRTUAL_ENV"]'
+)
+hatch env remove default
+# Activating a venv in a Bash kernel must not touch the prompt the kernel waits for.
+jupyter kernelspec list --json | python -c 'import json, sys; spec = json.load(sys.stdin)["kernelspecs"]["bash"]["spec"]; assert spec["env"]["VIRTUAL_ENV_DISABLE_PROMPT"] == "1", spec'
 /opt/hatch-venv/bin/python -m pip check
 
 /opt/calrissian-venv/bin/python - <<'PYTHON'