From 6c363cc179219727fe6f6b63970ce48e11794557 Mon Sep 17 00:00:00 2001 From: Babissimo Date: Wed, 23 Sep 2026 11:50:59 +0100 Subject: [PATCH 1/2] ADR: Python apps lock their dependencies with uv The standard installs every Python repo with `uv pip install -r requirements.txt`, which pins direct dependencies and re-resolves the rest on every install. For something that deploys, that lets the image, CI and each developer's venv run different trees: retina-server pinned 15 of the 54 third-party packages its image runs, and a developer venv had drifted from production on pydantic and websockets with nothing in the repo recording either choice. Apps become uv projects with a committed uv.lock, synced with --locked in CI and into a venv in images. Libraries are deliberately out of scope: whether they commit a lock is open in ClickUp 86cb49y4k, where the recommendation is that they should, and this record does not pre-empt it. Ticket: ClickUp 123zgec4jmx. Co-Authored-By: Claude Opus 5.5 --- .../2026-09-23-python-apps-lock-with-uv.md | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 docs/decisions/2026-09-23-python-apps-lock-with-uv.md diff --git a/docs/decisions/2026-09-23-python-apps-lock-with-uv.md b/docs/decisions/2026-09-23-python-apps-lock-with-uv.md new file mode 100644 index 0000000..e617f8e --- /dev/null +++ b/docs/decisions/2026-09-23-python-apps-lock-with-uv.md @@ -0,0 +1,58 @@ +# Python apps lock their dependencies with uv + +- **Date:** 2026-09-23. **Status:** proposed; accepted when this merges. +- **Scope:** Python apps. Whether the five vendored `retina-*` libraries commit a lock is a + separate question, open in ClickUp `86cb49y4k`, and this record does not settle it. +- **Ticket:** ClickUp `123zgec4jmx`. **Follow-up:** `123zgec4jn0`, the other apps. + +## 1. Context + +The `setup-repo` standard installs every Python repo the same way: +`uv pip install -r requirements.txt -r requirements-dev.txt`. The requirements files pin the +direct dependencies and nothing else, so every install resolves the rest of the tree afresh. + +For a deployed app that means the image, CI and each developer's venv can each run a different +tree. retina-server pinned 15 of the 54 third-party packages its image runs; pydantic, +SQLAlchemy, websockets and the rest were chosen again whenever the image's dependency layer +rebuilt. On +2026-09-23 a developer venv differed from production on pydantic (2.13.4 against 2.13.5) and +websockets (16.1.1 against 17.1), with nothing in either repository to say which was intended. + +## 2. Decision + +An **app** is a repo that runs as a deployed service or a tool, and that no other repo installs +as a dependency. Apps are uv projects: + +- `pyproject.toml` declares the runtime dependencies under `[project]` and the tooling under + `[dependency-groups] dev`. `[tool.uv] package = false` unless the app is itself installed. +- `uv.lock` is committed and is the only record of what gets installed. There is no + `requirements.txt`. +- CI runs `uv sync --locked`, which fails when the lock no longer matches `pyproject.toml`. +- Images run `uv sync --locked --no-dev` into a virtualenv put first on `PATH`, not into the + system site-packages: `uv sync` removes whatever the lock does not name, the base image's own + `pip` included. +- Libraries vendored as submodules are `[tool.uv.sources]` path entries, editable for local work + and installed with `--no-editable` in images. + +## 3. Consequences + +- A dependency change and its lock land in the same commit. So does a submodule bump that changes + a vendored library's own dependencies, which is the easy one to miss; `--locked` in CI catches + it. +- Upgrades are deliberate: `uv lock --upgrade-package `, or `uv lock --upgrade` for the + whole tree. Nothing relocks on its own, so a transitive security release reaches an app only + when someone relocks. No Python repo runs a dependency bot today. Dependabot's `uv` ecosystem + could do that job, and adopting it is a separate decision. +- Moving an app to a lock need not move any version. Seed the first `uv lock` with + `[tool.uv] constraint-dependencies` listing what production runs, then delete the constraints + and lock again: uv keeps a locked version until told to upgrade it, so the lock ends at the + deployed versions and the first image built from it matches the running one. +- The uv that writes a lock and the uv that reads it need not match. A lock written by 0.12.5 + syncs under 0.9.22. + +## 4. Adoption + +`setup-repo` gains a `python-app` stack that scaffolds §2. Its `python` stack stays as it is and +remains the scaffold for libraries. retina-server moves first. tower-finder-service, +retina-telemetry, retina-gui and node-infra's `mender-auto-accept` install with plain pip today +and move under `123zgec4jn0`. From 958a5e4029fbf940dcdc80fd52ac72314518db10 Mon Sep 17 00:00:00 2001 From: Babissimo Date: Wed, 23 Sep 2026 11:50:59 +0100 Subject: [PATCH 2/2] setup-repo: add a python-app stack that scaffolds a locked uv project The ADR in the previous commit makes apps uv projects; this is what lets a new repo start that way. `python-app` writes a pyproject.toml with [project], a dev dependency group and `package = false`, and a CI workflow that runs `uv sync --locked`. It ships no requirements files and no lock: the skill generates uv.lock at install time and tells the user to commit it, since CI fails without one. The existing `python` stack is untouched and is described as the library scaffold. pythonpath = ["."] is set because an unpackaged app is never installed, so tests import it from the repo root; that is the ImportError adopting-core-setup-repo documents for the library template. The scaffold test locks, syncs and runs the tests of a freshly scaffolded app when uv is available, which the setup-repo CI job has. Bumps core to 0.6.0. Co-Authored-By: Claude Opus 5.5 --- plugins/core/.claude-plugin/plugin.json | 2 +- plugins/core/skills/setup-repo/SKILL.md | 26 +++++++++++----- .../setup-repo/assets/ci/ci-python-app.yml | 31 +++++++++++++++++++ .../assets/stack/python-app/pyproject.toml | 31 +++++++++++++++++++ .../setup-repo/scripts/scaffold-repo.sh | 11 +++++-- tests/setup-repo/test-ci-assets.sh | 22 +++++++++++++ tests/setup-repo/test-python-assets.sh | 23 ++++++++++++++ tests/setup-repo/test-scaffold.sh | 25 +++++++++++++++ 8 files changed, 160 insertions(+), 11 deletions(-) create mode 100644 plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml create mode 100644 plugins/core/skills/setup-repo/assets/stack/python-app/pyproject.toml diff --git a/plugins/core/.claude-plugin/plugin.json b/plugins/core/.claude-plugin/plugin.json index 62fa199..1a1a95a 100644 --- a/plugins/core/.claude-plugin/plugin.json +++ b/plugins/core/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "core", - "version": "0.5.0", + "version": "0.6.0", "description": "Core Offworld Labs skills, commands, agents, and hooks shared across all repos.", "author": { "name": "Offworld Labs" diff --git a/plugins/core/skills/setup-repo/SKILL.md b/plugins/core/skills/setup-repo/SKILL.md index 506a232..d9f21e3 100644 --- a/plugins/core/skills/setup-repo/SKILL.md +++ b/plugins/core/skills/setup-repo/SKILL.md @@ -19,18 +19,26 @@ interactive parts and the report. `git init`; abort if the user declines. 2. **Determine the stack.** Detect from existing files: `pyproject.toml` / - `requirements*.txt` → `python`; `package.json` / `tsconfig.json` → TypeScript - (then ask whether it's `ts-frontend` — a React/Vite app — or `ts-backend` — a - Node service). If ambiguous or empty, ask the user to choose `python`, - `ts-frontend`, `ts-backend`, or `none`. + `requirements*.txt` / `uv.lock` → Python (then ask whether it's an app, + `python-app`, which runs as a service or tool and which no other repo installs, + or a library, `python`, which other repos install); `package.json` / + `tsconfig.json` → TypeScript (then ask whether it's `ts-frontend` — a React/Vite + app — or `ts-backend` — a Node service). If ambiguous or empty, ask the user to + choose `python-app`, `python`, `ts-frontend`, `ts-backend`, or `none`. + `python-app` follows `claude-shared/docs/decisions/2026-09-23-python-apps-lock-with-uv.md`. 3. **Scaffold the files.** Run the engine, which never overwrites existing files: `bash "$ENGINE" . ` Relay its `WRITTEN` / `SKIPPED` output to the user. 4. **Install dependencies.** - - **Python:** use `uv` (the org standard, a fast drop-in for pip that reads the - same `requirements.txt`): `uv venv && uv pip install -r requirements.txt -r requirements-dev.txt`. + - **python-app:** set `[project] name` in `pyproject.toml` to the repo's name, + then `uv lock && uv sync`. Remind the user to commit `uv.lock`: the CI + workflow runs `uv sync --locked` and fails without it. Add dependencies with + `uv add ` (`uv add --dev ` for tooling), never a requirements file. + There is no pip fallback; if `uv` is absent, skip and say so. + - **python (library):** use `uv`'s pip interface, which reads the same + `requirements.txt`: `uv venv && uv pip install -r requirements.txt -r requirements-dev.txt`. Fall back to `pip install -r requirements.txt -r requirements-dev.txt` in an active virtualenv if `uv` is absent. - **ts-frontend / ts-backend:** run `npm install` (this generates @@ -38,7 +46,8 @@ interactive parts and the report. `npm ci`). - **pre-commit (python & ts stacks):** after the stack deps are installed, register the git hook so the scaffolded `.pre-commit-config.yaml` runs on - every commit: `uvx pre-commit install` (or `pipx run pre-commit install`, or + every commit: `uv run pre-commit install` for `python-app`, whose dev group + carries pre-commit; otherwise `uvx pre-commit install` (or `pipx run pre-commit install`, or `pip install pre-commit && pre-commit install`). Skip with a note if pre-commit/uv is unavailable. Report the command and result; if the toolchain is unavailable, skip and note it @@ -48,7 +57,8 @@ interactive parts and the report. one-or-two-line description of what this repo does, then fill in the `Project Overview`, `Build & Test Commands`, and `Local Architecture` sections from their answer plus what was scaffolded (stack; lint/format via - `pre-commit run --all-files`; tests via `pytest` or `npm test`). If they skip, + `pre-commit run --all-files`; tests via `pytest`, `uv run pytest` for + `python-app`, or `npm test`). If they skip, leave the stub as-is. Keep CLAUDE.md under the 200-line ceiling noted in the template. diff --git a/plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml b/plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml new file mode 100644 index 0000000..02148e2 --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml @@ -0,0 +1,31 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + +jobs: + lint-and-test: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install uv + uses: astral-sh/setup-uv@v5 + + # --locked fails the job when uv.lock no longer matches pyproject.toml. + - name: Install dependencies + run: uv sync --locked + + - name: Pre-commit + run: uv run --locked pre-commit run --all-files --show-diff-on-failure + + - name: Pytest + run: uv run --locked pytest diff --git a/plugins/core/skills/setup-repo/assets/stack/python-app/pyproject.toml b/plugins/core/skills/setup-repo/assets/stack/python-app/pyproject.toml new file mode 100644 index 0000000..7e76710 --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/stack/python-app/pyproject.toml @@ -0,0 +1,31 @@ +[project] +name = "app" +version = "0.0.0" +requires-python = "==3.12.*" +dependencies = [] + +[dependency-groups] +dev = [ + "pre-commit>=4.0.0", + "pytest>=8.0.0", + "ruff>=0.8.0", +] + +# An app is run, not installed: uv syncs its dependencies and never the project. +[tool.uv] +package = false + +[tool.ruff] +target-version = "py312" +line-length = 120 + +[tool.ruff.lint] +select = ["E", "F", "W"] + +[tool.ruff.format] +quote-style = "double" + +[tool.pytest.ini_options] +testpaths = ["tests"] +# The project is never installed, so tests import it from the repo root. +pythonpath = ["."] diff --git a/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh b/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh index 669d199..d0f4cd3 100755 --- a/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh +++ b/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh @@ -2,7 +2,7 @@ # Deterministic file-scaffolding engine for the core:setup-repo skill. # Copies bundled assets into a target repo without clobbering existing files. # Usage: scaffold-repo.sh -# : python | ts-frontend | ts-backend | none +# : python | python-app | ts-frontend | ts-backend | none set -euo pipefail TARGET="${1:?target dir required}" @@ -10,7 +10,7 @@ STACK="${2:-none}" ASSETS="$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets" && pwd)" case "$STACK" in - python|ts-frontend|ts-backend|none) ;; + python|python-app|ts-frontend|ts-backend|none) ;; *) echo "unknown stack: $STACK" >&2; exit 2 ;; esac @@ -52,6 +52,13 @@ case "$STACK" in copy "$ASSETS/ci/ci-python.yml" "$TARGET/.github/workflows/ci.yml" copy "$ASSETS/precommit/python.yaml" "$TARGET/.pre-commit-config.yaml" ;; + python-app) + copy "$ASSETS/stack/python-app/pyproject.toml" "$TARGET/pyproject.toml" + copy "$ASSETS/stack/python/gitignore" "$TARGET/.gitignore" + copy "$ASSETS/stack/python/tests/.gitkeep" "$TARGET/tests/.gitkeep" + copy "$ASSETS/ci/ci-python-app.yml" "$TARGET/.github/workflows/ci.yml" + copy "$ASSETS/precommit/python.yaml" "$TARGET/.pre-commit-config.yaml" + ;; ts-frontend) copy "$ASSETS/stack/ts-frontend/package.json" "$TARGET/package.json" copy "$ASSETS/stack/ts-frontend/tsconfig.json" "$TARGET/tsconfig.json" diff --git a/tests/setup-repo/test-ci-assets.sh b/tests/setup-repo/test-ci-assets.sh index efb2958..227e7bd 100755 --- a/tests/setup-repo/test-ci-assets.sh +++ b/tests/setup-repo/test-ci-assets.sh @@ -32,6 +32,28 @@ assert setup_py and setup_py[0]["with"]["python-version"] == "3.12", setup_py print("ci-python.yml OK") EOF +APP_CI="$ROOT/plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml" +python3 - "$APP_CI" <<'EOF' +import sys +try: + import yaml +except ModuleNotFoundError: + print("pyyaml missing; skipping YAML parse"); sys.exit(0) +doc = yaml.safe_load(open(sys.argv[1])) +on = doc.get("on", doc.get(True)) +assert on["push"]["branches"] == ["main"] and "pull_request" in on, on +steps = doc["jobs"]["lint-and-test"]["steps"] +runs = "\n".join(s.get("run", "") for s in steps) +assert "uv sync --locked" in runs, runs +assert "uv run --locked pre-commit run --all-files" in runs, runs +assert "uv run --locked pytest" in runs, runs +# the lock is the only record of what gets installed +assert "uv pip" not in runs and "requirements" not in runs, runs +uses = [str(s.get("uses", "")) for s in steps] +assert any(u.startswith("astral-sh/setup-uv") for u in uses), uses +print("ci-python-app.yml OK") +EOF + grep -q "root = true" "$EC" grep -q "indent_size = 4" "$EC" # python grep -q "indent_size = 2" "$EC" # js/ts/yaml diff --git a/tests/setup-repo/test-python-assets.sh b/tests/setup-repo/test-python-assets.sh index d734dfa..2c63a51 100755 --- a/tests/setup-repo/test-python-assets.sh +++ b/tests/setup-repo/test-python-assets.sh @@ -20,3 +20,26 @@ grep -qE 'pytest>=8' "$PY/requirements-dev.txt" test -f "$PY/gitignore" && grep -q "__pycache__" "$PY/gitignore" test -f "$PY/tests/.gitkeep" echo "python assets OK" + +APP="$ROOT/plugins/core/skills/setup-repo/assets/stack/python-app" +python3 - "$APP/pyproject.toml" "$PY/pyproject.toml" <<'EOF' +import sys, tomllib +app = tomllib.load(open(sys.argv[1], "rb")) +lib = tomllib.load(open(sys.argv[2], "rb")) +project = app["project"] +assert project["requires-python"] == "==3.12.*", project +assert project["dependencies"] == [], project +dev = " ".join(app["dependency-groups"]["dev"]) +for tool in ("ruff", "pytest", "pre-commit"): + assert tool in dev, (tool, dev) +assert app["tool"]["uv"]["package"] is False, app["tool"]["uv"] +# lint config matches the library scaffold, so the two stacks format alike +assert app["tool"]["ruff"] == lib["tool"]["ruff"], (app["tool"]["ruff"], lib["tool"]["ruff"]) +assert app["tool"]["pytest"]["ini_options"]["pythonpath"] == ["."], app["tool"]["pytest"] +print("python-app pyproject.toml OK") +EOF +# an app has no requirements files: the lock is the only record +if ls "$APP" | grep -q '^requirements'; then + echo "python-app ships a requirements file" >&2; exit 1 +fi +echo "python-app assets OK" diff --git a/tests/setup-repo/test-scaffold.sh b/tests/setup-repo/test-scaffold.sh index 1bb23bc..4fd5237 100755 --- a/tests/setup-repo/test-scaffold.sh +++ b/tests/setup-repo/test-scaffold.sh @@ -106,4 +106,29 @@ for f in package.json tsconfig.json eslint.config.js vitest.config.ts .gitignore done grep -q '"react"' "$TMP_TSB/package.json" && { echo "ts-backend should not have react" >&2; exit 1; } python3 -c "import json; json.load(open('$TMP_TSB/package.json'))" + +# python-app: a uv project with no requirements files, which locks, syncs and +# passes its tests exactly as scaffolded +TMP_APP="$(mktemp -d)" +trap 'rm -rf "$TMP" "$TMP_NONE" "$BOGUS" "$TMP_TSF" "$TMP_TSB" "$TMP_APP"' EXIT +git -C "$TMP_APP" init -q +bash "$ENGINE" "$TMP_APP" python-app +for f in .claude/settings.json CLAUDE.md .editorconfig pyproject.toml .gitignore \ + tests/.gitkeep .github/workflows/ci.yml .pre-commit-config.yaml; do + test -e "$TMP_APP/$f" || { echo "MISSING (python-app): $f" >&2; exit 1; } +done +for f in requirements.txt requirements-dev.txt; do + if test -e "$TMP_APP/$f"; then echo "UNEXPECTED (python-app): $f" >&2; exit 1; fi +done +grep -q 'uv sync --locked' "$TMP_APP/.github/workflows/ci.yml" || { echo "python-app ci is not locked" >&2; exit 1; } +if command -v uv >/dev/null 2>&1; then + mkdir -p "$TMP_APP/src" + cp "$TMP/src/example.py" "$TMP_APP/src/example.py" + cp "$TMP/tests/test_example.py" "$TMP_APP/tests/test_example.py" + ( cd "$TMP_APP" && uv lock -q && uv sync -q --locked && uv run -q --locked pytest -q ) + echo "python-app uv OK" +else + echo "uv not installed; python-app lock skipped" +fi + echo "ALL CHECKS PASSED"