Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions docs/decisions/2026-09-23-python-apps-lock-with-uv.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Python apps lock their dependencies with uv

- **Date:** 2026-09-23. **Status:** proposed; accepted when this merges.
- **Scope:** Python apps. Whether the five vendored `retina-*` libraries commit a lock is a
separate question, open in ClickUp `86cb49y4k`, and this record does not settle it.
- **Ticket:** ClickUp `123zgec4jmx`. **Follow-up:** `123zgec4jn0`, the other apps.

## 1. Context

The `setup-repo` standard installs every Python repo the same way:
`uv pip install -r requirements.txt -r requirements-dev.txt`. The requirements files pin the
direct dependencies and nothing else, so every install resolves the rest of the tree afresh.

For a deployed app that means the image, CI and each developer's venv can each run a different
tree. retina-server pinned 15 of the 54 third-party packages its image runs; pydantic,
SQLAlchemy, websockets and the rest were chosen again whenever the image's dependency layer
rebuilt. On
2026-09-23 a developer venv differed from production on pydantic (2.13.4 against 2.13.5) and
websockets (16.1.1 against 17.1), with nothing in either repository to say which was intended.

## 2. Decision

An **app** is a repo that runs as a deployed service or a tool, and that no other repo installs
as a dependency. Apps are uv projects:

- `pyproject.toml` declares the runtime dependencies under `[project]` and the tooling under
`[dependency-groups] dev`. `[tool.uv] package = false` unless the app is itself installed.
- `uv.lock` is committed and is the only record of what gets installed. There is no
`requirements.txt`.
- CI runs `uv sync --locked`, which fails when the lock no longer matches `pyproject.toml`.
- Images run `uv sync --locked --no-dev` into a virtualenv put first on `PATH`, not into the
system site-packages: `uv sync` removes whatever the lock does not name, the base image's own
`pip` included.
- Libraries vendored as submodules are `[tool.uv.sources]` path entries, editable for local work
and installed with `--no-editable` in images.

## 3. Consequences

- A dependency change and its lock land in the same commit. So does a submodule bump that changes
a vendored library's own dependencies, which is the easy one to miss; `--locked` in CI catches
it.
- Upgrades are deliberate: `uv lock --upgrade-package <name>`, or `uv lock --upgrade` for the
whole tree. Nothing relocks on its own, so a transitive security release reaches an app only
when someone relocks. No Python repo runs a dependency bot today. Dependabot's `uv` ecosystem
could do that job, and adopting it is a separate decision.
- Moving an app to a lock need not move any version. Seed the first `uv lock` with
`[tool.uv] constraint-dependencies` listing what production runs, then delete the constraints
and lock again: uv keeps a locked version until told to upgrade it, so the lock ends at the
deployed versions and the first image built from it matches the running one.
- The uv that writes a lock and the uv that reads it need not match. A lock written by 0.12.5
syncs under 0.9.22.

## 4. Adoption

`setup-repo` gains a `python-app` stack that scaffolds §2. Its `python` stack stays as it is and
remains the scaffold for libraries. retina-server moves first. tower-finder-service,
retina-telemetry, retina-gui and node-infra's `mender-auto-accept` install with plain pip today
and move under `123zgec4jn0`.
2 changes: 1 addition & 1 deletion plugins/core/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "core",
"version": "0.5.0",
"version": "0.6.0",
"description": "Core Offworld Labs skills, commands, agents, and hooks shared across all repos.",
"author": {
"name": "Offworld Labs"
Expand Down
26 changes: 18 additions & 8 deletions plugins/core/skills/setup-repo/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,26 +19,35 @@ interactive parts and the report.
`git init`; abort if the user declines.

2. **Determine the stack.** Detect from existing files: `pyproject.toml` /
`requirements*.txt` → `python`; `package.json` / `tsconfig.json` → TypeScript
(then ask whether it's `ts-frontend` — a React/Vite app — or `ts-backend` — a
Node service). If ambiguous or empty, ask the user to choose `python`,
`ts-frontend`, `ts-backend`, or `none`.
`requirements*.txt` / `uv.lock` → Python (then ask whether it's an app,
`python-app`, which runs as a service or tool and which no other repo installs,
or a library, `python`, which other repos install); `package.json` /
`tsconfig.json` → TypeScript (then ask whether it's `ts-frontend` — a React/Vite
app — or `ts-backend` — a Node service). If ambiguous or empty, ask the user to
choose `python-app`, `python`, `ts-frontend`, `ts-backend`, or `none`.
`python-app` follows `claude-shared/docs/decisions/2026-09-23-python-apps-lock-with-uv.md`.

3. **Scaffold the files.** Run the engine, which never overwrites existing files:
`bash "$ENGINE" . <stack>`
Relay its `WRITTEN` / `SKIPPED` output to the user.

4. **Install dependencies.**
- **Python:** use `uv` (the org standard, a fast drop-in for pip that reads the
same `requirements.txt`): `uv venv && uv pip install -r requirements.txt -r requirements-dev.txt`.
- **python-app:** set `[project] name` in `pyproject.toml` to the repo's name,
then `uv lock && uv sync`. Remind the user to commit `uv.lock`: the CI
workflow runs `uv sync --locked` and fails without it. Add dependencies with
`uv add <pkg>` (`uv add --dev <pkg>` for tooling), never a requirements file.
There is no pip fallback; if `uv` is absent, skip and say so.
- **python (library):** use `uv`'s pip interface, which reads the same
`requirements.txt`: `uv venv && uv pip install -r requirements.txt -r requirements-dev.txt`.
Fall back to `pip install -r requirements.txt -r requirements-dev.txt` in an
active virtualenv if `uv` is absent.
- **ts-frontend / ts-backend:** run `npm install` (this generates
`package-lock.json` — remind the user to commit it, since the CI workflow uses
`npm ci`).
- **pre-commit (python & ts stacks):** after the stack deps are installed,
register the git hook so the scaffolded `.pre-commit-config.yaml` runs on
every commit: `uvx pre-commit install` (or `pipx run pre-commit install`, or
every commit: `uv run pre-commit install` for `python-app`, whose dev group
carries pre-commit; otherwise `uvx pre-commit install` (or `pipx run pre-commit install`, or
`pip install pre-commit && pre-commit install`). Skip with a note if
pre-commit/uv is unavailable.
Report the command and result; if the toolchain is unavailable, skip and note it
Expand All @@ -48,7 +57,8 @@ interactive parts and the report.
one-or-two-line description of what this repo does, then fill in the
`Project Overview`, `Build & Test Commands`, and `Local Architecture`
sections from their answer plus what was scaffolded (stack; lint/format via
`pre-commit run --all-files`; tests via `pytest` or `npm test`). If they skip,
`pre-commit run --all-files`; tests via `pytest`, `uv run pytest` for
`python-app`, or `npm test`). If they skip,
leave the stub as-is. Keep
CLAUDE.md under the 200-line ceiling noted in the template.

Expand Down
31 changes: 31 additions & 0 deletions plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: CI

on:
push:
branches: [main]
pull_request:

jobs:
lint-and-test:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install uv
uses: astral-sh/setup-uv@v5

# --locked fails the job when uv.lock no longer matches pyproject.toml.
- name: Install dependencies
run: uv sync --locked

- name: Pre-commit
run: uv run --locked pre-commit run --all-files --show-diff-on-failure

- name: Pytest
run: uv run --locked pytest
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
[project]
name = "app"
version = "0.0.0"
requires-python = "==3.12.*"
dependencies = []

[dependency-groups]
dev = [
"pre-commit>=4.0.0",
"pytest>=8.0.0",
"ruff>=0.8.0",
]

# An app is run, not installed: uv syncs its dependencies and never the project.
[tool.uv]
package = false

[tool.ruff]
target-version = "py312"
line-length = 120

[tool.ruff.lint]
select = ["E", "F", "W"]

[tool.ruff.format]
quote-style = "double"

[tool.pytest.ini_options]
testpaths = ["tests"]
# The project is never installed, so tests import it from the repo root.
pythonpath = ["."]
11 changes: 9 additions & 2 deletions plugins/core/skills/setup-repo/scripts/scaffold-repo.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@
# Deterministic file-scaffolding engine for the core:setup-repo skill.
# Copies bundled assets into a target repo without clobbering existing files.
# Usage: scaffold-repo.sh <target_dir> <stack>
# <stack>: python | ts-frontend | ts-backend | none
# <stack>: python | python-app | ts-frontend | ts-backend | none
set -euo pipefail

TARGET="${1:?target dir required}"
STACK="${2:-none}"
ASSETS="$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets" && pwd)"

case "$STACK" in
python|ts-frontend|ts-backend|none) ;;
python|python-app|ts-frontend|ts-backend|none) ;;
*) echo "unknown stack: $STACK" >&2; exit 2 ;;
esac

Expand Down Expand Up @@ -52,6 +52,13 @@ case "$STACK" in
copy "$ASSETS/ci/ci-python.yml" "$TARGET/.github/workflows/ci.yml"
copy "$ASSETS/precommit/python.yaml" "$TARGET/.pre-commit-config.yaml"
;;
python-app)
copy "$ASSETS/stack/python-app/pyproject.toml" "$TARGET/pyproject.toml"
copy "$ASSETS/stack/python/gitignore" "$TARGET/.gitignore"
copy "$ASSETS/stack/python/tests/.gitkeep" "$TARGET/tests/.gitkeep"
copy "$ASSETS/ci/ci-python-app.yml" "$TARGET/.github/workflows/ci.yml"
copy "$ASSETS/precommit/python.yaml" "$TARGET/.pre-commit-config.yaml"
;;
ts-frontend)
copy "$ASSETS/stack/ts-frontend/package.json" "$TARGET/package.json"
copy "$ASSETS/stack/ts-frontend/tsconfig.json" "$TARGET/tsconfig.json"
Expand Down
22 changes: 22 additions & 0 deletions tests/setup-repo/test-ci-assets.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,28 @@ assert setup_py and setup_py[0]["with"]["python-version"] == "3.12", setup_py
print("ci-python.yml OK")
EOF

APP_CI="$ROOT/plugins/core/skills/setup-repo/assets/ci/ci-python-app.yml"
python3 - "$APP_CI" <<'EOF'
import sys
try:
import yaml
except ModuleNotFoundError:
print("pyyaml missing; skipping YAML parse"); sys.exit(0)
doc = yaml.safe_load(open(sys.argv[1]))
on = doc.get("on", doc.get(True))
assert on["push"]["branches"] == ["main"] and "pull_request" in on, on
steps = doc["jobs"]["lint-and-test"]["steps"]
runs = "\n".join(s.get("run", "") for s in steps)
assert "uv sync --locked" in runs, runs
assert "uv run --locked pre-commit run --all-files" in runs, runs
assert "uv run --locked pytest" in runs, runs
# the lock is the only record of what gets installed
assert "uv pip" not in runs and "requirements" not in runs, runs
uses = [str(s.get("uses", "")) for s in steps]
assert any(u.startswith("astral-sh/setup-uv") for u in uses), uses
print("ci-python-app.yml OK")
EOF

grep -q "root = true" "$EC"
grep -q "indent_size = 4" "$EC" # python
grep -q "indent_size = 2" "$EC" # js/ts/yaml
Expand Down
23 changes: 23 additions & 0 deletions tests/setup-repo/test-python-assets.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,26 @@ grep -qE 'pytest>=8' "$PY/requirements-dev.txt"
test -f "$PY/gitignore" && grep -q "__pycache__" "$PY/gitignore"
test -f "$PY/tests/.gitkeep"
echo "python assets OK"

APP="$ROOT/plugins/core/skills/setup-repo/assets/stack/python-app"
python3 - "$APP/pyproject.toml" "$PY/pyproject.toml" <<'EOF'
import sys, tomllib
app = tomllib.load(open(sys.argv[1], "rb"))
lib = tomllib.load(open(sys.argv[2], "rb"))
project = app["project"]
assert project["requires-python"] == "==3.12.*", project
assert project["dependencies"] == [], project
dev = " ".join(app["dependency-groups"]["dev"])
for tool in ("ruff", "pytest", "pre-commit"):
assert tool in dev, (tool, dev)
assert app["tool"]["uv"]["package"] is False, app["tool"]["uv"]
# lint config matches the library scaffold, so the two stacks format alike
assert app["tool"]["ruff"] == lib["tool"]["ruff"], (app["tool"]["ruff"], lib["tool"]["ruff"])
assert app["tool"]["pytest"]["ini_options"]["pythonpath"] == ["."], app["tool"]["pytest"]
print("python-app pyproject.toml OK")
EOF
# an app has no requirements files: the lock is the only record
if ls "$APP" | grep -q '^requirements'; then
echo "python-app ships a requirements file" >&2; exit 1
fi
echo "python-app assets OK"
25 changes: 25 additions & 0 deletions tests/setup-repo/test-scaffold.sh
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,29 @@ for f in package.json tsconfig.json eslint.config.js vitest.config.ts .gitignore
done
grep -q '"react"' "$TMP_TSB/package.json" && { echo "ts-backend should not have react" >&2; exit 1; }
python3 -c "import json; json.load(open('$TMP_TSB/package.json'))"

# python-app: a uv project with no requirements files, which locks, syncs and
# passes its tests exactly as scaffolded
TMP_APP="$(mktemp -d)"
trap 'rm -rf "$TMP" "$TMP_NONE" "$BOGUS" "$TMP_TSF" "$TMP_TSB" "$TMP_APP"' EXIT
git -C "$TMP_APP" init -q
bash "$ENGINE" "$TMP_APP" python-app
for f in .claude/settings.json CLAUDE.md .editorconfig pyproject.toml .gitignore \
tests/.gitkeep .github/workflows/ci.yml .pre-commit-config.yaml; do
test -e "$TMP_APP/$f" || { echo "MISSING (python-app): $f" >&2; exit 1; }
done
for f in requirements.txt requirements-dev.txt; do
if test -e "$TMP_APP/$f"; then echo "UNEXPECTED (python-app): $f" >&2; exit 1; fi
done
grep -q 'uv sync --locked' "$TMP_APP/.github/workflows/ci.yml" || { echo "python-app ci is not locked" >&2; exit 1; }
if command -v uv >/dev/null 2>&1; then
mkdir -p "$TMP_APP/src"
cp "$TMP/src/example.py" "$TMP_APP/src/example.py"
cp "$TMP/tests/test_example.py" "$TMP_APP/tests/test_example.py"
( cd "$TMP_APP" && uv lock -q && uv sync -q --locked && uv run -q --locked pytest -q )
echo "python-app uv OK"
else
echo "uv not installed; python-app lock skipped"
fi

echo "ALL CHECKS PASSED"
Loading