Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ Use `uv run` to execute all commands (pytest, ruff, quarto). Do not use bare `py
Ruff is the linter and formatter. CI enforces both. Always run checks before committing:

``` bash
uv run ruff check src/ src/vip_tests/ selftests/ examples/
uv run ruff format --check src/ src/vip_tests/ selftests/ examples/
uv run ruff check src/ selftests/ examples/ docker/
uv run ruff format --check src/ selftests/ examples/ docker/
```

Or with just:
Expand All @@ -36,7 +36,7 @@ Or with just:
just check
```

Ruff rules: `E`, `F`, `I`, `UP`. Line length is 100. All Python directories (`src/`, `src/vip_tests/`, `selftests/`, `examples/`) must pass. CI pins ruff to version 0.15.0 -- do not change the version without updating `.github/workflows/ci.yml`.
Ruff rules: `E`, `F`, `I`, `UP`. Line length is 100. All Python directories (`src/`, which includes `src/vip_tests/`, plus `selftests/`, `examples/` and `docker/`) must pass. `docker/` is easy to forget and holds `docker/playwright-smoke.py`. CI pins ruff to version 0.15.0 -- do not change the version without updating `.github/workflows/ci.yml`.

Auto-fix before committing:

Expand Down Expand Up @@ -393,6 +393,5 @@ Register warning filters in `src/vip/plugin.py::pytest_configure` (via `config.a
- Reaching for a bare `pytest.skip()` when the real situation is "I could not check this". That is the failure mode #616 exists to close: an unverified deployment reporting itself as a passing one. If the product was configured and you still could not run the check, use `vip.attest.unproven()`.
- Using non-conventional PR titles (must be `type: description`).
- Relying on multi-line formatting to shorten lines -- `ruff format` will collapse list comprehensions back to one line if they fit within 100 chars. Extract a helper function instead.
- Importing a pytest-bdd step module (anything under `src/vip_tests/**` that calls `@scenario` / `scenarios()`) from inside a selftest. `@scenario` inspects the caller's frame at import time, so importing it mid-test raises `IndexError: list index out of range` — and only under some orderings, so it passes locally and fails in CI under `pytest-randomly`. Put the helper you want to test in `conftest.py` and import it from there, or assert via `--collect-only` in a subprocess the way `selftests/test_workbench_ordering.py` does.
- Running selftests with `-p no:randomly`. CI runs them randomized; disabling the plugin hides exactly the order-dependent failures it exists to catch.
- Importing a pytest-bdd step module (anything under `src/vip_tests/**` that calls `@scenario` / `scenarios()`) from inside a selftest. `@scenario` inspects the caller's frame at import time, so importing it mid-test raises `IndexError: list index out of range` — and only under some orderings, so it can pass in one run and fail in another as xdist redistributes tests across workers. Put the helper you want to test in `conftest.py` and import it from there, or assert via `--collect-only` in a subprocess the way `selftests/test_workbench_ordering.py` does.
- Bypassing `vip install` with raw `uv run playwright install --with-deps chromium` (or `playwright install chromium`) in setup recipes, Dockerfiles, CI workflows, or docs. The whole `vip uninstall` reversibility relies on the `.vip-install.json` manifest that only `vip install` writes -- a raw `playwright install` leaves no record. The only acceptable alternative is `uv run vip install --skip-system` (used by CI workflows where the runner already has system libs), which still records the Playwright cache.
15 changes: 10 additions & 5 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,21 @@ just format # ruff format
Without just, run ruff directly:

```bash
uv run ruff check src/ src/vip_tests/ # lint
uv run ruff format --check src/ src/vip_tests/ # format check
uv run ruff check --fix src/ src/vip_tests/ # auto-fix lint
uv run ruff format src/ src/vip_tests/ # reformat
uv run ruff check src/ selftests/ examples/ docker/ # lint
uv run ruff format --check src/ selftests/ examples/ docker/ # format check
uv run ruff check --fix src/ selftests/ examples/ docker/ # auto-fix lint
uv run ruff format src/ selftests/ examples/ docker/ # reformat
```

## Type checking

```bash
uv run mypy src/
just typecheck

# Without just. `--extra dev` matters: mypy lives in the dev extra, which a bare
# `uv sync` does not install. The path is `src/vip/`, not `src/` -- CI does not
# type-check `src/vip_tests/`.
uv run --extra dev mypy src/vip/
```

## The lockfile
Expand Down
12 changes: 7 additions & 5 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -48,28 +48,30 @@ check: lint format-check
fix: lint-fix format

# Run mypy type checker
# --extra dev: mypy is in the dev extra, which a bare `uv sync` does not install.
typecheck:
uv run mypy src/vip/
uv run --extra dev mypy src/vip/

# Run selftests with coverage
# --extra dev: pytest-cov is in the dev extra, which a bare `uv sync` does not install.
coverage:
uv run pytest selftests/ --cov=src/vip --cov-report=term-missing
uv run --extra dev pytest selftests/ --cov=src/vip --cov-report=term-missing

# Run selftests (no products required)
selftest *ARGS:
uv run pytest selftests/ {{ ARGS }}

# Run the full VIP test suite against configured products
test *ARGS:
uv run pytest tests/ {{ ARGS }}
uv run pytest src/vip_tests/ {{ ARGS }}

# Run tests for a specific product (connect, workbench, package_manager)
test-product PRODUCT:
uv run pytest tests/ -m {{ PRODUCT }}
uv run pytest src/vip_tests/ -m {{ PRODUCT }}

# Generate the Quarto report from product test results
report *ARGS:
uv run pytest tests/ {{ ARGS }}
uv run pytest src/vip_tests/ {{ ARGS }}
cd report && uv run quarto render

# Generate test catalog and feature matrix JSON for the website
Expand Down