Skip to content

Run contract tests under the right Python interpreter - #22

Open
MrJoy wants to merge 3 commits into
wandercom:mainfrom
MrJoy:fix/run-tests-interpreter
Open

MrJoy wants to merge 3 commits into
wandercom:mainfrom
MrJoy:fix/run-tests-interpreter

Conversation

@MrJoy

@MrJoy MrJoy commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

run_contract_tests ran python3 from PATH and ignored environment.python_path. When that python3 has no pytest, a passing test file comes back as total=0, errors=0, which looks like nothing went wrong. The same file, run on this machine where PATH resolves python3 to pyenv's 3.9:

OLD default: 0 0 0 []      # total, passed, errors
NEW default: 1 1 0

Changes:

  • Contract tests run under environment.python_path when one is set, and under sys.executable otherwise. EnvironmentSpec and resolve_environment now default to sys.executable too.
  • Pact's own purelib is no longer added to PYTHONPATH. It came in with 29ed47d so anyio would be importable from the wrong interpreter. Under sys.executable it's redundant. Under a custom python_path it mixes two environments' site-packages, which is bad news for compiled packages like pydantic_core.
  • No module named pytest now comes back as an environment error naming the interpreter, instead of 0/0.
  • pytest moves from the dev extra to the runtime dependencies. The harness now runs pytest under pact's own interpreter, so a pipx or uv tool install with no extras needs it. Checked with a bare uv pip install . into a fresh venv: pytest 9.1.1 comes along, and the same test file reports 1 1 0. The No module named pytest error still covers a custom python_path that lacks it.

Tests: the 8 new and changed tests in test_test_harness.py and test_environment.py all fail against the old code. Full suite: 9 failed, 2371 passed, 2 skipped, and the 9 are exactly the ones #21 fixes.

🤖 Generated with Claude Code

https://claude.ai/code/session_011FeZUmAZbyQWDJNAPkUA3S


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

The harness ran `python3` from `PATH` and ignored `environment.python_path`, then papered over the mismatch by injecting pact's site-packages.  When `PATH` had no pytest, a passing suite parsed as 0 tests and 0 errors.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011FeZUmAZbyQWDJNAPkUA3S
@MrJoy
MrJoy marked this pull request as ready for review September 28, 2026 19:25
The contract-test harness runs pytest under pact's own interpreter, so a pipx or `uv tool` install without the `dev` extra couldn't run tests at all.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011FeZUmAZbyQWDJNAPkUA3S

@adaptcom adaptcom Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confidence Score: 0/5

Summary

Review of 35ed5b2 is stale because the PR advanced to 85cda5a. That commit addresses pytest packaging; rerun validation after fixing the remaining interpreter-configuration propagation issue.

Important Files Changed

File Overview
src/pact/config.py Defaults execution environments to Pact’s interpreter
src/pact/test_harness.py Interpreter selection changes without propagating configuration from production callers
tests/test_environment.py Updates default-interpreter expectations
tests/test_test_harness.py Adds interpreter, PYTHONPATH isolation, and missing-pytest tests

↻ Re-run review · View in Adapt

Comment thread src/pact/test_harness.py
None of the 11 `run_contract_tests` callers passed `environment`, so a `python_path` set in `pact.yaml` or the global config never reached the harness.  They now pass the resolved `EnvironmentSpec`.  An unconfigured project resolves to the same interpreter and env as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXo6tGa9GdEhd8bZuX6cuo

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant