test: keep the suite off GitHub and away from real gh and zizmor (#548)

[mlieberman85]

Summary

Before this change, 111 tests in 37 files made 576 real gh calls and 76 real zizmor runs per suite run. Calls included gh api /repos/fake-owner/fake-repo, /user and /orgs/example. Results depended on network access, local gh auth and rate limits, so runs failed intermittently.

The change is test-only:

• tests/conftest.py stand-ins: a session fixture writes stand-in gh (exits 4, "not authenticated in tests") and zizmor (prints []).
• Autouse fixture: it puts the stand-ins first on PATH and installs RecordedGhApi({}), so unrecorded endpoints answer deterministically as ERROR unavailable. Tests that install their own responder, or override a tool with stand_in_tool, still win.
• New live marker: tests marked live or upstream are skipped unless the -m expression names that marker.
  • pytest tests/, CI's -m unit / -m integration, and release.yml's -m "not upstream" never run them.
  • The nightly -m upstream workflow and --update-hash keep working.
• Two test fixes: test_gh_api_status.py and test_utils.py::TestGithubCliErrors test the real gh code path by mocking subprocess.run, so they opt out of the default responder.
• Docs: docs/getting-started/testing.md gains a "Network and external tools" section.

Closes #548

Type of Change

• Bug fix (non-breaking change fixing an issue)
• New feature (non-breaking change adding functionality)
• Breaking change (fix or feature causing existing functionality to change)
• Documentation update
• Refactoring (no functional changes)

Testing

• Tests pass locally (uv run pytest tests/ -v): 5108 passed, 29 skipped
• Added tests for new functionality (if applicable)
• Linting passes (uv run ruff check .)

Determinism checks:

• The suite was run four times, with wrappers that log every real gh or zizmor call and every network connection. They recorded zero of each.
• The integration tests also pass with GitHub Actions environment variables set (333 passed).
• Suite time dropped from about 365 s to 201-237 s and was stable across runs.

AI assistance

• No AI assistance was used
• AI assistance was used

Claude (Claude Code, claude-opus-5-5) traced the network calls and made the change. This description was also drafted with Claude. The commit carries an Assisted-by: Claude:claude-opus-5-5 trailer.

Additional Notes

• upstream tests are now opt-in too. They fetch from raw.githubusercontent.com and used to run on every default run.
• tests/packaging/test_wheel_install_config.py still reaches PyPI (pip install, marked slow). It is left alone so release.yml keeps running it.
• Possible product follow-up: _classify_exec_failure labels any unexpected exit code as error_class=network. That covers grep exiting 2 on a missing directory and git exiting 128, so the class is misleading.

🤖 Generated with Claude Code