Repository navigation
test: trim redundant nox runs - #902
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 90552c6cf8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
|
|
||
|
|
||
| OPENAI_VERSIONS = _get_matrix_versions("openai") | ||
| OPENAI_ENDPOINT_VERSIONS = (OPENAI_VERSIONS[0], OPENAI_VERSIONS[-1]) |
There was a problem hiding this comment.
Restricting both sessions to latest and 1.71.0 leaves 16 checked-in fixtures permanently unread: the six btx/*.yaml files and two HTTP/2 streaming cassettes under each of cassettes/1.77.0/ and cassettes/1.92.0/. No remaining session opens those files, so the repository's unused-cassette audit will report them; remove these fixtures as part of narrowing the matrix.
AGENTS.md reference: AGENTS.md:L249-L253
Useful? React with 👍 / 👎.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Implements PR 6 from #900.
Changes
test_framework.py::test_run_evaluator_with_many_scorersin bothtest_autoevalsvariants andtest_braintrust_core, since that is the core test whose behavior changes when these packages are installed. The remaining core suite continues to run intest_core.test_openai_http2_streamingandtest_btx_openaiagainst OpenAIlatestand the oldest matrix version (1.71.0), removing1.92.0and1.77.0from those two sessions.This intentionally reduces CI coverage: package-specific core checks run one scorer test instead of the full core suite, and the two OpenAI sessions test the matrix endpoints rather than all four versions.
Validation
test_autoevalssessions passed.test_braintrust_corepassed.test_openai_http2_streaming(latest)and(1.71.0)passed.test_btx_openai(latest)and(1.71.0)passed.git diff --checkpassed; commit hooks passed.Closes #900