Skip to content

test: trim redundant nox runs #902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Abhijeet Prasad (AbhiPrasad) merged 2 commits into from
Oct 9, 2026
Merged

test: trim redundant nox runs #902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
90552c6
test: trim redundant nox runs
AbhiPrasad Oct 9, 2026
35f2d75
test: remove unused OpenAI cassettes
AbhiPrasad Oct 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions py/noxfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,7 @@ def test_instructor(session, version):


OPENAI_VERSIONS = _get_matrix_versions("openai")
OPENAI_ENDPOINT_VERSIONS = (OPENAI_VERSIONS[0], OPENAI_VERSIONS[-1])

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Delete cassettes orphaned by the endpoint-only sessions

Restricting both sessions to latest and 1.71.0 leaves 16 checked-in fixtures permanently unread: the six btx/*.yaml files and two HTTP/2 streaming cassettes under each of cassettes/1.77.0/ and cassettes/1.92.0/. No remaining session opens those files, so the repository's unused-cassette audit will report them; remove these fixtures as part of narrowing the matrix.

AGENTS.md reference: AGENTS.md:L249-L253

Useful? React with 👍 / 👎.



@nox.session()
Expand All @@ -320,7 +321,7 @@ def test_openai(session, version):


@nox.session()
@nox.parametrize("version", OPENAI_VERSIONS, ids=OPENAI_VERSIONS)
@nox.parametrize("version", OPENAI_ENDPOINT_VERSIONS, ids=OPENAI_ENDPOINT_VERSIONS)
def test_openai_http2_streaming(session, version):
_install_test_deps(session)
_install_matrix_dep(session, "openai", version)
Expand All @@ -331,7 +332,7 @@ def test_openai_http2_streaming(session, version):


@nox.session()
@nox.parametrize("version", OPENAI_VERSIONS, ids=OPENAI_VERSIONS)
@nox.parametrize("version", OPENAI_ENDPOINT_VERSIONS, ids=OPENAI_ENDPOINT_VERSIONS)
def test_btx_openai(session, version):
"""Run the BTX cross-language LLM-span spec tests (OpenAI provider)."""
_install_test_deps(session)
Expand Down Expand Up @@ -557,12 +558,11 @@ def test_pydantic_ai_wrap_openai(session, version):
@nox.session()
@nox.parametrize("version", AUTOEVALS_VERSIONS, ids=AUTOEVALS_VERSIONS)
def test_autoevals(session, version):
# Run all of our core tests with autoevals installed. Some tests
# specifically validate scores from autoevals work properly, so
# we need some tests with it installed.
# Only this test exercises the changed scorer behavior. Running the full
# core suite here duplicates test_core without extending package coverage.
_install_test_deps(session)
_install_matrix_dep(session, "autoevals", version)
_run_core_tests(session)
_run_tests(session, f"{SRC_DIR}/test_framework.py::test_run_evaluator_with_many_scorers")


# google-genai 1.29.0 has a broken async streaming path unless aiohttp is installed.
Expand Down Expand Up @@ -805,12 +805,11 @@ def test_api_codegen(session):

@nox.session()
def test_braintrust_core(session):
# Some tests do specific things if braintrust_core is installed, so run our
# common tests with it installed. Testing the latest (aka the last ever version)
# is enough.
# This is the core test whose scorer behavior changes with braintrust_core.
# The rest of the core suite already runs in test_core.
_install_test_deps(session)
_install_matrix_dep(session, "braintrust-core", LATEST)
_run_core_tests(session)
_run_tests(session, f"{SRC_DIR}/test_framework.py::test_run_evaluator_with_many_scorers")


@nox.session()
Expand Down
83 changes: 0 additions & 83 deletions py/src/braintrust/integrations/openai/cassettes/1.77.0/btx/attachments.yaml
View file
Open in desktop

This file was deleted.

144 changes: 0 additions & 144 deletions py/src/braintrust/integrations/openai/cassettes/1.77.0/btx/completions.yaml
View file
Open in desktop

This file was deleted.

Loading
Loading