Repository navigation
ci: add Python 3.15 to the CI matrix - #924
Merged
Merged
Conversation
Run static_checks, smoke, and the sharded nox job on Python 3.15. Fix two SDK behavior changes on 3.15: - PEP 810 lazy imports: _resolve_attr_path used inspect.getattr_static, which returns unreified types.LazyImportType proxies. Packages that use __lazy_modules__ (e.g. google-cloud-discoveryengine's GAPIC __init__) were silently skipped by auto_instrument(). Reify the proxy via getattr. - importlib.metadata now raises MetadataNotFound (a FileNotFoundError) for a lone .dist-info without METADATA. Catch it alongside PackageNotFoundError in the SDK version lookup (runs on every span) and integration version detection. Skip sessions on 3.15 whose dependencies cannot install yet (no cp315 wheels and sdists that fail to build, mostly PyO3 < 0.27, or Requires-Python < 3.15): dspy, deepagents, harbor, instructor, langchain, litellm(latest), openrouter(latest), transformers, openai_ddtrace. Gate the matching lint-group deps and pylint vendor installs, and skip linting those integrations' files when their package is absent. Bump pylint 4.0.5 -> 4.1.2 (astroid 4.3.4): astroid 4.0.4 mis-handles dataclasses.KW_ONLY on 3.15. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
# Conflicts: # py/nox_session_constraints.py # py/uv.lock
The advanced exclude-newer cutoff picks up pyarrow wheels for 3.15, so only dspy latest (fastuuid via litellm) is still blocked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
orjson 3.11.8 has no cp315 wheels, and its sdist fails to build on the runners' stable Rust (#![feature] on the stable channel), breaking test_litellm(1.74.0) on 3.15 for Linux and Windows. 3.12.0 ships cp315 wheels; keep 3.11.8 for older Pythons. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Abhijeet Prasad (AbhiPrasad)
added a commit
that referenced
this pull request
Oct 14, 2026
`import braintrust` now loads about 24% faster on Python 3.15. It uses PEP 810 `__lazy_modules__` to defer `jsonschema`, `tqdm`, `asyncio`, `slugify`, and `gzip` until first use. On 3.10–3.14 `__lazy_modules__` is an unused variable, so behavior there is unchanged apart from the two notes below. Follow-up to #924. ### Changes - `__lazy_modules__` declarations in `parameters`, `framework`, `framework2`, `logger`, `trace`, `workflow_eval`, `sandbox`, `audit`, `functions/stream`, `functions/invoke`, and `prompt_cache/disk_cache`. - `__init__.py` star-imports five of those modules, and they have no `__all__`: `logger`, `functions/stream`, `functions/invoke`, `framework2`, and `audit`. `from m import *` would reify their lazy names, so they bind the deferred module under a private alias (`import asyncio as _asyncio`). - `framework` and `logger` call `os.cpu_count()` instead of `multiprocessing.cpu_count()`. Both call it at import time, and the switch drops the `multiprocessing` import on every version. The only difference is when the CPU count can't be determined: the import used to raise, and now the thread pools fall back to their default size. ### Visible on all versions The accidental stdlib re-exports `braintrust.asyncio`, `braintrust.gzip`, `braintrust.slugify`, and `braintrust.cpu_count` no longer exist. Nothing in this repo uses them. Keeping them would mean adding `__all__` to `logger` and the other star-exported modules, which changes the public API far more. ### Measurements Median of 60 interleaved runs of `python -c "import braintrust"` on a Linux devbox: | Python | Before | After | Modules | |---|---|---|---| | 3.15.0 | 472 ms | 357 ms (−24%) | 511 → 403 | | 3.14.6 | 472 ms | 469 ms (noise) | 502 → 493 | Per step on 3.15: - `jsonschema` + `tqdm`: −15% - adding `asyncio`: −21% - adding `slugify` + `gzip`: −22% to −23% - adding `os.cpu_count()`: −24% The last two steps saved about 5 ms each, close to the noise floor. This box drifted ±10% between sequential pyperf runs, so the interleaved numbers are the primary measurement. `make bench-compare` on adjacent runs agrees: 3.15 is 1.23x faster and 3.14 is 1.02x faster. Over the same runs, the unchanged `python -c pass` control read 1.21x slower, which is pure drift. Not attempted: - `requests` (~100 ms): `api/_transport.py` subclasses `HTTPAdapter` at class-definition time. - `_generated_types`: it is star-exported, so lazy imports can't help it. ### Tests and benchmark - New pyperf benchmark `benchmarks/benches/bench_import_time.py`. It measures `python -c pass` and `import braintrust` separately, so interpreter startup can be subtracted. - Two new 3.15-only tests in `test_lazy_integration_imports.py`: - One collects every loaded `braintrust.*` module's `__lazy_modules__` and asserts none of those modules (nor `multiprocessing`) is in `sys.modules` after import. It fails on `main` and passes on this branch. - A smoke test runs a local `Eval` and checks `jsonschema` parameter validation through the deferred imports. ### OpenRouter test fix This PR made `test_openrouter_integration_setup_is_idempotent` fail on Windows with Python 3.10 only. The root cause is old: before 3.12, a nested import on one thread can make `importlib` raise a spurious `KeyError`. OpenRouter's SDK handles that by deleting and re-importing the sub-module, which creates a new `Embeddings` class. `setup()` then patches the new class while the test still holds the old one. Dropping `multiprocessing` from `import braintrust` likely changes what OpenRouter's GC finalizer imports for the first time, which surfaced the problem on Windows. The test now resolves the classes when it runs, the same way `setup()` does. Forcing that re-import on Linux 3.10 reproduces the exact CI failure, and the fix passes it. ### Validation (local, Linux) - `nox -s test_core`: 3.10.22 ✅ (1012 passed), 3.14.6 ✅ (1017 passed), 3.15.0 ✅ (1024 passed). - `pylint` and `test_types` on 3.15.0 ✅. pre-commit ✅. - Windows was not tested locally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- sfk:bot-coauthor --> Co-authored by @starfolkai[bot] --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI Summary
Adds Python 3.15 to the
static_checks,smoke, and shardednoxmatrices inchecks.yaml, and fixes two SDK behavior changes that only show up on 3.15.SDK fixes (3.15 only)
_resolve_attr_pathusesinspect.getattr_static, which on 3.15 returns the unreifiedtypes.LazyImportTypeproxy for names bound through__lazy_modules__. google-cloud-discoveryengine's GAPIC__init__already ships this way, soauto_instrument()silently returnedFalsefor it. We now reify the proxy with a plaingetattron the module; unresolvable proxies resolve toNone. On 3.10–3.14types.LazyImportTypedoesn't exist, so the new branch never runs.importlib.metadataraisesMetadataNotFound(aFileNotFoundError) for a lone.dist-infowithout METADATA (e.g. an interrupted install). Before 3.15,version()returnedNone.span_origin._sdk_version()(runs on every span),detect_module_version(), andBaseIntegration.detect_version()only caughtPackageNotFoundError. They now also catchFileNotFoundError.Both fixes went red → green, with tests skipped below 3.15. The lazy-import tests use a real PEP 810 package written to
tmp_path, not mocks.Sessions skipped on 3.15 (upstream packaging gaps)
These dependencies have no cp315 wheels, and their sdists fail to build (mostly PyO3 < 0.27, which refuses 3.15), or they declare
Requires-Python < 3.15. Caps live innox_session_constraints.pywith the blocking package noted, so they can be dropped as wheels land:test_litellm(latest),test_dspy(latest),test_harbortest_langchain,test_deepagentstest_instructortest_openrouter(latest)test_transformers(latest)test_openai_ddtracetest_langchain,test_openrouter, andtest_openai_ddtracenow call_skip_if_incompatible, so localnox -salso skips them on 3.15. Before this, only the CI sharder filtered them out.pylint on 3.15
python_version<'3.15'(livekit-blingfire has no cp315 dist at all). It addsjiter>=0.11; python_version>='3.15'so 3.15 isn't stuck on instructor'sjiter<0.11.E1120for everydataclasses.KW_ONLYconstructor call on 3.15. The bump surfaced one false positive (E1121on an explicitclassmethod.__func__call inpydantic_ai/patchers.py), now silenced with an inline disable.uv.lockThe jiter marker makes uv fork the resolution by Python version, hence the large diff. No package versions were added or removed. Before the pylint bump, I compared the effective pins for all 30 dependency groups × Python 3.10–3.14 × Linux/Windows against
main: 300/300 identical. The pylint bump then changes only pylint and astroid.test-litellmalso pinsorjson==3.12.0on 3.15 only. 3.11.8 has no cp315 wheels, and its sdist fails on the runners' stable Rust (#![feature]on the stable channel).Validation (local, Linux)
pylint✅,test_types✅,test_core✅ (1022 passed).test_google_discoveryenginefrom the lazy-import bug. With chore(deps): weekly dependency update #911's newer cutoff,test_dspy(2.6.0)installs and passes on 3.15, so 13 remain skipped. I re-probed all 13 with the caps lifted after the merge; they still fail to install.pylint✅ andtest_core✅ (1012 and 1017 passed).Note: some packages (pydantic-core 2.46.5, pyyaml, aiohttp, polyleven, …) build from sdist on 3.15, so the first 3.15 jobs will be slower until wheels land.
Follow-up (not in this PR)
__lazy_modules__is a no-op before 3.15. Deferringjsonschema,tqdm,asyncio,slugify, andgzipmeasured roughly 25% offimport braintruston 3.15 (about 298 ms → 224 ms). That's worth a separate PR with benchmarks.🤖 Generated with Claude Code
Co-authored by StarfolkAI (@starfolkai)[bot]