🧪 Run tests from sdist like downstreams

[webknjaz]

What do these changes do?

This allows us make sure everything they need is shipped through PyPI. And brings us closer to how pip install would build wheels in the wild. We now also run tests in CI from sdist.

Are there changes in behavior for the user?

A quality-of-life change — once released, the users' installers will be able to get new aiohttp versions in their pure-python form by default, even if there's no pre-built wheel for their runtime (like if they get back to a certain version in the future with an interpreter released years later).

Is it a substantial burden for the maintainers to support this?

It's actually bringing the infra closer to the rest of the repos. So it's a win.

Related issue number

And this should help us out getting CI/CD to:

• closes Please upload a pure python wheel to PyPi #7632
• closes Build pure-python wheel and sdist in CI/CD before other jobs like in frozenlist #13363
• Could we please get wheels for Python 3.15? #13353

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • The format is <Name> <Surname>.
  • Please keep alphabetical order, the file is sorted by names.
• Add a new news fragment into the CHANGES/ folder

  • name it <issue_or_pr_num>.<type>.rst (e.g. 588.bugfix.rst)

  • if you don't have an issue number, change it to the pull request
    number after creating the PR

    • .bugfix: A bug fix for something the maintainers deemed an
      improper undesired behavior that got corrected to match
      pre-agreed expectations.
    • .feature: A new behavior, public APIs. That sort of stuff.
    • .deprecation: A declaration of future API removals and breaking
      changes in behavior.
    • .breaking: When something public is removed in a breaking way.
      Could be deprecated in an earlier release.
    • .doc: Notable updates to the documentation structure or build
      process.
    • .packaging: Notes for downstreams about unobvious side effects
      and tooling. Changes in the test invocation considerations and
      runtime assumptions.
    • .contrib: Stuff that affects the contributor experience. e.g.
      Running tests, building the docs, setting up the development
      environment.
    • .misc: Changes that are hard to assign to any of the above
      categories.

  • Make sure to use full sentences with correct case and punctuation,
    for example:

    Fixed issue with non-ascii contents in doctest text files
    -- by :user:`contributor-gh-handle`.

    Use the past tense or the present tense a non-imperative mood,
    referring to what's changed compared to the last released version
    of this project.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.41%. Comparing base (77c5108) to head (9c7a5fc).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #13388      +/-   ##
==========================================
- Coverage   98.99%   98.41%   -0.59%     
==========================================
  Files         132      133       +1     
  Lines       49454    49746     +292     
  Branches     2572     2629      +57     
==========================================
- Hits        48959    48958       -1     
- Misses        371      663     +292     
- Partials      124      125       +1     

Flag	Coverage Δ
Autobahn	21.95% <ø> (-0.14%)	⬇️
CI-GHA	98.32% <ø> (-0.60%)	⬇️
OS-Linux	98.09% <ø> (-0.60%)	⬇️
OS-Windows	96.46% <ø> (-0.59%)	⬇️
OS-macOS	97.35% <ø> (-0.59%)	⬇️
Py-3.10	97.54% <ø> (-0.59%)	⬇️
Py-3.11	97.80% <ø> (-0.59%)	⬇️
Py-3.12	97.88% <ø> (-0.59%)	⬇️
Py-3.13	97.87% <ø> (-0.59%)	⬇️
Py-3.14	97.88% <ø> (-0.60%)	⬇️
Py-3.14t	96.97% <ø> (-0.59%)	⬇️
Py-pypy-3.11	96.82% <ø> (-0.60%)	⬇️
VM-macos	97.35% <ø> (-0.59%)	⬇️
VM-ubuntu	98.09% <ø> (-0.60%)	⬇️
VM-windows	96.46% <ø> (-0.59%)	⬇️
cython-coverage	38.15% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[codspeed-hq]

Merging this PR will improve performance by 19.76%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 83 untouched benchmarks
⏩ 83 skipped benchmarks¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	test_client_send_large_websocket_compressed_messages[isal.isal_zlib]	86.4 ms	72.2 ms	+19.76%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing webknjaz:testing/wheel-ci-cd-from-sdist (9c7a5fc) with master (77c5108)²}

Footnotes

1. 83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on master (616e937) during the generation of this report, so 77c5108 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[webknjaz]

@aiolibsbot review and keep in mind that this is intended to be very minimalist in scope/diff. Any improvements should be in follow-ups if at all possible.

[aiolibsbot]

@webknjaz: @aiolibsbot review and keep in mind that this is intended to be very minimalist in scope/diff. Any i...

Understood — scoped review, follow-ups noted as suggestions only.

Two blockers, both debug scaffolding, not design: - run: exit 1 in deploy, and if: always() on build-wheels. They interact. Deleting build-tarball plus if: always() removes the transitive pre-deploy gate. deploy then needs only build-wheels and pre-setup, both green on any upstream push. Its if checks repo ID only. So deploy becomes eligible on every master push and nightly run, calling create-release with ref_name == 'master'. The exit 1 is the sole thing blocking that today. Remove both together.

Everything else is a suggestion. MANIFEST.in additions check out — .mypy.ini declares files = aiohttp, docs/code, examples, tests, all already grafted.

[aiolibsbot]

PR Review — 🧪 Run tests from sdist like downstreams

Solid restructuring with two debug leftovers that must come out before merge.

The core design is right and carefully executed. Building the sdist once in build-pure-python-dists and fanning it out to test, autobahn, benchmark, test-mobile, build-wheels and lint-from-sdist removes six duplicated checkout + Restore llhttp + make cythonize sequences, and splitting lint into lint-from-git / lint-from-sdist is exactly the right seam — the two steps that genuinely need history (git diff --exit-code on runtime-deps.in, docs spelling against the installed dist) stay on the Git side, everything else moves. The MANIFEST.in additions are correct and non-obvious: I verified .mypy.ini declares files = aiohttp, docs/code, examples, tests, all of which are already grafted, so make mypy really does work from an unpacked sdist. The CHANGES/ fragments follow the repo's symlink convention (issue-numbered file + PR-numbered symlink) and are signed with the right handle.

• 🔴 deploy starts with - run: exit 1 — a debug leftover that would break every release.
• 🔴 build-wheels: if: always() plus the deletion of build-tarball removes the transitive pre-deploy gate, making deploy eligible on every master push, merge_group and nightly schedule run. The exit 1 is currently the only thing stopping a PyPI publish from ref_name == 'master', so the two must be fixed together, not one at a time.
• 🟢 Publishing a py3-none-any wheel is a real install-behaviour change (pip prefers it over the sdist on platforms with no binary wheel) — documented in 7632.packaging.rst but contradicted by the PR body's "Nope".
• 🟢 basename dist/*.tar.gz fails open — an unmatched glob propagates a literal dist/*.tar.gz filename into six downstream jobs.
• 🟢 wheel-filename output has no consumer.
• 🟢 benchmark now stacks a git checkout and an sdist unpack; it only works because python -Im pytest keeps the workspace copy off sys.path — worth a comment.
• 🟢 requirements/cython.txt as a global PIP_CONSTRAINT is inert for Cython (not a build requirement) but couples wheel builds to its multidict/typing-extensions pins.

Strip the two debug leftovers and this is ready; the rest are fine as follow-ups.

---

🔴 Blocking

1. Debug leftover: `- run: exit 1` as the first step of `deploy`
.github/workflows/ci-cd.yml:779

The deploy job now begins with an unconditional - run: exit 1.

Impact: every release attempt fails on the first step — no PyPI upload, no sigstore signing, no GitHub Release. If this lands, v4.x tag pushes are dead in the water until someone notices.

It is also load-bearing right now: it is the only thing stopping the un-gated deploy job described in the next finding from publishing on every master push. Removing it alone would be worse than leaving it, so both this and the build-wheels if: always() need to go together.

Fix: drop the step before merge.

    steps:
    - run: exit 1
    - name: Checkout

2. `build-wheels: if: always()` removes the tag-only gate on `deploy`
.github/workflows/ci-cd.yml:650-656

Before this PR the release gate worked transitively: deploy needed build-tarball and build-wheels, both of which needed pre-deploy, and pre-deploy carries if: fromJSON(needs.pre-setup.outputs.release-requested). On a non-tag event pre-deploy was skipped → both builders skipped → deploy skipped.

This PR breaks that chain in two places at once:

• build-tarball is deleted, so it is gone from deploy's needs.
• build-wheels gains if: always(), so it runs and succeeds even though pre-deploy was skipped.

deploy's remaining needs are build-wheels (now success) and pre-setup (success), and its only if: is needs.pre-setup.outputs.upstream-repository-id == github.repository_id — true for any upstream event. So deploy becomes eligible on every push to master, every merge_group, and the nightly schedule run, calling aio-libs/create-release with github.ref_name == 'master' and publishing to PyPI. Only the exit 1 above currently prevents that.

Secondary cost: if: always() also runs the full ~20-cell wheel matrix (including the QEMU ppc64le/riscv64/s390x musllinux cells) on every PR and commit.

Fix: drop if: always() and keep the needs: pre-deploy gate. If PR-time wheel builds are actually wanted, that is a separate decision and deploy must then get its own explicit fromJSON(needs.pre-setup.outputs.release-requested) guard (plus pre-deploy back in its needs) rather than relying on a skipped-dependency chain.

  build-wheels:
    if: always()
    name: Build wheels on ${{ matrix.os }} ${{ matrix.qemu }} ${{ matrix.musl }} ${{ matrix.platform }}
    runs-on: ${{ matrix.os }}
    needs:
    - build-pure-python-dists
    - pre-deploy

🟢 Suggestions

1. Publishing a `py3-none-any` wheel is a user-visible install-behaviour change
CHANGES/7632.packaging.rst:1-6

dist-pure is uploaded as an artifact named dist-pure, which matches deploy's pattern: dist-* + merge-multiple: true — so the pure-Python wheel is genuinely published to PyPI. The changelog fragment documents that intentionally, but the PR body answers "Are there changes in behavior for the user? Nope."

pip prefers any matching wheel over the sdist. So for CPython on a platform/ABI the binary matrix does not cover (32-bit Linux, FreeBSD, pre-manylinux_2_x glibc, a fresh CPython pre-release before the matrix is updated), users who previously compiled the C extensions from sdist will now silently get the pure-Python implementation — same API, materially slower parser/writer, with no warning.

PyPy is unaffected (setup.py already forces NO_EXTENSIONS = True for non-CPython, so a pure wheel is strictly better there).

This is a maintainer call, not a defect — but the PR body should say "yes, install behaviour changes for platforms without a binary wheel" so it gets reviewed as such rather than slipping in under "Nope".

Started publishing an additional pure-Python wheel alongside the existing
per-platform binary wheels and the ``sdist`` -- by :user:`webknjaz`.

2. Filename detection silently emits garbage if the glob does not match
.github/workflows/ci-cd.yml:127-128

basename dist/*.tar.gz has no set -euo pipefail and no failglob. If python -m build ever emits zero or more than one matching file, this step still succeeds:

• zero matches → bash passes the literal pattern through, and the output becomes dist/*.tar.gz, which every downstream job then feeds to pip install / cibuildwheel / checkout-python-sdist as a filename.
• two matches → basename prints two lines, producing a malformed multi-line $GITHUB_OUTPUT entry.

Since six downstream jobs now depend on these two outputs, a failure here should be loud and local rather than showing up as six confusing "file not found" errors.

A set -euo pipefail plus a shopt -s failglob (or an explicit count assertion) makes the failure land in the job that caused it.

        echo "sdist-filename=$(basename dist/*.tar.gz)" >> "$GITHUB_OUTPUT"
        echo "wheel-filename=$(basename dist/*.whl)" >> "$GITHUB_OUTPUT"

3. `benchmark` now does both a git checkout and an sdist unpack, with no explanation
.github/workflows/ci-cd.yml:509-520

The job keeps actions/checkout@v7 (now bare — the with: submodules: true block was reassigned to the new step) and then unpacks the sdist over that checkout, then additionally downloads dist-pure into dist/ for the pip install.

The result works — python -Im pytest runs in isolated mode, so sys.path excludes the CWD and import aiohttp resolves to the compiled site-packages install, not the extension-less source tree left in the workspace. But that is a non-obvious invariant: drop the -I and the benchmarks would silently measure the pure-Python fallback shipped in the sdist.

Worth a short comment on the checkout step saying it is retained for CodSpeed's git metadata, and on the pytest step noting that -I is what keeps the workspace copy from shadowing the installed extension modules.

    - name: Checkout project
      uses: actions/checkout@v7
    - name: Retrieve the sdist and unpack it into the workspace
      uses: re-actors/checkout-python-sdist@release/v3

4. `requirements/cython.txt` as a global build constraint is broader than intended
pyproject.toml:175-179

Two observations about pointing all four constraint vars at requirements/cython.txt:

• It is largely inert for the Cython pin. [build-system].requires is pkgconfig + setuptools >= 67.0 — Cython is not a build requirement, and the sdist ships pre-generated .c. So no wheel build from the sdist ever installs Cython, and PIP_BUILD_CONSTRAINT=cython.txt constrains a package that is never resolved.

• It drags in unrelated pins. requirements/cython.txt also pins multidict==6.7.1 and typing-extensions==4.16.0 (transitively via requirements/multidict.in). Those happen to match runtime-deps.txt and test-mobile.txt today, so nothing breaks — but the files are Dependabot-managed independently. A bump landing in one and not the other turns into a resolution conflict inside cibuildwheel, which is a slow and confusing place to debug.

A dedicated requirements/build-constraints.txt containing only the Cython pin would express the intent without the coupling. Happy for this to be a follow-up given the minimal-diff goal — flagging so it does not get forgotten.

[tool.cibuildwheel.environment]
PIP_BUILD_CONSTRAINT = "requirements/cython.txt"
PIP_CONSTRAINT = "requirements/cython.txt"
UV_BUILD_CONSTRAINT = "requirements/cython.txt"
UV_CONSTRAINT = "requirements/cython.txt"

---

Checklist

• No hardcoded secrets or credentials introduced
• Release/publish path remains correctly gated to tag pushes — critical ＃1, critical ＃2
• No debug/scaffolding code left in the diff — critical ＃1, critical ＃2
• Job permissions match what each job actually needs
• sdist contains everything the from-sdist jobs need (.mypy.ini, coverage configs, tests, Makefile, requirements, vendored llhttp)
• Shell steps fail loudly on unexpected input — suggestion ＃2
• Changelog fragments present and follow repo conventions
• PR description matches the actual behaviour change — suggestion ＃1
• Backward-compatible for existing consumers (install path) — suggestion ＃1

---

To rebase and address feedback, mention me: @aiolibsbot rebase critical (fixes 🔴 only), @aiolibsbot rebase important (fixes 🔴 + 🟡), or @aiolibsbot rebase --fix for all. (A bare @aiolibsbot rebase only rebases onto the base branch.)

---

Automated review by Kōan (Claude) HEAD=f9dc264 8 min 25s

[Dreamsorcerer]

Is it worth splitting into 2 jobs? I feel like keeping it in one job is probably enough (and reduces the number of runners needed at once).

[webknjaz]

This is a temporary solution, I didn't want to figure out a big refactoring in scope of this PR.

[webknjaz]

Backport PR to 3.15: #13413

[webknjaz]

Backport PR to 3.14: #13414