Build linux/macos arm64/aarch64 wheels on native runners by ddelange · Pull Request #1308 · MagicStack/asyncpg

ddelange · 2026-02-22T21:21:03Z

I did the initial implementation of the parallel release.yml in #954 and I'm providing some maintenance here :)

This PR contains the following commits:

switch to linux arm64 native runner (blog post) and macos x86 native runner (blog post)
fix CI rot related to setuptools deprecating pkg_resources
various CI speedups
fix CI rot related to cpython 3.9 on arm64 macos runners

the runner changes avoid emulation (qemu on linux, rosetta on macos), mainly for speedup (e.g. linux aarch64 wheel build is currently taking 30+ mins vs. 4 mins for x86). i left the setup-qemu step in place in case you want to add more wheel architectures for which no native runners exist

ddelange · 2026-02-22T22:29:44Z

+          # github does not provide 3.9 cpython for arm64 runners
+          - python-version: "3.9"
+            os: macos-15


fyi Python 3.9 is EOL since 31 Oct 2025, can consider dropping support

ddelange · 2026-05-16T10:46:57Z

@elprans this should fix the failing CI on master

ddelange · 2026-07-14T12:06:19Z

@elprans can i support in getting this merged?

ddelange · 2026-08-24T07:51:32Z

if uvloop doesnt have 3.15 wheels yet and we depend on it in our suite, i would wait for upstream support. happy to make the supported range in this PR 3.10-3.15 per your and my comment, but i would wait for the complete suite here to go green (when upstream has support) before merging.

honglei · 2026-08-24T08:17:31Z

Edit for lineage: GitHub cannot restore the comment I deleted, so the
following is a reconstruction of its original point, not a verbatim copy:

I initially suggested expanding #1308's supported Python range from
3.10-3.14 to 3.10-3.15 and incorporating the Python 3.15/3.15t work here,
based on the local compatibility testing.

That suggestion is withdrawn. I reconsidered the scope and opened #1352 as a
separate draft, so there is no need to expand #1308's supported range for that
work.

I also checked the current upstream state: uvloop 0.22.1 has no cp315 or
cp315t files on PyPI, and uvloop's test matrix currently stops at
3.14/3.14t. I will keep #1352 draft. Once uvloop supports 3.15, the temporary
uvloop exclusions and asyncpg's existing <3.15.0 dependency marker can be
updated together, and the complete asyncio/uvloop matrix must be green before
the PR is marked ready.

Sorry for deleting the earlier comment instead of editing it. I will preserve
the discussion lineage by editing comments going forward.

ddelange · 2026-08-24T09:50:42Z

please dont delete your github comments, edit them instead. deleting is bad for lineage and future reference

ddelange · 2026-09-16T09:58:32Z

fwiw there is now also windows-11-arm in case we want to add arm64 wheels for windows

elprans · 2026-09-23T20:18:44Z

@ddelange, please rebase

fwiw there is now also windows-11-arm in case we want to add arm64 wheels for windows

Sure!

setuptools 82.0.0 removed pkg_resources, breaking the build. Use packaging.requirements.Requirement instead, and switch the Cython dependency string to PEP 508 format (no parentheses).

- Switch to official PyPA build frontend for sdist ref https://pypi.org/project/build/ - Cancel running jobs when a new commit is pushed to a PR ref https://stackoverflow.com/questions/66335225#comment133398800_72408109 - Switch from setup-python to setup-uv ref https://github.com/astral-sh/setup-uv

ref https://github.com/MagicStack/asyncpg/actions/runs/22286259051/job/64465189709?pr=1308 failed with: ====================================================================== ERROR: test_tls_version (test_connect.TestSSLConnection) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/runner/work/asyncpg/asyncpg/asyncpg/_testbase/__init__.py", line 92, in wrapper self.loop.run_until_complete(coro) File "/Applications/Xcode_16.4.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete return future.result() File "/Users/runner/work/asyncpg/asyncpg/tests/test_connect.py", line 2024, in test_tls_version await self.connect( File "/Users/runner/work/asyncpg/asyncpg/asyncpg/connection.py", line 2443, in connect return await connect_utils._connect( File "/Users/runner/work/asyncpg/asyncpg/asyncpg/connect_utils.py", line 1209, in _connect addrs, params, config = _parse_connect_arguments(**kwargs) File "/Users/runner/work/asyncpg/asyncpg/asyncpg/connect_utils.py", line 893, in _parse_connect_arguments addrs, params = _parse_connect_dsn_and_args( File "/Users/runner/work/asyncpg/asyncpg/asyncpg/connect_utils.py", line 798, in _parse_connect_dsn_and_args ssl.minimum_version = _parse_tls_version( File "/Applications/Xcode_16.4.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/ssl.py", line 586, in minimum_version super(SSLContext, SSLContext).minimum_version.__set__(self, value) ValueError: Unsupported protocol version 0x304

elprans

👋 , I've rebased and fixed up myself. Thanks for working on this!

@elprans

## Improvements * Support Python 3.15 (MagicStack#1367, MagicStack#1378) (by @elprans in 1b11846 and 58c0c59) * Support PostgreSQL 19 (MagicStack#1379) (by @elprans in 8d2a5ae) * Make pool `min_size` a maintained connection floor and add `init_size` to control the number of connections opened at startup (MagicStack#1323). Idle connections at or below `min_size` are retained, and closed connections are replaced in the background. `init_size` defaults to `min_size`. To preserve the previous behavior, where the pool could drain to zero, replace `min_size=N` with `init_size=N, min_size=0`. (by @shatilov-diman in 67a1cbc) * Use `hashlib.pbkdf2_hmac()` for faster SCRAM password derivation (MagicStack#1359) (by @twelfthlabor in 00319a9) * Use the statement cache for `copy_records_to_table()` introspection (MagicStack#1360) (by @matemax in dd05865) ## Fixes * Clean up connections rejected by `target_session_attrs` immediately so an unresponsive rejected host cannot delay the selected connection, and cleanup tasks cannot outlive the event loop. Rejected connections are aborted without waiting for a graceful disconnect; servers may log "unexpected EOF on client connection" if the Terminate message is not delivered before the transport closes. * Fix a connection pool deadlock when releasing a connection after a protocol abort (MagicStack#1324) (by @pchaseh in 4a83cc2) * Honor `Connection.close(timeout=...)` while waiting for a pending cancellation (MagicStack#1361) (by @gyanu2507 in c06bcb4) * Prevent stuck cancellation from blocking connection teardown or pool release by bounding cancellation with the command or release timeout (MagicStack#1370) (by @elprans in 4cf0220) * Preserve the server error when the connection closes mid-operation (MagicStack#1369) (by @RitiGrover in 6c8f1a7) * Fix COPY cleanup when a records generator raises, preserving the original exception and avoiding protocol desynchronization and stalled rollback (MagicStack#1358) (by @CodingCossack in fdc6bc8) * Reparse unnamed statements before cursor binding when the statement cache is disabled (MagicStack#1364) (by @elprans in 302ba49) * Allow cursors in transactions started outside `Connection.transaction()`, such as with an explicit `BEGIN` (MagicStack#1350) (by @aryansk in 25aaaf5) * Remove the query logger when the `query_logger()` block raises (MagicStack#1373) (by @kratos0718 in 6a7fbd3) * Fix a segfault in `Record.get()` with an invalid number of positional arguments (MagicStack#1333) (by @apoorvdarshan in 40a3832) * Align SSL startup retries with libpq and improve SSL rejection diagnostics (MagicStack#1366). Retry alternative transports only before authentication succeeds. Direct TLS now requires `sslmode=require`, `verify-ca`, or `verify-full`, or an explicit `SSLContext`. (by @elprans in 78752c9) * Preserve password errors during SSL fallback when the fallback fails with a less specific authorization error (MagicStack#1331) (by @Mauricio0129 in 2d97947) * Handle cleartext password authentication when no password is supplied without raising an `AttributeError` (MagicStack#1351) (by @aryansk in 6be5915) * Set the `postgresql` ALPN protocol on SSL contexts created by asyncpg so direct TLS connections work with PostgreSQL 17 and later (MagicStack#1372). User-provided contexts must set this protocol for direct TLS connections. (by @DurandA in b21325d) * Link explicitly against `libm` on non-Windows platforms to fix undefined math symbol errors when loading the extensions (MagicStack#1305) (by @veeceey in db8ecc2) * Fix builds on FreeBSD 15 by linking against `libstdthreads` (MagicStack#1368) (by @elprans in 87bcaf9) ## Other * Replace `pkg_resources` with `packaging` in `setup.py` (MagicStack#1362) (by @elprans in 6695db7) * Build Linux and macOS ARM64 wheels on native runners and add Windows ARM64 wheel builds (MagicStack#1308) (by @ddelange in 1e8cc1c) * Add manual and PR-label-triggered dry runs to the release workflow (MagicStack#1374) (by @elprans in 1bd643e) * Clarify how `execute()` uses prepared statements and how to confirm an `INSERT` in the documentation (MagicStack#1336) (by @pctablet505 in cdf816c) * Fix typos in comments and docstrings (MagicStack#1332) (by @maxtaran2010 in bc3e246)

ddelange force-pushed the native-builds branch 9 times, most recently from 5be2c72 to dafc5bb Compare February 22, 2026 22:26

ddelange commented Feb 22, 2026

View reviewed changes

ddelange force-pushed the native-builds branch 4 times, most recently from 434ebc2 to 613099e Compare February 23, 2026 10:03

honglei mentioned this pull request Aug 24, 2026

Add Python 3.15 and 3.15t CI and wheels #1352

Closed

elprans mentioned this pull request Sep 23, 2026

Enable arm64 support for Windows. #1319

Closed

ddelange and others added 5 commits September 28, 2026 13:34

Build linux/macos arm64/aarch64 wheels on native runners

69273a9

Replace pkg_resources with packaging

1e7f0ec

setuptools 82.0.0 removed pkg_resources, breaking the build. Use packaging.requirements.Requirement instead, and switch the Cython dependency string to PEP 508 format (no parentheses).

Preserve stable Python coverage in PostgreSQL CI

409b214

elprans force-pushed the native-builds branch from 89b7a32 to 409b214 Compare September 28, 2026 20:45

Build Windows ARM64 wheels on native runner

4bd0295

elprans approved these changes Sep 28, 2026

View reviewed changes

elprans merged commit 1e8cc1c into MagicStack:master Sep 28, 2026
51 checks passed

elprans mentioned this pull request Oct 6, 2026

asyncpg v0.32.0 #1385

Merged

Navigation Menu

Repository navigation

Build linux/macos arm64/aarch64 wheels on native runners - #1308

Build linux/macos arm64/aarch64 wheels on native runners#1308
elprans merged 6 commits into
MagicStack:masterfrom
ddelange:native-builds

ddelange commented Feb 22, 2026 •

edited

Loading

Uh oh!

ddelange Feb 22, 2026

Uh oh!

ddelange commented May 16, 2026

Uh oh!

ddelange commented Jul 14, 2026

Uh oh!

ddelange commented Aug 24, 2026

Uh oh!

honglei commented Aug 24, 2026 •

edited

Loading

Uh oh!

ddelange commented Aug 24, 2026

Uh oh!

ddelange commented Sep 16, 2026

Uh oh!

elprans commented Sep 23, 2026

Uh oh!

elprans left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ddelange commented Feb 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ddelange Feb 22, 2026

Choose a reason for hiding this comment

Uh oh!

ddelange commented May 16, 2026

Uh oh!

ddelange commented Jul 14, 2026

Uh oh!

ddelange commented Aug 24, 2026

Uh oh!

honglei commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ddelange commented Aug 24, 2026

Uh oh!

ddelange commented Sep 16, 2026

Uh oh!

elprans commented Sep 23, 2026

Uh oh!

elprans left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ddelange commented Feb 22, 2026 •

edited

Loading

honglei commented Aug 24, 2026 •

edited

Loading