Repository navigation
Pyjo.Reactor.Winloop: a socket that loops watch, polled by one at a time, and a timer not due before its time - #6
Merged
Merged
Conversation
Documentation build overview
|
Contributor
Tests
|
libuv of Windows polls a socket with an exclusive request of AFD, which ends the request of any other loop for the same socket, also of a copy of it, and that loop polls it again, in a busy loop of C with no event at all. The workers of Pyjo.Server.Prethread watch the listen sockets of the manager on Windows, the same sockets in every thread: four idle loops took 240% of the processors and starved the process and the tests next to it, with or without the GIL. _Loop keeps the readers and the writers of each socket, and a registry of the process has the loops that watch it, the owner that libuv polls it for, and the loops that run and wait for it. The owner hands it over to the next one after each of its events, so that the connections go to every worker in turn, and at the end of its run; a loop changes the polls of libuv only in its thread. A socket that one loop watches goes to libuv as before. Closes #5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pyjoyment of Git main with pyjoyment/Pyjoyment#550 and pyjoyment/Pyjoyment#552, the tests of a page of GitHub without its title in the tutorial and of the servers of Hot and Reload that did not answer or did not stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dex4er
force-pushed
the
shared-sockets
branch
from
November 5, 2026 14:07
45af24c to
fc4b016
Compare
Pyjoyment of Git main with pyjoyment/Pyjoyment#553, a sleep of Pyjo.Steps in its tests on an event loop of libuv, whose timer may be due a millisecond early, and pyjoyment/Pyjoyment#554, the data of a connection that the daemon has removed discarded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libuv counts a timer in whole milliseconds from uv_now, its clock rounded down, and Winloop rounds the delay to them, as uvloop does, so a timer, as asyncio.sleep, was due up to a millisecond before its time, where the loop of asyncio is never early: a sleep of 2 ms took 1.5 ms. call_later of _Loop, which call_at of Winloop calls too, rounds the delay up to a millisecond and adds the one that the clock loses, as _Loop of Pyjo.Reactor.Uvloop does, after MagicStack/uvloop#755. The subtest of Pyjo-Reactor-Uvloop measures with time.perf_counter, which time.monotonic is on Windows since Python 3.13, and which counted in the ticks of the system of 15.6 ms before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Loops of Winloop that watch one socket, as the workers of
Pyjo.Server.Prethreadwatch the listen sockets of the manager on Windows, polled it in a busy loop, which starved the process and the tests next to it: the hangs oft/Pyjo/Server_Prethread.py,t/Pyjo/Server_FastCGI.pyandt/Pyjo/Server_Lambda.pyof Pyjoyment on the reactor, on 3.15t in CI.Closes #5
Changes
Pyjo/Reactor/Winloop.py:_Loopkeeps the readers and the writers of each socket, and a registry of the process,_SOCKETS, has the loops that watch it, the owner that libuv polls it for, and the loops that run and wait for it. The owner hands it over to the next one after each of its events, so the connections go to every worker in turn, and at the end of its run; a loop that starts to run takes it from an owner that does not run. A loop changes the polls of libuv only in its thread, withcall_soon_threadsafewhen another loop hands a socket over to it. A socket that one loop watches goes to libuv as before, only the reader or the writer that changes, and a run looks at the sockets of the loop only while some socket is shared. The module documentation says that reactors in threads take turns with a socket that they share.Pyjo/Reactor/Winloop.py:_Loop.call_later, whichcall_atof Winloop calls too, rounds the delay up to a millisecond and adds one, as_Loopof Pyjo.Reactor.Uvloop does (uv.lock: Pyjoyment of Git main at b6d623e, and a timer of uvloop not due before its time Pyjo-Reactor-Uvloop#8, Fix timers firing up to 1ms early MagicStack/uvloop#755). libuv counts a timer in whole milliseconds ofuv_now, its clock rounded down, and Winloop rounds the delay, so a timer, asasyncio.sleep, was due up to 1 ms before its time, where the loop of asyncio is never early: a sleep of 2 ms took 1.5 ms. Tests of Pyjoyment work around it since Tests: a sleep of Pyjo.Steps on an event loop of libuv, whose timer may be due a millisecond early Pyjoyment#553. The module documentation says that a timer may be up to 2 ms late instead.typings/winloop/__init__.pyi:call_later,remove_reader,remove_writer,call_soon_threadsafe,closeandis_closed, which_Loopcalls.t/Reactor_Winloop.py:Timer not due before its timeof Pyjo-Reactor-Uvloop, 150 timers of 2 ms each of the reactor,call_later,call_atandasyncio.sleep, measured withtime.perf_counter, whichtime.monotonicis on Windows since Python 3.13, and which counted in ticks of 15.6 ms before; it fails without the change in 9 of 9 runs on 3.12, 3.15 and 3.15tt/Reactor_Winloop.py:A socket that reactors in threads watch: four reactors in threads watch one listen socket; idle, they take less than half of a processor, which fails without the change, and 40 connections go to all of them.AGENTS.md: why, and how, with the measurements, for both.uv.lock: Pyjoyment 2cc52c7 → 7b09754 of Gitmain, with Tests: a page of GitHub without its title skipped in the tutorial, a server of Reload ended with its process group, and process id files of their own Pyjoyment#550 and Tests: the log and the processes of a server of Hot or Reload that did not answer or did not stop Pyjoyment#552, the tests of a page of GitHub without its title in the tutorial and of the servers of Hot and Reload that did not answer or did not stop, Tests: a sleep of Pyjo.Steps on an event loop of libuv, whose timer may be due a millisecond early Pyjoyment#553, a sleep ofPyjo.Stepsin its tests on an event loop of libuv, and Pyjo.Server.Daemon: the data of a connection that the daemon has removed, whose stream reads on while it closes gracefully, is discarded Pyjoyment#554, the data of a connection that the daemon has removed discardedMeasurements
On Windows 11 ARM64, 4 loops in threads that watch one listen socket, idle for 2 seconds, then 40 connections:
winloop.Loop_Loopof this changet/Pyjo/Server_Prethread.pyof Pyjoyment, 4 copies at once,PYTHONDEVMODE=1:Validation
t/of the repository withPYTHONDEVMODE=1, on 3.12, 3.15 and 3.15t; the new subtests fail without their changesPYTHONDEVMODE=1,-j 4, on 3.15 and on 3.15t: 434 files, 7020 tests pass.trunk/trunk.yaml; Biome does not run on this machine, Trunk Check of CI checksAGENTS.mdt/of the repository with Pyjoyment 7b09754 ofuv.lockCapture the standard errorandTimeoutoft/Pyjo/Prove.pymay still fail now and then, a process of the script not killed on the runner (Tests on Windows: a stream whose peer aborted the connection, and Pyjo.Prove before Python 3.15 Pyjoyment#545)🤖 Generated with Claude Code