[ekmartin]

Fixes the lost wakeup reproduced by #229, and includes that PR's test commit. The fix is in the second commit.

The problem

Wakeups from WorkerPool are addressed to the fiber, not to the work item. Ruby can run GC finalizers inside the scheduler's block hook before the fiber suspends. If a finalizer offloads a blocking operation (e.g. Tempfile's finalizer closing and unlinking its file), the same fiber re-enters WorkerPool#call. The outer work's wakeup then resumes the nested wait, which:

1. treated the wake as spurious and cancelled the nested operation. If it hadn't started, it was skipped, and rb_nogvl returned as if it had run, so the unlink was silently lost.
2. consumed the only wakeup for the outer operation, which then never woke.

Why a plain wake used to cancel

The cancel-on-wake comes from the first version of the wait loop (0345ded). It wrapped block in rb_protect and cancelled whenever the fiber woke without the work being complete. The intended case was an exception (cancel, then wait for the worker to drain). A wake without an exception took the same path because the two weren't distinguished. 84e03fb kept that shape, and #209 split the loop into a wait part and an rb_ensure part, still cancelling on plain wakes in the wait part.

I couldn't find anything that relies on a plain wake meaning "cancel":

• Async: Task#stop/#cancel raise Async::Cancel into the fiber (Fiber.scheduler.raise → Selector#raise → Fiber#raise), and with_timeout raises too.
• fiber_interrupt raises.
• Thread#raise/#kill become exceptions when the fiber is resumed, because fiber_switch checks pending interrupts.

All of these unwind through rb_ensure, which still cancels the work and drains it. A wake without an exception and without completion is a stale unblock (cf. socketry/async#467 and #469, where Async now retries io_wait/process_wait on stale wakeups) or this re-entrancy. RMagick's rmagick_gvl.cpp documents the consequence: "IO::Event::WorkerPool can also cancel a call that no worker has started without raising, when the fiber wakes up early right after a task is stopped, and Ruby then returns as if the call had run". RMagick re-runs such calls to work around it.

So the distinction is now explicit: an exception or other non-local exit cancels and drains, and any other wake just keeps waiting.

The fix

• worker_pool_work_wait loops until the work's own wakeup has been accounted for, and never cancels.
• Each pool keeps a fiber → innermost-waiting-work table (st_table, protected by the GVL). Each work item records the outer work the same fiber was already waiting for.
• When the fiber wakes without an exception, the wakeup is attributed to the innermost work in that chain that has completed but hasn't been woken yet. If there is none, it's a spurious wakeup and is ignored.
• When a call returns (normally or after draining on an exception), it re-issues rb_fiber_scheduler_unblock for any outer work whose wakeup it consumed. The outer block then resumes once its work is complete, whether that happened before, during or after the nested call.

The accounting keeps this invariant: pending wakeups for the fiber ≥ completed-but-not-woken work in its chain. So a wait never blocks when its work is done and no wakeup is coming. It doesn't depend on GC timing or the number of workers.

Alternatives considered

• Only removing the cancellation (while (!completed) block) isn't enough. I tried it: the spurious-wakeup test passes, but 4 of the re-entrancy tests still time out, because the nested wait still consumes the outer wakeup.
• Always re-issuing a wakeup when a nested call returns and outer work has completed is simpler, but it re-issues even when the outer wakeup is still pending, which adds stale wakeups. Those cause their own problems (Sleep/IO timeout can return early under concurrent per-fiber timeouts (async 2.42.0) async#467). Re-issuing only consumed wakeups avoids that.
• Storing the per-fiber state on the fiber (a fiber local or hidden ivar) would also cover nesting across two different WorkerPool instances. But it either exposes a raw pointer to Ruby code or depends on the fiber not being frozen. A per-pool table is invisible to Ruby and Ractor-safe, since pools aren't shareable. Nesting across pools doesn't happen with a scheduler that uses one pool.

Scheduler / Ruby level

The underlying issue is that Ruby runs finalizers (and other deferred work) inside fiber scheduler hooks, which re-enters the scheduler from arbitrary points. Deferring finalizers until the fiber has actually suspended, or masking them inside scheduler hooks, would be the more general fix, and it's out of scope here. One consequence that WorkerPool can't fix: if an exception (e.g. Async::Cancel or a timeout) is raised into the fiber while it's in a nested wait inside a finalizer, the nested operation is correctly cancelled and drained. But Ruby discards exceptions from finalizers, so the outer wait doesn't see the exception and keeps waiting for its operation to complete.

Testing

ReentrantBlockScheduler from #229 now takes a list of nested durations, so it can nest more than once. It also takes an optional hook that runs before the nested call. New tests:

• outer work completes before the nested call starts (asserted via completed_count)
• outer work completes during the nested call (the original test, single worker)
• nested work finishes first (two workers)
• two levels of nesting, with one and with three workers
• an exception raised while nested (two workers): both operations are cancelled, the exception propagates through block, and both are drained before call returns (call_count == completed_count == 2, current_queue_size == 0)
• a spurious unblock of the waiting fiber doesn't cancel the operation

Without the fix, 5 of these fail. The nested-finishes-first and exception-while-nested tests pass before and after, as regression guards.

• macOS (KQueue, Ruby 4.0.1): full suite: 323 passed, 21 skipped. worker_pool.rb passes with IO_EVENT_SELECTOR=Select and KQueue, and in 25 back-to-back runs.
• Linux (Docker ruby:4.0, with liburing): worker_pool.rb passes with EPoll, URing and Select. Full suite: 419 passed, 21 skipped.
• RuboCop: no offenses.

Benchmark

There's no worker pool benchmark in the repository, so I used an ad-hoc one: 100 fibers × 200 busy(duration: 0) calls through TestScheduler, best of 5. There's no measurable difference:

workers	before	after
1	9.2–9.6 µs/call	9.6–9.7 µs/call
4	10.4–10.5 µs/call	10.5–10.6 µs/call

With Async and real Tempfile finalizers

Setup: async 2.46.0 with ASYNC_SCHEDULER_WORKER_POOL=true (single worker). The test creates 20 garbage Tempfiles, then a task inside with_timeout(3) does 20 File.writes. A block wrapper runs GC.start, so the finalizers' close/unlink happen inside block:

• before: hangs in 5 of 5 runs, past the task's own 3 s timeout (killed after 20 s)
• after: completes in 5 of 5 runs (40 re-entrant calls; call_count == completed_count)

I also tried a Rack 3.2 multipart workload under Async with a single worker and large allocations, without forcing GC inside block. With natural GC timing, finalizers never landed inside a WorkerPool block locally (0 re-entrant calls), so it didn't reproduce the hang on either build.

🤖 Generated with Claude Code

[Watson1978]

FYI, I reported the Ruby side of this to https://bugs.ruby-lang.org/issues/22428.
When a scheduler cancels an operation before it starts, rb_nogvl returns the zeroed result as if the function had run.
This PR removes the trigger in io-event, and the Ruby ticket is about the silent success itself.