Skip to content

Make io_uring cancellation submission exception-safe #228

Open

Description

Problem

Cancellation can unwind a waiting fiber before its original kernel operation has completed if obtaining the cancellation SQE raises.

This was found while reviewing #210, but the shared cancellation implementation already exists on main (fb8c691); it is not introduced by Futex support.

Cause

  1. Completion_cancel_async sets cancellation_pending = true before calling io_get_sqe.
  2. If the submission queue is full, io_get_sqe calls io_uring_submit_now. This can yield or raise, including when io_uring_submit returns an error.
  3. Waiting_cancel_and_wait invokes Completion_cancel_async outside its protected draining loop.

If cancellation submission raises, cleanup can unwind without observing the original operation's completion. Stack-backed waiting state and, depending on the caller, buffer locks/references can be released while the operation is still outstanding. The completion also remains marked as having a pending cancellation even though no cancellation SQE was prepared.

The existing read cleanup, write cleanup, and waitid cleanup use this shared helper. These paths are implicated by code inspection; the fault-injected reproduction below used the Futex branch.

Reproduction observed

Reproduced for both single and vector Futex waits on #210 at 9944c6d, using Linux, Ruby head, and liburing 2.6:

  1. Submit an original wait with selector.select(0).
  2. Fill the 64-entry submission queue with other waits without calling select.
  3. Use an isolated LD_PRELOAD shim to make io_uring_submit return -EIO.
  4. Raise into the original waiting fiber, triggering cancellation cleanup.
  5. Observe that the fiber exits with Errno::EIO and its Futex can be closed.
  6. A separate Futex referencing the same address can still wake one kernel waiter.

Thus the Ruby wait has finished and released its reference while the kernel wait remains pending. The subprocess disabled GC and used Process.exit! without processing further completions to avoid dereferencing the abandoned wait state. This demonstrates premature cleanup under injected submission failure; it is not a naturally occurring crash reproduction.

Proposed direction

Make requesting cancellation non-raising and non-yielding:

  • Try to acquire an SQE directly, without flushing.
  • If the queue is full, retain the completion on a selector-owned pending-cancellation list without allocating. Track the deferred request explicitly and prevent completion reuse.
  • Have the event loop prepare and submit deferred cancellations before blocking.
  • Retain the waiting frame and any kernel-visible memory until the original operation's CQE has been observed. Track the cancellation CQE separately before recycling the completion.

Simply moving the call inside rb_protect is insufficient: a failed submission must not leave an untracked cancellation that later calls silently skip.

Unrecoverable ring errors also need an explicit selector-level shutdown policy. A failed cancellation submission is not proof that the original operation has stopped accessing memory.

Regression coverage

Cover cancellation with a full submission queue, injected submission errors, and interruption while submission yields. Verify that outstanding operations retain their wait state and memory, cancellation is not silently lost, and completion records are not reused prematurely. Keep this fix separate from #210.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions