Skip to content

Explore MPMC queue contention, wakeup, and storage optimizations #298

Description

@tisonkun

Goal

Explore measurable performance improvements to the competing MPMC queues introduced in #265. Preserve their delivery, strict bounded capacity, cancellation, and disconnection contracts. The existing mutex-based implementation is a valid baseline; a lock-free rewrite is not a prerequisite.

Candidates to investigate

  1. Queue and waiter coordination. Shared uses a queue mutex plus separate sender and receiver semaphores, each with its own waiter lock. Profile lock acquisitions, cache contention, and retries after a notified task loses the race. Compare this with keeping queue state and waiter bookkeeping under one mutex before introducing more elaborate synchronization.
  2. Separate bounded and unbounded internals where useful. Both currently share capacity: Option<usize> and both semaphore fields, although unbounded sends never wait for capacity. A smaller unbounded state and a bounded-specific capacity model may simplify hot paths. Public endpoint types can stay unchanged.
  3. Bounded storage allocation. The current VecDeque grows lazily while the queue mutex is held. Compare preallocation or a fixed-capacity buffer with the current approach, including construction cost and large, mostly empty capacities. Preserve the exact requested logical capacity.
  4. Unbounded burst handling and memory retention. Measure allocation frequency and retained memory after a burst drains. Investigate chunked storage, incremental reclamation, or limited internal batching if measurements justify them. MPSC's receiver-private batch relies on a unique receiver; do not copy it into MPMC without preserving message accessibility and correct cancellation/drop behavior for competing receivers.
  5. Waiter allocation and scheduling overhead. Profile repeated registration, redundant waker clones, and wake-to-poll round trips under empty/full transitions. Optimize only measured costs while preserving notification transfer on cancellation and the Waker contract in AGENTS.md.

These are hypotheses to test, not a requirement to implement every candidate. Coordinate capacity-accounting changes with the bounded-reservation work in #297.

Measurement and acceptance

Start with the existing MPMC ecosystem benchmarks, which compare asyncband, async-channel, and Flume across 1P/1C, 1P/8C, 8P/1C, and 8P/8C. Keep producer/consumer behavior and message counts comparable.

  • Cover bounded and unbounded queues, small and larger capacities, steady traffic, bursts, and idle-to-active transitions. Include workloads where receivers stop at different times rather than only consuming equal quotas.
  • Report reproducible before/after results with commit, toolchain, hardware, runtime/thread setup, and benchmark parameters. Measure throughput and latency; include allocations and peak/retained memory for storage changes.
  • Explain which cost each change removes and report regressions across the other topologies. Land independently reviewable improvements rather than one combined backend rewrite.
  • Preserve deterministic cancellation and disconnect regressions, exactly-once delivery, and bounded-capacity checks; use the relevant cargo x workflows and Miri where appropriate.

Follow-up to #211 and #265; part of #206. This issue does not add public APIs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions