worker_threads: proposal for Persistent Worker Runtime with stateful workers and outbox dispatch

[FelipeMiiller]

What is the problem this feature will solve?

Node.js' single-threaded Event Loop excels at asynchronous non-blocking I/O. However, synchronous CPU-bound operations (cryptography, AST/JSON parsing, image manipulation, template rendering, ML embeddings) freeze the main Event Loop, leading to latency spikes and request timeouts.

Currently, developers face several limitations when trying to offload work:

1. Manual new Worker() per task induces high memory allocation and V8 Isolate bootstrap latency (~20ms per task).
2. Existing userland worker pools (piscina, workerpool) treat worker threads strictly as disposable, stateless function runners, discarding in-memory worker heap state between tasks.
3. In-process background tasks & side-effects (e.g. the Transactional Outbox pattern: sending an email or webhook after writing to the DB) have no standard core abstraction. Developers either do naive fire-and-forget on the main thread (risking Event Loop blocking during template compilation and unhandled promise crashes) or are forced to add external infrastructure like Redis + BullMQ / RabbitMQ just for simple in-process background jobs.

In our empirical benchmarks, executing CPU tasks synchronously on the main thread froze the Event Loop for 2,207ms, completely starving incoming I/O during computation.

What is the feature you are proposing to solve the problem?

We propose adding a built-in Persistent Worker Runtime to Node.js (either as node:worker_runtime or as an enhancement under node:worker_threads), operating on the core axiom:

"The Event Loop coordinates. Persistent Workers execute."

Core Capabilities:

1. Dual Execution Model:
   • runtime.execute(task): Interactive request-response mode. Offloads computation and awaits the result with near-zero Event Loop lag (~6.5ms in benchmarks).
   • runtime.dispatch(task): Background dispatch mode for Transactional Outbox / fire-and-confirm patterns. Returns an unblocked TaskHandle immediately, allowing sub-millisecond HTTP responses while exposing .onComplete(cb) and .onError(cb) for asynchronous confirmation.
2. Bounded Concurrency Batch Execution (executeAll):
   • Native Promise.all semantics, but bounded strictly to the worker pool's hardware capacity (e.g., 4 workers processing 40 tasks in parallel without CPU thrashing).
3. Stateful Workers with Persistent L1 Heap Memory:
   • Dedicated workers can retain warm in-memory heaps (caches, loaded models, compiled WASM modules) across tasks with worker affinity. Benchmarks show a 33.2x latency speedup (1.58ms vs 52.37ms per query) compared to stateless reloading.
4. Diagnostic & Backpressure Integration:
   • Native AsyncResource (node:async_hooks) propagation to preserve OpenTelemetry / APM distributed tracing context across threads.
   • Non-blocking queue backpressure with timeout (TaskQueueTimeoutError) to prevent process OOM.
   • Supervisor with automatic crash detection and worker isolate replacement.
5. Zero-Copy Memory Transfer:
   • Full support for transferList to transfer ArrayBuffer instances in <0.1ms without memory copying.

Reference Implementation:

We have developed a functional, production-grade reference implementation written in pure vanilla JavaScript (ESM, zero external runtime dependencies) with 100% passing tests on Node.js 22/24, 9 Architecture Decision Records (ADRs), and empirical benchmarks:
👉 https://github.com/FelipeMiiller/persistent-worker-runtime

What alternatives have you considered?

1. Userland libraries (piscina, workerpool):
   While Piscina is excellent for stateless function execution, it lacks native support for persistent stateful workers with warm L1 memory affinity, in-process outbox dispatch with asynchronous event confirmation, and standard zero-dependency stdlib availability.
2. External Queues (Redis + BullMQ / RabbitMQ / SQS):
   Heavy operational and infrastructure overhead for applications that only require reliable in-process background job offloading.
3. libuv Threadpool (UV_THREADPOOL_SIZE):
   Only handles internal asynchronous I/O and specific C++ crypto/zlib operations; cannot be used to run arbitrary userland JavaScript code.
4. Child Processes (child_process.fork):
   Full OS process sandboxing incurs 30-50MB RAM per process and higher IPC serialization latency compared to lightweight V8 Isolates with zero-copy transferList.

[everett1992]

I can't really tell what you're asking for

[FelipeMiiller]

What I propose adding to core: a new module node:persistent_worker_runtime (or an opt-in helper exported from node:worker_threads — open to whatever the TSC prefers in terms of API surface) that wraps the raw Worker / MessagePort plumbing to deliver three things new Worker() does not give you today:

A worker pool with elastic sizing + worker affinity — runtime.execute({ type, payload, fn, affinityKey }) / runtime.dispatch(...). State is preserved per-worker across tasks via the state argument, so e.g. a warmed-up ML model or parsed lookup table survives 1000 invocations on the same worker (33× latency speedup in our benchmarks vs stateless reload).

Bounded concurrency batch dispatch — runtime.executeAll([task, task, ...]) and runtime.dispatchAll([...]) give you a Promise.all-style API but enforce a hard cap on the worker pool's workers config so 40 bursts to a 4-worker pool do not fork 40 microtasks.

Backpressure and lifecycle hooks — TaskQueueTimeoutError when the queue is saturated + maxQueueSize so users do not OOM under burst. runtime.on('worker:recycled', cb) for the watchdog pattern. Native AsyncResource propagation so OpenTelemetry spans cross thread boundaries without bespoke plumbing.

What I am NOT proposing: replacing worker_threads or adding a one-size-fits-all abstraction. node:worker_threads already exposes the primitives; this module composes them into the "persistent worker pool" shape most apps actually want, the same way node:fs/promises composes node:fs into promise-based ergonomics.