NetRunner

Safe OS process execution for Elixir. Zero zombie processes, NIF-based backpressure, PTY support, and cgroup isolation.

NetRunner combines NIF-based async I/O (enif_select) with a persistent shepherd binary that watches over child processes. Three layers of cleanup guarantee zero zombies under every failure mode — BEAM crash, GenServer crash, or resource leak.

Installation

def deps do
  [
    {:net_runner, "~> 1.3"}
  ]
end

Requires a C compiler (gcc or clang) and make.

Quick Start

# Run a command, collect output
{output, 0} = NetRunner.run(~w(echo hello))
# => {"hello\n", 0}

# Pipe data through stdin
{output, 0} = NetRunner.run(~w(cat), input: "from elixir")
# => {"from elixir", 0}

# Nonzero exit status
{"", 1} = NetRunner.run(~w(false))

# Multi-word commands with arguments
{output, 0} = NetRunner.run(["sh", "-c", "echo $HOME"])

Streaming

Lazy, demand-driven streams with backpressure — data stays in the OS pipe buffer until you consume it, so you can stream gigabytes without OOM:

# Stream stdout chunks with backpressure
NetRunner.stream!(~w(cat /usr/share/dict/words))
|> Stream.filter(&String.starts_with?(&1, "elixir"))
|> Enum.to_list()

# Pipe input through a command
NetRunner.stream!(~w(tr a-z A-Z), input: "hello world")
|> Enum.join()
# => "HELLO WORLD"

# Sort lines
NetRunner.stream!(~w(sort), input: "cherry\napple\nbanana\n")
|> Enum.join()
# => "apple\nbanana\ncherry\n"

# Process large files without loading into memory
File.stream!("huge.csv")
|> NetRunner.stream!(~w(grep ERROR))
|> Stream.each(&IO.write/1)
|> Stream.run()

# Non-raising variant
{:ok, stream} = NetRunner.stream(~w(sort), input: "c\nb\na\n")
Enum.to_list(stream)
# => ["a\nb\nc\n"]

Timeouts and Limits

# Kill process after 500ms
{:error, :timeout} = NetRunner.run(~w(sleep 100), timeout: 500)

# Cap output size — kills process if exceeded
{:error, {:max_output_exceeded, _partial}} =
  NetRunner.run(["sh", "-c", "yes"], max_output_size: 1000)

# Custom kill escalation: SIGTERM → wait 2s → SIGKILL
NetRunner.run(~w(my_server), kill_timeout: 2000, timeout: 10_000)

Input Validation and Error Returns

run/2 and stream/2 return tagged errors for bad input instead of crashing. NUL bytes inside cmd or args are rejected early (they are undefined in argv on the C side).

# Empty executable
{:error, {:invalid_cmd, _}} = NetRunner.run([""])

# NUL byte in an argument
{:error, {:invalid_args, _}} = NetRunner.run(["echo", "he\0llo"])

# Same behaviour for streaming
{:error, {:invalid_args, _}} = NetRunner.stream(["echo", "he\0llo"])

# Unknown signal atoms come back as tagged errors, not raises
{:error, :unknown_signal} = NetRunner.Signal.resolve(:sigwhatever)
{:error, :unknown_signal} = NetRunner.Signal.resolve(99)

Working with Binary Output

stdout is delivered as a BEAM binary, not a String. It is safe to pass bytes containing NUL, high-bit, or anything else through the pipeline.

# NUL bytes round-trip unchanged
{out, 0} = NetRunner.run(["sh", "-c", ~S|printf 'a\0b\0c'|])
byte_size(out)          # => 5
out == "a\0b\0c"        # => true

# UTF-8 boundaries straddle chunks fine — just concatenate and then
# decode.
"héllo\n" =
  NetRunner.stream!(~w(echo héllo))
  |> Enum.join()

Process API

For fine-grained control over the OS process lifecycle:

alias NetRunner.Process, as: Proc

# Start a process
{:ok, pid} = Proc.start("cat", [])

# Write to stdin
:ok = Proc.write(pid, "hello world")
:ok = Proc.close_stdin(pid)

# Read from stdout (blocks until data available)
{:ok, "hello world"} = Proc.read(pid)
:eof = Proc.read(pid)

# Wait for exit
{:ok, 0} = Proc.await_exit(pid)

Incremental reads and writes

{:ok, pid} = Proc.start("cat", [])

# Write in chunks — useful for feeding large data
:ok = Proc.write(pid, "chunk 1 ")
:ok = Proc.write(pid, "chunk 2 ")
:ok = Proc.write(pid, "chunk 3")
:ok = Proc.close_stdin(pid)

# Read comes back in whatever chunks the OS delivers
{:ok, data} = Proc.read(pid)
# data => "chunk 1 chunk 2 chunk 3" (may come in multiple reads)

Signals and process groups

{:ok, pid} = Proc.start("sleep", ["100"])
:ok = Proc.kill(pid, :sigterm)
{:ok, 143} = Proc.await_exit(pid)  # 128 + SIGTERM(15)

# Signals kill the entire process group (catches grandchildren)
{:ok, pid} = Proc.start("sh", ["-c", "sleep 100 & sleep 100 & wait"])
:ok = Proc.kill(pid, :sigkill)
# All three processes (sh + both sleeps) are killed

Supported signals: :sigterm, :sigkill, :sigint, :sighup, :sigusr1, :sigusr2, :sigstop, :sigcont, :sigquit, :sigpipe.

Checking process state

{:ok, pid} = Proc.start("sleep", ["10"])

Proc.alive?(pid)   # => true
Proc.os_pid(pid)   # => 12345 (the actual OS PID)

Proc.kill(pid, :sigkill)
Proc.await_exit(pid)

Proc.alive?(pid)   # => false

Stats

Every process tracks I/O statistics automatically:

{:ok, pid} = Proc.start("cat", [])
Proc.write(pid, "hello")
Proc.close_stdin(pid)
Proc.read(pid)
Proc.await_exit(pid)

stats = Proc.stats(pid)
stats.bytes_in     # => 5       (bytes written to stdin)
stats.bytes_out    # => 5       (bytes read from stdout)
stats.bytes_err    # => 0       (bytes read from stderr, :consume mode)
stats.read_count   # => 1       (number of read calls)
stats.write_count  # => 1       (number of write calls)
stats.duration_ms  # => 3       (wall-clock time)
stats.exit_status  # => 0       (exit code)

Tying an OS process to an owner

If the calling process crashes, the OS process it launched should go with it. Pass :owner to have the Process GenServer monitor a pid; on :DOWN it SIGKILLs the child and stops cleanly. NetRunner.stream/2 does this automatically with self().

# Spawn a long-lived command tied to the caller
parent = self()

spawn(fn ->
  {:ok, pid} = Proc.start("sleep", ["30"], owner: self())
  send(parent, {:os_pid, Proc.os_pid(pid)})
  exit(:boom)   # caller dies → Process SIGKILLs sleep, stops itself
end)

Per-call kill timeout

Tune the SIGTERM→SIGKILL escalation window per-process. Useful when a command has its own graceful shutdown hook you want to honour, or when you need a fast hard-kill.

# Give my_server 10s to drain on SIGTERM before SIGKILL
{:ok, pid} = Proc.start("my_server", [], kill_timeout: 10_000)

# Or make it effectively immediate for tests
{:ok, pid} = Proc.start("sleep", ["100"], kill_timeout: 100)

PTY Mode

Run commands with a pseudo-terminal for programs that require a TTY. PTY mode is designed for interactive and long-running programs — shells, REPLs, curses apps.

# Programs see a real terminal
{:ok, pid} = Proc.start("python3", ["-c", "import sys; print(sys.stdout.isatty())"], pty: true)
{:ok, data} = Proc.read(pid)
# data =~ "True"

# Interactive REPL
{:ok, pid} = Proc.start("python3", ["-i"], pty: true)
Proc.write(pid, "print(1 + 2)\n")
{:ok, data} = Proc.read(pid)

# Resize the terminal window
Proc.set_window_size(pid, 40, 120)

# Clean up when done (PTY doesn't support independent stdin close)
Proc.kill(pid, :sigkill)
Proc.await_exit(pid)

PTY caveats

PTY mode differs from pipe mode in important ways:

No independent stdin close — the PTY is a single bidirectional FD. Use kill/2 to terminate.
Echo — the terminal echoes input back by default, so reads include what you wrote.
Fast-exiting commands — if a command exits before you call read/1 (e.g., in iex), the PTY buffer may be lost. PTY mode is meant for long-running programs. For simple commands, use pipe mode (the default).
Line buffering — the terminal line discipline buffers input until \n by default.

# DON'T: Use PTY for simple commands in iex (data may be lost)
{:ok, pid} = Proc.start("echo", ["hi"], pty: true)
# ... time passes while you type ...
Proc.read(pid)  # => :eof (too late, PTY torn down)

# DO: Use pipe mode (default) for simple commands
{:ok, pid} = Proc.start("echo", ["hi"])
{:ok, "hi\n"} = Proc.read(pid)

# DO: Use PTY for interactive programs that need a terminal
{:ok, pid} = Proc.start("bash", [], pty: true)
Proc.write(pid, "echo hello\n")
{:ok, data} = Proc.read(pid)  # works — bash stays alive

Daemon Mode

Run long-lived processes under a supervision tree. Automatically drains stdout/stderr to prevent pipe blocking:

# In your supervisor
children = [
  {NetRunner.Daemon,
   cmd: "redis-server",
   args: ["--port", "6380"],
   on_output: :log,
   name: MyApp.Redis}
]

Supervisor.start_link(children, strategy: :one_for_one)

# Interact with the daemon
NetRunner.Daemon.os_pid(MyApp.Redis)   # => 12345
NetRunner.Daemon.alive?(MyApp.Redis)   # => true
NetRunner.Daemon.write(MyApp.Redis, "PING\r\n")

Output handling options:

:discard (default) — silently consume output to prevent pipe blocking
:log — log each chunk via Logger.info
fun/1 — custom callback for each chunk

# Custom output handler
{:ok, daemon} = NetRunner.Daemon.start_link(
  cmd: "tail",
  args: ["-f", "/var/log/system.log"],
  on_output: fn chunk -> MyApp.LogIngester.ingest(chunk) end
)

Graceful shutdown: on terminate/2, sends SIGTERM, waits 5 seconds, then SIGKILL.

cgroup Support (Linux)

Isolate child processes in a cgroup v2 hierarchy for resource control:

{:ok, pid} = Proc.start("my_worker", [],
  cgroup_path: "net_runner/job_123"
)

The shepherd creates the cgroup directory, moves the child into it, and cleans up on exit (kills all processes via cgroup.kill, then removes the directory). No-op on macOS.

Command DSL

Bundle an executable, default args, and default options into a reusable %NetRunner.Command{}. Both run/2 and stream/2 accept it, and call-site options override the defaults.

alias NetRunner.Command

# Inline construction
cmd = Command.new("curl", ["-sS"], timeout: 30_000)
{body, 0} = NetRunner.run(cmd, args: ["https://example.com"])

# Extend at call time (args append; opts merge with runtime winning)
listing = Command.new("ls", ["-la"])
{out, 0} = NetRunner.run(listing, args: ["/tmp"])

# `defcommand` in your own module captures a reusable template:
defmodule MyCmds do
  use NetRunner.Command

  defcommand :curl, "curl", ["-sS", "--max-time", "30"]
  defcommand :echo, "echo"
end

{out, 0} = NetRunner.run(MyCmds.echo(["hi"]))
{:ok, stream} = NetRunner.stream(MyCmds.curl(["https://example.com"]))

Error Handling Cheatsheet

case NetRunner.run(["my_tool", arg], timeout: 5_000) do
  {output, 0} ->
    {:ok, output}

  {_partial, status} when status != 0 ->
    {:error, {:nonzero_exit, status}}

  {:error, :timeout} ->
    {:error, :took_too_long}

  {:error, {:max_output_exceeded, partial}} ->
    {:error, {:too_much_output, byte_size(partial)}}

  {:error, {:invalid_cmd, msg}} ->
    {:error, {:bad_cmd, msg}}

  {:error, {:invalid_args, msg}} ->
    {:error, {:bad_args, msg}}
end

Parallel Execution

Every NetRunner process is fully independent — no shared state, no singleton bottleneck:

# Process files in parallel
files
|> Task.async_stream(fn file ->
  {out, 0} = NetRunner.run(["ffprobe", "-hide_banner", file])
  {file, out}
end, max_concurrency: System.schedulers_online())
|> Enum.to_list()

# Fan-out pattern
urls
|> Task.async_stream(fn url ->
  {body, 0} = NetRunner.run(["curl", "-s", url], timeout: 30_000)
  body
end, max_concurrency: 20)
|> Enum.to_list()

Why NetRunner?

System.cmd uses Erlang ports which tie stdin/stdout lifecycle together, have no backpressure (mailbox flooding), and leave zombies when programs ignore stdin EOF. This was filed as ERL-128 and marked Won't Fix.

Feature	System.cmd	MuonTrap	Exile	NetRunner
No zombies (BEAM SIGKILL)	-	Yes	-	Yes
NIF async I/O + backpressure	-	-	Yes	Yes
Close stdin independently	-	-	Yes	Yes
Process group kills	-	Yes	-	Yes
PTY / terminal emulation	-	-	-	Yes
cgroup isolation (Linux)	-	Yes	-	Yes
Per-process stats	-	-	-	Yes
Daemon mode (supervision)	-	Yes	-	Yes

Options Reference

`NetRunner.run/2`

Option	Type	Default	Description
`:input`	binary \| `Enumerable` of iodata	`nil`	Data to write to stdin. Written concurrently with reading stdout, so an input larger than the OS pipe buffers does not deadlock. Stdin is closed after the last chunk.
`:input_buffer`	integer	`0`	Bytes of stdin coalescing for a lazy `:input` enumerable. `0` writes through element-by-element (what interactive stdin wants); a positive value batches elements into writes of up to that many bytes (elements are never split), e.g. `input_buffer: 65_536` for `File.stream!` line input. Eager lists always coalesce; a plain binary is a single write.
`:timeout`	integer	`nil`	Wall-clock timeout in ms (`run/2` only)
`:max_output_size`	integer	`nil`	Max bytes to collect (`run/2` only)
`:output`	atom	`:binary`	Result shape for collected stdout (`run/2` only): `:binary` concatenates chunks into one binary; `:iodata` returns the collected chunks as iodata and skips the final flatten (an extra full-size allocation + copy for large outputs). The `max_output_exceeded` partial is always a binary.
`:stderr`	atom	`:consume`	`:consume` (drained internally), `:capture` (drained, and the retained tail is returned as a third tuple element: `{output, exit_status, stderr}`) or `:disabled`
`:stderr_tail_bytes`	integer	`8192`	Cap on the retained stderr tail; `0..1_048_576`. `0` retains nothing.
`:env`	map	`nil`	Environment variables for the child: `%{"NAME" => "value"}` sets, `%{"NAME" => nil}` unsets. The child's executable is resolved against the modified environment, so an `:env`-supplied `PATH` changes which binary runs — pass absolute command paths when `:env` comes from untrusted input.
`:pty`	boolean	`false`	Use pseudo-terminal
`:kill_timeout`	integer	`5000`	SIGTERM→SIGKILL escalation timeout in ms
`:cgroup_path`	string	`nil`	cgroup v2 path (Linux only); must sit under a `net_runner/` prefix and be under 256 bytes

Unknown options raise ArgumentError (Keyword.validate!/2) instead of being silently ignored. stream!/2 accepts the same options minus :timeout, :max_output_size, and :output, and raises NetRunner.Error (with the original reason in :reason) on spawn failure or a mid-stream read error.

`NetRunner.Process.start/3`

Accepts all options above except :input, :input_buffer, :timeout, :max_output_size, :output, and :stderr :capture (use :consume + NetRunner.Process.stderr_tail/1). NetRunner.Process.shutdown/3 owns the SIGTERM→await→SIGKILL escalation for callers that manage processes directly.

Architecture

BEAM Process (NetRunner.Process GenServer)
    |
    |-- Port.open("priv/shepherd", [:nouse_stdio, :exit_status])
    |     |
    |     v
    |   Shepherd Binary (stays alive for child's lifetime)
    |     |-- fork() → child process (execvp)
    |     |-- Passes pipe FDs to BEAM via UDS + SCM_RIGHTS
    |     |-- poll() loop: watches UDS + signal pipe (SIGCHLD)
    |     |-- BEAM dies (POLLHUP) → SIGTERM → SIGKILL child
    |     |-- Child dies (SIGCHLD) → notify BEAM, exit
    |
    |-- NIF (enif_select on raw O_NONBLOCK FDs, normal schedulers)
    |     |-- Demand-driven backpressure via OS pipe buffers
    |
    v
NetRunner.Watcher (belt-and-suspenders with shepherd)
    |-- Monitors GenServer, kills OS process if GenServer crashes

Three layers of zombie prevention:

Shepherd — detects BEAM death via POLLHUP on UDS, kills child process group
Watcher — detects GenServer crash via Process.monitor, kills child via NIF
NIF resource destructor — closes FDs on GC, child sees broken pipe

Performance

Medians on an idle Apple M1 Max (10 cores), OTP 29, default VM flags:


spawn (two `fork`+`exec`, UDS handshake, FD passing)	~4.4–7.2 ms
stdout throughput, 64 KiB reads	~2000 MB/s (`run/2`), ~2800 MB/s (`stream!/2`)
stdin+stdout round trip, 16 MiB through `cat`	~1160 MB/s (`run/2`), ~1130 MB/s (`stream!/2`)
stderr drain, `:consume` mode with an 8 KiB tail	~930–970 MB/s
`NetRunner.run(["/bin/echo", "hi"])` end to end	~6 ms

Spawn is a one-time cost and it is two execs by design — the shepherd's is the zero-zombie guarantee. One fork+exec on this host measures 2532 µs via System.cmd/2 and 2655 µs via a raw Port, so ~5.3 ms is the floor for this architecture. See Architecture.

I/O after that is limited by the pipe, not by NetRunner. Reads are sized at exactly one pipe buffer (65 536 bytes); see ADR-9 in Decisions before changing that constant.

Every NIF call is a bounded, non-blocking syscall on a normal scheduler. Readiness comes from enif_select, integrated with BEAM's epoll/kqueue, so an idle process costs nothing. NetRunner used to run these calls on dirty IO schedulers; each call then paid a scheduler handoff, which on a machine whose dirty schedulers are contended cost milliseconds rather than nanoseconds and capped throughput at single-digit MiB/s. See ADR-6 in Decisions.

Absolute numbers are host-specific — re-measure on your own hardware before relying on them.

Documentation

API Reference (HexDocs)
Architecture — component diagrams, spawn sequence, zombie prevention
Protocol — shepherd ↔ BEAM byte protocol spec
Backpressure — how demand-driven I/O works
Decisions — architecture decision records
Comparison — feature matrix vs alternatives

Requirements

Elixir ~> 1.17
Erlang/OTP 27+
C compiler (gcc or clang)
make
Linux or macOS

License

Apache-2.0 — see LICENSE.

Name		Name	Last commit message	Last commit date
.github		.github
bench		bench
c_src		c_src
docs		docs
lib		lib
priv		priv
test		test
.credo.exs		.credo.exs
.formatter.exs		.formatter.exs
.gitignore		.gitignore
.tool-versions		.tool-versions
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
mix.exs		mix.exs
mix.lock		mix.lock

Navigation Menu

Repository files navigation

NetRunner

Installation

Quick Start

Streaming

Timeouts and Limits

Input Validation and Error Returns

Working with Binary Output

Process API

Incremental reads and writes

Signals and process groups

Checking process state

Stats

Tying an OS process to an owner

Per-call kill timeout

PTY Mode

PTY caveats

Daemon Mode

cgroup Support (Linux)

Command DSL

Error Handling Cheatsheet

Parallel Execution

Why NetRunner?

Options Reference

`NetRunner.run/2`

`NetRunner.Process.start/3`

Architecture

Performance

Documentation

Requirements

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

NetRunner

Installation

Quick Start

Streaming

Timeouts and Limits

Input Validation and Error Returns

Working with Binary Output

Process API

Incremental reads and writes

Signals and process groups

Checking process state

Stats

Tying an OS process to an owner

Per-call kill timeout

PTY Mode

PTY caveats

Daemon Mode

cgroup Support (Linux)

Command DSL

Error Handling Cheatsheet

Parallel Execution

Why NetRunner?

Options Reference

NetRunner.run/2

NetRunner.Process.start/3

Architecture

Performance

Documentation

Requirements

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

`NetRunner.run/2`

`NetRunner.Process.start/3`