Benchmark

Makefile

@@ -1,4 +1,4 @@
-.PHONY: all build test test-cover race vet fmt fmt-check lint tidy run check clean ci help devcert
+.PHONY: all build test test-cover race vet fmt fmt-check lint tidy run check clean ci help devcert bench
 
 BINARY := bin/gobalancer
 PKG := ./...
@@ -42,6 +42,14 @@ check: build
 run: build
 	./$(BINARY) run -c $(CONFIG)
 
+bench:
+	bash test/bench/env.sh
+	bash test/bench/e1.sh
+	bash test/bench/e2.sh
+	bash test/bench/e3.sh
+	bash test/bench/e4.sh
+	bash test/bench/e5.sh
+
 ci: fmt-check vet test
 
 clean:

docs/benchmark.md

@@ -0,0 +1,140 @@
+# Benchmarks
+
+Reproducible measurements of GoBalancer — what it costs and how it behaves under load. Everything here can be regenerated with `make bench`.
+
+## How the benchmarks work
+
+- **Bare processes over loopback.** The balancer, the backends, and the load generator all run as plain processes on one machine, talking over localhost — no Docker in the path. This measures the balancer's own overhead, not network or container costs.
+- **Open-loop load.** The load generator (`test/loadgen`) sends requests at a fixed rate no matter how fast replies come back.
+- **Percentiles.** Each run reports p50 (median), p99, and p999 latency plus the success rate. An average hides the slow tail, percentiles show it.
+- **Warm-up discarded.** The first few seconds of each run (connection setup, pool fill) are thrown away, so we measure steady state.
+
+Run everything with:
+
+```bash
+make bench
+```
+
+Results land in `test/bench/results/`.
+
+## Environment
+
+All numbers below were measured on:
+
+```
+cpu:   13th Gen Intel(R) Core(TM) i5-13420H
+cores: 12
+mem:   15Gi
+go:    go1.26.5
+host:  Linux 7.0.11-76070011-generic x86_64
+```
+
+---
+
+## E1 — How much does GoBalancer cost?
+
+**Question.** When you put GoBalancer in front of a backend, how much latency does it add per request?
+
+**Method.** One backend with a fixed 5 ms delay. Send 1000 requests/second for 30 seconds, three ways: straight to the backend (the baseline), through GoBalancer in L7 (HTTP) mode, and through it in L4 (TCP) mode. Then compare.
+
+**Result.**
+
+| target | requests | success | throughput | p50 | p99 | p999 | max |
+|--------|---------:|--------:|-----------:|----:|----:|-----:|----:|
+| direct     | 28911 | 100% | 964/s | 5.591 ms | 6.464 ms | 7.363 ms | 10.163 ms |
+| through L7 | 29011 | 100% | 967/s | 6.013 ms | 7.008 ms | 8.672 ms | 11.469 ms |
+| through L4 | 29293 | 100% | 976/s | 5.777 ms | 6.598 ms | 7.750 ms | 10.106 ms |
+
+Overhead added by GoBalancer (latency minus the direct baseline):
+
+| mode | added at p50 | added at p99 |
+|------|-------------:|-------------:|
+| L4 | +0.19 ms | +0.13 ms |
+| L7 | +0.42 ms | +0.54 ms |
+
+**What it proves.** On this test workload, GoBalancer adds less than 0.5 ms of p50 latency per request. L4 adds 0.19 ms at p50 and 0.13 ms at p99, while L7 adds 0.42 ms at p50 and 0.54 ms at p99. All requests succeeded, and the p999 latency remained within 3 ms of the median in both proxy modes.
+
+---
+
+## E2 — Does goroutine-per-connection hold up?
+
+**Question.** GoBalancer gives every connection its own goroutines. Does that model still work with thousands of connections at once, or does it fall apart?
+
+**Method.** Run GoBalancer in L4 mode in front of a backend that hangs on every request, so connections stay open. Open a growing number of connections at once — 100 up to 10,000 — and at each level read two numbers straight off the balancer's `/metrics`: its live goroutine count (`go_goroutines`) and its memory use (`process_resident_memory_bytes`). L4 is used here because its byte-relay has no HTTP-level timeout, so the slow backend keeps every connection parked.
+
+**Result.**
+
+| connections | goroutines | memory |
+|------------:|-----------:|-------:|
+| 0 (idle) | 12    | 12.8 MB  |
+| 100      | 312   | 16.9 MB  |
+| 500      | 1512  | 25.3 MB  |
+| 1000     | 3012  | 32.9 MB  |
+| 2000     | 6012  | 51.1 MB  |
+| 5000     | 15012 | 107.5 MB |
+| 10000    | 30012 | 189.5 MB |
+
+Every connection was established; none failed.
+
+**What it proves.** Goroutine usage scales linearly with the number of active L4 connections at exactly `12 + 3 × connections`in this test. Memory usage increases by approximately 18 KB per connection, reaching 189.5 MB at 10,000 connections. All 10,000 connections were established successfully, with no connection failures.
+
+---
+
+## E3 — When does least-connections beat round-robin?
+
+**Question.** When one backend is slow, does the choice of balancing algorithm actually matter?
+
+**Method.** Three backends: two fast (5 ms) and one slow (150 ms). Send 1000 requests/second for 30 seconds through GoBalancer (L7), once with `round_robin` and once with `least_connections`, then compare the latencies. The slow backend is 150 ms (not slower) so it stays under the proxy's 300 ms response-header timeout and is not treated as failed.
+
+**Result.**
+
+| algorithm | p50 | p90 | p99 | mean |
+|-----------|----:|----:|----:|-----:|
+| round_robin       | 6.6 ms | 151 ms | 153 ms | 54.7 ms |
+| least_connections | 6.2 ms | 6.9 ms | 152 ms | 10.1 ms |
+
+Both served every request successfully at ~960/s.
+
+**What it proves.** With two 5 ms backends and one 150 ms backend, least-connections reduces p90 latency from 151 ms to 6.9 ms and mean latency from 54.7 ms to 10.1 ms compared with round-robin. The slow backend received approximately 1.6% of requests under least-connections, compared with approximately 33% under round-robin. p99 remained approximately 152 ms for both algorithms.
+
+---
+
+## E4 — Is the 1/N remap claim real?
+
+**Question.** When one backend is removed, how many keys does consistent hashing move, compared with naive modulo-N hashing?
+
+**Method.** Map 100,000 keys across 10 backends using GoBalancer's consistent-hash ring, then drop each backend in turn and count how many keys change backend. Repeat with naive `hash(key) % N` for comparison. Averaging over all 10 possible single-backend drops removes the luck of which backend is dropped. Pure computation — no network or processes.
+
+**Result.**
+
+| method | keys moved (avg) | per-drop range |
+|--------|-----------------:|---------------:|
+| consistent_hash | 10.0% | 4.3% – 16.0% |
+| modulo_n        | 90.1% | — |
+
+Ideal for 10 backends is 1/N = 10.0%.
+
+**What it proves.** Removing 1 of 10 backends moves an average of 10.0% of keys under consistent hashing, matching the 1/N ideal, versus 90.1% under modulo-N — roughly 9× fewer. The average of exactly 10.0% means only the dropped backend's own keys move; no other keys are disturbed. The per-drop range of 4.3% to 16.0% reflects uneven key distribution at the production vnode count: individual backends own between 4.3% and 16.0% of the key space.
+
+---
+
+## E5 — What does the health loop buy?
+
+**Question.** When a backend is sick — accepting connections but not responding in time — does the health loop keep traffic away from it?
+
+**Method.** Three backends: two healthy (5 ms) and one sick (60 s delay, past the proxy's 300 ms header timeout, so every request to it times out). Send 1000 requests/second for 30 seconds through GoBalancer (L7, round_robin), once with the passive path enabled (`fall = 3`) and once with it disabled (`fall = 1000000`, i.e. active-only). Round-robin is used so the health loop is the only mechanism that can route around the sick backend.
+
+**Result.**
+
+| health | p50 | p90 | p99 | p999 | mean |
+|--------|----:|----:|----:|-----:|-----:|
+| passive+active | 6.1 ms | 6.7 ms | 10.5 ms | 308 ms | 8.3 ms |
+| active-only    | 6.5 ms | 307 ms | 309 ms | 312 ms | 106 ms |
+
+Both served every request successfully at ~965/s.
+
+**What it proves.** With one sick backend and round-robin balancing, enabling the passive path reduces mean latency from 106 ms to 8.3 ms and p90 from 307 ms to 6.7 ms. With the passive path disabled, round-robin sends approximately one-third of requests to the sick backend; each times out after 300 ms and is retried, so a third of all requests are slow for the entire run. With the passive path enabled, the sick backend is evicted after 3 failures and traffic goes only to the healthy backends. In GoBalancer the passive path is the only mechanism that evicts a backend — the active prober only readmits — so active-only means no eviction at all. The passive+active p999 of 308 ms reflects brief flapping: the active prober's TCP-only probe succeeds against the sick backend (it accepts connections), readmitting it momentarily before the passive path evicts it again.
+
+## Limitations
+
+All traffic runs on a single machine over loopback. This measures GoBalancer's own overhead — CPU and memory — not real-world network behaviour. On a real network the round-trip time between machines dwarfs the sub-millisecond cost measured here. Read these as "how much does the balancer add," not "how fast is a request in production."

test/bench/connhold/main.go

@@ -0,0 +1,56 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"flag"
+	"fmt"
+	"io"
+	"net/http"
+	"os"
+	"sync"
+	"time"
+)
+
+// connhold opens N concurrent connections to the target and holds them open for `hold`, then exits.
+// Used to measure how the balancer's goroutine count and memory scale with the number of live connections.
+// Keep-alive is disabled, so each request uses its own fresh connection -- conns == sockets.
+func main() {
+	target := flag.String("target", "http://127.0.0.1:8080/", "url to hold connections against")
+	conns := flag.Int("conns", 100, "number of concurrent connections")
+	hold := flag.Duration("hold", 8*time.Second, "how long to hold them open")
+	flag.Parse()
+
+	client := &http.Client{
+		Transport: &http.Transport{DisableKeepAlives: true},
+	}
+
+	ctx, cancel := context.WithTimeout(context.Background(), *hold)
+	defer cancel()
+
+	var mu sync.Mutex
+	var failed int
+	var wg sync.WaitGroup
+
+	for range *conns {
+		wg.Go(func() {
+			req, _ := http.NewRequestWithContext(ctx, http.MethodGet, *target, nil)
+			resp, err := client.Do(req)
+			if err != nil {
+				// A dial/connection error -> we never held this connection.
+				// A ctx timeout -> we held it to the deadline -- that is success.
+				if !errors.Is(err, context.DeadlineExceeded) && !errors.Is(err, context.Canceled) {
+					mu.Lock()
+					failed++
+					mu.Unlock()
+				}
+				return
+			}
+			io.Copy(io.Discard, resp.Body)
+			resp.Body.Close()
+		})
+	}
+	wg.Wait()
+
+	fmt.Fprintf(os.Stderr, "conns=%d failed=%d\n", *conns, failed)
+}

test/bench/e1.sh

@@ -0,0 +1,108 @@
+#!/usr/bin/env bash
+# E1 - How much does GoBalancer cost?
+# Compares latency of hitting a backend directly vs through GoBalancer (L7 and L4).
+# Runs everything as bare host processes over loopback, so we measure the balancer,
+# not Docker's network.
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+OUT="$ROOT/test/bench/results"
+mkdir -p "$OUT"
+
+GOBAL="$ROOT/bin/gobalancer"
+BACKEND="$ROOT/bin/testbackend"
+LOADGEN="$ROOT/bin/loadgen"
+REPORT="$ROOT/bin/report"
+
+BACKEND_PORT=9101
+LISTEN="127.0.0.1:8080"
+RATE="${RATE:-1000}"
+DURATION="${DURATION:-30s}"
+WARMUP="${WARMUP:-3s}"
+
+echo "building binaries..."
+go build -o "$GOBAL"   ./cmd/gobalancer
+go build -o "$BACKEND" ./testbackend
+go build -o "$LOADGEN" ./test/loadgen
+go build -o "$REPORT"  ./test/bench/report
+
+pids=()
+cleanup() { for p in "${pids[@]:-}"; do kill "$p" 2>/dev/null || true; done; }
+trap cleanup EXIT
+
+wait_up() { 
+  for _ in $(seq 1 100); do
+    curl -s -o /dev/null "$1" && return 0
+    sleep 0.1
+  done
+  echo "timed out waiting for $1" >&2
+  exit 1
+}
+
+# One backend with a fixed 5ms delay: its own variance stays out of the delta we measure.
+NAME=b1 PORT="$BACKEND_PORT" DELAY=5ms "$BACKEND" &
+pids+=($!)
+wait_up "http://127.0.0.1:$BACKEND_PORT/health"
+
+BAL_PID=""
+run_balancer() { # mode
+  local cfg="$OUT/e1-$1.yaml"
+  if [ "$1" = "l7" ]; then
+    cat > "$cfg" <<EOF
+mode: l7
+listen: "$LISTEN"
+balancer: round_robin
+routes:
+  - match: { path_prefix: "/" }
+    pool: default
+pools:
+  default:
+    - addr: "127.0.0.1:$BACKEND_PORT"
+EOF
+  else
+    cat > "$cfg" <<EOF
+mode: l4
+listen: "$LISTEN"
+balancer: round_robin
+pools:
+  default:
+    - addr: "127.0.0.1:$BACKEND_PORT"
+EOF
+  fi
+  "$GOBAL" run -c "$cfg" -metrics-addr 127.0.0.1:9095 -log-level error >/dev/null 2>&1 &
+  BAL_PID=$!
+  pids+=("$BAL_PID")
+  wait_up "http://$LISTEN/"
+}
+
+stop_balancer() {
+  kill "$BAL_PID" 2>/dev/null || true
+  wait "$BAL_PID" 2>/dev/null || true
+  sleep 0.5
+}
+
+measure() { # target outfile
+  "$LOADGEN" -target "$1" -rate "$RATE" -duration "$WARMUP" -out /dev/null >/dev/null 2>&1 || true
+  "$LOADGEN" -target "$1" -rate "$RATE" -duration "$DURATION" -out "$2"
+}
+
+echo "E1: direct-to-backend..."
+measure "http://127.0.0.1:$BACKEND_PORT/" "$OUT/e1-direct.csv"
+
+echo "E1: through L7..."
+run_balancer l7
+measure "http://$LISTEN/" "$OUT/e1-l7.csv"
+stop_balancer
+
+echo "E1: through L4..."
+run_balancer l4
+measure "http://$LISTEN/" "$OUT/e1-l4.csv"
+stop_balancer
+
+echo ""
+{
+  echo "target count ok% thr(req/s) p50 p90 p99 p999 mean max"
+  "$REPORT" -label direct     -in "$OUT/e1-direct.csv"
+  "$REPORT" -label through-l7 -in "$OUT/e1-l7.csv"
+  "$REPORT" -label through-l4 -in "$OUT/e1-l4.csv"
+} | { column -t 2>/dev/null || cat; } | tee "$OUT/e1.txt"