Adapt to benchmark hardware

[nigoroll]

Description

Changes to adapt to the hardware as per rules in https://www.http-arena.com/docs/add-framework/implementation-rules/frameworks/standard/

• Setting worker count to match CPU cores
• Configuring connection pool sizes

---

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command	Description
/benchmark -f <framework>	Run every test the framework subscribes to
/benchmark -f <framework> -t <test>	Run one test only
/benchmark -f <framework> --save	Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save	Run one test and save results
/benchmark -f <framework> --compare <other>	Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,...	Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save	No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test>	Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.

---

Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

[github-actions]

👋 Heads up! This PR modifies the following frameworks:

• Vinyl Cache → @gquintard

[nigoroll]

/benchmark -f vinyl-cache

[github-actions]

👋 Benchmark request received. A collaborator will review and approve the run.

[github-actions]

Benchmark Results

Framework: vinyl-cache | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	793,226	6621.0%	306MiB	+0.5%	+64.5%
baseline	4096	732,818	6654.7%	926MiB	-8.1%	+199.7%
pipelined	512	2,284,027	6654.7%	252MiB	-1.8%	+131.2%
pipelined	4096	2,147,086	6527.1%	439MiB	-10.6%	+145.3%
limited-conn	512	377,425	6576.3%	435MiB	-9.8%	+115.3%
limited-conn	4096	368,315	6585.0%	1.1GiB	-10.5%	+387.6%

Full log

  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.35ms   1.03ms   3.03ms   5.05ms   6.78ms

  1878293 requests in 5.00s, 1878298 responses
  Throughput: 375.55K req/s
  Bandwidth:  59.70MB/s
  Status codes: 2xx=1878298, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1878286 / 1878298 responses (100.0%)
  Reconnects: 187738
  Per-template: 625963,626298,626025
  Per-template-ok: 625963,626298,626025
[info] CPU 6514.1% | Mem 429MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.34ms   1.03ms   3.02ms   5.02ms   6.67ms

  1887120 requests in 5.00s, 1887127 responses
  Throughput: 377.25K req/s
  Bandwidth:  59.98MB/s
  Status codes: 2xx=1887127, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1887118 / 1887127 responses (100.0%)
  Reconnects: 188704
  Per-template: 629104,629105,628909
  Per-template-ok: 629104,629105,628909
[info] CPU 6576.3% | Mem 435MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.36ms   1.04ms   3.05ms   5.07ms   6.75ms

  1876538 requests in 5.00s, 1876547 responses
  Throughput: 375.21K req/s
  Bandwidth:  59.66MB/s
  Status codes: 2xx=1876547, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1876542 / 1876547 responses (100.0%)
  Reconnects: 187666
  Per-template: 625697,625497,625348
  Per-template-ok: 625697,625497,625348
[info] CPU 6557.7% | Mem 441MiB

=== Best: 377425 req/s (CPU: 6576.3%, Mem: 435MiB) ===
[info] input BW: 29.16MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/512/vinyl-cache.json
httparena-bench-vinyl-cache
httparena-bench-vinyl-cache

==============================================
=== vinyl-cache / limited-conn / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   9.33ms   5.61ms   21.40ms   36.80ms   180.40ms

  1841858 requests in 5.00s, 1839240 responses
  Throughput: 367.73K req/s
  Bandwidth:  58.61MB/s
  Status codes: 2xx=1839240, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1839231 / 1839240 responses (100.0%)
  Reconnects: 182206
  Per-template: 613214,615365,610652
  Per-template-ok: 613214,615365,610652
[info] CPU 6401.4% | Mem 1.1GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   9.77ms   6.88ms   22.00ms   37.50ms   60.10ms

  1843922 requests in 5.00s, 1841577 responses
  Throughput: 368.05K req/s
  Bandwidth:  58.73MB/s
  Status codes: 2xx=1841577, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1841565 / 1841577 responses (100.0%)
  Reconnects: 182599
  Per-template: 613362,616511,611692
  Per-template-ok: 613362,616511,611692
[info] CPU 6585.0% | Mem 1.1GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   9.93ms   6.99ms   22.40ms   38.10ms   63.50ms

  1832133 requests in 5.00s, 1828137 responses
  Throughput: 365.30K req/s
  Bandwidth:  58.30MB/s
  Status codes: 2xx=1828137, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1828137 / 1828137 responses (100.0%)
  Reconnects: 180948
  Per-template: 609305,611373,607459
  Per-template-ok: 609305,611373,607459
[info] CPU 6366.2% | Mem 1.2GiB

=== Best: 368315 req/s (CPU: 6585.0%, Mem: 1.1GiB) ===
[info] input BW: 28.45MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/vinyl-cache.json
httparena-bench-vinyl-cache
httparena-bench-vinyl-cache
[info] skip: vinyl-cache does not subscribe to json
[info] skip: vinyl-cache does not subscribe to json-comp
[info] skip: vinyl-cache does not subscribe to json-tls
[info] skip: vinyl-cache does not subscribe to upload
[info] skip: vinyl-cache does not subscribe to api-4
[info] skip: vinyl-cache does not subscribe to api-16
[info] skip: vinyl-cache does not subscribe to static
[info] skip: vinyl-cache does not subscribe to static-tls
[info] skip: vinyl-cache does not subscribe to async-db
[info] skip: vinyl-cache does not subscribe to crud
[info] skip: vinyl-cache does not subscribe to fortunes
[info] skip: vinyl-cache does not subscribe to baseline-h2
[info] skip: vinyl-cache does not subscribe to static-h2
[info] skip: vinyl-cache does not subscribe to baseline-h2c
[info] skip: vinyl-cache does not subscribe to json-h2c
[info] skip: vinyl-cache does not subscribe to baseline-h3
[info] skip: vinyl-cache does not subscribe to static-h3
[info] skip: vinyl-cache does not subscribe to gateway-64
[info] skip: vinyl-cache does not subscribe to gateway-h3
[info] skip: vinyl-cache does not subscribe to production-stack
[info] skip: vinyl-cache does not subscribe to unary-grpc
[info] skip: vinyl-cache does not subscribe to unary-grpc-tls
[info] skip: vinyl-cache does not subscribe to stream-grpc
[info] skip: vinyl-cache does not subscribe to stream-grpc-tls
[info] skip: vinyl-cache does not subscribe to echo-ws
[info] skip: vinyl-cache does not subscribe to echo-ws-pipeline
[info] skip: vinyl-cache does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[purged stale] zix-grpc.json (zix-grpc)
[purged stale] zix-ws.json (zix-ws)
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/vinyl-cache.json - 6 new, 6 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536