Backend Stack Benchmark

One app, many backend stacks, one specification.

This repo compares backend stacks on equal terms. Every backend implements the same address-book API from the same written spec, over the same SQLite database, with the same SQL, cache and compression. A load generator then measures how many simulated users each one can serve, and a parity check proves they all return the same responses.

Holding everything else constant leaves the runtime and the web framework as the only variables: Rust vs Go vs Node vs Java, and so on. Along the way the repo also became a record of how to make each stack fast, and of what doesn't help.

It's also an experiment in AI-written code. Every backend was generated by Claude from the same written requirements. How good is that code, measured by the same spec, tests and load? Feedback is welcome on both questions.

Why SQLite

Each backend embeds SQLite and reads a local database file. There's no database server and no network hop between the app and its data.

That was a deliberate choice. With Postgres or MySQL, every request includes a network round trip, and connection pools and server tuning would dominate the results. With SQLite in the process, a query is a function call, so the numbers show what the runtime itself costs:

• its HTTP stack
• JSON encoding
• threading or process model
• garbage collector
• how its SQLite driver is built

It also keeps the setup trivial: clone, build, run. The data is realistic, though: 110,000 contacts with emails, phones, addresses and tags, full-text search, and a database of about 90 MB.

The app

An address book: browse a sorted list, jump through an A–Z index, search as you type, open and edit contacts, tag and favorite them. The API has 13 routes (see 03-api.md).

The frontend (frontend/, Vue 3 + Tailwind) is only there to demo the functionality and to make the workload realistic. The load generator replays the frontend's real request patterns. Every backend serves the built frontend, so you can open any of them in a browser and use the same app.

How it works

1. One spec. docs/requirements/ is a complete, language-neutral specification: architecture, schema and SQL, SQLite tuning, every route and error message, the cache and gzip rules, the data generator and how to verify a backend. Backends are written from the spec, not by copying another backend. backend/rust/ is the reference implementation the spec was derived from.

2. Shared schema. Every backend applies the migrations in backend/migrations/, so schemas can't drift.

3. Same work everywhere. Every backend uses:

   • the same SQLite pragmas and one-writer / many-readers access
   • the same 64 MiB response cache (with ETags and read-your-writes)
   • gzip at level 6
   • a 4-core budget

   Without that, the comparison would mean nothing.

4. Proof of sameness. Each backend's seeder must produce byte-identical data (a checksum), and bench/parity.py compares 71 responses against the Rust server, plus 42 contract checks. Every stack passes all of them.

5. Measurement. loadtest/ simulates users who browse, search as they type, open contacts and edit (5%), with ~3 s pauses. bench/run.py builds each stack, verifies it, ramps users until p99 latency passes 2 s, and writes reports to results/ (Markdown and HTML, with charts).

Results

Latest runs on an Apple M5 Max, with the load generator on the same machine. Capacity is the number of simulated users at which p99 latency reaches 100 ms. There are two sets of numbers, one per way of running the stacks. Never compare a figure from one set with a figure from the other.

In Docker (the default)

Each stack in its Debian 13 container, held to exactly 4 CPUs by the kernel (--cpus=4), with the load generator on the Docker network.

Stack	Capacity	Single contact	Uncached list page	Idle memory	Throttled at 5,000 users
C# (ASP.NET Core)	≥ 6,000	62,100 req/s	17,200 req/s	41 MB	1%
C (libmicrohttpd)	≥ 6,000	88,000 req/s	26,200 req/s	3 MB	11%
Go (net/http)	~5,300	58,400 req/s	19,300 req/s	15 MB	14%
Rust (Axum)	~5,150	73,600 req/s	20,900 req/s	13 MB	36%
Node (Fastify)	~4,850	38,000 req/s	14,700 req/s	91 MB	62%
Java (Spring Boot)	~4,700	34,100 req/s	13,100 req/s	176 MB	62%
Rails (direct SQL)	~2,850	15,400 req/s	8,400 req/s	116 MB	38%
Python (FastAPI)	~2,450	59,000 req/s	14,900 req/s	222 MB	78% at 4,000 users
Rails + ActiveRecord	~1,150	4,400 req/s	1,300 req/s	116 MB	57% from 2,000 users

Throttled is the share of 100 ms slices in which the kernel paused the container for going over 4 CPUs. Stacks with garbage collectors, JITs or helper threads (Java, Node, Rails) burst past the quota and lose tail latency even when their average CPU is under 4. That's the same limit a Kubernetes CPU limit applies, which is why Docker mode is the default.

Full reports: results/summary.md, with each stack's report a click away (for example Rust). These runs were flagged busy: macOS's own background indexing (corespotlightd, mediaanalysisd) kept starting while the machine sat idle overnight, so treat the exact numbers as noisy. The pattern held across two full runs. Python was rerun on its own on 2026-10-05, after its reads moved to the event loop, and Rust on 2026-10-06, after capping its blocking pool and switching to jemalloc (see below); both runs were quiet. In a same-day rerun next to it, Rails reached ~2,300 (flagged busy), so treat the two as roughly level.

Native

Each stack with its own toolchain on macOS (--mode native), limited to 4 cores by its runtime settings only (GOMAXPROCS=4, 4 workers, and so on).

Stack	Capacity	Single contact	Uncached list page	Idle memory
C# (ASP.NET Core)	≥ 6,000	79,100 req/s	21,500 req/s	63 MB
C (libmicrohttpd)	≥ 6,000	48,500 req/s	25,500 req/s	3 MB
Java (Spring Boot)	~5,600	71,800 req/s	17,500 req/s	176 MB
Go (net/http)	~5,500	55,400 req/s	20,100 req/s	4 MB
Rust (Axum)	~5,100	71,500 req/s	27,300 req/s	3 MB
Node (Fastify)	~5,050	38,800 req/s	16,600 req/s	102 MB
Python (FastAPI)	~3,050	13,500 req/s	2,800 req/s	211 MB
Rails (direct SQL)	~3,000	18,900 req/s	11,200 req/s	199 MB
Rails + ActiveRecord	~1,950	5,700 req/s	1,600 req/s	199 MB

Native mode's budget is softer: Python, on its old thread-pool design, used ~11 cores at 4,000 users, which is why it scored ~3,050 here against ~2,050 in Docker at the time. Its native row predates the event-loop change, and Rust's predates its blocking-pool cap and jemalloc; neither has been rerun. Full reports: results/native/summary.md. The C# and Java native runs were flagged because other programs were busy during their low-user levels.

Read both sets as a comparison between stacks, not as server sizing: a cloud vCPU is slower than an M5 core. Running on Linux also exposed a real bug in the C backend that macOS had hidden (see backend/c/README.md). Your own numbers are one command away: see DEVELOPER.md.

What we learned about performance

The first versions of most stacks had one avoidable bottleneck, and it was rarely the language. Finding it changed the ranking more than the choice of language did:

Stack	Before	After	The bottleneck
Go	~1,500	~5,500	CGO_CFLAGS replaced the default -O2, so SQLite ran unoptimized
C#	~3,250	≥ 6,000	SQLite's memory statistics: a global lock on every malloc, serializing the reader threads
Node	~2,900	~5,050	4 cluster processes meant 4 caches that each missed ~58% of the time. Now one process with reader threads, on a driver whose SQLite has no global locks
Rust	~3,800	~5,100	The cache policy (TinyLFU) rejected fresh entries, the bundled SQLite had a global page-cache lock, and slow gzip ran on the async threads
Python	~2,050	~3,050	The same memory-statistics lock, switched off at runtime before import sqlite3
Rust (Docker)	35,000 req/s	73,600 req/s	Single contact. Tokio's blocking pool grew to ~80 threads queueing for 4 read connections, 3 context switches per query, which is expensive in Docker's VM; glibc's malloc cost another ~25%. Capped the pool at DB_READERS and switched to jemalloc. Capacity stayed ~5,150–5,250 users
Python (Docker)	~2,050	~2,450	Handlers ran on 4 threads per worker, but sqlite3 drops and retakes the GIL on every row, so the threads and the event loop queued for it. Reads now run on the event loop: single contact 5,800 → 59,000 req/s
Rails	~2,250	~3,000	The sqlite3 gem holds Ruby's GVL during queries, so extra threads per worker only added switching
Java	~5,250	~5,600	Parallel GC instead of G1, leaving the 4 cores to the query threads
C	resets at 6,000	0 errors	SQLite ran on the HTTP I/O threads, so bursts left nobody accepting connections

The lessons that carry over to any project:

• Check how your SQLite is compiled. Prebuilt copies often take a process-wide lock on every allocation or page fetch, which quietly turns "4 reader threads" into one. PRAGMA compile_options tells you.
• Cache hit rate beats raw speed. A cache that rejects new entries, or several processes that each keep their own, can cost more than the language saves.
• Keep blocking and CPU-heavy work off the threads that serve connections. That applies to SQLite calls and gzip.
• More threads is rarely the fix. Find the shared resource first: a pool, a lock or a cache. In Python and Ruby the interpreter lock is one: Python's handler threads cost 10x on single requests.
• Measure carefully. Profilers distort latency, lock waits don't show up in CPU profiles, and macOS's own background indexing can take 3+ cores in the middle of a run.

The full method, tools and pitfalls are in docs/optimizing.md, and every change tried on every stack, dated and with its effect, is in docs/optimization-log.md. Each backend/<stack>/README.md has an Optimizing section with what was tried there, including what didn't help.

Writing a backend with AI

Every backend here was written by Claude (Claude Code) from the spec, and the repo is set up so the next one can be too. CLAUDE.md gives the assistant the project rules, the doc map and the benchmarking pitfalls. The spec gives it everything it needs, and the checks tell it (and you) when it's done.

A typical request:

Add a Kotlin + Ktor backend in backend/kotlin-ktor/ on the next free port. Build it from docs/requirements/ only, not from the other backends. Pass the seed checksum and parity check, add a bench.json, and run the quick benchmark profile.

The definition of done is the acceptance checklist in 06-verification.md:

• VER-1: identical seed data
• VER-2: "71 identical, 0 mismatched; 42/42 contract checks passed"
• VER-3: a clean load test
• VER-4/5/6: registered with the suite, plus a README for the stack
• VER-7: compared against the existing stacks, with any large gap investigated

Because the checks are mechanical, you can trust the result without reading every line.

Optimizing with AI

Optimizing works the same way. Point Claude at one stack and the method:

Optimize the Python backend. Follow docs/optimizing.md: reproduce outside the suite, compare single-endpoint throughput, probe the read pool and the cache, and change one thing at a time. Record what helped and what didn't in its README.

What made this work:

• Ask for evidence, not guesses. A temporary probe (wait time against query time, cache misses) found most bottlenecks in one run.
• A/B one change at a time against the full user ramp, and repeat close calls: run-to-run noise at high load can be ±40 ms.
• Keep the machine quiet while measuring. The suite flags runs where other programs were busy. Don't trust flagged numbers.
• Write the lessons down: in the stack's README, in docs/optimizing.md, and in the spec when they apply to every backend. That way the next session, human or AI, starts where the last one stopped.

Feedback welcome

I'm not an expert in all of these stacks, and getting better at tuning them is one of the things I want from this project. If you know one of them well, I'd like to hear from you:

• Performance: a setting, library or pattern that would make a stack faster while doing the same work. Every speed-up so far came from one specific fix, and there are surely more.
• Code quality: whether a backend reads like idiomatic code in its language and framework, or like something an experienced developer would rewrite.
• Correctness: anything a backend gets wrong that the parity check misses.

The second question is how good AI is at writing production-quality code from a common set of requirements. So far I'm impressed: every stack passes the same seed checksum, the 71 parity checks and the 42 contract checks, and the bottlenecks it found were real. I'd still like other people's judgment on the quality and functionality, especially in the stacks you know best.

Open an issue or a pull request; CONTRIBUTING.md has the ground rules (in short: same work as the spec, the checks pass, and before and after numbers for performance changes). The project is MIT licensed.

Running it

DEVELOPER.md is the full guide, from a fresh clone to tuning a stack with Claude. The short version needs only Docker and Python 3.10+:

docker compose --profile rust up --build                 # API + UI on http://127.0.0.1:7878
docker compose --profile rust run --rm rust seed 110000  # fill it with fake contacts

python3 bench/run.py --stacks go,rust --profile quick     # benchmark two stacks in Docker (~3 min each)
python3 bench/run.py                                     # every stack (~6 min each)
open results/summary.html                                # the comparison

Add --mode native to benchmark with each stack's own toolchain instead (results in results/native/). Each stack's README has its native build and run commands: Rust · Go · C# · C · Java · Node · Python · Rails. More on the suite in bench/README.md; the image rules are in 07-containers.md.

Repository layout

backend/
  migrations/        shared schema, applied by every backend
  rust/              reference implementation (Axum)      :7878
  python/            FastAPI                               :7879
  node/              Fastify                               :7880
  rails/             Rails, direct SQL and ActiveRecord    :7881 / :7882
  go/                net/http                              :7883
  csharp/            ASP.NET Core                          :7884
  c/                 libmicrohttpd                         :7885
  java-spring/       Spring Boot                           :7886
frontend/            Vue 3 + Tailwind demo UI
loadtest/            simulated-user and single-endpoint load generator (Rust)
bench/               benchmark suite, report renderer, parity check
results/             published reference results: Docker (default) here, native in results/native/
compose.yaml         run any stack in Docker by hand
logs/                benchmark progress log (git-ignored)
docs/
  requirements/      the spec every backend is built from
  optimizing.md      how to find and fix a stack's bottlenecks
  plans/             what's planned next, and how Docker mode was planned
CLAUDE.md            instructions for AI assistants working in this repo
DEVELOPER.md         setup and usage guide: Docker first, native second, then tuning with Claude
CONTRIBUTING.md      how to contribute; LICENSE: MIT