test(bench): measure single-user latency and max throughput

[kyscott18]

Summary

The benchmarks now measure the two numbers that matter:

1. How quickly Typewriter responds to a single user.
2. The maximum number of mutations per second.

• One benchmark, harness.benchmark.ts: it uses Harness.sol as the Typewriter app with the production defaults (FIFO sequencing, 300 ms submit, 200 ms block polling). Submit and block polling run as they do in production. It sends 1,000 mutations one at a time to measure latency. Then it sends 10,000 mutations at the same time to measure maximum throughput.
• Removed: counter.benchmark.ts, the shared benchmark utilities, and PLAN_execute_many.md. The plan describes files that do not exist now.
• New bun run benchmark script in packages/typewriter: it builds the release typewriter-evm addon before the benchmark runs. The test script builds a debug addon, and a benchmark on the debug addon is approximately 12 times slower (187 mutations/s).
• typewriter-evm execute benchmark: no changes from main.

Results (m8a.large, release addon)

• Single user, 1,000 sequential mutations: average 5.21 ms, p50 4.24 ms, p99 71.73 ms
• Maximum throughput, 10,000 concurrent mutations: 2,276 mutations/s

Known issues found by this benchmark

• Submit increases latency: submit holds withSpeculativeStateLock, and acceptBatch uses the same lock. In a test run with submit turned off, single-user p99 was 12.85 ms. With submit on, it was 756 ms (debug addon for both runs).
• Submit failure after the 10,000-mutation burst: the runtime stops with a fatal error, halt: OutOfGas(MemoryLimit) from execute. The probable cause is that submit puts all mutations accepted since the last submit into one transaction, with no size limit. The benchmark test passes, but bun test reports one unhandled error. A fix for this is not in this PR.

Test plan

• bun run typecheck
• biome check on the changed files
• DATABASE_URL=... bun run benchmark in packages/typewriter

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable new issue remains.

Summary

The PR replaces the counter benchmark with a harness benchmark that measures single-user latency and concurrent throughput.

• The Harness benchmark now separates single-call latency from concurrent throughput.
• The repository no longer includes the executeMany planning document.

_{Reviews (2) · Last reviewed commit: "test(bench): measure single-user latency..." · Reviewed by Greptile}