Benchmarking is a game — how to read a throughput claim
Yakovenko is explicit that his own headline number is a benchmark, not a promise. Benchmarks flood the system with non-overlapping, easily parallelized transactions under ideal conditions, in the same way a chip foundry's process node names its smallest feature rather than a typical one. Real workloads like a central limit order book carry heavier reads and writes and behave differently.
- Benchmarks use ideal, non-overlapping, easily parallelized transactions
- Compared to a foundry quoting its smallest achievable process node
- Serum's order books use megabyte reads and writes per order
- Hitting a limit is treated as a three-to-four-week optimization, not a wall
- Honest reading: the benchmark bounds the theory, not the experience
“there's a lot of caveats there because benchmarking is like I worked at a hardware company, benchmarking is a game”
“So when we benchmark, you flood the system with as many non-overlapping, easy parallelized transactions as you can.”