article 2026-11-09 16 min 3925 views

Measuring p99 correctly in FastAPI under keep-alive

A practical walkthrough of benchmarks with numbers and failure modes.

benchmarksfastapiasync

Setup

You need a reproducible load profile before arguing about frameworks. Same payload, same concurrency, same machine.

Method

  1. Warm the process for 30 seconds.
  2. Run fixed concurrency (not “as fast as possible”).
  3. Record p50/p95/p99 and error rate separately.
# example shape — replace with your harness
hey -n 20000 -c 64 https://fast-api.online/api/v1/status

Results shape

Metric Low concurrency High concurrency
RPS stable collapses when pools exhaust
p99 dominated by GC/IO dominated by queueing

Takeaways for Measuring p99 correctly in FastAPI under keep-alive

  • Fix the bottleneck you measured, not the blog post you liked.
  • Pool size and worker count interact; change one variable at a time.
  • Publish the command line with the chart or the chart is decoration.

Related