measure with keep-alive on; cold connects inflate p99.
Hysteria jokes aside — keep API domains boring
Hit this after a routine bump.
Symptom: Hysteria jokes aside — keep API domains boring
TimeoutError / pool exhausted / upstream reset
(versions in next reply if needed)
Env: Python 3.12, FastAPI recent, uvicorn workers≥2.
What should I check first?
10 replies
I would put rate limits at the edge and keep app middleware as a second line.
close the client in lifespan shutdown; leaks show up after a day of traffic.
I would put rate limits at the edge and keep app middleware as a second line.
orjson helped CPU a few percent; DB still the ceiling.
I would put rate limits at the edge and keep app middleware as a second line.
dropping pool_size per worker fixed it for us on the same box size.
minimal repro with ASGITransport would help more than screenshots.
measure with keep-alive on; cold connects inflate p99.
align client timeout below proxy timeout or you get retry storms.