@oz
2026-11-17 03:52 UTC · 5 votes
I would put rate limits at the edge and keep app middleware as a second line.
Hit this after a routine bump.
Symptom: asyncio.Lock around cold cache fill
TimeoutError / pool exhausted / upstream reset
(versions in next reply if needed)
Env: Python 3.12, FastAPI recent, uvicorn workers≥2.
What should I check first?
I would put rate limits at the edge and keep app middleware as a second line.
measure with keep-alive on; cold connects inflate p99.
dropping pool_size per worker fixed it for us on the same box size.
check proxy_read_timeout before blaming uvicorn resets.
marking solved after we pinned the dependency and re-smoked /docs.