Adaptive Concurrency Limits Using Little's Law and Gradient2 Queue Controllers
Preventing retry storms and queue saturation in storage RPC gateways without hardcoded max_inflight magic numbers.
1. Why Static Concurrency Limits Fail Under Backend Degradation
A static limit of `max_inflight = 512` works when backend service time is 1 ms (supporting 512,000 req/s via Little's Law `L = λW`), but when an NVMe rebuild doubles service time to 10 ms, 512 in-flight requests represent 500+ queued requests waiting purely in buffers. The Gradient2 controller tracks the ratio between an exponentially weighted short-term RTT and a long-term baseline RTT (`gradient = rtt_long / rtt_short`), shrinking the concurrency window immediately when queueing delay emerges.
gradient := clamp(emaLongRTT / emaShortRTT, 0.5, 1.0)
newLimit := float64(currentLimit)*gradient + queueTolerance
currentLimit = int(clamp(newLimit, minLimit, maxLimit))