Adaptive Concurrency Limits Using Little's Law and Gradient2 Queue Controllers

Preventing retry storms and queue saturation in storage RPC gateways without hardcoded max_inflight magic numbers.

1. Why Static Concurrency Limits Fail Under Backend Degradation

A static limit of `max_inflight = 512` works when backend service time is 1 ms (supporting 512,000 req/s via Little's Law `L = λW`), but when an NVMe rebuild doubles service time to 10 ms, 512 in-flight requests represent 500+ queued requests waiting purely in buffers. The Gradient2 controller tracks the ratio between an exponentially weighted short-term RTT and a long-term baseline RTT (`gradient = rtt_long / rtt_short`), shrinking the concurrency window immediately when queueing delay emerges.

gradient := clamp(emaLongRTT / emaShortRTT, 0.5, 1.0)
newLimit := float64(currentLimit)*gradient + queueTolerance
currentLimit = int(clamp(newLimit, minLimit, maxLimit))
← Older Note (2024-07-14)Inside Linux cgroup v2 memory.high Throttling and Direct Reclaim BackpressureNewer Note (2024-10-12) →Profiling Hardware TLB Misses with perf stat and 2MiB Hugetlbfs Backing Pools

← Back to all notes · View full chronological archive (24)