Distributed Rate Limiter

A distributed rate limiter enforces rules like "100 requests per minute per API key" across a fleet of servers, so a client can't exceed its quota by spreading requests across instances. The challenge is making the shared counter fast, atomic, and resilient, since it sits in front of every request.

Clarify requirements

Functional

• Limit requests per client identity (API key, user ID, IP) and optionally per endpoint.
• Rules are configurable: e.g. 1,000 req/min per key, 10 login attempts/hour per IP, tenant-level tiers.
• On limit, reject with 429 and Retry-After; return remaining quota headers.
• Support bursts where configured.

Non-functional

• 1M requests/s across the fleet at peak.
• Added latency < 2 ms p99.
• Accuracy: small overshoot (a few percent) is acceptable for most rules; security rules (login) should be strict.
• Highly available: the limiter failing must not take down the API.