Requirements
- Limit requests per key (API key, tenant, user, IP) per time window. Return
429with aRetry-Afterheader. - Low overhead (under 1 ms), accurate across many gateway instances, and tolerant of Redis blips.
Algorithms
| Algorithm | How | Pros / cons |
|---|---|---|
| Fixed window | Counter per minute | Simple; allows a 2× burst at window edges |
| Sliding window log | Store timestamps | Exact; memory-heavy |
| Sliding window counter | Weighted current + previous window | Good accuracy, cheap. Common choice |
| Token bucket | Tokens refill at rate r, bucket size b | Allows controlled bursts. Very common for APIs |
| Leaky bucket | Queue drained at a constant rate | Smooths output; adds latency |
Architecture
diagram
- Run the check in a Lua script so read-modify-write is atomic in Redis.
- Key design:
rl:{tenantId}:{route}keeps tenant keys on the same Redis slot. - Fail open or fail closed? Usually fail open for availability, with an alert. Fail closed for expensive or abusable endpoints.
- Layers: edge/CDN for IP floods, gateway for API-key quotas, and in-service limits for expensive operations like bulk jobs.
Architect's tip
In multi-tenant SaaS, rate limits are also a fairness tool. Combine per-tenant request limits with per-tenant concurrency limits on background jobs so one big tenant can't monopolise workers.
Cheatsheet
The whole topic on one page. Click to open full screen.
Sources & further learning
Videos, courses, docs and books I recommend for this topic.
Scaling your API with rate limitersArticle · Stripe Engineering · stripe.comHow we built rate limiting capable of scaling to millions of domainsArticle · Cloudflare · blog.cloudflare.comRate Limiting patternDocs · Azure Architecture Center · learn.microsoft.comDesign a rate limiter — video searchVideo · YouTube · youtube.com
Related topics
Architecture PatternsUnderstood
API Gateway & Backend-for-Frontend (BFF)
A single entry point that handles cross-cutting concerns (auth, rate limits, routing); BFFs go further with one tailored backend per client type.
Intermediate5 sources
Architecture PatternsApplied in production
Multi-Tenant SaaS Architecture
Serve many customers from shared infrastructure while guaranteeing isolation of data, performance and configuration per tenant.
Advanced4 sources
Architecture PatternsUnderstood
Circuit Breaker, Retry, Timeout & Bulkhead
The resilience toolkit — fail fast, retry safely, bound waiting, and isolate resources so one slow dependency can't take down the whole system.
Intermediate6 sources