ADevGuide 
A retrying mobile app, a buggy integration, or a credential-stuffing attack can turn a healthy API into an overloaded one. Rate limiting defines how much of a shared resource one client may consume, and the hard part is making that decision correctly across many gateway instances.
This guide compares the main algorithms and the production trade-offs interviewers expect. It follows API Gateway in the Backend Developer Interview Guide. Looking for a Spring Boot implementation? See API Rate Limiting with Spring Boot.
Table of Contents
Open Table of Contents
What a Rate Limiter Protects
Rate limiting protects availability, fairness, and cost. Define the key first: IP address is useful for anonymous traffic, user ID fits authenticated APIs, and an API key fits third-party integrations. Limits may be per route, tenant, method, or a weighted combination when one operation costs much more than another.
Return 429 Too Many Requests with clear retry guidance. Do not treat rate limiting as authentication; a valid user can still exceed a fair quota.
Request Flow
flowchart TD
A[Client request] --> B[Gateway]
B --> C[Derive limit key]
C --> D[(Atomic limiter state)]
D --> E{Allowed?}
E -->|Yes| F[Route to service]
E -->|No| G[429 with retry guidance]
G --> H{Client backs off?}
H -->|Yes| A
H -->|No| I[Stop]
F --> I
classDef flow fill:#e8f0fe,stroke:#1a73e8,stroke-width:2px,color:#000000;
class A,B,C,D,E,F,G,H,I flow;Fixed Window
A fixed-window counter stores one count per key and time bucket. It is cheap and works for coarse internal quotas, but it has a boundary problem: a client can use the entire quota at the end of one minute and again at the start of the next. That can nearly double the intended short-term burst.
Sliding Window
A sliding-window log stores request timestamps and removes expired entries. It is accurate but memory-heavy at high volume. A sliding-window counter approximates the current total from the current and previous buckets, trading a little precision for much lower memory. Choose it when fairness near boundaries matters more than a simple implementation.
Token Bucket
Token bucket is a strong default for user-facing APIs. Each key has a capacity and a refill rate; a request spends a token, and an empty bucket rejects it. It permits controlled bursts while limiting the long-term average. Store token count and last-refill timestamp, then refill lazily during the request instead of running a timer for every client.
Distributed Enforcement
Per-instance counters are incorrect when a client can reach multiple gateways. Use a shared store such as Redis and make refill/check/decrement one atomic operation, commonly a Lua script. Decide failure behavior deliberately: fail closed for sensitive or expensive actions, or fail open for low-risk read traffic where availability is more important. Monitor rejected requests, store latency, hot keys, and the ratio of limiter failures. For a Java implementation, see API rate limiting in Spring Boot; for the wider learning path, browse the Backend category.
Interview Questions
1. Which algorithm would you choose for a bursty public API?
I would start with token bucket because it allows a bounded burst while preserving a long-term rate. It uses constant state per key and has a direct Redis implementation. If the requirement is strict fairness at a window boundary, I would discuss a sliding-window counter instead.
2. Why is a local in-memory limiter insufficient?
A client can be routed to several instances, each with its own counter, and exceed the global quota. Shared atomic state fixes that correctness issue but adds latency and availability dependency. The interview answer should name both sides of the trade-off.
3. What should a client do after a 429?
The server should expose retry guidance where appropriate; the client should use bounded exponential backoff with jitter. Immediate retries create a retry storm precisely when capacity is scarce. For non-idempotent writes, retries also need an idempotency contract.
4. Can one limit fit every endpoint?
No. A cheap cached read and an expensive report-generation request consume different resources. Use route-level and weighted limits, then add concurrency limits for work that is slow rather than merely frequent.
5. Where should the limiter run?
The gateway is a good first enforcement point because it rejects traffic before downstream work. Services may still enforce local domain-specific quotas as defense in depth, especially when internal callers bypass the public edge.
Conclusion
- Rate limiting protects fairness, availability, and cost.
- Fixed windows are simple but permit boundary bursts.
- Sliding windows improve fairness at greater state cost.
- Token bucket is usually the best default for controlled API bursts.
- Distributed limits require atomic shared state and intentional failure behavior.
The next topic in this series covers idempotency in REST APIs - how to make retry-prone writes safe. Revisit HTTP methods for the underlying request semantics.
References
- Amazon API Gateway Request Throttling - AWS https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html
- Rate Limiting - MDN Web Docs https://developer.mozilla.org/en-US/docs/Glossary/Rate_limit
YouTube Videos
- “System Design Interview: Rate Limiting Algorithms That Get You Hired” https://www.youtube.com/watch?v=cs_orOjuaPM