
A payment request times out, so one client retries. Then a hundred application instances retry together at one second, two seconds, and four seconds. The dependency receives synchronized traffic exactly when it is recovering. Exponential backoff with jitter turns retries from a synchronized storm into a bounded recovery strategy.
This guide covers retryable errors, backoff formulas, jitter algorithms, retry budgets, idempotency, and circuit-breaker interaction. Continue from timeouts, circuit breakers, and idempotency in REST APIs.
Table of Contents
Open Table of Contents
When Retries Help
Retry only when another attempt can plausibly change the outcome: a connection reset, temporary 503, rate limit with a Retry-After, or transient leader election. Do not retry malformed requests, authentication failures, deterministic validation errors, or a non-idempotent operation without an idempotency boundary.
flowchart TD
A[Request fails] --> B{Classify error}
B -->|Permanent| C[Return or dead-letter]
B -->|Transient and deadline remains| D[Check retry budget]
B -->|Unknown| E[Fail safe and alert]
D -->|Available| F[Sleep with jitter]
F --> G[Retry with same idempotency key]
G --> H{Success?}
H -->|Yes| I[Return result]
H -->|No| B
D -->|Exhausted| C
Retries consume the caller’s time, dependency capacity, connection slots, and sometimes money. A retry policy is therefore a capacity policy, not just an error-handling convenience.
Exponential Backoff and Jitter
A capped exponential delay can be expressed as min(cap, base * 2^attempt). With a 100 ms base and a 2 s cap, the nominal schedule is 100 ms, 200 ms, 400 ms, 800 ms, 1,600 ms, then 2,000 ms. The cap prevents a single request from waiting forever.
Jitter randomizes that delay:
| Algorithm | Formula | Use and trade-off |
|---|---|---|
| Full jitter | random(0, exponential delay) | Strongly spreads a fleet; some attempts retry early |
| Equal jitter | half delay + random(0, half delay) | Preserves a minimum wait; less spread |
| Decorrelated jitter | random(base, previous * 3), capped | Avoids repeated synchronization; harder to reason about |
Full jitter is a strong default for many clients. Honor server-provided Retry-After when it is safe, then still apply a local deadline and maximum. Never use a fixed sleep in a fleet unless the request rate is tiny and the dependency explicitly asks for it.
Retry Budgets
An attempt limit alone is not enough. Three retries on every request can triple traffic during an outage. A retry budget caps retries as a fraction of normal request volume, for example no more than 10 retry attempts per 100 original requests in a rolling minute. When the budget is exhausted, fail fast or queue work for later.
Propagate one deadline through the call chain. If an upstream has 300 ms left, a downstream client must not start a 500 ms timeout and three retries. Budget each hop from the remaining deadline and leave time for serialization and response delivery.
Idempotency and Layering
Retries can make an already-successful operation happen twice when the response is lost. Use an idempotency key stored with the business result for payment, order creation, webhook delivery, and other side effects. A database unique constraint or inbox table should make the key durable, not only an in-memory map.
Choose one retry owner per dependency path. If an HTTP client, service mesh, application, and message consumer all retry, a single failure can be multiplied by every layer. The owning layer should classify errors, honor deadlines, record attempts, and expose the policy to operators.
Implementation Sketch
async function retry<T>(operation: () => Promise<T>, deadlineMs: number) {
const started = Date.now();
const baseMs = 100;
const capMs = 2_000;
const maxAttempts = 4;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
try {
return await operation();
} catch (error) {
if (!isTransient(error) || attempt === maxAttempts - 1) throw error;
const exponential = Math.min(capMs, baseMs * 2 ** attempt);
const delay = Math.floor(Math.random() * exponential); // Full jitter.
if (Date.now() - started + delay >= deadlineMs) throw error;
await sleep(delay);
}
}
throw new Error("unreachable");
}
The example is incomplete without an idempotency key, metrics, and a shared retry budget. It also assumes operation is safe to repeat. For a queue consumer, prefer a durable retry topic or delayed queue when sleeping would block a partition or worker.
Observability and Testing
Record attempt number, error class, delay, remaining deadline, budget decision, idempotency key, and final outcome. Alert on retry ratio, retry traffic, budget exhaustion, and time spent in retries. A healthy success rate can hide a dependency that is surviving only because every request is retried twice.
Test a synchronized fleet, Retry-After, clock skew, cancellation during sleep, duplicate success after a timeout, a full retry budget, and a dependency that fails longer than the deadline. Load-test the dependency with retries enabled; a no-retry benchmark is not a resilience benchmark.
Interview Questions
1. Why add jitter to exponential backoff?
Without jitter, clients that observe the same outage often retry on the same schedule and create synchronized spikes. Jitter spreads attempts over a window, giving the dependency a chance to recover. It also reduces contention when many clients compete for a resource.
2. Which errors should be retried?
Retry transient transport errors, selected 5xx responses, and rate limits when the operation is safe and the deadline remains. Do not retry deterministic validation or authorization failures. The dependency contract, idempotency model, and error classification should drive the policy.
3. What is a retry budget?
A retry budget limits extra attempts relative to original traffic or a time window. It prevents retries from multiplying load during an outage and can force callers to fail fast or defer work. Track budget consumption as a first-class reliability metric.
4. How do retries interact with a circuit breaker?
Retries handle a small number of transient failures; the breaker stops calls when the failure signal persists. Once the breaker opens, retry loops should stop rather than repeatedly calling a dependency that the breaker rejected. Both mechanisms need the same deadline and failure classification.
5. How do you retry a payment safely?
Send a stable idempotency key with every attempt and make the provider or a local durable record return the same business result for that key. A timeout is ambiguous: the first request may have succeeded. Never rely on a client-side counter alone to prevent duplicate charges.
Conclusion
Safe retries are selective, deadline-aware, idempotent, jittered, and budgeted. Exponential backoff reduces pressure, jitter avoids synchronization, and a retry budget prevents an outage from multiplying traffic. Pair the policy with timeouts, circuit breakers, durable status, and observability so recovery behavior is predictable.
References
- AWS Reliability Pillar: Control and limit retry calls
- AWS Builders’ Library: Timeout, retries, and backoff with jitter
- Google Cloud: Retry strategy
YouTube Videos
- “Retry Strategies: Exponential Backoff & Jitter Explained” - Engineering with Utsav [https://www.youtube.com/watch?v=NByH-cau97A]