Skip to content
ADevGuide Logo ADevGuide
Go back

A Downstream Service Times Out: How Do You Contain the Failure?

Updated:

By Pratik Bhuite | 10 min read

Hub: Java / Interview Fundamentals

Series: Backend Interview Mastery

Last updated: Sep 19, 2026

Part 65 of 88 in the Backend Interview Mastery

Key Takeaways

On this page
Reading Comfort:

Containing a downstream service timeout with deadlines and a circuit breaker

One slow dependency can consume every request thread in its callers and create a cascading outage. Demonstrate a deadline budget, bounded concurrency, a circuit breaker, and a user-appropriate fallback. See timeouts, retries, and circuit breakers.

The Interview Question

“The pricing service starts timing out. Your checkout API has a five-second SLO. How do you contain the failure and decide what customers see?”

Clarifying Questions

Ask whether pricing is required to authorize checkout, the dependency’s error budget, request fan-out, and whether stale prices are allowed. Assume a 2-second caller deadline, three downstream calls, and no safe stale price for payment capture.

The Naive Design and Why It Fails

Retrying each call three times with a five-second timeout multiplies load and guarantees the caller misses its SLO. Unbounded connection pools and synchronous waits spread the outage to unrelated routes. Returning cached prices without a freshness contract can charge the wrong amount.

Better Design

Propagate a deadline, allocate a per-call budget, limit concurrent calls, and open a circuit after a rolling failure threshold. Fail checkout closed when price correctness is required, but allow catalog browsing to show a clearly labeled stale value.

flowchart TD
    A[Request deadline 2s] --> B[Bulkhead for pricing]
    B --> C{Circuit state}
    C -->|Closed| D[One bounded call]
    C -->|Open| E[Fast fallback or retryable error]
    D --> F{Response before deadline?}
    F -->|Yes| G[Continue checkout]
    F -->|No| E

Budget the Request, Not Just the Socket

The five-second endpoint SLO is not a five-second timeout for every dependency. In this checkout path, choose the stated two-second caller deadline so the client does not wait through the entire SLO. If local validation and persistence need 300 ms, leave roughly 1.7 seconds for the pricing call and reserve the remaining time for response serialization and network variance. A fan-out of three pricing calls should either run in parallel under one shared deadline or be reduced to one batch request; giving each child its own independent five-second timeout defeats the caller’s budget.

Propagate an absolute deadline, not only a timeout duration. Each service computes the remaining time and refuses work when the budget is already exhausted:

deadline = request.deadline
remaining = deadline - now()
if remaining <= safety_margin:
    return retryable_timeout

call pricing with min(remaining, pricing_budget)
retry only if the operation is idempotent and remaining time permits backoff

The same boundary applies to connection acquisition, DNS, TLS, database queries, and response writes. A socket timeout without a pool timeout still allows requests to pile up waiting for a connection.

Choosing the Fallback by Business Invariant

Caller operationSafe fallbackUnsafe fallback
Product browsingVersioned stale price with a visible freshness timestampPresenting a stale value as the checkout total
Cart displayShow the last known estimate and request revalidationCapturing payment against an unverified total
CheckoutRetryable error or PENDING_REPRICE stateGuessing a price to keep the UI successful
RecommendationsOmit the module or show a cached listBlocking the whole page until recommendations recover

This is why “always return cached data” is not resilience. The fallback must preserve the invariant that matters to that operation.

Containing Recovery Traffic

When the dependency recovers, do not close the circuit and release every waiting request. Keep a small half-open probe set, increase it gradually, and enforce a retry budget at the caller. A caller making 1,000 original requests per second with three retries can create 4,000 QPS during recovery. Track original and retry traffic separately, and coordinate the reopening rate with the downstream service’s advertised capacity.

Use bulkheads per route or dependency, not one process-wide pool. A slow pricing call should not consume the connections used for health checks, order reads, or operator traffic. Every rejected request should carry a correlation ID and a retryable error that clients can handle without an immediate tight loop.

Failure Handling and Recovery

Use jittered, idempotent retries only for transient errors and only within the remaining deadline. Emit dependency metrics and trip the circuit before all threads block. Half-open probes should be few; close the circuit only after success and latency recover.

Scaling and Trade-offs

If 1,000 requests per second each fan out to three pricing calls, the dependency sees 3,000 QPS before retries. A retry budget of 10% caps amplification at 300 extra QPS. Bulkheads protect other endpoints, but rejected work reduces feature availability; explain that trade-off.

Interview Follow-Ups

1. How do you choose a timeout?

Start from the endpoint SLO, subtract local processing and downstream budgets, and use measured tail latency. Propagate the deadline so nested services cannot outlive the request. Revisit it as traffic and latency distributions change.

2. When should you retry?

Retry only transient failures, with bounded exponential backoff and jitter, when the operation is idempotent or has a deduplication key. Do not retry validation errors, authentication failures, or an expired deadline.

3. What is a bulkhead?

It limits resources for one dependency or route so its failure cannot consume the entire process. Use separate pools, queues, or concurrency semaphores. Tie the limit to capacity and observed rejection rates.

4. What fallback is safe for payments?

If the amount depends on the dependency, a stale or guessed value is unsafe. Return a retryable error or place the order in a pending state that re-prices before capture. A fallback is safe only when its correctness contract is explicit.

5. How do you avoid a retry storm during recovery?

Enforce caller retry budgets, add jitter, use circuit half-open limits, and gradually increase traffic. Coordinate with dependency rate limits. Monitor retries separately from original calls so amplification is visible.

Practice Redis failure handling, backpressure, and service discovery.

References

  1. Microsoft: Circuit breaker pattern
  2. AWS: Timeouts, retries and backoff

YouTube Videos

  1. “Circuit Breaker Pattern Explained” - Hussein Nasser [https://www.youtube.com/watch?v=rvKKj7JU92g]

Share this post on:

Next in Series

Continue through the [object Object] with the next recommended article.

Related Posts

Keep Learning with New Posts

Subscribe through RSS and follow the project to get new series updates.

Was this guide helpful?

Share detailed feedback

Previous Post
Distributed Transaction Failure: Order Succeeds but Payment Fails
Next Post
How Would You Prevent Duplicate Payments?