Skip to content

Interview PrepResilience · Advanced

Resilience Patterns ​

18. Why do timeouts matter, and how should you set them? ​

The short answer

Every call over the network, to a database, an API or a cache, can hang. Without a timeout, a slow dependency holds your connections, memory and request slots until the whole service runs out and stops answering anyone. A timeout turns "hang forever" into a fast, handleable error. Set each timeout from the dependency's normal latency, a little above its p99, not to a round number like 30 seconds. Pass a deadline down the call chain, so inner calls never wait longer than the user's request has left.

ts
export async function getShippingQuote(orderId: string, signal: AbortSignal) {
  const res = await fetch(`${config.SHIPPING_URL}/quotes/${orderId}`, {
    signal: AbortSignal.any([signal, AbortSignal.timeout(800)]),                // (1)
  })
  if (!res.ok) throw new HttpError(502, 'Shipping quote failed')
  return res.json()
}

app.get('/api/checkout/:id', async (req, res) => {
  const deadline = AbortSignal.timeout(2_000)                                   // (2)
  const [cart, quote] = await Promise.all([
    getCart(req.params.id, deadline),
    getShippingQuote(req.params.id, deadline),
  ])
  res.json({ cart, quote })
})
  1. Stop at 800 ms, or earlier if the request's own deadline fires first. AbortSignal.any combines both.
  2. The whole request gets 2 seconds. Every call inside shares that budget, so the user never waits longer, however many dependencies there are.
TimeoutTypical valueNote
Connect0.5 to 2 secondsA healthy service accepts connections quickly
Database query1 to 5 secondsUse statement_timeout (database timeouts)
Internal API callJust above its p99, such as 300 to 800 msMeasure it
Whole incoming request2 to 10 secondsBelow the load balancer's own timeout

Inner timeouts must be shorter than outer ones. If the load balancer gives up after 30 seconds but your app waits 60 seconds on a dependency, the app keeps working on requests nobody is waiting for.

19. What is the Retry pattern, and how should backoff be designed? ​

The short answer

Retrying means trying a failed call again, because many failures are temporary: a dropped connection, a brief overload, a failover. Only retry idempotent operations, or ones protected by an idempotency key, and only on retryable errors, such as timeouts, 502, 503, 504 and 429, never on 400 or 401. Wait between attempts with exponential backoff, doubling the delay each time, plus jitter, randomness, so thousands of clients do not retry in sync. Cap the number of attempts and the total time, and respect Retry-After.

ts
const RETRYABLE = new Set([408, 429, 500, 502, 503, 504])

export async function withRetry<T>(fn: () => Promise<T>, { attempts = 4, baseMs = 100, maxMs = 2_000 } = {}): Promise<T> {
  for (let i = 0; ; i++) {
    try {
      return await fn()
    } catch (err: any) {
      const status = err.status as number | undefined
      const retryable = status === undefined || RETRYABLE.has(status)          // (1)
      if (!retryable || i === attempts - 1) throw err
      const cap = Math.min(maxMs, baseMs * 2 ** i)                              // (2)
      const delay = err.retryAfterMs ?? Math.random() * cap                     // (3)
      await sleep(delay)
    }
  }
}
  1. Network errors have no status and are retryable. Client errors such as 400 are not: retrying will not fix bad input.
  2. Exponential growth with a cap: 100, 200, 400, 800 ms, never above 2 seconds.
  3. Full jitter: a random wait between zero and the cap, so clients spread out. A server's Retry-After overrides it.

Why jitter matters, as a picture. Without jitter, 10,000 clients that all failed at 10:00:00.000 all retry at exactly 10:00:00.100, then 10:00:00.300, hitting the recovering server with the same spike again and again. With jitter, their retries spread evenly over the window.

The same function, as a coding question, is in the JavaScript classics.

20. What is the Circuit Breaker pattern? ​

The short answer

A circuit breaker sits in front of calls to a dependency and stops calling it when it keeps failing, so your service fails fast instead of waiting on timeouts, and the struggling dependency gets room to recover. It has three states. Closed: calls pass through and failures are counted. Open: after too many failures, calls are rejected immediately for a cool-down period. Half-open: after the cool-down, one trial call is allowed; success closes the breaker, failure opens it again. Pair it with a fallback, such as cached data or a friendly error.

closedcalls passfailures countedopenfail immediatelyno calls senthalf-openone trial callthreshold hitcool-down overtrial succeeds: closetrial fails: open again
Stop knocking on a door that is not answering. While open, callers get an instant error and the dependency gets no traffic. After the cool-down, one call tests whether it has recovered.
ts
type State = 'closed' | 'open' | 'half-open'

export class CircuitBreaker {
  private state: State = 'closed'
  private failures = 0
  private openedAt = 0
  private readonly threshold: number
  private readonly coolDownMs: number

  constructor(threshold = 5, coolDownMs = 10_000) {
    this.threshold = threshold
    this.coolDownMs = coolDownMs
  }

  async call<T>(fn: () => Promise<T>, now = Date.now()): Promise<T> {
    if (this.state === 'open') {
      if (now - this.openedAt < this.coolDownMs) throw new Error('circuit open')   // (1)
      this.state = 'half-open'                                                  // (2)
    }
    try {
      const result = await fn()
      this.state = 'closed'                                                     // (3)
      this.failures = 0
      return result
    } catch (err) {
      this.failures++
      if (this.state === 'half-open' || this.failures >= this.threshold) {      // (4)
        this.state = 'open'
        this.openedAt = now
      }
      throw err
    }
  }

  get current() { return this.state }
}
  1. Open: reject at once, without touching the dependency.
  2. Cool-down over: let one call through as a test.
  3. Any success closes the breaker and resets the count.
  4. Open on a failed trial, or when failures reach the threshold.

What it does, with a threshold of 3 and a 10-second cool-down. Three failed calls at time 1 s leave it closed, closed, then open. A fourth call is rejected with circuit open. A call at 5 s is still rejected. A successful call at 12 s, after the cool-down, closes it again. Real libraries, such as opossum for Node, add a failure-rate window, timeouts and metrics.

21. What is the Bulkhead pattern? ​

The short answer

A bulkhead isolates resources so a failure in one part cannot drown the rest, like the watertight compartments in a ship's hull. You give each dependency or type of work its own limited pool: separate connection pools, separate concurrency limits, separate worker queues, or separate instances. If the recommendations API hangs, it can fill only its own 10 slots; checkout, with its own pool, keeps working. Without bulkheads, one slow dependency can consume every connection and thread the service has.

ts
import pLimit from 'p-limit'

const limits = {
  recommendations: pLimit(10),                                                 // (1)
  payments: pLimit(50),
}

export const getRecommendations = (userId: string) =>
  limits.recommendations(() => recClient.forUser(userId))                       // (2)

export const chargeCard = (orderId: string, pence: number) =>
  limits.payments(() => paymentsClient.charge(orderId, pence))
  1. A separate concurrency limit for each dependency. At most 10 recommendation calls run at once, however many requests arrive.
  2. Calls beyond the limit wait in that dependency's own queue, so they never take slots that payments needs. Combine with a timeout, so the queue does not grow forever.
Bulkhead atExample
Inside a processA concurrency limit per dependency, as above
Connection poolsA separate database pool for reports and for the API
Queues and workersSeparate queues for "send email" and "process payment"
DeploymentSeparate instances for the public API and the admin API
TenantsHeavy customers on their own capacity

22. How do you design retries and circuit breakers without causing cascading failures? ​

The short answer

Badly designed retries multiply load exactly when a system is weakest: if each of five layers retries three times, one user request can become 243 calls at the bottom. So: retry at one layer only, usually the one closest to the caller; use backoff with jitter and a retry budget, such as retries at most 10% of traffic; never retry when the breaker is open; keep timeouts shorter at each inner layer; make operations idempotent; and shed load early with rate limits. A circuit breaker then stops calls entirely when a dependency is clearly down.

1 userrequestgateway3 triesorders3 × 3 = 9inventory9 × 3 = 27DB27 callsfix: retry at one layer, with a budget, and stop when the breaker opens
Retries multiply at every layer. Three layers that each retry three times turn one request into 27 calls on an already struggling database. That is how a small slowdown becomes a full outage.

The rules, as a checklist:

  1. Retry in one place. Inner services fail fast; the outermost caller decides whether to retry.
  2. Backoff with jitter, and few attempts, two or three at most (question 19).
  3. A retry budget. If more than about 10% of recent calls are already retries, stop retrying.
  4. Breakers before retries. If the breaker is open, do not retry at all.
  5. Shrinking timeouts inward, under one overall deadline (question 18).
  6. Idempotency keys on writes, so a retry never doubles a payment.
  7. Load shedding at the edge when overloaded (question 23).

23. What are load shedding and backpressure? ​

The short answer

Load shedding means deliberately rejecting some requests when a service is overloaded, quickly, with 503 or 429, so it can serve the rest well instead of serving everyone badly. Shed the least important work first, such as background syncs before checkouts. Backpressure means a slow consumer signals upstream to slow down, instead of letting work pile up in memory until it crashes: bounded queues that refuse new items when full, streams that pause, and 429 with Retry-After. Both keep a system working at its capacity rather than collapsing beyond it.

ts
let inFlight = 0
const MAX_IN_FLIGHT = 200

app.use((req, res, next) => {
  const critical = req.path.startsWith('/api/checkout')
  if (inFlight >= MAX_IN_FLIGHT && !critical) {                                 // (1)
    res.set('Retry-After', '2')
    return res.status(503).json({ error: 'Busy, try again shortly' })
  }
  inFlight++
  res.on('close', () => { inFlight-- })                                         // (2)
  next()
})
  1. Above 200 requests in progress, turn away non-critical work at once. Checkout keeps priority.
  2. Count down when each request finishes, successfully or not.

Why rejecting helps. Past capacity, a server that accepts everything makes every request slow, until they all time out and nobody is served. Rejecting 20% quickly lets the other 80% complete normally.

Backpressure you already use: Node streams pause the producer when the consumer's buffer is full (backpressure in streams), and a bounded job queue rejects new jobs instead of growing without limit.

24. How do you handle dependency outages and partial system failures? ​

The short answer

Assume every dependency will fail, and decide in advance how each feature behaves when it does. Protect calls with timeouts, limited retries, circuit breakers and bulkheads. Give each feature a fallback: serve cached or default data, hide the optional part of the page, queue the work for later, or show a clear message. Separate critical paths, such as checkout, from optional ones, such as recommendations, so an optional failure never breaks a critical flow. Monitor dependencies separately, so you know which one is failing.

Dependency downFallback
RecommendationsShow bestsellers from a cached list, or hide the panel
Review servicePage renders without reviews
Email providerQueue the email; send when it recovers
Shipping quote APIUse a flat-rate estimate, confirm later
Payment providerCannot fake it: clear message, a second provider if the business needs it
Redis cacheRead from the database, with a concurrency limit
ts
async function productPage(id: string) {
  const product = await getProduct(id)                                          // critical: let failures propagate
  const [reviews, recs] = await Promise.allSettled([                            // (1)
    reviewsBreaker.call(() => reviewsClient.forProduct(id)),
    recsBreaker.call(() => recsClient.similar(id)),
  ])
  return {
    product,
    reviews: reviews.status === 'fulfilled' ? reviews.value : null,             // (2)
    recommendations: recs.status === 'fulfilled' ? recs.value : await cachedBestsellers(),
  }
}
  1. Promise.allSettled waits for both optional calls, without failing if one rejects.
  2. Each optional part degrades on its own, while the product itself still renders.

25. How do you design a highly available backend system? ​

The short answer

Remove every single point of failure and make failure handling automatic. Run several instances of each stateless service across several availability zones, behind a load balancer with health checks. Keep state in managed, replicated stores with automatic failover, plus backups you have restored. Deploy with rolling or blue-green releases and fast rollback. Protect services with timeouts, breakers, bulkheads and rate limits. Use queues so spikes and outages delay work instead of losing it. Monitor and alert on user-facing symptoms, and practise failures. Then quantify it: each extra "nine" of availability costs much more.

load balancer, health checksavailability zone Aapp × 3DB primaryRedis primaryqueue, replicatedavailability zone Bapp × 3DB standbyRedis replicaqueue, replicatedsync
Everything at least twice, in two places. If a whole zone fails, the load balancer sends traffic to the other zone's instances, and the standby database is promoted automatically. No single machine or zone can take the service down.

What the nines mean in downtime:

AvailabilityDowntime per yearDowntime per month
99%3.65 days7.3 hours
99.9%8.8 hours43.8 minutes
99.95%4.4 hours21.9 minutes
99.99%52.6 minutes4.4 minutes

Availability multiplies. If a request needs three services that are each 99.9% available, the request is available only 99.9% × 99.9% × 99.9% ≈ 99.7% of the time. Every synchronous dependency you add lowers the total, which is a strong argument for fallbacks and asynchronous work.

The checklist:

  1. Stateless app instances, at least two per zone.
  2. Health checks that remove bad instances quickly (health checks).
  3. Replicated data stores with tested automatic failover.
  4. Backups, and restores practised on a schedule.
  5. Rolling deploys, graceful shutdown, and one-click rollback.
  6. Timeouts, breakers, bulkheads, rate limits.
  7. Queues for work that can wait.
  8. Alerts on symptoms users feel: error rate and latency.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Interview Prep
Architecture & Distributed Systems
Structure
Data across services
Resilience
Communication
Security between services
Algorithms
Other topics
Nothing picked yet