Appearance
Service Communication
26. When should services communicate synchronously, and when asynchronously?
The short answer
Synchronous communication, an HTTP or gRPC call that waits for the answer, fits when the caller needs the result right now to continue, such as "is this coupon valid?" while the user waits. Its cost is coupling: if the callee is slow or down, so is the caller. Asynchronous communication, a message or event put on a queue or broker, fits when the work can happen later or many parties need to know, such as sending a receipt or updating search. It absorbs spikes and outages, at the cost of eventual consistency and harder debugging. Most systems use both.
| Synchronous: HTTP, gRPC | Asynchronous: queue, events | |
|---|---|---|
| Caller waits | Yes | No |
| Callee down | Caller fails or degrades | Messages wait in the queue |
| Coupling in time | Both must be up together | Independent |
| Consistency | Immediate answer | Eventual |
| Good for | Reads, validations the user waits for | Side effects, notifications, long jobs |
| Debugging | One request, one trace | Messages flowing through several hops |
A rule of thumb that interviewers like. Use synchronous calls for queries the user is waiting on. Use asynchronous messages for side effects of something that already happened. Placing an order checks stock synchronously, then publishes order.placed for email, analytics and loyalty asynchronously.
27. What is the difference between REST, gRPC and GraphQL?
The short answer
REST models resources as URLs with HTTP methods and usually JSON; it is simple, cacheable, and understood by every tool, so it is the default for public APIs. gRPC defines services and messages in a .proto file, generates typed clients, and sends compact binary Protocol Buffers over HTTP/2 with streaming; it is fast and strict, ideal between internal services. GraphQL exposes one endpoint and a typed schema, and lets the client choose exactly which fields it wants in one request; it suits apps with many screens needing different shapes of data, but makes caching, rate limiting and query cost harder.
| REST | gRPC | GraphQL | |
|---|---|---|---|
| Contract | OpenAPI, optional | .proto, required | Schema, required |
| Format | JSON, text | Protocol Buffers, binary | JSON |
| Transport | HTTP/1.1 or 2 | HTTP/2 | HTTP |
| Browser support | Native | Needs gRPC-Web or a proxy | Native |
| HTTP caching | Easy with GET and ETag | No | Hard: usually one POST endpoint |
| Over and under fetching | Common | Fixed messages | Client picks fields |
| Streaming | Server-Sent Events, chunked responses | Built in, both directions | Subscriptions |
| Best for | Public APIs, simple services | Internal service-to-service calls | Front ends with varied data needs |
The same request three ways:
text
REST GET /orders/8812?fields=id,status,total
gRPC orders.v1.OrderService/GetOrder { order_id: "8812" }
GraphQL POST /graphql { order(id: "8812") { id status total items { name qty } } }GraphQL's traps to mention: a client can send a deeply nested query that costs the server enormously, so limit depth and cost; naive resolvers cause N+1 database queries, so batch with DataLoader (N+1); and authorization must be checked per field and object, not per endpoint.
28. When would you use WebSockets, Server-Sent Events or polling?
The short answer
Polling asks the server every few seconds whether anything changed; it is simple and works everywhere, but wastes requests and adds delay. Long polling holds each request open until there is news. Server-Sent Events, SSE, keep one HTTP response open and let the server push text events to the browser, with automatic reconnection; ideal for one-way updates such as notifications, order tracking and AI output streamed token by token. WebSockets open a full two-way channel for both sides to send at any time; needed for chat, multiplayer games and collaborative editing.
| Polling | Server-Sent Events | WebSocket | |
|---|---|---|---|
| Direction | Client asks | Server to client | Both ways |
| Protocol | Plain HTTP | Plain HTTP, text/event-stream | Upgrade to ws:// or wss:// |
| Reconnect | Every request is new | Built into the browser | You write it |
| Through proxies and CDNs | Easy | Usually easy | Sometimes needs configuration |
| Good for | Rare updates, simplest option | Notifications, live status, streamed text | Chat, games, live collaboration |
Server-Sent Events in Express:
ts
app.get('/api/orders/:id/events', requireUser, (req, res) => {
res.writeHead(200, {
'Content-Type': 'text/event-stream', // (1)
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
})
const send = (status: string) => res.write(`event: status\ndata: ${JSON.stringify({ status })}\n\n`) // (2)
const unsubscribe = orderStatus.subscribe(req.params.id, send)
const ping = setInterval(() => res.write(': keep-alive\n\n'), 25_000) // (3)
req.on('close', () => { clearInterval(ping); unsubscribe() }) // (4)
})ts
// browser
const source = new EventSource('/api/orders/8812/events')
source.addEventListener('status', (e) => render(JSON.parse(e.data).status))- The SSE content type keeps the response open as a stream of events.
- Each event is a few
field: valuelines ending in a blank line. - A comment line every 25 seconds stops proxies closing an idle connection.
- Clean up when the browser disconnects, or the subscription leaks.
Scaling either one. Each open connection lives on one server instance. To send a message to a user connected to another instance, instances share events through Redis Pub/Sub or a broker (Redis Pub/Sub).
29. What is service discovery?
The short answer
Service discovery is how a service finds the current network address of another service, when instances start, stop and move all the time. Instances register themselves, or the platform registers them, in a registry, and callers look up a healthy instance by name. In client-side discovery, the caller queries the registry and picks an instance itself. In server-side discovery, the caller talks to a stable name or load balancer that routes for it. In Kubernetes this is built in: a Service gets a DNS name such as orders.default.svc.cluster.local that always points to healthy pods.
| Approach | How | Examples |
|---|---|---|
| DNS | A stable name resolves to current instances | Kubernetes Services, cloud internal DNS |
| Load balancer | One stable address; it tracks instances | AWS ALB target groups |
| Registry with client-side lookup | Instances register; clients query and choose | Consul, Eureka |
| Service mesh | A sidecar proxy beside each service handles discovery and routing | Istio, Linkerd |
Health is part of discovery. A registry that returns a crashed instance sends traffic into a black hole. Registration must be tied to health checks, so failing instances are removed within seconds (readiness checks).
30. What is an API Gateway, and what responsibilities should it own?
The short answer
An API gateway is the single entry point that sits in front of your services for external clients. It should own cross-cutting edge concerns: TLS termination, routing requests to the right service, authentication of the incoming token, rate limiting and quotas, request size limits, CORS, response compression, request IDs, and access logging and metrics. It should not own business logic, data transformation that belongs to a service, or orchestration of many calls; that turns it into a fragile monolith that every team must change. Examples are Kong, AWS API Gateway, Azure API Management, Envoy and NGINX.
The services behind the gateway still check permissions themselves, as described in securing microservices.
| Gateway should | Gateway should not |
|---|---|
| Verify the token's signature and expiry | Decide whether this user may refund this order |
| Rate limit per client or API key | Join data from three services into one response |
Route /orders/* to the orders service | Contain per-feature business rules |
| Add request IDs and log every call | Become a deploy bottleneck for every team |
31. What is the Backend for Frontend (BFF) pattern?
The short answer
A Backend for Frontend is a small server-side layer built for one specific client, such as the web app, the iOS app or the partner portal, usually owned by that client's team. It calls the underlying services, combines and reshapes their data into exactly what that client's screens need, and handles client-specific concerns such as session cookies for the browser or smaller payloads for mobile. It keeps general-purpose services free of client-specific code. In browser security, a BFF also holds the OAuth tokens on the server, so the browser only has a session cookie (tokens in a BFF).
| Without a BFF | With a BFF |
|---|---|
| Mobile app makes 6 calls to render one screen | 1 call to the mobile BFF, which makes 6 nearby calls |
| Services grow fields only one client uses | Client-specific shaping lives in its BFF |
| Browser holds access tokens | BFF holds tokens; browser holds an HttpOnly cookie |
When not to use one: a single client, or several clients needing the same data. Then a BFF is just an extra hop. Next.js route handlers and server components often act as the web app's BFF already.
32. What is a message queue?
The short answer
A message queue is a buffer that holds messages from producers until consumers take them, usually one consumer per message. The producer adds a job, such as "resize image 991", and returns immediately; a worker picks it up when it can, processes it, and acknowledges it, and only then is it removed. If the worker crashes before acknowledging, the message is delivered again. Queues absorb spikes, let you scale workers independently, and let work survive outages. Examples are RabbitMQ, Amazon SQS, Azure Service Bus, and BullMQ on Redis.
A job queue in Node, with BullMQ on Redis:
ts
import { Queue, Worker } from 'bullmq'
const images = new Queue('images', { connection })
app.post('/api/uploads', async (req, res) => {
const upload = await saveOriginal(req)
await images.add('resize', { uploadId: upload.id }, {
jobId: `resize:${upload.id}`, // (1)
attempts: 5, backoff: { type: 'exponential', delay: 1_000 }, // (2)
})
res.status(202).json({ id: upload.id, status: 'processing' }) // (3)
})
new Worker('images', async (job) => {
await resizeAndStore(job.data.uploadId) // (4)
}, { connection, concurrency: 4 })- A job ID derived from the upload, so adding the same job twice creates only one.
- Retries with backoff for temporary failures.
202 Accepted: the work has been accepted, not finished.- The worker must be idempotent: a retried resize just overwrites the same output file.
33. What is the publish/subscribe (Pub/Sub) pattern?
The short answer
In publish/subscribe, a publisher sends a message to a topic, not to a specific receiver, and every subscriber to that topic gets its own copy. The publisher does not know who, or how many, are listening. That is the difference from a work queue, where each message goes to one worker. Pub/sub is how events reach many services, such as order.placed reaching email, warehouse and analytics. Durable pub/sub, such as Kafka topics or SNS feeding SQS queues, keeps messages for subscribers that are offline; simple pub/sub, such as Redis Pub/Sub, does not.
| Work queue | Pub/sub | |
|---|---|---|
| Each message goes to | One consumer | Every subscriber |
| Purpose | Share work across workers | Tell many parties about one event |
| Adding a consumer | Splits the work further | Adds one more copy |
| Example | Resize-image jobs | order.placed events |
Both at once, the common real shape: each subscribing service gets its own copy of every event, and inside that service, several instances share that copy like a work queue. Kafka does this with consumer groups; SNS does it by fanning out to one SQS queue per service.
34. How do Kafka and RabbitMQ differ?
The short answer
RabbitMQ is a message broker: it routes messages through exchanges into queues, pushes them to consumers, and deletes each message once it is acknowledged. It shines at task queues, flexible routing, per-message acknowledgement and retries. Kafka is a distributed log: messages are appended to partitioned topics and kept for a set time, whether read or not; each consumer group tracks its own position, its offset, and can replay history. It shines at very high throughput, event streaming, multiple independent readers and replay. Pick RabbitMQ for jobs and commands, Kafka for event streams and data pipelines.
| RabbitMQ | Kafka | |
|---|---|---|
| Model | Broker with exchanges and queues | Partitioned, replicated log |
| After a message is read | Removed once acknowledged | Kept until retention expires |
| Replay old messages | No, by default | Yes, by moving the offset |
| Ordering | Per queue | Per partition only |
| Throughput | High | Very high, millions per second per cluster |
| Routing | Rich: direct, topic, fanout, headers | By topic and partition key |
| Per-message retry and delay | Built in | You build it with retry topics |
| Typical use | Task queues, RPC-style commands | Event streaming, logs, analytics, change data capture |
Ordering in Kafka is per partition. All events with the same key, such as one orderId, go to the same partition, so they stay in order. Events for different orders may interleave. Choose the key so that whatever must stay in order shares it.
Recent changes to mention: Kafka 4.0, released in 2025, runs only in KRaft mode, its own built-in consensus, so ZooKeeper is no longer needed. RabbitMQ 4 replaced the old mirrored queues with quorum queues for replication.
35. What is a Dead Letter Queue (DLQ)?
The short answer
A dead letter queue is where messages go when they cannot be processed after a set number of attempts, because they are malformed, refer to data that does not exist, or keep hitting a bug. Moving them aside stops one "poison" message from blocking the queue or being retried forever, while keeping it for investigation instead of losing it. You must monitor the DLQ and alert when it grows, inspect the messages, fix the cause, and then redrive them back into the main queue. A DLQ nobody watches is just a slower way of losing data.
ts
import { Worker, UnrecoverableError } from 'bullmq'
const worker = new Worker('payments', async (job) => {
const event = PaymentEventSchema.safeParse(job.data)
if (!event.success) throw new UnrecoverableError('Malformed payment event') // (1)
await applyPayment(event.data)
}, { connection })
worker.on('failed', async (job, err) => {
if (job && (job.attemptsMade >= (job.opts.attempts ?? 1) || err instanceof UnrecoverableError)) {
await deadLetters.add('payments', { original: job.data, error: err.message, failedAt: new Date().toISOString() }) // (2)
metrics.increment('payments.dead_lettered') // (3)
}
})- A message that can never succeed skips the remaining retries.
- Copy it, with the error and time, to the dead letter queue.
- Count it, so an alert fires when dead letters appear.
| Failure kind | Retry? | Then |
|---|---|---|
Timeout, 503, connection reset | Yes, with backoff | DLQ after the attempt limit |
| Malformed message, schema mismatch | No | DLQ immediately |
| Bug in the consumer | A few times | DLQ; fix the code; redrive |
| Referenced record not found yet | Yes, with delay | It may arrive shortly; DLQ if not |
36. How do you ensure idempotency in distributed systems?
The short answer
In distributed systems messages and requests are delivered at least once: networks drop acknowledgements, clients retry, relays republish. "Exactly once" delivery is not achievable across systems, so you make processing idempotent: doing it twice has the same effect as once. Give every operation a unique ID, an idempotency key or message ID, and record processed IDs in the same transaction as the effect, so a duplicate is detected and skipped. Use natural idempotence where you can, such as SET status = 'paid' instead of balance = balance + 10, and unique constraints as the final guard.
ts
export async function handlePaymentSucceeded(msg: { id: string; orderId: string; amountPence: number }) {
await withTransaction(async (tx) => {
const { rowCount } = await tx.query(
'INSERT INTO processed_messages (message_id) VALUES ($1) ON CONFLICT DO NOTHING', [msg.id]) // (1)
if (rowCount === 0) return // (2)
await tx.query("UPDATE orders SET status = 'paid', paid_pence = $1 WHERE id = $2", [msg.amountPence, msg.orderId]) // (3)
})
}- Try to record the message ID. The primary key on
message_idmakes a second insert do nothing. - Already processed? Stop. The duplicate has no effect.
- The effect commits together with the record of it. If the transaction fails, neither happens, and the redelivered message is processed normally.
| Technique | Where |
|---|---|
| Idempotency key from the client | Payment and order APIs (idempotency keys) |
| Processed-message table in the same transaction | Event and queue consumers |
| Naturally idempotent writes | SET status = 'paid', upserts, PUT with full state |
| Unique constraints | One order per checkout session ID |
| Version numbers | Ignore an event older than the state you have |
Kafka's "exactly-once" applies inside Kafka: an idempotent producer plus transactions make reading, processing and writing back to Kafka atomic. The moment your consumer writes to a database or calls an API, you are back to at-least-once, and need the techniques above.