Skip to content

Interview PrepScaling · Core

Scaling Node.js ​

7. What are vertical scaling and horizontal scaling? ​

The short answer

Vertical scaling, scaling up, means giving one machine more power: more CPU, memory or faster disks. It is simple, needs no code changes, but has a ceiling, gets expensive, and the machine stays a single point of failure. Horizontal scaling, scaling out, means adding more machines or instances behind a load balancer. It has almost no ceiling and survives the loss of an instance, but the app must be stateless: sessions, uploads and caches must live in shared stores, not in one process's memory. Web servers usually scale out; databases usually scale up first.

VerticalHorizontal
HowBigger machineMore machines
LimitThe biggest machine you can buyPractically none
Survives one machine failingNoYes
Code changesNoneApp must be stateless
Downtime to scaleOften a restartNone; add instances
Typical forDatabases, first stepsWeb and API servers, workers

Node specifically. One Node process runs JavaScript on one core. On an 8-core machine, a single process leaves 7 cores idle for JavaScript. So even "vertical" scaling for Node means running several processes on the machine (question 10).

8. What is a load balancer, and how does it distribute traffic? ​

The short answer

A load balancer sits in front of several app instances and spreads incoming requests across them, so no single instance is overloaded, and it removes unhealthy instances using health checks, so users are not sent to a broken one. Common algorithms are round robin, each instance in turn; least connections, the instance with the fewest open requests; weighted, more traffic to bigger instances; and hashing, the same client or key always to the same instance. Layer 4 balancers route TCP connections by address and port; layer 7 balancers understand HTTP and can route by path, host or header.

clientsload balancerleast connectionshealth checks every 5 sinstance 1 · healthyinstance 2 · healthyinstance 3 · healthyinstance 4 · failed checkno traffic
Spread the load, skip the sick. Requests go to whichever healthy instance has the least work. An instance failing its health check is taken out of rotation until it passes again.
AlgorithmPicksGood when
Round robinNext instance in turnRequests cost about the same
Least connectionsFewest open requestsRequest times vary a lot
WeightedIn proportion to capacityInstances of different sizes
Hash of IP or keySame instance for same clientSticky sessions, local caches
Layer 4Layer 7
SeesIP addresses and portsFull HTTP: path, host, headers, cookies
Can route by URLNoYes
TLSPasses through, or terminatesTerminates and inspects
ExamplesAWS NLBAWS ALB, Nginx, Envoy

9. What is PM2, and why is it used? ​

The short answer

PM2 is a process manager for Node on a plain server or VM. It starts your app, restarts it if it crashes, runs several copies in cluster mode to use every CPU core, does zero-downtime reloads by restarting copies one at a time, starts apps on boot, rotates logs, and shows CPU and memory per process. It suits apps deployed directly on VMs. In containers and Kubernetes, the orchestrator already restarts and scales processes, so you usually run one plain node process per container instead of PM2.

js
// ecosystem.config.cjs
module.exports = {
  apps: [{
    name: 'orders-api',
    script: 'dist/server.js',
    instances: 'max',                         // (1)
    exec_mode: 'cluster',
    max_memory_restart: '600M',               // (2)
    kill_timeout: 10000,                      // (3)
    env: { NODE_ENV: 'production', PORT: 3000 },
  }],
}
bash
pm2 start ecosystem.config.cjs
pm2 reload orders-api       # zero-downtime: restarts workers one by one
pm2 startup && pm2 save     # start again after a reboot
  1. One process per CPU core, sharing port 3000 through Node's cluster module.
  2. Restart a process that grows past 600 MB, a safety net for a slow leak, not a fix.
  3. Give each process 10 seconds to finish in-flight requests before it is killed (graceful shutdown).

10. How does Node.js clustering differ from using multiple containers or processes? ​

The short answer

The cluster module, which PM2 uses, forks several worker processes on one machine that share one listening port; a primary process hands connections to the workers. It uses all cores on that machine, but every worker dies with the machine, and it adds one more layer of process management. Multiple containers, one Node process each, spread across many machines and let the orchestrator do restarts, scaling, health checks and rolling updates. In Kubernetes or any container platform, prefer one process per container and scale the number of containers.

Cluster module or PM2One process per container
Spread acrossCores of one machineCores of many machines
RestartsPrimary process or PM2Orchestrator
Health checks per processLimitedPer container
Rolling deploypm2 reloadBuilt into the orchestrator
Resource limits per processSharedCPU and memory limits per container
Use whenPlain VMsKubernetes, ECS, Cloud Run

Do not combine them blindly. Running PM2 cluster mode inside a container that Kubernetes also scales hides crashes from Kubernetes and makes CPU limits confusing. The details of Node's options are in worker threads, cluster and child processes.

11. How do you handle one million API requests per day? ​

The short answer

Start with the arithmetic: one million a day is about 12 requests per second on average, perhaps 50 to 100 at peak. A single well-written Node instance handles hundreds to thousands of simple requests per second, so the volume itself is modest. The real work is making it reliable: at least two instances behind a load balancer, a connection pool to an indexed database, caching for hot reads, timeouts and rate limits, background queues for slow work, and monitoring of p99 latency and errors. Then design so you can add instances when traffic grows tenfold.

The numbers, step by step:

  1. A day has 86,400 seconds.
  2. 1,000,000 ÷ 86,400 ≈ 11.6 requests per second on average.
  3. Traffic is uneven; peaks are often 3 to 10 times the average, so about 35 to 115 per second.
  4. If each request takes 50 ms of server time, one process handling one request at a time would manage 20 per second. Node is not one at a time: while one request waits on the database, others run. Realistic capacity per Node process for I/O-bound JSON APIs is in the hundreds to low thousands per second.

The design that answers the question:

ConcernChoice
Availability2 or more instances in two zones, behind a load balancer
DatabasePostgreSQL with indexes, a pool of 10 to 20 per instance, a read replica if reads dominate
CachingRedis for hot reads, CDN for public responses
Slow workQueue it: emails, image processing, reports
ProtectionTimeouts, rate limits per client, request size limits
Visibilityp95 and p99 latency, error rate, saturation dashboards and alerts
GrowthStateless instances, autoscaling on CPU or request rate

Saying the number out loud is what interviewers listen for. "A million a day is only about 12 per second" shows you check before you over-engineer.

12. How would you scale a Node.js application horizontally? ​

The short answer

Make every instance stateless and interchangeable, then run many behind a load balancer and add or remove them by load. Concretely: move sessions to Redis or use tokens; store uploads in object storage, not local disk; use a shared cache instead of in-process caches for anything that must be consistent; make scheduled jobs run once, with a lock or a separate worker; keep WebSocket fan-out in a shared broker; read config from the environment; add health checks and graceful shutdown; and autoscale on CPU, request rate or queue length. Then watch the shared parts, usually the database, which become the next bottleneck.

load balancernode instancenode instancenode instancenode instanceno state inside instances; everything shared lives belowRedissessions, cacheobject storageuploadsPostgreSQLdataqueue + workersslow jobs, cron
Identical boxes, shared state outside. Because no instance keeps anything another needs, the load balancer can send any request anywhere, and instances can be added or killed at any time.
State that breaks scalingMove it to
express-session memory storeRedis store (distributed sessions)
Files written to local diskObject storage (uploads)
In-process cache that must be consistentRedis, or short TTLs
setInterval cron in every instanceOne scheduler, a lock, or a queue
WebSocket clients on different instancesRedis Pub/Sub or a broker for fan-out
Rate-limit counters in memoryRedis (question 13)

13. How would you implement rate limiting in a distributed system? ​

The short answer

Keep the counters in a shared store, usually Redis, so all instances count against the same limit, and update them atomically, with INCR plus an expiry or a Lua script. Key the limit by API key, user or IP. Return 429 with Retry-After. Layer it: a coarse per-IP limit at the CDN or gateway, and per-user and per-endpoint limits in the app. Decide what happens when Redis is unavailable, usually allowing requests with a local fallback limit and an alert. The algorithms and code are covered in detail elsewhere in this guide.

The two detailed answers:

  1. For one Express service with many instances, express-rate-limit with a Redis store, and why in-memory counters fail: rate limiting across instances.
  2. The algorithms and a token bucket written as one atomic Redis script: a distributed rate limiter.
LayerLimitPurpose
CDN or WAFPer IP, coarseStop floods before they reach you
API gatewayPer API key or planQuotas and fair use
AppPer user and per sensitive routeBrute force, expensive endpoints

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Interview Prep
Production & Operations
Performance
Scaling
Edge
Deploying
Delivery
Observability
Other topics
Nothing picked yet