Appearance
Scaling Node.js
7. What are vertical scaling and horizontal scaling?
The short answer
Vertical scaling, scaling up, means giving one machine more power: more CPU, memory or faster disks. It is simple, needs no code changes, but has a ceiling, gets expensive, and the machine stays a single point of failure. Horizontal scaling, scaling out, means adding more machines or instances behind a load balancer. It has almost no ceiling and survives the loss of an instance, but the app must be stateless: sessions, uploads and caches must live in shared stores, not in one process's memory. Web servers usually scale out; databases usually scale up first.
| Vertical | Horizontal | |
|---|---|---|
| How | Bigger machine | More machines |
| Limit | The biggest machine you can buy | Practically none |
| Survives one machine failing | No | Yes |
| Code changes | None | App must be stateless |
| Downtime to scale | Often a restart | None; add instances |
| Typical for | Databases, first steps | Web and API servers, workers |
Node specifically. One Node process runs JavaScript on one core. On an 8-core machine, a single process leaves 7 cores idle for JavaScript. So even "vertical" scaling for Node means running several processes on the machine (question 10).
8. What is a load balancer, and how does it distribute traffic?
The short answer
A load balancer sits in front of several app instances and spreads incoming requests across them, so no single instance is overloaded, and it removes unhealthy instances using health checks, so users are not sent to a broken one. Common algorithms are round robin, each instance in turn; least connections, the instance with the fewest open requests; weighted, more traffic to bigger instances; and hashing, the same client or key always to the same instance. Layer 4 balancers route TCP connections by address and port; layer 7 balancers understand HTTP and can route by path, host or header.
| Algorithm | Picks | Good when |
|---|---|---|
| Round robin | Next instance in turn | Requests cost about the same |
| Least connections | Fewest open requests | Request times vary a lot |
| Weighted | In proportion to capacity | Instances of different sizes |
| Hash of IP or key | Same instance for same client | Sticky sessions, local caches |
| Layer 4 | Layer 7 | |
|---|---|---|
| Sees | IP addresses and ports | Full HTTP: path, host, headers, cookies |
| Can route by URL | No | Yes |
| TLS | Passes through, or terminates | Terminates and inspects |
| Examples | AWS NLB | AWS ALB, Nginx, Envoy |
9. What is PM2, and why is it used?
The short answer
PM2 is a process manager for Node on a plain server or VM. It starts your app, restarts it if it crashes, runs several copies in cluster mode to use every CPU core, does zero-downtime reloads by restarting copies one at a time, starts apps on boot, rotates logs, and shows CPU and memory per process. It suits apps deployed directly on VMs. In containers and Kubernetes, the orchestrator already restarts and scales processes, so you usually run one plain node process per container instead of PM2.
js
// ecosystem.config.cjs
module.exports = {
apps: [{
name: 'orders-api',
script: 'dist/server.js',
instances: 'max', // (1)
exec_mode: 'cluster',
max_memory_restart: '600M', // (2)
kill_timeout: 10000, // (3)
env: { NODE_ENV: 'production', PORT: 3000 },
}],
}bash
pm2 start ecosystem.config.cjs
pm2 reload orders-api # zero-downtime: restarts workers one by one
pm2 startup && pm2 save # start again after a reboot- One process per CPU core, sharing port 3000 through Node's cluster module.
- Restart a process that grows past 600 MB, a safety net for a slow leak, not a fix.
- Give each process 10 seconds to finish in-flight requests before it is killed (graceful shutdown).
10. How does Node.js clustering differ from using multiple containers or processes?
The short answer
The cluster module, which PM2 uses, forks several worker processes on one machine that share one listening port; a primary process hands connections to the workers. It uses all cores on that machine, but every worker dies with the machine, and it adds one more layer of process management. Multiple containers, one Node process each, spread across many machines and let the orchestrator do restarts, scaling, health checks and rolling updates. In Kubernetes or any container platform, prefer one process per container and scale the number of containers.
| Cluster module or PM2 | One process per container | |
|---|---|---|
| Spread across | Cores of one machine | Cores of many machines |
| Restarts | Primary process or PM2 | Orchestrator |
| Health checks per process | Limited | Per container |
| Rolling deploy | pm2 reload | Built into the orchestrator |
| Resource limits per process | Shared | CPU and memory limits per container |
| Use when | Plain VMs | Kubernetes, ECS, Cloud Run |
Do not combine them blindly. Running PM2 cluster mode inside a container that Kubernetes also scales hides crashes from Kubernetes and makes CPU limits confusing. The details of Node's options are in worker threads, cluster and child processes.
11. How do you handle one million API requests per day?
The short answer
Start with the arithmetic: one million a day is about 12 requests per second on average, perhaps 50 to 100 at peak. A single well-written Node instance handles hundreds to thousands of simple requests per second, so the volume itself is modest. The real work is making it reliable: at least two instances behind a load balancer, a connection pool to an indexed database, caching for hot reads, timeouts and rate limits, background queues for slow work, and monitoring of p99 latency and errors. Then design so you can add instances when traffic grows tenfold.
The numbers, step by step:
- A day has 86,400 seconds.
- 1,000,000 ÷ 86,400 ≈ 11.6 requests per second on average.
- Traffic is uneven; peaks are often 3 to 10 times the average, so about 35 to 115 per second.
- If each request takes 50 ms of server time, one process handling one request at a time would manage 20 per second. Node is not one at a time: while one request waits on the database, others run. Realistic capacity per Node process for I/O-bound JSON APIs is in the hundreds to low thousands per second.
The design that answers the question:
| Concern | Choice |
|---|---|
| Availability | 2 or more instances in two zones, behind a load balancer |
| Database | PostgreSQL with indexes, a pool of 10 to 20 per instance, a read replica if reads dominate |
| Caching | Redis for hot reads, CDN for public responses |
| Slow work | Queue it: emails, image processing, reports |
| Protection | Timeouts, rate limits per client, request size limits |
| Visibility | p95 and p99 latency, error rate, saturation dashboards and alerts |
| Growth | Stateless instances, autoscaling on CPU or request rate |
Saying the number out loud is what interviewers listen for. "A million a day is only about 12 per second" shows you check before you over-engineer.
12. How would you scale a Node.js application horizontally?
The short answer
Make every instance stateless and interchangeable, then run many behind a load balancer and add or remove them by load. Concretely: move sessions to Redis or use tokens; store uploads in object storage, not local disk; use a shared cache instead of in-process caches for anything that must be consistent; make scheduled jobs run once, with a lock or a separate worker; keep WebSocket fan-out in a shared broker; read config from the environment; add health checks and graceful shutdown; and autoscale on CPU, request rate or queue length. Then watch the shared parts, usually the database, which become the next bottleneck.
| State that breaks scaling | Move it to |
|---|---|
express-session memory store | Redis store (distributed sessions) |
| Files written to local disk | Object storage (uploads) |
| In-process cache that must be consistent | Redis, or short TTLs |
setInterval cron in every instance | One scheduler, a lock, or a queue |
| WebSocket clients on different instances | Redis Pub/Sub or a broker for fan-out |
| Rate-limit counters in memory | Redis (question 13) |
13. How would you implement rate limiting in a distributed system?
The short answer
Keep the counters in a shared store, usually Redis, so all instances count against the same limit, and update them atomically, with INCR plus an expiry or a Lua script. Key the limit by API key, user or IP. Return 429 with Retry-After. Layer it: a coarse per-IP limit at the CDN or gateway, and per-user and per-endpoint limits in the app. Decide what happens when Redis is unavailable, usually allowing requests with a local fallback limit and an alert. The algorithms and code are covered in detail elsewhere in this guide.
The two detailed answers:
- For one Express service with many instances,
express-rate-limitwith a Redis store, and why in-memory counters fail: rate limiting across instances. - The algorithms and a token bucket written as one atomic Redis script: a distributed rate limiter.
| Layer | Limit | Purpose |
|---|---|---|
| CDN or WAF | Per IP, coarse | Stop floods before they reach you |
| API gateway | Per API key or plan | Quotas and fair use |
| App | Per user and per sensitive route | Brute force, expensive endpoints |