Skip to content

Interview PrepCloud

Compute, Containers and Serverless ​

559. How do you choose between virtual machines, a container service, Kubernetes, and serverless functions? ​

The short answer

Each step up the ladder hands more work to the cloud and gives you less control. Virtual machines give full control of the operating system, and you patch and scale everything. Serverless container services run your image with almost no platform work. Kubernetes gives the most flexible platform for many services, but someone must run it. Functions run small pieces of code on events and scale to zero. Pick the most managed option that still meets your needs, and move down only for a concrete reason.

virtual machinesyou run the OS,patching, scalingEC2, Azure VMsKubernetesyou run the platformand its add-onsEKS, AKS, GKEserverless containersyou give an image;it runs and scalesContainer Apps, Cloud Runfunctionsyou give code;it runs per eventLambda, Azure Functionsless work for you, more for the providermore control, more flexibility
A ladder of handing work over. Going right, the provider takes over the operating system, then the platform, then the servers entirely. Going left, you gain control, and you pay for it in operations work.

A quick guide:

OptionChoose it whenWatch out for
Virtual machinesSpecial OS needs, licensed software, legacy appsPatching, scaling and recovery are yours
KubernetesMany services, a platform team, need for its extensionsRunning and upgrading the cluster (question 309)
Serverless containersWeb apps and APIs, small teams, spiky loadFewer controls, platform limits
FunctionsShort event-driven jobs, glue between servicesTime limits, cold starts, cost at steady high load
Platform services, PaaSA standard web app with no container needLess control of the runtime (question 566)

Questions that decide it:

  1. How many services, and who runs the platform? Two services and three developers rarely justify a Kubernetes cluster.
  2. How steady is the load? Spiky and often idle suits scale-to-zero options. Steady high load is cheaper on reserved machines or nodes.
  3. How long does work run? Functions have time limits, such as 15 minutes on AWS Lambda.
  4. What does it need from the OS? Special kernels, drivers, GPUs or licensed software point to virtual machines or Kubernetes.
  5. Portability. A container image runs on any of the container options, and can move between them later.

A sensible default for a new web service in a small team: a serverless container service. Move to Kubernetes when the number of services, or a specific need, makes it worth running.

560. What are autoscaling groups and virtual machine scale sets? ​

The short answer

They are managed groups of identical virtual machines built from one template. You set the minimum, maximum and desired number of machines, and the group keeps that many healthy ones running, replacing any that fail. Scaling rules add machines when load rises, for example when average CPU passes 60%, and remove them when it falls. A load balancer in front sends traffic to whichever machines are currently in the group. AWS calls them Auto Scaling groups, Azure calls them virtual machine scale sets.

loadbalancerscale set: min 2, max 10VM 1VM 2VM 3 newtemplate: image, size, start-up scriptevery VM is built from thisscaling ruleaverage CPUabove 60%: add onemetrics
Many identical machines, one set of rules. Every machine comes from the same template. Metrics drive the rule, the rule changes the count, and the load balancer always sends traffic to whichever machines exist now.

The parts:

  1. A template, called a launch template in AWS and a model in Azure: the machine image, size, network, identity and a start-up script.
  2. Size limits: minimum, maximum and desired count.
  3. Health checks: from the load balancer or the machine itself. An unhealthy machine is replaced.
  4. Scaling rules: when to add or remove machines.
  5. Zones: spread across availability zones, so losing a zone leaves machines in the others.

Kinds of scaling rule:

KindHow it worksExample
Target trackingKeep a metric near a valueKeep average CPU around 50%
Step or thresholdAdd or remove when a metric crosses a lineAbove 70% CPU for 5 minutes: add 2
ScheduledChange counts at set times10 machines from 8 a.m. on weekdays
PredictiveLearn the daily pattern and scale aheadAdd machines before the morning rush

What makes this work: stateless machines. Any machine may be removed at any time, so nothing important may live only on it. Sessions go to a shared store, files go to object storage, and logs are shipped out (question 571).

Updating the template does not change running machines on its own. A rolling update, called an instance refresh in AWS, replaces machines in batches, much like a Kubernetes rolling update (question 350).

561. What are spot or preemptible instances, and how do you use them safely? ​

The short answer

Spot instances are the cloud's spare capacity sold at a deep discount, often 60 to 90% off, with one condition: the provider can take them back when it needs the capacity, with only a short warning. AWS gives two minutes, while Azure and Google Cloud give about 30 seconds. Use them for work that can be interrupted and restarted, such as batch jobs, CI runners and stateless services spread over many machines, and never as the only home of something that must stay up.

ProviderNameWarning before removal
AWSSpot InstancesAbout 2 minutes
AzureSpot Virtual MachinesAs little as 30 seconds
Google CloudSpot VMs, formerly preemptibleAbout 30 seconds

Good uses:

  • Batch and data processing, where a lost job is simply retried.
  • CI runners: a cancelled build reruns elsewhere.
  • Stateless web services, as part of a mix with normal machines.
  • Kubernetes node pools for workloads that tolerate interruption (question 612).

Bad uses: a single database, a single server, long jobs with no checkpoints, anything holding state only in memory.

Using them safely:

  1. Mix them. Keep a base of normal on-demand or reserved machines, and add spot capacity on top.
  2. Diversify. Ask for several machine sizes and several zones, so one size running out does not take all your capacity.
  3. Watch for the warning. The metadata service announces the coming removal (question 550). Use the time to stop taking new work, finish or save current work, and leave the load balancer.
  4. Checkpoint long work, so a restarted job continues instead of starting over.
  5. In Kubernetes, drain the node on warning, and use PodDisruptionBudgets so not every copy of a service goes at once (question 400).

562. What are serverless container services such as Azure Container Apps, AWS Fargate, and Google Cloud Run? ​

The short answer

They run your container image without you running any servers or cluster. You give the image, the CPU and memory it needs, and scaling rules, and the service runs as many copies as needed, gives them an HTTPS address, and handles patching of everything underneath. Cloud Run and Container Apps can scale to zero when idle and scale on requests or queue length. Fargate is the serverless way to run containers for AWS ECS, and EKS on AWS, where you still define tasks and services but no machines.

What you provide, and what the service does:

You provideThe service does
A container imagePulls and runs it
CPU and memory per copyFinds capacity for it
Scaling rulesAdds and removes copies
Environment, secrets, identityInjects them at start-up
Port and health checksGives an HTTPS address, routes traffic, restarts failures

How the three differ:

Azure Container AppsGoogle Cloud RunAWS Fargate with ECS
Scale to zeroYesYesNo; a service keeps its set count
Scales onHTTP, queues, events, through KEDARequests, CPUCPU, memory, custom metrics
Revisions and traffic splitYesYesThrough deployments and load balancer rules
Built onKubernetes, hidden from youGoogle's own platformAWS's own platform

KEDA, Kubernetes Event-driven Autoscaling, scales on outside events such as queue length (question 496). Container Apps uses it underneath.

Why it fits many teams: it gives most of what people use Kubernetes for, which is rolling out images, scaling, HTTPS and health checks, without a cluster to upgrade (question 621).

Where it falls short: very specific networking, custom operators, privileged or GPU-heavy workloads in some services, and dozens of services that need a shared platform with its own policies.

563. What is serverless computing, and how do functions like AWS Lambda and Azure Functions work? ​

The short answer

Serverless means you run code without managing any servers, and you pay only while the code runs. With functions, you upload a small piece of code and connect it to a trigger, such as an HTTP request, a new file in storage, a message on a queue, or a timer. When the event happens, the platform starts an instance of your function, passes it the event, and runs it. Many events at once mean many instances in parallel, and no events means no instances and no cost.

HTTP requestfile uploadedqueue messagetimer, every hourfunctions platforminstanceinstanceinstancemore if neededdatabase,other services
Events in, instances on demand. Each trigger delivers an event. The platform starts instances as needed, runs the code once per event or batch, and removes idle instances afterwards.

An example: an order pipeline. When a customer pays, the payment service puts a message on a queue. A function picks it up and sends the confirmation email. Nobody runs a server for this, and at 3 a.m., when nobody orders, it costs nothing.

How billing works. You pay per request and per unit of compute time, measured as memory size multiplied by run time, rounded to the millisecond on Lambda. A function with 512 MB that runs for 200 ms is billed for 0.1 gigabyte-seconds.

Key limits, worth knowing by heart:

AWS LambdaAzure Functions
Longest single run15 minutesDepends on the plan: minutes on consumption plans, longer on premium or dedicated plans
Memory128 MB to 10 GB; CPU grows with memoryDepends on the plan
Scale to zeroYesYes, on consumption-style plans

Functions should be stateless. An instance may be reused for the next event, or thrown away at any moment. Anything that must last goes to a database, storage or a queue.

Instances are reused. After a run, the platform keeps the instance warm for a while. Code outside the handler, such as creating a database client, runs once per instance and is then reused, which is why you put set-up there (question 564).

564. What is a cold start, and how do you reduce it? ​

The short answer

A cold start is the extra delay when a function, or a scaled-to-zero container, has no warm instance ready. Before it can handle the event, the platform must find capacity, load your code or image, start the runtime, and run your set-up code. That can add from about a hundred milliseconds to several seconds. Reduce it by keeping packages small, doing less at start-up, choosing faster-starting runtimes, and paying to keep some instances always warm, with provisioned concurrency, minimum instances or premium plans.

coldfind capacityload the codestart runtimeyour set-up codehandle the eventwarmhandle the eventred: the platform's work; amber: shaped by your code and runtime; green: the real work
Where the time goes. A cold start pays for every step before the real work. A warm instance skips straight to handling the event.

When it happens:

  • The first request after a quiet period.
  • A sudden burst, when many new instances start at once.
  • After a new deployment, since old warm instances are replaced.

Ways to reduce it, cheapest first:

  1. Smaller packages and images. Less to download and load (question 72).
  2. Less work at start-up. Create clients lazily, load only what is needed, avoid big frameworks for tiny functions.
  3. Faster runtimes. Node.js, Python and Go usually start faster than the JVM or .NET without tuning. Lambda SnapStart saves a snapshot of an initialised instance for Java, Python and .NET, and resumes from it.
  4. More memory on Lambda, because CPU grows with memory, so start-up code runs faster.
  5. Keep instances warm, for a fee: Lambda provisioned concurrency, Azure Functions always-ready or pre-warmed instances, Cloud Run and Container Apps minimum instances.

Is it worth fixing? For a nightly batch job, a two-second cold start does not matter. For an API that users wait on, it does. Measure the slowest requests, the 99th percentile, not the average, because cold starts hide in the tail (question 629).

565. When should you not use serverless functions? ​

The short answer

Avoid functions for work that runs longer than their time limits. Avoid them for steady high load, where always-on compute is cheaper, and for workloads that need very low latency on every single request. They also fit badly with long connections and in-memory state. And hundreds of tiny functions can be harder to understand than one service. Also be careful that unlimited scaling does not overwhelm a database that cannot scale with it.

SituationWhy functions struggleBetter choice
Jobs longer than 15 minutesHard time limitsContainers, batch services
Steady high traffic all dayPer-run pricing costs more than always-onContainers, reserved machines
Strict latency on every requestCold starts in the tailAlways-warm containers
WebSockets, long connectionsShort-lived instancesContainers, managed WebSocket services
In-memory caches or stateInstances come and goA service with a shared cache
Very many tiny functionsHard to trace, deploy and reason aboutA few well-shaped services

The database trap. A traffic spike makes the platform start 1,000 instances. Each opens a database connection. The database, sized for 200 connections, falls over. Fix it with a connection pooler or proxy, such as RDS Proxy, by limiting the function's concurrency, or by putting a queue in between so work is processed at a steady rate.

The cost crossover. Functions are very cheap for spiky or low traffic. As traffic becomes steady and high, a few always-on containers become cheaper. Do the arithmetic with real numbers before deciding, and revisit it as traffic grows.

Where functions shine: glue between services, file processing on upload, scheduled jobs, webhooks, and spiky low-volume APIs.

566. What are platform services such as Azure App Service and AWS Elastic Beanstalk, and when do they fit? ​

The short answer

They are platform as a service offerings for web apps: you hand over your code, or a container, and the platform provides the servers, runtime, load balancing, scaling, TLS certificates, deployment slots and logs. They fit standard web apps and APIs where the team wants to focus on code, not infrastructure. You give up some control over the operating system and networking, and very custom or multi-service systems may fit containers or Kubernetes better.

What you get, for example with Azure App Service:

  • Runtimes ready to use: .NET, Node.js, Python, Java, PHP, or your own container.
  • Scaling: more instances by rule or schedule.
  • Custom domains and managed TLS certificates.
  • Deployment slots: deploy to a "staging" slot, warm it up, then swap it with production instantly. Swapping back is the rollback, a form of blue-green deployment (question 281).
  • Built-in authentication, logs and health checks.

AWS Elastic Beanstalk gives a similar experience on top of normal AWS resources: it creates the machines, load balancer and scaling group for you, from your application bundle.

Platform serviceServerless containersKubernetes
You provideCode or a containerA containerContainers and manifests
Platform workVery littleVery littleA lot
FlexibilityLowestMediumHighest
Good forClassic web appsModern web apps and APIsMany services, special needs

A practical note. The lines are blurring. App Service runs containers, and serverless container services run web apps. Choose by how much control you need and what your team already knows.

567. What is immutable infrastructure, and what are golden images? ​

The short answer

Immutable infrastructure means you never change a running server. To update anything, you build a new image, start new servers from it, move traffic to them, and remove the old ones. A golden image is a pre-built machine image with the operating system, patches, agents and settings already in place, built automatically from code with tools such as Packer, and used as the starting point for every server. Every server is then identical and reproducible, and a rollback is simply returning to the previous image.

The old way, and its problem. Servers were updated in place: log in, install a patch, change a setting. Over months, each server became slightly different. This is called configuration drift. Nobody could rebuild a server exactly, and "it works on server 3 but not on server 4" became normal.

The immutable way:

  1. Build an image from code: a base OS, plus patches, security agents and your settings.
  2. Scan and test the image, and give it a version.
  3. Roll out new machines from it, through the scale set's rolling update (question 560).
  4. Remove old machines.
  5. To roll back, deploy the previous image version.

This is exactly how containers work. An image is built once and never changed while running. Golden images apply the same idea to whole virtual machines (question 28).

Tools:

ToolDoes
HashiCorp PackerBuilds images for any cloud from a template
EC2 Image BuilderAWS's managed image pipeline
Azure VM Image Builder, Compute GalleryAzure's image building and sharing

Baking versus frying. Baking puts everything into the image in advance, so machines start fast and identical. Frying installs things at start-up with a script, which is more flexible but slower and can fail on start-up. Most teams bake the slow, stable parts, and fry only the small environment-specific parts (question 569).

568. How do you choose a machine size or instance type? ​

The short answer

Machine types are grouped into families by their balance of CPU and memory: general purpose, compute-optimised, memory-optimised, storage-optimised, and accelerated ones with GPUs. Start from what the app actually uses, measured under realistic load, pick the family that matches its shape, and pick the smallest size that leaves sensible headroom. Then keep resizing as real usage data comes in, because most cloud machines are bigger than they need to be.

FamilyBalanceGood forAWS exampleAzure example
General purposeEven CPU and memoryWeb servers, most appsMD
Compute optimisedMore CPU per GBBatch, encoding, busy APIsCF
Memory optimisedMore memory per CPUDatabases, caches, analyticsRE
Storage optimisedFast local disksSearch, some databasesIL
AcceleratedGPUs or other chipsMachine learning, graphicsP, GN
BurstableLow baseline, short burstsSmall, mostly idle machinesTB

Burstable machines run at a low baseline and save up "CPU credits" while idle, spending them on short bursts. They are cheap for small, quiet workloads, and a nasty surprise if the load is steady, because the credits run out and the machine slows down.

How to choose, step by step:

  1. Measure CPU, memory, disk and network under realistic load.
  2. Match the shape. An app using 2 CPUs and 14 GB of memory is memory-heavy: a memory-optimised type fits better than a bigger general one.
  3. Pick the current generation. Newer generations usually give more performance for the same price.
  4. Consider ARM processors, such as AWS Graviton or Azure Cobalt. They are often cheaper for the same work, if your software and images support ARM (question 82).
  5. Leave headroom, then resize with data. Cloud tools such as AWS Compute Optimizer and Azure Advisor suggest right-sizing.

For Kubernetes nodes, the same thinking applies to node pools, plus one more factor: larger nodes waste less on system overhead per pod, but losing one hurts more (question 612).

569. How do start-up scripts like cloud-init and user data configure a new machine? ​

The short answer

When a cloud machine boots for the first time, it can run a script or configuration you provide, called user data in AWS and custom data in Azure. On Linux, a tool called cloud-init reads it and does the work: installing packages, writing files, creating users, and starting services. It is how a generic image becomes a specific server, for example joining a cluster or fetching configuration. Keep it small and quick, and put the slow, stable parts in the image instead.

An example, cloud-init configuration for a worker that processes orders:

yaml
#cloud-config
package_update: true
packages:
  - docker.io
write_files:
  - path: /etc/order-worker/env
    content: |
      QUEUE_NAME=orders
      REGION=westeurope
runcmd:
  - systemctl enable --now docker
  - docker run -d --restart=always --env-file /etc/order-worker/env registry.shop.example/order-worker:1.8.2

The first line, #cloud-config, tells cloud-init what kind of content this is. Then, in order:

  1. package_update and packages refresh the package list and install Docker.
  2. write_files writes the worker's settings to a file.
  3. runcmd runs commands at the end: start Docker, then start the worker container, restarted automatically if it stops.

Good habits:

  • No secrets in user data. It can be read from the machine's metadata by anything on the machine, and often by anyone who can view the machine's settings in the cloud console. Fetch secrets at runtime with the machine's identity (question 553).
  • Keep it short. Every minute here is a minute before the new machine can take traffic, which matters a lot when scaling up during a spike.
  • Bake the stable parts into a golden image, such as Docker and agents (question 567).
  • Logs are in /var/log/cloud-init-output.log when something goes wrong.

570. How do you handle retries and failures in event-driven serverless code? ​

The short answer

Event platforms deliver each event at least once, so your function may see the same event twice, and they retry automatically when it fails. So make every function idempotent: processing the same event twice must have the same effect as processing it once. Report failures per message rather than failing a whole batch, send events that keep failing to a dead-letter queue after a few attempts, and alert on that queue. Never let a broken message block the ones behind it for ever.

Idempotent means doing it twice gives the same result as doing it once. Setting an order's status to "paid" is idempotent. Adding £10 to a balance is not.

An example: an AWS Lambda function processing order messages from an SQS queue, in TypeScript:

ts
import type { SQSEvent, SQSBatchResponse } from 'aws-lambda'
import { markOrderPaid, alreadyProcessed, rememberProcessed } from './orders'

export async function handler(event: SQSEvent): Promise<SQSBatchResponse> {
  const failures: { itemIdentifier: string }[] = []

  for (const record of event.Records) {                         // (1)
    const { orderId, paymentId } = JSON.parse(record.body)
    try {
      if (await alreadyProcessed(paymentId)) continue           // (2)
      await markOrderPaid(orderId, paymentId)
      await rememberProcessed(paymentId)
    } catch (err) {
      console.error('failed', { orderId, paymentId, err })
      failures.push({ itemIdentifier: record.messageId })        // (3)
    }
  }

  return { batchItemFailures: failures }                         // (4)
}
  1. A batch. The queue delivers several messages in one call, here up to ten.
  2. Skip duplicates. If this payment was already handled, do nothing. That makes a repeated delivery harmless. In real code, the check and the update should happen in one database transaction, or use a unique constraint on paymentId, so two copies running at once cannot both pass the check.
  3. Record only the failed message. One bad message should not force the good ones to be retried.
  4. Report partial failure. With "report batch item failures" turned on, only the listed messages go back to the queue to be retried. It prints nothing for the successful ones.
orders queueat-least-oncefunctionidempotentdead-letter queueafter 3 attemptsalerta person looks① deliver② failed: retry later③ too many failures④
Retry, then park. ① The queue delivers messages. ② Failed ones return and are retried after a delay. ③ After a set number of attempts, the message moves to a dead-letter queue. ④ An alert tells a person, who fixes the cause and replays it.

Why "at least once"? Guaranteeing exactly one delivery across machines that can fail is very hard. Platforms choose to deliver again when unsure, and leave duplicate handling to you. That is why idempotency matters.

Good habits:

  1. A dead-letter queue on every queue and asynchronous trigger, with an alarm when it is not empty.
  2. Retries with backoff: wait longer between each attempt, so a struggling dependency can recover.
  3. A visibility timeout longer than the function's run time. Otherwise the queue thinks the message was lost and hands it to another instance while the first is still working.
  4. Replay tooling: once the bug is fixed, move messages from the dead-letter queue back to the main queue.
  5. Logs with IDs, such as orderId and paymentId, so one message can be followed through every step (question 634).

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Interview Prep
Docker, CI/CD, Kubernetes & Cloud
Your project
Docker
CI/CD
Kubernetes
Cloud
Observability
Scenarios
Other topics
Nothing picked yet