Appearance
13.1 — The Cloud Mental Model
The cloud is other people's computers, rented to you over the internet. A cloud provider is a company that owns huge buildings full of computers. The three largest are Amazon Web Services (AWS), Microsoft Azure and Google Cloud. You rent a share of their computers, you pay by the second, and you hand them back when you no longer need them. You never see the machines. You control them through a website called the console, or by sending commands from your own laptop.
A computer that does work for other programs is called a server. Here is what goes wrong when a team rents servers without knowing what is underneath.
A team rents three servers for its website. It wants three so that if one breaks, the other two keep the site running. Keeping spare copies like this is called redundancy. When you create a server, the console asks where to put it and offers a default answer. The team accepts the default three times. So all three servers land in the same group of buildings. The provider calls such a group an availability zone, and section 1 defines it properly. One night the power fails in that zone, and all three servers go down together.
Nothing was misconfigured. They bought three copies of a machine and believed they had bought resilience. Those are different things. Resilience means the system keeps working when a part of it fails. Three copies in one place share the same power and the same cooling, so one failure takes all three. Resilience needs copies that fail separately. To get that, you have to know how the buildings underneath the console are laid out. Almost every rule in this Part comes from that layout.
1. What is physically there
A data centre is a building full of servers. The servers sit in tall metal frames called racks. The building has more than one power line coming in, so it survives the loss of one. It has batteries and diesel generators for when all the power lines fail. It has cooling, because thousands of computers in one room produce a great deal of heat. And it has optical fibre cables to the internet, entering from more than one direction. There is nothing exotic in it. It is a warehouse with very good electricity.
An availability zone is one or more data centres that share a failure domain. A failure domain is the set of things that break together. Inside one zone, the buildings share power, cooling and a site, so if a transformer fails or a cooling system floods, everything in that zone is at risk at the same moment. Each zone is built so that its power and cooling are separate from every other zone.
A region is a cluster of availability zones in one geography, typically three or more, physically separated by kilometres but linked by dedicated high-bandwidth fibre.
That last sentence is the whole design, so take it in two halves.
Far enough apart. The zones are kilometres apart, so one flood, fire or substation failure cannot take out two of them.
Close enough together. Light in fibre covers about 200 kilometres in a millisecond. So at tens of kilometres, a message goes from one zone to another and back in one to two milliseconds. That is fast enough for synchronous replication. Replication means keeping a second copy of your data on another machine. Synchronous means the first machine waits for the second before it carries on. So your database can wait for a second zone to confirm every write before telling the user "saved", and the user does not notice the wait (Chapter 10.5).
Put the halves together and you get the cloud's whole promise of high availability. A copy in another zone fails separately, and it is close enough to be kept exactly up to date.
Three rules follow directly from this layout.
Spread your copies across zones within one region. This is how you get availability, which means the system is up when a user wants it. It is cheap, the copies are only a millisecond or two apart, and it survives the most common large failure, which is the loss of one zone. The team in the opening story needed exactly this. One server in each of three zones would have kept the site running.
Spread across regions only when you have one of three specific reasons. The first reason is disaster recovery: you want to survive the loss of a whole region. The second is data residency: a law says that some data, such as the records of German customers, must be stored inside a particular country. The third is latency, which is the time a user waits for a reply. A user in Sydney gets faster replies from a server in Sydney than from one in London. Without one of these reasons, stay in one region. A message between regions takes tens to hundreds of milliseconds, which is too slow for synchronous replication. And the provider charges for traffic between regions.
A region can fail as a whole. Each region has its own control plane. That is the provider's own software that creates, starts, stops and connects your resources in that region. Regional control planes have failed in the past. When one does, a system that lives in a single region has no way to repair itself from the inside.
Two related terms. Edge locations, also called points of presence, are small facilities placed close to users. They do not run your servers. They hold copies of your files, so a user's request is answered from nearby (Chapter 13.4). A provider has hundreds of edge locations and only dozens of regions. Local zones put a small number of servers close to one large city. They are for the few applications where even a region's distance makes the reply too slow.
2. The service models
A provider sells its computers in four forms. They differ in how much of the work the provider does for you. Take one example through all four: you want to run an online shop.
IaaS stands for infrastructure as a service. The provider rents you the raw parts: virtual machines, disks and networks. A virtual machine is a share of a real server that behaves like a complete computer of your own. You get an empty machine. You install the operating system updates, the database and the shop's code, and you keep all of it running. Everything from the operating system upwards is your job.
PaaS stands for platform as a service. You hand over only the shop's code. The platform finds a machine, installs what the code needs, starts it, and restarts it when it crashes. You never log in to a server. Azure App Service, AWS App Runner and Google Cloud Run are platforms of this kind.
SaaS stands for software as a service. You do not write a shop at all. You sign up for a finished one, such as Shopify, and use it through your browser. The vendor runs everything. Gmail and Slack are SaaS too.
FaaS stands for functions as a service, and it is the main form of what people call serverless. You hand over one small function, for example "work out the delivery price for this basket". The platform runs it each time a request arrives and charges you only for those moments. There are still servers, but you never see or manage them. Chapter 13.3 covers it.
Each step up this ladder removes work from you, and it also removes control. On a virtual machine you can change anything. On a platform you can change only what the platform lets you change. The trade is usually a good one. A service that the provider runs for you is called a managed service, and a managed database or a managed load balancer is nearly always the right choice. (A load balancer shares incoming requests across several servers, Chapter 10.15.) The trade stops being good in two cases. The first is when you need the service to behave in a way it does not offer. The second is when you use so much of it that the extra price the provider charges for managing it becomes a large sum of money.
3. The shared responsibility model
When you rent from a provider, the two of you split the job of keeping things secure. This split is called the shared responsibility model. People often get it wrong, and that mistake is how most cloud data leaks happen.
The provider is responsible for security of the cloud. That covers the buildings, the hardware, and the network cables and switches between the machines. It covers the hypervisor, which is the software that slices one real server into several virtual machines and keeps them apart. And it covers the provider's own software inside each managed service.
You are responsible for security in the cloud. That covers your data, the rules for who may access it, your network rules and your application. It also covers patching, which means installing security updates, for every piece of software that you manage yourself.
Where the line sits depends on the service, and that is the part people miss:
| Service type | Provider handles | You handle |
|---|---|---|
| Virtual machine | Hardware, hypervisor | OS patching, firewall rules, application, data |
| Managed database | OS, engine patching, backups | Schema, access control, encryption choices, data |
| Serverless function | Everything below your code | Code, permissions, dependencies, data |
| SaaS | Almost everything | Access control and data |
Read down the last column. As you move down the table the provider takes over more of the work, but every row still ends with your data. Nobody else decides who is allowed to read it.
That explains the leaks you read about in the news every year. Providers offer object storage, a service that keeps files of any size. Each container of files is called a bucket (Chapter 13.4). In nearly every one of those leaks, the owner of a bucket set it to be readable by anyone on the internet, and somebody found it. Nobody broke into the provider's storage. The storage did exactly what its owner told it to do.
Two practical consequences follow.
A provider holds compliance certifications. These are documents from outside auditors saying that the provider meets a security standard. They cover the provider's half only. Your application does not become compliant because it runs on a certified provider.
And "managed" never means you can stop thinking about backups. A managed database does take automatic backups. But three questions are still yours. Are the backups kept for long enough? Has anyone ever tried restoring one? Can someone who steals your password delete them?
4. What "cloud native" actually means
People often say "cloud native" when they only mean "modern". The phrase has a real meaning: software designed for a place where any single machine may break, may be swapped for another, and may be added or removed at any moment.
In your own office a server might run for five years. In the cloud, the machine running your code can be replaced tonight, and the provider will not ask you first. Each running copy of your application is called an instance. Software that is built for the cloud expects its instances to come and go. That expectation leads to seven habits.
- Stateless where possible. State is anything a program remembers between one request and the next, such as who is logged in or what is in a basket. A stateless instance remembers nothing of its own. So any instance can answer any request, and losing one instance loses nothing (Chapter 10.2).
- State lives in managed services, not on the instance's own disk. The basket goes into a database. When the instance is replaced, the basket is still there.
- Configuration comes from the environment. Settings such as the database address are handed to the program when it starts. They are not written into the program itself. So the same build runs unchanged in testing and in production.
- Health checks and graceful shutdown. A health check is a small request the platform sends to ask "are you working?". Graceful shutdown means that an instance told to stop first finishes the requests it has already accepted. Together they let the platform move your instances around without users noticing (Chapter 9.9.7).
- Horizontal scaling. When load grows, you add more instances. You do not move to one bigger machine.
- Infrastructure as code. The servers and networks are described in files, so a whole environment can be rebuilt exactly (Chapter 13.8).
- Observability built in. The program sends out its logs and measurements as it runs. You cannot log in to a machine that no longer exists to find out what went wrong (Chapter 10.10).
"Cloud native" describes how the software is designed. It does not describe which tools run it. Suppose an application keeps its users' login sessions in a file on its own disk. When that machine is replaced, every user is logged out. That application is not cloud native, however modern the tools around it are.
Moving an application onto virtual machines without changing it is also a fair choice. It is called lift and shift. It is often the right first step, because it gets you out of your own data centre quickly. Just call it lift and shift, and do not call it cloud native.
5. On-premises versus cloud, without the marketing
Cloud advantages that are real:
On-premises means the servers sit in a building that your own company owns or rents. That was the only option before the cloud.
Elasticity. You pay for what you use, and you can add or remove machines in minutes. This is the biggest real change. On-premises, more capacity meant a purchase order and about eight weeks of waiting for delivery. In the cloud it is one command.
No capital expenditure. Capital expenditure is money spent up front to buy equipment. A small company no longer has to buy servers before it has a single customer, so it can attempt things it could never have afforded.
Managed services. A small team can run a database, a message queue and a load balancer that used to need a specialist for each.
Global reach. You can start servers in a region on another continent in minutes.
Cloud disadvantages that are equally real:
It is more expensive at steady high utilisation. Elasticity is the product, and if your load is flat and predictable, you are paying for flexibility you do not use. Several well-known companies have moved workloads back on-premises and published the arithmetic.
Egress charges. Egress means data leaving the provider's network. Sending data in is free. Sending data out is charged for every gigabyte. The providers set it up this way on purpose. Once you have stored a large amount of data with one provider, moving it all to another one is expensive.
Lock-in through managed services. Lock-in means you cannot leave without rebuilding. A plain virtual machine runs much the same on any provider. A system built on one provider's own queue, database, login service and functions does not, because the other providers' versions work differently.
Less control. You do not decide how fast the underlying hardware is. You do not decide when the provider does its maintenance. And when something goes wrong below the level you can see, you cannot investigate it yourself.
A sensible position is to use the cloud and know how you would leave. Where a portable choice costs you little, take it. For example, PostgreSQL is a database that every provider offers, so prefer it over a database that only one provider sells. Where a provider's own service is clearly better, use it, and accept on purpose that it ties you to them.
Multi-cloud means running the same system on two providers at once. Teams sometimes do this so that they can never be locked in. It usually costs more than the lock-in it avoids. You can only use the features that both providers have. And you must learn, run and pay for two of everything.
6. Storage performance, and IOPS
When you rent a disk in the cloud, you choose it by three numbers, and you pay according to them.
IOPS stands for input/output operations per second. One operation is one read or one write. IOPS matters when a program does many small reads and writes at scattered places on the disk. A database works this way, because the rows it needs are spread all over its files (Chapter 7.3.1).
Throughput is how many megabytes move per second. It matters when a program reads large files from start to finish. A job that scans a whole year of sales records works this way.
Latency is how long one single operation takes.
They are tied together by simple arithmetic. Throughput is IOPS multiplied by the size of each operation. A database reads in pages of 16 KB, so 3,000 IOPS of 16 KB pages moves 3,000 × 16 KB = 48 MB per second. A log archive reads in 1 MB chunks, so the same 3,000 IOPS would move 3 GB per second, which no volume will do. So a volume's throughput limit stops the archive long before its IOPS limit does.
Latency ties in the same way. If each operation takes 1 millisecond and your application sends one at a time, you can never exceed 1,000 IOPS, whatever the volume is rated for. To use 3,000 IOPS you must keep about three operations in flight at once. The number in flight is called the queue depth. So "the disk is slow" is often "the application only ever asks for one thing at a time".
The workload decides which number you buy. A database wants IOPS and low latency. A log archive wants throughput and cheap capacity.
The tiers you will meet:
- Local instance storage — physically attached, fastest, and it disappears when the instance stops. For caches and scratch space only.
- Network block storage — a virtual disk over the network. Survives the instance, and you provision IOPS and throughput.
- Network file storage — a shared filesystem several machines can mount.
- Object storage — HTTP-addressed blobs, effectively unlimited, cheapest, and not a filesystem (Chapter 13.4).
Watch out for burst credits. A cloud disk is called a volume. Many of the cheaper volume types have two speeds. They have a low guaranteed speed, called the baseline. They can also run much faster for a short time, which is called a burst. Each fast operation spends a credit from a saved-up balance. When the balance reaches zero, the volume drops back to the baseline. A database that was fast for three weeks and then became slow for good, with no code change, has nearly always run out of burst credits. It looks exactly like a mysterious bug in the application.
Here is how the credits work on AWS's older gp2 volume type, which is still common:
- The volume has a baseline of 3 IOPS for every GiB of size, and never less than 100. (A GiB, or gibibyte, is 1,024 × 1,024 × 1,024 bytes. It is about 7% larger than a gigabyte.) A 100 GiB volume has a baseline of 300 IOPS.
- It holds a bucket of up to 5.4 million credits. One credit pays for one operation above the baseline.
- While you are under the baseline, the bucket refills. While you are over it, the bucket drains.
- While the bucket has credits, the volume can burst up to 3,000 IOPS. When the bucket is empty, you are held at the baseline.
So the question "how long until my database falls off the cliff" is a division. The code below does it for the orders database:
ts
// How long can a 100 GiB gp2 volume run the orders database at 2,000 IOPS?
type Gp2Volume = { sizeGiB: number; creditBalance: number };
const MAX_CREDITS = 5_400_000; // (1)
const BURST_IOPS = 3_000;
function baselineIops(v: Gp2Volume): number {
return Math.min(16_000, Math.max(100, 3 * v.sizeGiB)); // (2)
}
function minutesUntilThrottled(v: Gp2Volume, demandIops: number): number {
const baseline = baselineIops(v);
if (demandIops <= baseline) return Infinity; // (3)
const served = Math.min(demandIops, BURST_IOPS);
const drainPerSecond = served - baseline; // (4)
return v.creditBalance / drainPerSecond / 60;
}
const ordersDisk: Gp2Volume = { sizeGiB: 100, creditBalance: MAX_CREDITS };
console.log("baseline IOPS:", baselineIops(ordersDisk));
console.log("minutes at 2,000 IOPS:", minutesUntilThrottled(ordersDisk, 2_000).toFixed(1));
console.log("minutes at 250 IOPS:", minutesUntilThrottled(ordersDisk, 250));(1) A full bucket holds 5.4 million credits, and a new volume starts full. (2) The baseline is 3 IOPS per GiB, never below 100 and never above 16,000. (3) If the database asks for less than the baseline, the bucket never drains, so the answer is "never". (4) While bursting, the bucket loses the difference between what you use and what you earn, once every second.
It prints:
baseline IOPS: 300
minutes at 2,000 IOPS: 52.9
minutes at 250 IOPS: InfinityRead what that means for a real system. The database runs a nightly job at 2,000 IOPS. For the first 53 minutes the job is fast. After that it is held at 300 IOPS, nearly seven times slower, until the load drops and the bucket refills. As data grows, the job runs longer each night, so one night it crosses the 53-minute line and "suddenly" becomes slow. Nothing in the code changed.
The fix is a volume type without credits. AWS's newer gp3 gives a flat 3,000 IOPS and 125 MiB/s at any size, with no bucket at all, and it is usually cheaper than gp2. Azure and Google have equivalent fixed-performance disk types.
7. Cost, as an engineering property
The size of a cloud bill depends on how the system is designed. So the engineers who design the system control the bill, far more than the finance team does.
You are charged for five kinds of thing.
- Compute time. You pay for each second a machine runs. The price drops sharply if you promise to keep using it, or if you let the provider take it back at short notice. Both are described just below.
- Storage. You pay for each gigabyte you keep for a month. Faster kinds of storage cost more.
- Data transfer. Data coming in is free. Data going out to the internet is charged. Data moving between zones is charged, and between regions it is charged more.
- Requests. Some services charge per million requests. Object storage and functions both do.
- The extra price for managed services. A managed database costs clearly more than a plain virtual machine on which you run the same database yourself. The provider is doing the backups, the updates and the repairs, so the extra is usually worth paying.
There are three ways to buy compute, and choosing well saves a lot of money.
On-demand is the full price with no promise. It is right when you cannot predict the load, and for anything you might switch off.
Reserved instances and savings plans are a promise. You commit to a certain amount of use for one or three years, and you save 30–70%. This is right for the steady part of your load. A simple test: anything you have run without a break for six months should be committed.
Spot instances, which Google calls preemptible, are the provider's spare machines at 60–90% off. The catch is that the provider can take one back with only a short warning. This is right for work that can be stopped and picked up again from a saved point. Long calculations that save their progress are a good fit, such as the training runs of Chapter 12.5.7. It is wrong for a server that answers users, because users would see the interruptions.
Four design decisions do most to decide the bill.
Traffic between zones. Suppose your system is many small services that call each other all day, and they are spread across three zones. Every gigabyte that crosses from one zone to another is charged. You are paying for your own services to talk to each other. The fix is zone-aware routing: each service first tries a copy of the other service in its own zone, and crosses zones only when that copy is down. This is often a large saving.
Data passing through a NAT gateway. Chapter 5.10 placed most servers in a private network, where nothing on the internet can reach them. For such a server to reach out to the internet, its traffic passes through a device called a NAT gateway. The provider charges for every gigabyte the NAT gateway handles. A job that pulls terabytes out of object storage through a NAT gateway is a well-known bill surprise. The fix is a VPC endpoint. It is a private doorway from your network straight to the provider's own services, so the data never touches the NAT gateway. The endpoint removes the charge completely (Chapter 5.10).
Storage tiers and lifecycle rules. Object storage comes in price tiers. The hot tier is for files you read often, and it is the most expensive to keep data in. A lifecycle rule moves or deletes files automatically as they age. Data that nobody has read in ninety days should not be sitting in the hot tier.
Idle resources. These are things you pay for and do not use. Typical examples are a disk that is attached to nothing, a load balancer everyone forgot, a stopped machine whose disk is still charged, and a test environment left running all weekend. Cleaning these up is dull work, and it is usually the largest saving you can make right away.
A worked example shows why the network lines surprise people. These are AWS's published prices in its largest region in 2026, and the other providers are similar:
| Item | Price |
|---|---|
| Traffic between zones | $0.01 per GB, charged on each side |
| NAT gateway, data processed | $0.045 per GB |
| NAT gateway, running | $0.045 per hour |
| Public IPv4 address | $0.005 per hour |
Now take an order service in one zone and a copy of its database, called a replica, in another zone. They exchange 50 GB a day. Each gigabyte is billed once as it leaves one zone and once as it arrives in the other, so it costs $0.02. That is 50 × $0.02 × 30 = $30 a month for one pair of services. A system of forty services that all talk to each other can easily reach thousands of dollars a month, and none of it shows up on the bill as "compute".
Next, a nightly job copies 1 TB a day from object storage through a NAT gateway. The NAT gateway charges $0.045 for each of the roughly 1,000 GB. That is $45 a day, about $1,350 a month, to move data that never left the provider. Now add the VPC endpoint described above, so the data skips the NAT gateway. For object storage the endpoint is free, so this charge falls to zero. One configuration change saves about $16,000 a year.
Tag everything, so that each team and each service can see what it costs. A tag is a label you attach to a resource, such as team=checkout. The bill can then be split by tag. FinOps is the name for the practice of managing cloud cost, and most of it is exactly this: showing each cost to the people who are able to change the design that causes it. Also set an alert for unusual spending. Then a job that has gone wrong and is burning money is caught on its first day, and not when the invoice arrives a month later.
What the interviewer will push on
"What is the difference between a region and an availability zone, and why does it matter?" A zone is a failure domain: its buildings share power, cooling and a site. A region is several zones separated by kilometres but linked with 1–2 ms fibre. That latency is what makes synchronous cross-zone replication possible, which is the whole basis of cloud high availability. Multi-zone for availability, multi-region only for disaster recovery, residency or user latency.
"Explain shared responsibility." The provider secures of the cloud and you secure in it. The line moves per service. Then the point that matters: every row ends with your data, which is why every public-bucket incident is a customer misconfiguration rather than a provider breach.
"When is the cloud the wrong choice?" When the load is steady, predictable and high, you are paying for elasticity you never use. It is also wrong when egress charges make up most of the bill. A strong answer adds two things. The cost of leaving is built into the pricing on purpose. And multi-cloud usually costs more than the lock-in it avoids.
"How would you reduce a cloud bill?" Give the steps in order of value.
- Delete idle resources.
- Commit the steady part of the load with savings plans.
- Move interruptible work to spot instances.
- Cut cross-zone and NAT gateway traffic with zone-aware routing and VPC endpoints.
- Apply storage lifecycle rules.
- Tag everything, so each cost reaches the team that can change it.
"Your database got slow after three weeks with no deploy. What do you check?" Check two things. The first is whether the storage volume has run out of burst credits. The second is whether the data the database reads most often has grown too large to fit in the memory the database uses as its cache, which Chapter 7.3.1 calls the buffer pool. Both problems build up gradually. Neither involves a code change. Both look like a mysterious bug.
One thing to volunteer: point out that three instances in one availability zone are three copies of a machine and give no redundancy. Add that this happens through a default setting, and nobody chooses it on purpose. It is the simplest example of why you have to understand the physical layout underneath the console.
Recall
- A data centre is a building. An availability zone is a shared failure domain. A region is several zones kilometres apart with 1–2 ms fibre between them. That is far enough to fail independently and close enough for synchronous replication. Every availability rule follows from that.
- Multi-zone for availability; multi-region only for disaster recovery, residency or latency, because cross-region is tens of milliseconds and charged.
- Shared responsibility: provider secures of the cloud, you secure in it, and the line moves per service. Every row ends with your data. That is why every public-bucket leak is a customer misconfiguration.
- "Cloud native" means designed for unreliable, replaceable, dynamically allocated machines: stateless, state in managed services, config from the environment, health checks and graceful shutdown, horizontal scaling, infrastructure as code, observability. Lift and shift is legitimate — call it what it is.
- Cloud wins on elasticity, no capital expenditure, managed services and reach. It loses on steady high utilisation, egress charges, lock-in and control. Know your exit; multi-cloud usually costs more than the lock-in it avoids.
- IOPS for small random operations, throughput for large sequential ones, and they trade off. Local instance storage disappears with the instance. Burst credit exhaustion is the cause of "the database got slow after three weeks with no deploy".
- Cost is architecture: cross-zone chatter, NAT gateway data processing (fixed by VPC endpoints), storage tiers, and idle resources. Commit the steady baseline, use spot for interruptible work.
- Tag everything. FinOps is mostly about showing each cost to the people who can change the design. Add an anomaly alert that catches a runaway job on day one.
Self-test: Why are zones kilometres apart rather than adjacent or continents apart? · Where does the responsibility line move between a virtual machine and a managed database? · Name two real disadvantages of cloud that are not marketing. · What is the difference between IOPS and throughput, and which does a database want? · Which two gradual failures make a system slow with no deploy? · What is the first thing to do about an unexplained bill?
Next: this page treated the network as a line on the bill. 13.2 opens it up. It takes the private networks of Chapter 5.10 and shows what a company runs into once it has several of them. It covers how to hand out address ranges, why IPv4 addresses now cost money, and how traffic enters and leaves a private network.