Appearance
2.1.2 — User Mode, Kernel Mode & the System Call
Our photo-upload service saves every photo with one line: writeFileSync("uploads/photo-001.jpg", photo). That line looks like it writes to the disk. It does not, and it cannot. The upload service is an ordinary program, and the CPU will not let an ordinary program send a single command to the disk.
So how does the photo get saved? The program asks the kernel to save it. This page is about that asking: the wall the CPU builds between ordinary programs and the kernel, and the one guarded door through the wall, called the system call. Understand this one door and many later mysteries become simple. You will see why reading a file one byte at a time is thousands of times slower than reading it in chunks, and why a crash in your program spares the machine while a crash in the kernel kills it.
1. Two levels of power, built into the CPU
Page 2.1.1 ended with a promise: the CPU itself keeps the kernel and ordinary programs apart. Here is how.
Every modern CPU has at least two privilege levels. A privilege level is a setting inside the CPU, stored in a couple of bits of one of its registers, that decides which instructions the CPU is willing to run right now. People also call these levels modes. On Intel and AMD processors (the x86 family) they are called rings, numbered 0 to 3.
- Kernel mode is the trusted level. On x86 it is ring 0. Code running in kernel mode may run any instruction, read and write any memory, and talk directly to devices. Only the kernel runs here.
- User mode is the ordinary level. On x86 it is ring 3. Code running in user mode may do arithmetic, make decisions, and read and write the memory it was given. Your browser, your Node server and every command you type run here.
x86 has four rings, but almost every operating system uses only two of them, ring 0 and ring 3. The phone in your pocket uses ARM processors, which name their levels differently. ARM calls user mode EL0 and kernel mode EL1, where EL stands for "exception level". The idea is identical.
The memory the kernel uses is called kernel space. The memory ordinary programs use is called user space. People also use these two words to mean "the world of the kernel" and "the world of ordinary programs".
What the CPU refuses to do in user mode
The rule is enforced on every single instruction. Here are the kinds of instruction that are privileged, meaning the CPU runs them only in kernel mode:
- Stop the CPU. On x86 this is the
hltinstruction. - Turn interrupts off. If a program could do this, it could stop the kernel's timer from ever taking the CPU back, and keep the CPU forever.
- Change which memory a program can see. The tables that decide this are covered in 2.5.1. If a program could change them, it could see everyone's memory.
- Talk to devices directly. On x86 some devices are reached through special
inandoutinstructions, and others through memory addresses that only the kernel is allowed to use.
What happens if a user-mode program tries one anyway? The CPU does not run it. Instead it stops the program on the spot and jumps into the kernel. This jump is called a trap. The kernel looks at what happened, sees that an ordinary program tried something forbidden, and normally ends that program. On Linux the program is sent a signal that kills it (2.2.3 covers signals). The rest of the machine is untouched.
This is not a polite request the kernel makes. It is a wall in the silicon. A user-mode program physically cannot run a privileged instruction, because the transistors that would carry it out check the privilege bits first. You met the same idea in 1.2, where a well-designed state machine makes illegal states impossible to reach. Here the hardware makes the dangerous powers impossible to use from the ordinary level.
So a program cannot open a file, send a packet or ask for memory by itself. Yet programs do those things all the time. The answer is the door in the wall.
2. Three ways into the kernel
Before we look at the door itself, it helps to see that the CPU can jump from user mode into the kernel for three different reasons. All three use the same hardware machinery. They differ in why the jump happens.
| Way in | Who causes it | Example |
|---|---|---|
| System call | the program, on purpose | "please write this file" |
| Exception | the program, by accident | dividing by zero, a bad memory address |
| Interrupt | a device or the timer | "the disk has finished", "your time is up" |
A system call is the program asking for something on purpose. It is the subject of this page.
An exception is the program doing something the CPU cannot complete. Dividing by zero is one. Touching a memory address the program was not given is another. The CPU stops and hands the problem to the kernel. Sometimes the kernel can fix it and let the program carry on; 2.5.3 shows that most memory exceptions are fixed this way, invisibly. Sometimes it cannot, and the program is ended. The trap from section 1, when a program tries a privileged instruction, is also an exception.
An interrupt comes from outside the program entirely. A device has finished its work, or the timer has fired. The program had nothing to do with it. The kernel handles the event and then usually lets the program carry on as if nothing happened. 2.7.1 covers interrupts.
Hold this table in mind. Whenever the kernel "takes over", it is always one of these three.
3. The system call, step by step
A program that needs something only the kernel can do does not do it. It asks the kernel to do it. That request is a system call, usually shortened to syscall. The set of system calls is the kernel's whole public menu. Everything a program can ask the operating system for is one of them. Linux has a few hundred. On a modern 64-bit PC there are roughly 380, with names like read, write, openat, close, mmap, clone and execve.
The word "call" makes it sound like an ordinary function call. It is very different. An ordinary function call jumps to another place in the same program, at the same privilege level. A system call must raise the CPU from user mode to kernel mode. And it must do that without letting the program choose where in the kernel it lands. If a program could jump to any address it liked in the kernel, it could leap past every safety check. So the crossing is narrow and controlled.
Here is exactly what happens when our upload service writes 3 bytes, the text ok\n, to its standard output on a 64-bit Linux PC. Standard output is where a program's printed text goes; 2.8.1 explains it fully. For now it is simply "the screen", and the kernel knows it as number 1.
Step 1: put the request in registers. A register is one of the CPU's own tiny storage slots (1.5). The program puts four numbers into four agreed registers:
| Register | Holds | Value here |
|---|---|---|
rax | which system call | 1, meaning write |
rdi | first argument: where to write | 1, standard output |
rsi | second argument: where the bytes are | the address of ok\n |
rdx | third argument: how many bytes | 3 |
Every system call has a number. On 64-bit Linux, read is 0, write is 1, close is 3 and openat is 257. The agreement about which register holds what is called the calling convention, the same idea you saw in the trace in 1.5.
Step 2: run the syscall instruction. This one instruction is the door. It does two things at the same instant, which nothing else can do. It switches the CPU into kernel mode, and it jumps to one fixed address inside the kernel. The program cannot choose that address. The kernel wrote it into a special CPU register while the machine was booting, and the instruction always goes there. One door, one doorman. On ARM processors the door is an instruction called svc, and older x86 programs used int 0x80, but the idea is the same.
Step 3: the kernel checks everything, then does the work. Now running in kernel mode, the kernel's entry code reads rax to find out which system call was asked for. It looks the number up in a table to find the right handler. Then comes the most important part: it checks every argument, because it trusts nothing the program handed it.
- Is descriptor 1 really open in this program?
- Is the address in
rsireally inside this program's own memory, and not inside the kernel or another program? - Is the length sensible?
Only after those checks pass does it do the real work. It copies the 3 bytes out of the program's memory into its own memory, with a careful copying routine that stops safely if the address turns out to be bad. Then it passes them to whatever is behind descriptor 1, which here is the terminal.
Step 4: return to user mode. The kernel puts the result into rax. On success that is the number of bytes written, here 3. On failure it is a negative error number. Then it runs the matching instruction, sysret, which drops the CPU back to user mode and continues the program at the very next instruction after its syscall. To the program, it looks as if an ordinary function returned.
syscall instruction switches to kernel mode and jumps to the kernel's single entry point. ③ The kernel finds the handler, checks every argument, and does the work. ④ sysret returns to user mode with the result in a register.Read the figure once more with the colours. Blue and amber happen on the program's side. Purple, red and green happen inside the kernel. The red box is the reason the wall is safe: the kernel checks before it acts. Then the green arrow carries the answer back across the wall.
This round trip, from user mode into kernel mode and back, is called a mode switch. Remember that name. Section 6 shows that it costs real time, and that cost explains a great deal of performance engineering.
4. File descriptors: how a program names an open file
Step 1 put the number 1 in rdi to mean "standard output". Notice that the program did not pass a name. It passed a small number. That number is a file descriptor, usually shortened to fd, and you will meet it in every remaining chapter of this Part.
Here is the idea. When a program opens a file, the kernel does the real work. It finds the file, checks the program is allowed to open it, and keeps all the details in its own memory: which file it is, how far the program has read, whether it was opened for reading or writing. The program never sees any of that. The kernel hands back just one small whole number. From then on the program says "read from descriptor 3", and the kernel looks up entry 3 in that program's table to find out what it really means.
Think of a coat check. You hand over your coat and get a numbered ticket. The cloakroom keeps the coat. You only hold the number. This design keeps the kernel in charge, because a program can only name things the kernel has already given it a ticket for. It also means descriptors belong to one program. Your descriptor 3 and my descriptor 3 are unrelated tickets from two different cloakrooms.
Here is the upload service opening a photo and printing the ticket it gets back.
ts
import { openSync, closeSync } from "node:fs";
const fd: number = openSync("photo.jpg", "r"); // (1)
console.log("file descriptor we were given:", fd); // (2)
closeSync(fd); // (3)
// Prints on Windows: file descriptor we were given: 3(1)openSyncmakes theopenatsystem call. The kernel opens the file and returns a descriptor. In TypeScript its type is justnumber, because that is all it is.(2)On Windows this printed 3. Why not 0? Because every program starts with three descriptors already open: 0 is standard input, 1 is standard output and 2 is standard error (2.8.1 covers all three). The kernel always gives out the lowest free number, so the first file a simple program opens gets 3. On Linux, Node prints a bigger number, because Node opens a dozen or so descriptors for its own machinery before your first line of code runs.(3)closeSyncmakes theclosesystem call. The kernel throws the ticket away and the number becomes free again. A program that keeps opening files and never closes them eventually runs out of numbers. On Linux the usual limit is 1,024 per program unless raised, and the error isEMFILE, "too many open files".
Three tables behind one number
Behind that small number the kernel keeps three layers of records. The picture shows them.
- The descriptor table belongs to one program. Each slot holds a pointer.
- The open-file entry is created each time a file is opened. It remembers the current position ("the next read starts at byte 65,536") and whether the file was opened for reading or writing.
- The file's own record describes the file on disk: its size, its owner, and where its data lives. 2.6.1 calls this record an inode.
Why three layers and not one? Because they can be shared in different ways. If the upload service opens photo.jpg twice, it gets two descriptors and two open-file entries, each with its own position, both pointing to the same file. If it starts a child program, the child gets a copy of the descriptor table whose slots point to the same open-file entries, so parent and child share one position. 2.2.2 shows why that sharing matters when a program starts another.
Sockets are file descriptors too
Unix treats almost everything as a file, an idea 2.6.3 explores. So a file descriptor might stand for a file on disk, the keyboard, a pipe between two programs, or a socket. A socket is one end of a network connection. When a browser connects to our upload service, the kernel gives the service a new descriptor for that connection. The service then uses the very same read and write system calls on it that it would use on a file. Part 5 explains how sockets carry data across a network. Here you only need one sentence: a socket is a network connection you talk to with a file descriptor.
5. You have been making system calls all along
You almost never write a syscall instruction yourself. Here is the chain of layers that hides it, because seeing the chain once removes the mystery from "high-level" code.
When our upload service calls writeFileSync:
- Your TypeScript calls Node's
fsmodule. - Node's own C++ code receives the call and passes it to a library called libuv, which Node uses for all its file and network work (3.8.2).
- libuv calls the
open,writeandclosefunctions in the C standard library, usually called libc. On Linux the common one is called glibc. - The libc function is a tiny wrapper. It puts the numbers in the right registers and runs the
syscallinstruction. - The kernel does the work, as in section 3.
Every print, every file read and every network call in every language reaches the kernel through a chain like this. Python, Java and Go each have their own middle layers, but they all end at the same door.
Windows has the same chain with different names. Programs call functions in a system library, and the lowest one, ntdll.dll, contains the wrappers that run the syscall instruction. Microsoft changes the Windows system call numbers between versions, so programs are expected to always go through ntdll.dll and never use the numbers directly. Linux promises the opposite: its system call numbers never change, so a program built twenty years ago still works.
When a system call fails
Step 4 said that a failed system call returns a negative error number. libc turns that number into a variable called errno and a short code name. Node turns it into an error object you can catch.
ts
import { openSync } from "node:fs";
try {
openSync("uploads/photo-999.jpg", "r"); // (1)
} catch (err) {
const e = err as NodeJS.ErrnoException;
console.log(e.code); // (2)
console.log(e.syscall); // (3)
console.log(e.message);
}
// Prints:
// ENOENT
// open
// ENOENT: no such file or directory, open 'uploads/photo-999.jpg'
// (on Windows the message shows the full path instead)(1)The photo does not exist, so the kernel refuses theopenrequest.(2)ENOENTis the error's code name. It means "error: no entry", which is Unix's old way of saying "no such file or directory". Other codes you will meet often areEACCES(permission denied),EEXIST(already exists) andEMFILE(too many open files).(3)Node even tells you which system call failed. When an error message names a system call, it is telling you the kernel itself said no.
Watching the system calls happen
On Linux there is a tool called strace that prints every system call a program makes, as it makes it. Run strace -f node upload-service-startup.ts and, among hundreds of lines, you will see the photo being saved. Trimmed, it looks like this:
text
openat(AT_FDCWD, "uploads/photo-001.jpg", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 21
write(21, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5242880) = 5242880
close(21) = 0Read each line as "system call name, arguments in brackets, then the result after the equals sign". The first line opens the file for writing, creating it if needed, and the kernel hands back descriptor 21. The second line writes 5,242,880 bytes, which is 5 MB, to descriptor 21, and the kernel reports that all of them were written. The third line closes it. (This trace is illustrative. The descriptor number and exact flags vary by machine and Node version.)
Running strace ls once and watching the openat, read and write calls scroll past is one of the best ten minutes you can spend in computing. 2.8.5 uses strace to debug real problems.
6. What a system call costs
A mode switch is not free. Here is everything the CPU and kernel must do for every single system call:
- Save the program's registers, so the program can continue later exactly where it was.
- Switch the privilege level and jump to the entry point.
- Find the handler and check every argument.
- Do the actual work.
- Restore the registers and switch back to user mode.
Steps 1, 2, 3 and 5 are pure overhead. They happen even if the "work" in step 4 is tiny. So the question is: what happens if a program makes millions of tiny system calls?
Here is the upload service reading the same 5 MB photo two ways. The first reads 1 byte per call. The second reads 64 KB (65,536 bytes) per call.
ts
// read-cost.ts — read the same 5 MB photo two ways and time both
import { openSync, readSync, closeSync, writeFileSync } from "node:fs";
writeFileSync("photo.jpg", Buffer.alloc(5 * 1024 * 1024, 7)); // a 5 MB test file
function readWithChunk(path: string, chunkSize: number): { calls: number; ms: number } {
const fd: number = openSync(path, "r"); // (1)
const buffer = Buffer.alloc(chunkSize); // (2)
let calls = 0;
const start = performance.now();
while (true) {
const got = readSync(fd, buffer, 0, chunkSize, null); // (3)
calls++;
if (got === 0) break; // (4)
}
const ms = performance.now() - start;
closeSync(fd); // (5)
return { calls, ms };
}
const small = readWithChunk("photo.jpg", 1); // one byte per call
const big = readWithChunk("photo.jpg", 64 * 1024); // 64 KB per call
console.log(`1-byte reads: ${small.calls} calls, ${small.ms.toFixed(0)} ms`);
console.log(`64 KB reads: ${big.calls} calls, ${big.ms.toFixed(1)} ms`);
console.log(`slower by about ${(small.ms / big.ms).toFixed(0)} times`);
// Printed on one Windows laptop with Node 22 (your numbers will differ):
// 1-byte reads: 5242881 calls, 11624 ms
// 64 KB reads: 81 calls, 1.0 ms
// slower by about 11220 times(1)Open the file once and get a descriptor.(2)Make a buffer, a block of the program's own memory where the kernel will put the bytes it reads. Its size is the chunk size.(3)readSyncmakes onereadsystem call. It asks for up tochunkSizebytes and returns how many it actually got.(4)When the file is finished,readreturns 0. That is how a program knows it has reached the end. This is why the 1-byte version made 5,242,881 calls, one more than the number of bytes: the last call is the one that returns 0.(5)Close the descriptor.
The two versions read exactly the same bytes. One took about 11 seconds and the other took about 1 millisecond. The cost was set by the number of system calls, not by the number of bytes. Each 1-byte call cost a couple of microseconds, and five million of them add up to seconds. Some of that time is Node's own bookkeeping on each call, not only the kernel crossing, but the lesson is the same either way: every crossing has a fixed price, so cross less often.
This one fact explains several things you will meet again:
- Languages buffer by default. When you print a line, most languages collect output in a buffer in your program and send it to the kernel in one big
writelater. Node's streams read files in 64 KB chunks for exactly this reason (3.8.4). - The crossing got more expensive in 2018. The Meltdown attack, a cousin of the Spectre attack from 1.5 section 5, let ordinary programs read kernel memory through a timing trick. The main fix, called KPTI (kernel page-table isolation), hides the kernel's memory from user mode completely and swaps it back in on every system call. That made each crossing slower, on some workloads noticeably.
- Some "system calls" skip the crossing entirely. Asking for the current time is so common that the kernel maps a small piece of read-only code and data straight into every program's memory. Calls like
clock_gettimerun there, in user mode, with no mode switch at all. This shared piece is called the vDSO (virtual dynamic shared object). - Modern interfaces batch the crossings. Linux's
io_uringlets a program queue thousands of input and output requests in memory it shares with the kernel, with almost no system calls. 2.7.3 explains it.
Every one of these tricks is the same idea: crossing the wall costs time, so cross it less.
7. Why a crash in your program spares the machine
Suppose our upload service has a bug and follows a pointer to a memory address it was never given. The CPU cannot complete the access. That is an exception, from section 2. The CPU jumps into the kernel. The kernel sees an ordinary program touching memory that is not its own. It ends that one program, takes back its memory and its descriptors, and carries on. Every other program keeps running. On Linux you see the message Segmentation fault, which 2.5.3 explains in full.
Now suppose the same bug is inside the kernel, perhaps in a disk driver. The code is running in kernel mode. There is no higher level above it to catch the mistake. The kernel cannot trust its own memory any more, so the only safe move is to stop everything. On Linux this is a kernel panic. On Windows it is the famous blue screen.
Same bug, completely different damage. The only difference is which side of the wall it happened on.
The wall is also the security boundary
Because every request from user mode must go through the checks in step 3, the kernel enforces all the separation on the machine. It keeps one program from another, one user from another, and one container from another. That makes the most dangerous kind of security bug a privilege escalation: a flaw that tricks the kernel into running the attacker's code in kernel mode. Once that happens, the attacker is the referee and owns the machine. One missed check on one pointer in one system call can be enough. When you later read about container escapes (14.1.3) or sandbox breakouts, you are reading about holes in this wall.
What the interviewer will push on
"Walk me through what happens when a program calls read()." They are checking whether you know it is not an ordinary function call. A good answer has four beats: the number and arguments go into registers, the syscall instruction switches to kernel mode at a fixed entry point, the kernel checks every argument before doing anything, and sysret returns with the result. The tell of real understanding is mentioning the checks and why the entry point is fixed. The common wrong answer stops at "the OS reads the file".
"Why is reading a file one byte at a time slow?" They want the cost model: each call pays a fixed price for the mode switch, so the number of calls matters more than the number of bytes. A strong answer gives a rough figure, such as "a few hundred nanoseconds to a couple of microseconds per call", and names the cure: read in big chunks, or use a buffered reader. The wrong answer blames the disk. The disk is not even involved after the first read, because the file is already cached in memory (2.6.2).
"What is a file descriptor?" They are checking whether you know it is a per-program index into a kernel table, not a pointer or a file name. Bonus points for saying why the first file you open is usually 3, and for naming the EMFILE error when a program leaks descriptors.
"Why does a segfault kill only your program, but a driver bug crashes the machine?" This tests whether you understand that protection comes from the privilege levels. User-mode faults are caught by the kernel above them. A kernel-mode fault has nothing above it.
"How can a program get the time without a system call?" Few candidates know the vDSO. Saying "the kernel maps the time into every program's memory, so clock_gettime never crosses the wall" shows you have looked under the hood.
Something to volunteer that nobody asks for: Linux promises never to change its system call numbers, while Windows changes them and forces programs through ntdll.dll. That one design choice is why an old Linux program still runs on a new kernel, and why containers can ship their own userland while sharing the host's kernel.
Recall
- The CPU has privilege levels: kernel mode (x86 ring 0, only the kernel) and user mode (ring 3, every program). The wall between kernel space and user space is checked by the hardware on every instruction, and a forbidden instruction causes a trap into the kernel.
- The kernel takes over in three ways: a system call (on purpose), an exception (by accident) and an interrupt (from a device or the timer).
- A system call puts a number and arguments in registers, runs
syscallto enter kernel mode at one fixed entry point, lets the kernel check every argument and do the work, thensysretreturns the result. The round trip is a mode switch. - A file descriptor is a small per-program number naming an open file, pipe or socket; 0, 1 and 2 are already open, so the first file is usually 3. Behind it sit the descriptor table, the open-file entry (with the position) and the file's own record.
- Every crossing has a fixed cost, so the count of system calls dominates: buffering, the vDSO, and batching interfaces all exist to cross less. A user-mode crash is caught by the kernel; a kernel-mode crash is a kernel panic.
Self-test: Why must the privilege levels be enforced by the hardware? Name the three ways into the kernel and who causes each. What are the four steps of a system call, and why is the entry point fixed? Why was the first file descriptor 3? Why was reading one byte at a time about ten thousand times slower?
Quiz Bank
FoundationalWhat is the difference between user space and kernel space, and why does it exist?
User space is where ordinary programs run, in the CPU's restricted user mode (ring 3 on x86). Code there can do arithmetic and use its own memory, but cannot run privileged instructions or talk to devices. Kernel space is where the kernel runs, in kernel mode (ring 0), where any instruction and any memory is allowed.
It exists for safety. A buggy or hostile program must not be able to read another program's memory, keep the CPU forever, or wreck a device. The separation is enforced by the CPU itself on every instruction, so no program can promote itself.
FoundationalWhat is a system call?
A system call is the controlled way an ordinary program asks the kernel to do something only the kernel may do, such as reading a file, getting memory, opening a network connection or starting a program. The set of system calls is the kernel's whole public menu. The program puts a call number and arguments in registers and runs the syscall instruction, which switches the CPU to kernel mode and jumps to one fixed kernel entry point. The kernel checks the arguments, does the work, and returns the result to user mode.
AppliedTrace what happens, step by step, when a program calls read().
- Through a libc wrapper, the program puts the number for
read(0 on 64-bit Linux) intorax, and the descriptor, the buffer address and the byte count intordi,rsiandrdx. - It runs the
syscallinstruction. The CPU switches to kernel mode and jumps to the kernel's single entry point. This is a mode switch. - The kernel looks up the handler for call 0. It checks that the descriptor is open in this program, that the buffer address is inside this program's own memory, and that the count is sensible.
- It does the work: it gets the data, from the memory cache if it is already there or from the device if not, and copies it into the program's buffer.
- It puts the number of bytes read into
raxand runssysret, returning to user mode at the instruction aftersyscall. The program sees an ordinary function return.
AppliedWhy is reading a file one byte at a time far slower than reading in large chunks, even though the total bytes are the same?
Because each read is a mode switch with a fixed cost: saving registers, switching privilege level, finding the handler, checking the arguments and switching back. That overhead is the same whether the call reads 1 byte or 64 KB.
Reading a 5 MB file one byte per call makes over five million crossings. Reading it 64 KB per call makes 81. In a measured run, the first took about 11 seconds and the second about 1 millisecond. The number of system calls decides the time, not the number of bytes. That is why languages and runtimes buffer input and output by default.
InterviewWhy does a user program crashing with a segfault not bring down the machine, while a kernel bug does?
A user program runs in ring 3. When it does something illegal, such as following a bad pointer, the CPU raises an exception and jumps into the kernel, which acts as a referee above it. The kernel ends just that program and takes back its memory and descriptors. Everything else keeps running.
A kernel bug runs in ring 0. There is no level above it to catch the mistake, and the kernel can no longer trust its own memory. The only safe action is to stop everything: a kernel panic on Linux, the blue screen on Windows. The damage differs entirely because of which side of the wall the fault happened on.
InterviewWhat is a file descriptor, and why is the first file a program opens usually number 3?
A file descriptor is a small whole number that indexes the program's own table of open things inside the kernel. Each slot points to an open-file entry, which records the current position and the access mode, and that entry points to the file itself. The program only ever holds the number, like a coat-check ticket, so it can only name things the kernel has already opened for it.
Every program starts with three descriptors open: 0 for standard input, 1 for standard output and 2 for standard error. The kernel always hands out the lowest free number, so the first new file usually gets 3. Descriptors are per program, so two programs can both have a descriptor 3 that mean different things. Leaking descriptors eventually fails with EMFILE, too many open files.
StaffA latency-sensitive service does millions of tiny network reads, and its profile shows most time inside the kernel. Using this page, list the ways to cut system-call overhead.
The service is bound by crossings, not by work. Every lever is a version of "cross the wall less often, or more cheaply".
- Batch and buffer. Read bigger chunks per call, or merge many small logical reads into fewer large ones.
- Use interfaces built for many connections.
epolllets one call report many ready connections, andio_uringlets a program submit and collect thousands of operations through shared memory with almost no system calls (2.7.3). - Use batched calls. Linux has calls such as
recvmmsgthat receive many network messages in one crossing. - Avoid needless crossings. Make sure hot paths such as reading the clock use the vDSO rather than a real system call.
- Know the per-crossing cost. Mitigations such as KPTI make each crossing dearer, and their cost depends on the CPU.
- Move work to the kernel side.
sendfilesends file data to a socket without copying it through the program (2.7.4).
The staff framing: find that throughput is limited by the number of crossings, then attack the count and the price per crossing, not the number of bytes.
Scenario Drill
DrillYou run code uploaded by untrusted users on your servers. Explain why the CPU privilege levels are your foundation of safety, and where they are not enough on their own.
The privilege levels are the first wall. Untrusted code runs in user space, so the hardware stops it from touching other programs' memory, commanding devices or stopping the CPU. Every dangerous action must go through a system call, which the kernel checks and can refuse. A buggy or hostile program can crash itself, and the kernel cleans up, without hurting the host.
But user mode alone is not enough. A user-mode program can still do harm with allowed system calls. It can read any file it has permission to read, open network connections, start other programs, or use up all the CPU and memory. It can also try a privilege escalation, hunting for a kernel bug in how some system call checks its arguments, to get into kernel mode and take over.
So real sandboxes add more walls, all enforced by the kernel:
- seccomp limits which system calls the program may make at all;
- namespaces give it its own separate view of processes, files and the network (14.1.3);
- cgroups cap the CPU and memory it can use;
- and it often runs inside a container or a small virtual machine (14.1.2), so even a kernel escape hits another wall.
The lesson: the privilege boundary is the enforcement mechanism, but safety comes from narrowing what the kernel will do on the untrusted program's behalf, so the door has fewer ways to be abused.
Next: every page so far has assumed the kernel is already running. But when you press the power button, the kernel is just a file on a disk, and the CPU does not yet know how to read a file system to find it. Page 2.1.3 follows the machine from the first instruction after power-on to the moment the first program starts.