Skip to content

Volume ITechnology

2.3.1 — Threads Inside the Kernel ​

Making a thumbnail is pure CPU work. The photo-upload service has four photos to resize, and the laptop has eight CPU cores. Here is what happens when it does the work on one strand of execution, and then on four, all inside one process.

ts
// thumbnail-threads.ts — CPU-heavy thumbnail work, first on one thread, then on four
import { Worker, isMainThread, parentPort, threadId, workerData } from "node:worker_threads";
import { createHash } from "node:crypto";
import { cpus } from "node:os";

function fakeResize(seed: number): string {                      // (1) pure CPU work, about 1 s
  let h = String(seed);
  for (let i = 0; i < 600_000; i++) h = createHash("sha256").update(h).digest("hex");
  return h.slice(0, 8);
}

if (isMainThread) {
  console.log(`cores: ${cpus().length}, process ${process.pid}, main thread id ${threadId}`);

  let t = performance.now();
  for (let i = 0; i < 4; i++) fakeResize(i);                     // (2) one after another
  console.log(`4 photos on 1 thread:  ${((performance.now() - t) / 1000).toFixed(1)} s`);

  t = performance.now();
  await Promise.all([0, 1, 2, 3].map((i) => new Promise((resolve) => {
    const w = new Worker(new URL(import.meta.url), { workerData: i }); // (3)
    w.on("message", resolve);
  })));
  console.log(`4 photos on 4 threads: ${((performance.now() - t) / 1000).toFixed(1)} s`);
} else {
  parentPort!.postMessage(fakeResize(workerData));               // (4)
  console.log(`  worker thread id ${threadId} ran inside process ${process.pid}`);  // (5)
}

// Printed on an 8-core Windows laptop with Node 22 (your numbers will differ):
// cores: 8, process 26424, main thread id 0
// 4 photos on 1 thread:  3.5 s
//   worker thread id 3 ran inside process 26424
//   worker thread id 2 ran inside process 26424
//   worker thread id 4 ran inside process 26424
// 4 photos on 4 threads: 1.4 s
//   worker thread id 1 ran inside process 26424
  • (1) fakeResize stands in for real image work. It hashes a string 600,000 times, which keeps one CPU core completely busy for about a second.
  • (2) First, the main thread resizes the four photos one after another. That took 3.5 seconds.
  • (3) Then we start four Workers. Each Node worker is a real operating-system thread, and each runs this same file. The isMainThread check sends it down the other branch.
  • (4) Each worker resizes one photo and sends the result back to the main thread as a message.
  • (5) Look at the printed process number. All four workers ran inside the same process as the main thread, 26424. They are not separate programs. They are four extra strands of execution inside one program. And together they finished in 1.4 seconds instead of 3.5, because the kernel ran them on four cores at the same time.

The extra strands are threads. This page shows what a thread is to the kernel, how Linux builds one, what one costs, and the different ways languages arrange their threads. (One detail in the output: worker 1's message appears last only because workers' printed output is passed to the main thread to print, and arrives a little late.)

1. A process is a container, a thread is a strand of execution ​

2.2.1 described a process as two things bundled together:

  1. A container of resources: its memory, its open files, its PID, the user it runs as.
  2. A strand of execution: the CPU registers and the program counter, marching through the instructions one after another, plus a stack of function calls.

A thread is the second thing, pulled out and multiplied. One process can hold several threads. Each thread has its own strand of execution: its own program counter, its own registers and its own stack. But all the threads in a process share the one container: the same code, the same global variables, the same heap and the same open files.

ONE PROCESS (PID 26424)shared by every threadcode · global variables · heap · open file descriptors · PIDmain threadits own stackregisters +program counterworker 1its own stackregisters +program counterworker 2its own stackregisters +program counter
Figure 1 — Threads inside one process. The green band is shared by every thread. Each thread owns only its stack and its registers. Sharing memory makes threads fast to coordinate, and it is also what makes them dangerous.

That sharing is the whole point, and the whole danger.

  • The point: two threads in one process can pass data simply by reading and writing the same variable. No copying, no channel, no kernel in between. Compare that with 2.2.4, where two processes needed a pipe or a socket just to exchange a file name.
  • The danger: two threads touching the same data at the same moment can corrupt it. A counter of uploads can lose updates. 2.4.1 shows exactly how, at the level of single CPU instructions.

The words concurrency and parallelism belong here too. Concurrency means several tasks are in progress over the same period, taking turns. Parallelism means several tasks are running at the very same instant, on different cores. Our four workers were parallel, because the laptop had cores to spare. 9.5.1 covers the difference from the design side. This page stays with the kernel.

2. To Linux, a thread is just a task that shares ​

Here is a surprise. The Linux kernel has no separate idea of a "thread" and a "process". It has one kind of thing, a task, each with its own task_struct record (2.2.1). What makes some tasks "threads of the same process" is simply what they share.

The system call that makes a new task is clone. It takes a set of flags, and each flag says "share this thing with the parent instead of copying it":

FlagMeaning when set
CLONE_VMshare the same memory
CLONE_FILESshare the same table of open files
CLONE_FSshare the current directory
CLONE_SIGHANDshare the same signal handlers
CLONE_THREADjoin the parent's thread group

Now fork and threads become two settings of one dial.

  • fork from 2.2.2 is clone with none of these flags. The child gets a copy of everything, so it is a separate process.
  • Creating a thread is clone with all of them. The new task shares the memory, the files and the handlers, and joins the same group. When a C program calls pthread_create, or Node starts a Worker, this is what happens underneath on Linux.

Two kinds of ID ​

Every task has its own ID, called a thread ID or TID. A process is a group of tasks, and the group's ID, the thread group ID or TGID, is the TID of its first thread. The confusing part is the names:

  • What programs call the PID, the number process.pid returns, is really the TGID. That is why all four workers above reported the same PID.
  • Each thread's own number is its TID. On Linux, ps -L -p 26424 lists every thread of process 26424 with its TID, and the directory /proc/26424/task/ holds one entry per thread.

This design arrived in Linux 2.6 in 2003. Before that, an older thread library gave each Linux thread its own separate PID, and many tools behaved oddly as a result. The current model, where a process is a group of tasks sharing memory, fixed that.

The scheduler schedules threads ​

Because every thread is its own task, the scheduler picks threads, not processes. A process with four busy threads gives the scheduler four things to place on cores. That is how our four workers ran on four cores at once. It is also why "this process uses 400 % CPU" is a normal thing for top to print on a multicore machine: four threads each using one whole core.

3. What a thread costs ​

A thread is much cheaper than a process, because there is no new memory map to set up and nothing to copy. But it is not free.

Creating one costs a clone system call, a new task_struct, and a new stack. On Linux that is typically tens of microseconds, against a hundred or more for fork. (These are rough figures; they depend on the machine.)

Its stack is the main memory cost. On Linux the default stack for a new thread reserves 8 MB of address space. That sounds huge, but reserving address space is not the same as using RAM (2.2.1 showed that unused address space costs nothing). The kernel gives the stack real memory only as the thread actually uses it, usually a few kilobytes. Each stack also has a guard page at its end, so a thread that recurses too deeply crashes instead of silently overwriting its neighbour's stack.

Switching between two threads of the same process is a little cheaper than switching between processes. The memory map stays the same, so the CPU keeps its saved address translations (2.5.2). The registers still have to be saved and loaded, and the scheduler still has to run.

Many threads still add up. Ten thousand threads, each touching tens of kilobytes of stack, use hundreds of megabytes. When they all wake at once, the scheduler has ten thousand tasks to juggle, and each switch leaves cold caches behind. That is the wall 2.7.2 calls the C10k problem.

A Node Worker costs much more than a plain thread, because each one starts its own copy of the JavaScript engine with its own heap and its own event loop. Starting one takes tens of milliseconds and several megabytes. That is part of why our four workers finished in 1.4 seconds rather than the ideal 0.9: some of the time went on starting them. Node workers are for a few long-lived, CPU-heavy jobs, not for one-per-request.

4. Threads the kernel runs for itself ​

Not every thread belongs to your programs. The kernel runs many threads of its own, called kernel threads. They run entirely in kernel mode and have no user-space memory. They do background work: writing changed data from memory to disk, handling the second half of interrupts (2.7.1), moving memory between cores, and so on.

Run ps -ef on a Linux machine and you will see them with their names in square brackets, such as [kworker/0:1], [ksoftirqd/0] and [kthreadd]. The number after the slash is usually the CPU core the thread is tied to. They are scheduled exactly like your threads, by the same scheduler.

5. Who manages the threads: three models ​

So far, every thread was a kernel task. But a language can also run its own threads on top of the kernel's, juggling them itself. There are three arrangements, and each language you meet picks one.

1:1N:1 (green threads)M:Nlanguagekernelone kernel thread eachmany on one kernel threadmany spread over a few
Figure 2 — Three ways to map a language's threads onto the kernel's. The circles are the threads the program creates. The purple boxes are kernel threads, the only things the kernel's scheduler can see.

1:1. Every thread your program creates is one kernel thread. C's pthread_create, Java's ordinary threads, C#'s threads and Node's Workers all work this way. It is simple and it uses every core. The kernel schedules them, and if one thread waits for the disk, the kernel just runs another. The cost is that each thread is a full kernel task, so you cannot have millions of them.

N:1, also called green threads. The language runs many of its own lightweight threads on a single kernel thread, switching between them itself, without the kernel knowing. Early versions of Java did this. Switching is extremely cheap, because no system call is needed. But there are two fatal problems. First, everything runs on one kernel thread, so only one core is ever used. Second, if any green thread makes a system call that blocks, such as a slow disk read, the one kernel thread blocks, and every green thread stops with it. That is why this model fell out of favour.

M:N. The language runs many lightweight threads, M of them, spread across a smaller pool of kernel threads, N of them, usually about one per core. The language's own scheduler moves its lightweight threads between the kernel threads. When one is about to block in a system call, the scheduler moves the others off that kernel thread so they keep running. This gives cheap threads and every core, at the price of a complicated scheduler inside the language.

  • Go works this way. Its lightweight threads, called goroutines, start with a stack of only a few kilobytes that grows as needed, so a Go server can run hundreds of thousands of them.
  • Java's virtual threads, added in Java 21 in 2023, brought the same model to Java.
  • Erlang's processes, from 2.2.1, are also lightweight threads of this kind, despite the name.

Where does Node fit? Your JavaScript runs on one main thread, plus any Workers you start, all 1:1. Node gets its concurrency mostly from the event loop instead of from threads: one thread handles thousands of connections by never waiting (3.8.1). Underneath, Node's library libuv also keeps a small pool of ordinary kernel threads for work that cannot be done without blocking, such as some file operations (3.8.2).

6. Memory that belongs to one thread only ​

Sometimes each thread needs its own copy of a variable, even though everything else is shared. The classic example is errno, the error code from 2.1.2. If two threads both made failing system calls at the same moment and shared one errno, each could read the other's error. So C libraries keep errno in thread-local storage: a small area of memory that every thread has its own copy of, found through a special CPU register that the kernel sets for each thread when it switches to it.

Node workers take this much further. Each worker has its own JavaScript engine and its own heap, so ordinary JavaScript variables are never shared between workers at all. A global variable in the main thread and the same global in a worker are two different variables. Workers exchange data by sending messages, which are copied, as in line (4) above. The one exception is a SharedArrayBuffer, a block of raw memory that several workers can genuinely see at once. 2.4.1 uses one to show what goes wrong when two threads write the same memory.

7. How many threads should you use? ​

The demo used four workers for four photos. Would 200 workers make it faster? No, and the reason decides almost every thread-pool setting you will ever choose.

For CPU-bound work, like resizing photos, threads only help up to the number of cores. Eight cores can run eight threads at the same instant. A ninth thread does not get a ninth core. It just takes turns with the others, adding context switches and cold caches for no gain. So the right number is about the number of cores.

For I/O-bound work, like waiting for the network or the disk, threads spend most of their time blocked, using no CPU. So many more threads than cores can be useful, because at any moment most of them are waiting. But each still costs memory and scheduling, which is why event loops, which need no thread per waiting job, usually beat thousands of blocked threads (2.7.2).

The honest method is to measure. Run the real workload with 4, 8, 12 and 16 threads and pick the number where throughput stops rising.

What the interviewer will push on ​

"What is the difference between a process and a thread?" They are checking whether you can name what is shared and what is private. Threads share code, globals, heap and open files; each has its own stack, registers and program counter. A strong answer adds the consequence on both sides: sharing makes data exchange free and makes races possible.

"How does Linux implement threads?" This separates people who have looked underneath. The answer is that Linux has only tasks, created by clone, and a thread is a task that shares memory, files and signal handlers with its parent through clone flags. fork is the same call with nothing shared. Bonus: getpid returns the thread group ID, so all threads report the same PID.

"What are green threads, and why did they fall out of favour? What does Go do instead?" Green threads are many language threads on one kernel thread. They switch cheaply, but they cannot use more than one core, and one blocking system call stops them all. Go uses M:N: goroutines spread over a few kernel threads, with the runtime moving them off a thread that is about to block.

"Why doesn't adding threads always make a program faster?" CPU-bound work cannot run on more threads than cores at once, so extra threads only add switching and cache misses. Shared data also needs locks, and contention serialises the threads.

"How many threads should a pool have?" Roughly the core count for CPU-bound work, more for I/O-bound work, and measured rather than guessed.

Something to volunteer that nobody asks for: a new thread's 8 MB default stack is reserved address space, not used memory. The kernel gives it real RAM only as the stack grows. That is why a program can create a thousand threads without using 8 GB, and why "threads are expensive because of their stacks" is only partly true.

Recall

  • A thread is a strand of execution inside a process: its own stack, registers and program counter, sharing the process's code, globals, heap and open files. Sharing makes data exchange free and races possible.
  • In Linux every thread is a task made by clone; flags such as CLONE_VM and CLONE_FILES decide what is shared. fork shares nothing; a thread shares everything. getpid returns the group ID; each thread has its own TID. The scheduler schedules threads.
  • A thread is cheaper than a process, but each has a stack (8 MB reserved by default, little actually used), and thousands of threads cost memory and switching. A Node Worker is far heavier, with its own engine and heap.
  • Kernel threads such as [kworker/0:1] do the kernel's background work.
  • Threading models: 1:1 (pthreads, Java threads, Node workers), N:1 green threads (one core, one blocking call stops all), and M:N (Go goroutines, Java virtual threads). Thread-local storage gives each thread its own copy of variables such as errno.

Self-test: What does a thread share with the other threads in its process, and what does it own? How does Linux tell a thread from a process if both are tasks? Why did all four workers print the same process number? Why do green threads stall when one of them reads a slow disk? Why would 200 workers be slower than 8 for resizing photos on an 8-core laptop?

Quiz Bank ​

FoundationalWhat exactly do threads of the same process share, and what does each have of its own?

Threads of one process share the code, the global and static variables, the heap, the open file descriptors, the current directory, and the process's identity, including its PID. So an object one thread creates on the heap can be used by another thread directly.

Each thread owns its stack, holding its local variables and function calls, its CPU registers, and its program counter. It may also have thread-local storage, its own copy of certain variables such as errno. This split is why passing data between threads is trivial and why two threads writing the same data can corrupt it.

FoundationalWhy is creating a thread cheaper than creating a process?

A new process needs its own memory map, set up with copy-on-write, its own copy of the descriptor table and its own identity. A new thread needs none of that: it shares the existing memory map and files, so the kernel only creates a new task record, a stack and a register set. Switching between threads of the same process is also slightly cheaper, because the memory map does not change, so the CPU keeps its cached address translations.

AppliedHow are threads implemented in Linux?

Linux has one kind of schedulable thing, the task, each with its own task_struct. New tasks are created by the clone system call. Its flags choose what the new task shares with its creator. CLONE_VM shares memory, CLONE_FILES shares open files, CLONE_SIGHAND shares signal handlers, and CLONE_THREAD joins the same thread group.

A thread is a task created with all those flags, so it shares nearly everything. fork is a task created with none of them, so it is a separate process. Each task has its own ID (the TID); the group's ID is what getpid returns, which is why all threads of a process report the same PID. The scheduler works on tasks, so it schedules threads individually.

InterviewExplain 1:1, many-to-one and many-to-many threading models.

1:1: each language thread is one kernel thread, as with pthreads, Java platform threads and Node workers. The kernel schedules them on every core, and a blocking call blocks only that thread, but each thread is a full kernel task, so millions are not possible.

Many-to-one (green threads): many language threads run on one kernel thread, switched by the language itself. Switching is very cheap, but only one core is used, and one blocking system call stops every thread, which is why the model was abandoned.

Many-to-many (M:N): many lightweight threads run over a small pool of kernel threads, about one per core, with the language's scheduler moving them around and away from threads about to block. Go's goroutines and Java's virtual threads work this way, giving very cheap threads that still use every core.

InterviewWhy doesn't adding more threads always make a program faster?

Two reasons. For CPU-bound work, only as many threads can run at once as there are cores. Extra threads take turns on the same cores, adding context switches and cache misses, so throughput can fall. For any work on shared data, threads must synchronise with locks, and past a point more threads mean more waiting for locks rather than more progress.

The right number depends on the work: about the core count for CPU-bound jobs, more for I/O-bound jobs whose threads mostly wait. Measure throughput at several pool sizes rather than guessing.

StaffA team plans to handle each incoming request on its own new Node Worker so CPU-heavy requests do not block the event loop. What is wrong with the plan and what would you do instead?

A Node Worker is not a cheap thread. Each one starts a separate JavaScript engine with its own heap and event loop, taking tens of milliseconds and several megabytes. Starting one per request adds that delay to every request, and under load it creates far more threads than cores, so CPU-bound work slows down from switching while memory climbs.

Instead, create a fixed pool of workers when the service starts, about one per core minus one for the main thread, and send CPU-heavy jobs to idle workers through a queue. Requests that do not need heavy CPU stay on the event loop. If the queue grows, the service is genuinely short of CPU, and the answer is more machines or a separate service for the heavy work, not more threads on the same cores.

Scenario Drill ​

DrillThe photo-upload service resizes photos on a pool of worker threads on an 8-core machine. A colleague raises the pool from 8 to 200 threads to go faster, and throughput drops. Explain what is happening and how to choose the size.

Resizing is CPU-bound, so the speedup comes only from running threads on separate cores, and there are 8 cores. With 200 threads, at most 8 run at any instant. The other 192 are ready and waiting, and the scheduler keeps switching between them. Every switch costs time, and leaves the next thread with cold caches, because its image data is not in the cache. 200 stacks and 200 copies of working buffers also use more memory, and any shared structures add lock contention.

The result is more coordination with no extra parallelism, so throughput falls. For CPU-bound work, a pool of about the core count, here 8, or 7 to leave the main thread room, is right. If each job also reads and writes files, a few extra threads can hide that waiting. Measure throughput at 8, 10, 12 and 16 and take the plateau. By contrast, a pool of mostly waiting threads, such as one making slow network calls, can usefully be much larger than the core count.

Next: with many threads ready and only a few cores, something has to decide which thread runs on which core, and for how long. Page 2.3.2 works through the classic answers, with real numbers, to show what each one costs.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Volume I — Technology
Part 0 — Retention & Notation
Part 1 — Foundations of Computation
Part 2 — Operating Systems
2.1 The Kernel
2.2 Processes
2.3 Threads & Scheduling
2.4 Concurrency Under the Hood
2.5 Memory Management
2.6 File Systems
2.7 Input/Output
2.8 Linux for Engineers
2.9 Windows & macOS in Practice
Part 3 — Languages, Compilers & Runtimes
3.6 JavaScript & V8
3.7 TypeScript
3.8 Node.js Internals
3.13 Python & CPython
3.14 FastAPI & Production Python
Part 4 — Data Structures & Algorithms
4.0 Classic Problems — Before NeetCode
4.4 Arrays & Hashing — Problems
4.5 Two Pointers
4.6 Sliding Window
4.8 Stack — Problems
4.9 Linked List — Problems
4.11 Binary Search — Problems
4.13 Trees
4.14 Tree — Problems
4.15 Trie — Problems
4.17 Heap — Problems
4.18 Backtracking — Problems
4.19 Graphs
4.20 Graph — Problems
4.21 Advanced Graph — Problems
4.23 1-D DP — Problems
4.24 2-D DP — Problems
4.26 Greedy — Problems
4.27 Interval — Problems
4.28 Math & Geometry — Problems
4.30 Bit Manipulation — Problems
Part 5 — Networking & the Web
5.3 IP
5.4 Transport
5.6 HTTP
Part 6 — Frontend Engineering
6.1 Browser Internals
6.2 HTML & CSS Mastery
6.3 JavaScript in the Browser
6.4 React, from First Component to Full Shop
6.5 Frameworks & Rendering
6.8 UI Engineering & Patterns
Part 7 — Databases & Storage
7.2 SQL Mastery
7.3 Storage Engine Internals
7.4 Transactions
7.5 NoSQL Taxonomy
7.8 Analytics & Big Data
Part 8 — Security, Identity & Cryptography
8.2 Cryptography
8.3 Protocols of Trust
8.4 Identity: AuthN & AuthZ
8.5 Application Security
8.6 Infrastructure Security
Part 9 — Software Design (LLD)
9.2 OOP Foundations
9.3 Design Principles
9.4 Design Patterns
9.5 Concurrency
9.6 API Design
9.7 LLD Interview Machines
9.9 Node/Express Production Playbook
Part 10 — Distributed Systems & HLD
10.7 Consistency & Consensus
10.8 Messaging & Microservices
10.14 Caching
Building blocks from the design seat
Part 11 — System Design Case Studies
Part 12 — AI & ML Engineering
12.1 What AI Engineering Is
12.2 The Math You Need
12.3 Classical Machine Learning
12.4 Data for ML
12.5 Deep Learning
12.6 Embeddings & Vector Search
12.7 Transformers & LLMs
12.8 Building with LLMs
12.9 Retrieval-Augmented Generation
12.10 Agents
12.11 Adapting Models
12.12 Production AI
12.13 Multimodal AI
12.14 AI Security & Governance
12.15 Case Studies & Capstone
Part 13 — Cloud, DevOps & Infrastructure
13.11 Git, from First Commit to Team Workflow
Part 14 — Containers: Docker & Kubernetes
14.1 Foundations
14.2 Docker
14.3 Kubernetes
14.4 Kubernetes on Azure
Part 15 — The Craft & Career
Nothing picked yet