Skip to content

Volume ITechnology

2.5.4 — mmap & Memory Allocators ​

The photo-upload service builds two million small records, one per photo, each with an ID, a caption and some tags. Then it throws them all away, and asks the garbage collector to clean up.

ts
// heap-vs-rss.ts — freeing objects shrinks the heap faster than it shrinks the process
const mb = (n: number) => (n / 2 ** 20).toFixed(0).padStart(4) + " MB";
const show = (label: string) => {
  const m = process.memoryUsage();
  console.log(`${label.padEnd(22)} heap used ${mb(m.heapUsed)}   resident ${mb(m.rss)}`);  // (1)
};

show("at start");
let photos: { id: number; caption: string; tags: string[] }[] = [];
for (let i = 0; i < 2_000_000; i++)                                                   // (2)
  photos.push({ id: i, caption: "photo " + i, tags: ["holiday", "beach"] });
show("2 million records");

photos = [];                                                                          // (3)
(globalThis as any).gc();                                                             // (4)
show("after freeing them");

// Run with: node --expose-gc heap-vs-rss.ts
// Prints:
// at start               heap used    7 MB   resident   54 MB
// 2 million records      heap used  281 MB   resident  352 MB
// after freeing them     heap used    7 MB   resident  314 MB
  • (1) heapUsed is how much of Node's object heap holds live objects. rss is the process's resident set size (2.2.1): how much of its memory is actually in RAM.
  • (2) Two million small objects fill about 274 MB of heap.
  • (3) We drop every reference to them, so they are garbage.
  • (4) gc() forces a garbage collection, which Node allows only when started with --expose-gc. Normally it runs by itself.

The heap went straight back to 7 MB. But the process still holds 314 MB of RAM. The objects are gone, yet the memory they lived in has not been given back to the operating system.

That gap is the subject of this page. Between "the kernel hands out 4 KB pages" (2.5.1) and "your program asks for a 40-byte object" there are two layers. The lower one, mmap, asks the kernel for whole regions of pages, and can even make a file appear in memory. The upper one, the memory allocator, carves those regions into the small pieces programs actually use, and decides when, if ever, to give pages back.

1. Two layers between your object and RAM ​

A program creates objects of a few bytes to a few megabytes, constantly. Every new, every string, every array is one. The kernel only deals in whole pages, through system calls, and system calls are expensive (2.1.2). Asking the kernel for every 40-byte object would be thousands of times too slow. So the work is split:

  1. The kernel hands out large regions of virtual memory, a page at a time underneath, when a program asks with a system call.
  2. An allocator, which runs inside the program in user mode, takes a large region once and hands out small pieces of it many times, with no system call at all. When a piece is freed, the allocator keeps it to reuse.

In C, the allocator is the library function malloc and its partner free. In Node, there are two allocators stacked up: V8's garbage-collected heap holds your JavaScript objects (3.4), and underneath it, and for things like Buffer contents, Node uses the C library's allocator. Both get their pages from the kernel in the same way.

2. mmap: asking the kernel for a region ​

The main system call for getting memory is mmap, short for memory map. It asks the kernel: "make this much address space usable in my process, with these permissions". The kernel records a new memory region (2.5.3) and returns its starting address. As 2.5.1 showed, no RAM is used yet. Frames appear only as pages are touched.

mmap comes in two flavours.

  • Anonymous mappings are plain memory with no file behind it, filled with zeros when first touched. This is how allocators get memory for the heap.
  • File mappings make a file's contents appear in the process's memory. Section 3 explains these.

And each mapping is either private or shared. Changes to a private mapping stay inside this process, using copy-on-write if the pages came from somewhere shared. Changes to a shared mapping are seen by every process that maps the same thing, and, for a file, are written back to the file. A shared anonymous mapping is exactly the shared memory of 2.2.4.

Its partner, munmap, removes a region and gives its frames back to the kernel.

There is an older interface too. A Unix process's heap originally grew by moving a boundary called the program break, with the system calls brk and sbrk. The heap was one region that could only grow or shrink at its top end. Allocators still use it for small allocations on Linux, but mmap is the general tool. The C library on Linux uses brk for small requests and switches to a separate mmap for each request of 128 KB or more, by default. A large block on its own mapping can be handed straight back to the kernel when freed, which is not possible for a block in the middle of the brk heap.

3. Memory-mapped files ​

A file mapping makes a file look like an array in memory. After mapping a 1 GB file, the program can simply read byte 700,000,000 of that array, and that is byte 700,000,000 of the file. There is no read call, no buffer and no copy.

What happens underneath is the page-fault machinery of 2.5.3:

  1. The program touches an address in the mapped region.
  2. The page is not present, so a page fault happens.
  3. The kernel looks in the page cache, its in-memory copy of file data. If that part of the file is there, it maps that very frame into the process: a minor fault. If not, it reads that part of the file from disk first: a major fault.
  4. The instruction is retried, and the program reads the byte.

Writes to a shared file mapping change the page in the page cache, and the kernel writes it back to the file later.

This gives memory-mapped files some real strengths.

  • No copying. With read, the data is copied from the page cache into the program's buffer. With a mapping, the program reads the page cache's own frames directly.
  • Random access is natural. Jumping to any position is just indexing an array.
  • The kernel manages the memory. Only touched pages use RAM, and under pressure the kernel can simply drop clean file pages, since they can be read again. A program can map a file far larger than RAM.
  • Sharing is free. Every process mapping the same file sees the same frames.

That last point is how every program on your machine is loaded. The program's code and every shared library it uses, such as the C library, are mapped from their files. A hundred processes using the C library all point at the same frames of it. That is the sharing 2.5.1 described.

Memory-mapped files have costs too, and they are why many databases avoid relying on them:

  • Page faults are hidden blocking. A read from a mapped file that misses the page cache stops the thread for a disk read, at a moment the program did not choose and cannot easily see or control.
  • Errors arrive as signals. If the file shrinks, or the disk fails, touching a missing page cannot return an error code. The process gets a signal called SIGBUS instead.
  • The program loses control of what stays in memory. The kernel, not the program, decides which pages to evict.

Node has no built-in way to memory-map a file. For large files it offers streams instead, which read in fixed-size chunks (3.8.4).

4. Inside an allocator ​

Now the upper layer. An allocator gets large regions from the kernel and must answer two kinds of request, fast, millions of times a second:

  • malloc(40): "give me 40 bytes".
  • free(pointer): "I am done with this piece".

Chunks, free lists and size classes ​

The allocator divides its regions into chunks. Each chunk has a small hidden header just before the address it returns, recording the chunk's size. That is how free knows how big a piece is, when you only give it a pointer.

Freed chunks are kept on free lists so they can be reused. To avoid searching one long list for a chunk of the right size, allocators keep separate lists for separate size classes: one list of 16-byte chunks, one of 32-byte chunks, one of 48-byte chunks, and so on, with wider classes for bigger sizes. A request for 40 bytes is rounded up to the 48-byte class and served from that list in a few instructions.

When no chunk of the right size is free, the allocator can split a bigger free chunk into two. When two free chunks sit next to each other, it can merge them back into one bigger chunk, called coalescing, so that large requests can still be met later.

Fragmentation: wasted space of two kinds ​

No allocator uses memory perfectly. The waste has a name, fragmentation, and it comes in two kinds.

  • Internal fragmentation is waste inside a chunk. A 40-byte request served from a 48-byte class wastes 8 bytes. Small per object, but multiplied by millions.
  • External fragmentation is waste between chunks. Suppose the heap has 100 MB free, but scattered in thousands of 4 KB gaps between live objects. A request for one 1 MB block cannot be met from any gap, even though plenty of memory is free in total.

External fragmentation is also why the process in our opening demo kept its memory. The allocator can only give a page back to the kernel if every object on that page is free. After a big burst of allocations and frees, the surviving objects tend to be scattered, keeping many pages partly used. Even when whole regions are free, allocators often keep them on purpose, because the program is likely to need memory again soon and fetching it back from the kernel costs system calls, page faults and, on many-core machines, the TLB shootdowns of 2.5.2. An allocator gives unused pages back with system calls such as madvise, which tells the kernel "these pages are free, take their frames", and different allocators are more or less eager about it.

Allocators and threads ​

In a program with many threads, one shared set of free lists would need a lock (2.4.2) on every malloc and free. That lock would become the most contended lock in the program. So modern allocators give each thread its own cache of free chunks, and only touch shared structures occasionally, in batches.

The C library on Linux, called glibc, uses several arenas that different threads use. Two well-known replacement allocators go further: jemalloc, written for FreeBSD and used widely by large web companies, and tcmalloc, from Google, where "tc" stands for thread-caching. Swapping a program's allocator for one of these, without changing any of its code, sometimes cuts its memory use or speeds it up noticeably. The usual reason is less fragmentation or less contention.

5. How the kernel allocates its own memory ​

The kernel needs memory too: for page tables, for each process's control block, for the page cache. It cannot call malloc. It manages physical frames directly with two allocators of its own.

The buddy allocator: frames in powers of two ​

Physical frames are handed out by the buddy allocator. It only deals in blocks whose size is a power of two pages: 1, 2, 4, 8 pages and so on. Every block has a buddy, the other half of the bigger block it was split from. Here is a small version, with 16 pages of memory.

ts
// buddy.ts — how the kernel hands out physical memory in power-of-two blocks
const TOTAL = 16;                                   // 16 pages of 4 KB = 64 KB, to keep it small
const free = new Map<number, number[]>([[16, [0]]]); // block size -> start pages of free blocks

function allocate(pages: number): number {
  let size = 1;
  while (size < pages) size *= 2;                   // (1) round up to a power of two
  let s = size;
  while (s <= TOTAL && !(free.get(s)?.length)) s *= 2;   // (2) smallest free block that is big enough
  if (s > TOTAL) throw new Error("out of memory");
  let start = free.get(s)!.shift()!;
  while (s > size) {                                // (3) split it in half until it fits
    s /= 2;
    if (!free.has(s)) free.set(s, []);
    free.get(s)!.push(start + s);                   //     the upper half becomes a free "buddy"
  }
  console.log(`asked for ${pages} page(s), got ${size} at page ${start}, wasted ${size - pages}`);
  return start;
}

function release(start: number, size: number): void {
  while (size < TOTAL) {
    const buddy = start ^ size;                     // (4) the buddy's address differs in one bit
    const list = free.get(size) ?? [];
    const i = list.indexOf(buddy);
    if (i < 0) break;                               // buddy is in use: stop merging
    list.splice(i, 1);                              // (5) merge with the buddy
    start = Math.min(start, buddy);
    size *= 2;
  }
  if (!free.has(size)) free.set(size, []);
  free.get(size)!.push(start);
  console.log(`freed; largest free block is now ${Math.max(...[...free].filter(([, l]) => l.length).map(([s]) => s))} pages`);
}

const a = allocate(3);   // a photo buffer needing 3 pages
const b = allocate(1);
const c = allocate(4);
release(b, 1);
release(a, 4);
release(c, 4);

// Prints:
// asked for 3 page(s), got 4 at page 0, wasted 1
// asked for 1 page(s), got 1 at page 4, wasted 0
// asked for 4 page(s), got 4 at page 8, wasted 0
// freed; largest free block is now 4 pages
// freed; largest free block is now 8 pages
// freed; largest free block is now 16 pages

Walk through it with the output.

  • (1) A request for 3 pages is rounded up to 4. That wasted page is internal fragmentation, printed as "wasted 1".
  • (2) At the start, the only free block is all 16 pages.
  • (3) So the allocator splits it: 16 into two 8s, then the first 8 into two 4s. It hands out pages 0–3 and keeps pages 4–7 and 8–15 as free blocks. Each split leaves a free buddy behind.
  • The second request, 1 page, splits the free block 4–7 down to a single page, and gets page 4. The third request, 4 pages, splits 8–15 and gets pages 8–11.
  • (4) Here is the neat part. A block's buddy is found by flipping one bit of its address. For a block of size 4 starting at page 0, the buddy starts at 0 \oplus 4 = 4, where \oplus, read "x-or", flips the bits that are set in the second number. For page 8 it is 8 \oplus 4 = 12. No searching is needed.
  • (5) When a block is freed, the allocator checks whether its buddy is free. If it is, the two merge into one block twice the size, and the check repeats one level up. After all three blocks are freed, everything has merged back into one 16-page block, which is exactly what the last line shows.

The buddy system is fast and fights external fragmentation well, because free neighbours always merge. Its weakness is internal fragmentation: asking for 9 pages costs 16. Linux shows how many free blocks of each size it has in the file /proc/buddyinfo.

The slab allocator: caches of identical objects ​

The kernel allocates enormous numbers of small, identical objects: process control blocks, file records, network buffers. Asking the buddy allocator for a whole page for each would waste most of every page. So the kernel keeps slab caches: for each kind of object, a set of pages already divided into slots exactly that object's size. Allocating a new process control block just takes a free slot from the right cache. Freeing it puts the slot back. The idea is the same as a user-space allocator's size classes, but with one class per kind of object. On Linux, /proc/slabinfo lists the caches, with names like task_struct and dentry.

6. Measuring memory honestly ​

You now have every piece needed to read memory numbers correctly. Tools such as ps and top show several, and they mean different things.

MeasureWhat it counts
VSZ (virtual size)all address space the process has mapped, touched or not
RSS (resident set size)pages of the process that are in RAM now, shared ones included
PSS (proportional set size)RSS, with each shared page split between its sharers
USS (unique set size)only pages no other process shares

VSZ is often huge and means little, because untouched address space costs nothing (2.5.1). RSS is the usual number, but it counts shared libraries in full for every process. PSS and USS answer the real capacity question: "how much would I free by killing this process?".

For a Node program, process.memoryUsage() adds Node's own view on top. heapUsed and heapTotal describe V8's object heap. external and arrayBuffers cover memory held outside it, such as Buffer contents. And rss is the kernel's resident set size for the whole process. The opening demo showed the most important lesson about them: heapUsed falling does not mean rss will fall, because the allocators underneath keep pages for reuse. A process whose RSS stays high after a burst is usually not leaking. A process whose RSS climbs steadily under a constant load usually is.

What the interviewer will push on ​

"What is the role of malloc, compared with the kernel's mmap?" They are checking the two-layer picture. mmap and brk are system calls that give whole regions of pages, which is expensive per call. malloc runs in user space, carves those regions into small chunks, and reuses freed ones, so most allocations need no system call. The tell is mentioning size classes, and the default switch to a separate mmap for large blocks.

"What are memory-mapped files, and when would you use them?" Mapping a file into the address space so that reading memory reads the file, with pages loaded by page faults from the page cache. Use them for random access to large files, and for sharing read-only data between processes. Know the costs: hidden blocking on major faults, SIGBUS on errors, and little control over eviction.

"What is fragmentation?" Internal: waste inside a block from rounding up. External: free memory scattered in pieces too small to use. Strong answers connect external fragmentation to why a process's RSS does not shrink after freeing.

"Explain the buddy allocator." Power-of-two blocks, split in halves to satisfy requests, merged with their buddy when both are free, and the buddy found by flipping one bit. Bonus for naming the slab allocator for small kernel objects.

"My Node service's heap is back to normal but RSS is still high. Is it leaking?" Probably not. The allocators keep freed pages for reuse. Watch whether RSS keeps growing under steady load, which would point to a real leak.

Something to volunteer that nobody asks for: swapping a program's allocator, for example to jemalloc, is one of the few performance changes that needs no code change at all. It sometimes cuts memory use or contention noticeably, which is why large services often try it.

Recall

  • Two layers sit between your object and RAM: the kernel gives whole regions with system calls (mmap, brk), and a user-space memory allocator (malloc) carves them into small pieces without system calls.
  • mmap makes anonymous or file-backed, private or shared regions. Memory-mapped files load pages through page faults from the page cache, with no copying; costs are hidden blocking, SIGBUS on errors and no control over eviction. Programs and shared libraries are loaded this way.
  • Allocators keep chunks with hidden size headers on free lists by size class, split and merge chunks, and give each thread its own cache (glibc arenas, jemalloc, tcmalloc).
  • Fragmentation is internal (rounding up inside a block) or external (free memory in unusable pieces); it, and deliberate reuse, explain why RSS stays high after objects are freed.
  • The kernel allocates frames with the buddy allocator (power-of-two blocks; the buddy is found by flipping one bit) and small kernel objects with slab caches. Read memory as VSZ, resident set size, PSS and USS, which mean different things.

Self-test: Why would calling the kernel for every small object be too slow? What happens, step by step, when a program reads a byte of a memory-mapped file for the first time? Give an example of internal and of external fragmentation. Work out the buddy of a 4-page block at page 8. Why did RSS stay at 314 MB after the heap fell to 7 MB?

Quiz Bank ​

FoundationalWhat is the role of a memory allocator like malloc, compared with the kernel's mmap?

The kernel hands out memory only in whole pages, through system calls such as mmap and brk, and system calls are expensive. Programs allocate small objects constantly. A memory allocator runs inside the program in user mode: it gets large regions from the kernel occasionally, divides them into chunks, and serves each malloc from those regions, usually in a few instructions with no system call. Freed chunks go on free lists for reuse rather than back to the kernel. The allocator also handles fragmentation and decides when to return unused pages to the kernel.

FoundationalWhat is a memory-mapped file?

A memory-mapped file is a file made to appear as a region of a process's memory with mmap. Reading an address in the region reads the corresponding byte of the file, and writes to a shared mapping change the file. The kernel loads pages lazily: touching an unloaded page causes a page fault, and the kernel maps the matching frame from its page cache, reading it from disk first if needed. There is no read call and no copy into a separate buffer, and every process mapping the same file shares the same frames. Programs and shared libraries are loaded this way.

AppliedWhat is the difference between internal and external fragmentation?

Internal fragmentation is space wasted inside an allocated block because requests are rounded up to a fixed size, such as a 40-byte request served from a 48-byte size class, or a 3-page request served by a 4-page buddy block.

External fragmentation is free memory split into pieces that are each too small for a request, even though the total free space is large, such as 100 MB free in scattered 4 KB gaps when a 1 MB block is needed. Merging free neighbours reduces external fragmentation; using finer size classes reduces internal fragmentation.

InterviewExplain how the buddy allocator works.

The buddy allocator manages physical memory in blocks whose sizes are powers of two pages. To satisfy a request, it rounds up to the next power of two, takes the smallest free block that is large enough, and repeatedly splits it in half, keeping each unused half on the free list for its size. The two halves of a split are buddies. When a block is freed, the allocator checks whether its buddy is also free, and if so merges them into one block twice the size, repeating upwards. A block's buddy is found by flipping the one address bit equal to the block size, so no searching is needed. It fights external fragmentation well, at the cost of internal fragmentation from rounding up.

InterviewWhy does a process's memory not go down after it frees most of its objects?

Freeing an object returns it to the allocator, not to the kernel. The allocator can only give a page back if every object on it is free, and after a burst of allocation the survivors are usually scattered, keeping many pages partly used: external fragmentation. Allocators also keep fully free regions on purpose, expecting the program to need memory again, because getting it back from the kernel costs system calls, page faults and possibly TLB shootdowns. So the garbage-collected heap shrinks, but the resident set size stays high. A steady climb under constant load, by contrast, suggests a real leak.

StaffA multi-threaded service spends noticeable time inside malloc and free, and its memory keeps growing slowly although its live data is stable. What would you investigate and try?

Time inside malloc and free in a many-threaded program usually means contention on shared allocator structures, or expensive paths such as splitting, merging and returning memory to the kernel. Slow growth with stable live data usually means fragmentation: freed memory scattered so that it cannot be reused for new requests of different sizes, or per-thread caches holding memory that other threads cannot use.

Investigate allocation sizes and frequencies, and compare the resident set size against the live data the program reports. Then try three things:

  1. A different allocator, such as jemalloc or tcmalloc. It can be swapped in without changing code, and often reduces both contention and fragmentation.
  2. Tuning the number of arenas or the thread-cache size.
  3. Less churn in the program itself, by reusing buffers from a pool instead of allocating and freeing them per request. Measure each change, because allocator behaviour depends heavily on the workload.

Scenario Drill ​

DrillThe photo-upload service must produce a report from a 50 GB file of upload records on a machine with 16 GB of RAM. A colleague reads the whole file into memory and the job crawls or is killed. Using this page and the previous one, give two approaches that work and explain why.

Reading 50 GB into memory on a 16 GB machine makes the working set far larger than RAM. The kernel either swaps constantly, which is thrashing, or the out-of-memory killer ends the job.

Approach 1: stream it. Read and process the file in bounded chunks of a few megabytes, keeping only running totals in memory. The working set stays small and constant however big the file is. This suits a report, which reads every record once in order. In Node, a readable stream over the file does exactly this (3.8.4).

Approach 2: memory-map it. Map the file with mmap and treat it as one large array. The kernel loads only the pages actually touched and can drop clean pages whenever memory runs short, because they can be read from the file again. This suits random access into the file, such as looking up particular records, and it is how some databases work. It needs a language with access to mmap.

Both obey one rule: never make the working set larger than RAM. Streaming bounds it explicitly; mapping lets the kernel bound it automatically.

Next: memory-mapped files, the page cache and swap all led to the same place: the disk. 2.6.1 opens the file system, and shows what a file really is on disk, why a folder is just a special kind of file, and how a tiny fixed-size record can describe a file of terabytes.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Volume I — Technology
Part 0 — Retention & Notation
Part 1 — Foundations of Computation
Part 2 — Operating Systems
2.1 The Kernel
2.2 Processes
2.3 Threads & Scheduling
2.4 Concurrency Under the Hood
2.5 Memory Management
2.6 File Systems
2.7 Input/Output
2.8 Linux for Engineers
2.9 Windows & macOS in Practice
Part 3 — Languages, Compilers & Runtimes
3.6 JavaScript & V8
3.7 TypeScript
3.8 Node.js Internals
3.13 Python & CPython
3.14 FastAPI & Production Python
Part 4 — Data Structures & Algorithms
4.0 Classic Problems — Before NeetCode
4.4 Arrays & Hashing — Problems
4.5 Two Pointers
4.6 Sliding Window
4.8 Stack — Problems
4.9 Linked List — Problems
4.11 Binary Search — Problems
4.13 Trees
4.14 Tree — Problems
4.15 Trie — Problems
4.17 Heap — Problems
4.18 Backtracking — Problems
4.19 Graphs
4.20 Graph — Problems
4.21 Advanced Graph — Problems
4.23 1-D DP — Problems
4.24 2-D DP — Problems
4.26 Greedy — Problems
4.27 Interval — Problems
4.28 Math & Geometry — Problems
4.30 Bit Manipulation — Problems
Part 5 — Networking & the Web
5.3 IP
5.4 Transport
5.6 HTTP
Part 6 — Frontend Engineering
6.1 Browser Internals
6.2 HTML & CSS Mastery
6.3 JavaScript in the Browser
6.4 React, from First Component to Full Shop
6.5 Frameworks & Rendering
6.8 UI Engineering & Patterns
Part 7 — Databases & Storage
7.2 SQL Mastery
7.3 Storage Engine Internals
7.4 Transactions
7.5 NoSQL Taxonomy
7.8 Analytics & Big Data
Part 8 — Security, Identity & Cryptography
8.2 Cryptography
8.3 Protocols of Trust
8.4 Identity: AuthN & AuthZ
8.5 Application Security
8.6 Infrastructure Security
Part 9 — Software Design (LLD)
9.2 OOP Foundations
9.3 Design Principles
9.4 Design Patterns
9.5 Concurrency
9.6 API Design
9.7 LLD Interview Machines
9.9 Node/Express Production Playbook
Part 10 — Distributed Systems & HLD
10.7 Consistency & Consensus
10.8 Messaging & Microservices
10.14 Caching
Building blocks from the design seat
Part 11 — System Design Case Studies
Part 12 — AI & ML Engineering
12.1 What AI Engineering Is
12.2 The Math You Need
12.3 Classical Machine Learning
12.4 Data for ML
12.5 Deep Learning
12.6 Embeddings & Vector Search
12.7 Transformers & LLMs
12.8 Building with LLMs
12.9 Retrieval-Augmented Generation
12.10 Agents
12.11 Adapting Models
12.12 Production AI
12.13 Multimodal AI
12.14 AI Security & Governance
12.15 Case Studies & Capstone
Part 13 — Cloud, DevOps & Infrastructure
13.11 Git, from First Commit to Team Workflow
Part 14 — Containers: Docker & Kubernetes
14.1 Foundations
14.2 Docker
14.3 Kubernetes
14.4 Kubernetes on Azure
Part 15 — The Craft & Career
Nothing picked yet