Skip to content

Volume ITechnology

3.13.13 — Memory, Garbage Collection & the GIL ​

An image-resizing service runs on an 8-core server. The team adds four threads so it can resize four photos at once, and measures. It is no faster. Meanwhile, its memory graph climbs slowly every hour until the process is restarted at night. Both symptoms come from the same place: how CPython manages objects in memory. CPython frees most objects the instant they are no longer used, by counting references. That counting is not safe to do from several threads at once, so CPython protects it with one big lock, the Global Interpreter Lock or GIL, which lets only one thread run Python code at a time. This page explains reference counting, the cycle collector that cleans up what counting misses, how to find memory leaks, and then the GIL: why it exists, what it costs, and the new free-threaded Python that removes it.

1. Reference counting: free it the moment nobody needs it ​

3.4 introduced reference counting as one of the main ways languages manage memory. CPython is the classic example. Every object's header (3.13.2) holds a count of how many references point at it. Binding a name, putting the object in a list or passing it to a function adds one. Unbinding, removing or returning takes one away. When the count reaches zero, the object is freed immediately.

python
import sys

class Order:
    def __init__(self, order_id):
        self.order_id = order_id

    def __del__(self):                        # (1)
        print(f"  freed {self.order_id}")

order = Order("ORD-1")
print(sys.getrefcount(order))                 # (2)
alias = order
print(sys.getrefcount(order))                 # (3)
del alias
del order                                     # (4)
print("after del")
text
2
3
  freed ORD-1
after del
  • (1) __del__ runs just before an object is destroyed, which makes the moment visible.
  • (2) The name order holds one reference, and passing the object to getrefcount temporarily adds another, so it reports 2.
  • (3) A second name adds one more.
  • (4) Deleting the last name drops the count to zero, and the object is freed on that line, before the next print.

That immediacy is a real strength. Files, sockets and locks are released as soon as the last reference goes, memory use is predictable, and there are no long pauses. Inside the interpreter, every place that copies or drops a pointer calls the C macros Py_INCREF or Py_DECREF. That happens millions of times per second, and it is the reason behind the GIL in section 5.

Immortal objects ​

Some objects are used everywhere: None, True, False, small integers. Constantly updating their counts from every thread wasted time. Since Python 3.12 (PEP 683), these are immortal: their count is set to a special huge value and is never changed, and they are never freed.

python
import sys
print(sys.getrefcount(None))
text
3221225472

That number is not a real count of references. It is the "immortal" marker.

2. The weakness of counting: reference cycles ​

Counting has one blind spot. If two objects point at each other, each always has a count of at least one, even when nothing else in the program can reach them.

python
import gc

class Node:
    def __init__(self, name):
        self.name = name
        self.partner = None

    def __del__(self):
        print(f"  freed {self.name}")

gc.disable()                                    # (1)
customer = Node("customer")
order = Node("order")
customer.partner = order                        # (2)
order.partner = customer
del customer, order                             # (3)
print("both unreachable, but not freed")
print("collected:", gc.collect())               # (4)
gc.enable()
text
both unreachable, but not freed
  freed customer
  freed order
collected: 2
  • (1) We switch off the automatic collector so we can watch it work.
  • (2) A cycle: each object holds a reference to the other.
  • (3) Both names are gone, so nothing in the program can reach either object. But each still has a count of 1, from its partner. Counting alone will never free them.
  • (4) gc.collect() runs the cycle collector, which finds and frees them.

Cycles are common in real code: a parent and child that point at each other, an object that stores one of its own bound methods, an exception that holds its traceback, which holds the frame, which holds the exception.

How the cycle collector finds garbage ​

CPython's cycle collector, added in Python 2.0, only looks at container objects, the ones that can hold references, such as lists, dicts, sets and class instances. Numbers and strings cannot form cycles, so they are skipped. Here is its algorithm, simplified.

  1. For each container in the group being collected, copy its reference count into a scratch field.
  2. For each container, look at every object it references. If that target is also in the group, subtract one from the target's scratch count. This removes the references that come from inside the group.
  3. Any object whose scratch count is still above zero is referenced from outside the group, by a variable, a module or the call stack. It is definitely alive. Mark it, and everything reachable from it, as alive.
  4. Everything not marked is reachable only from inside the group. It is garbage. Free it.

Figure 1 runs the algorithm on our two nodes, plus one live order that a variable still points at.

unreachable cyclecustomercount 1 → 0ordercount 1 → 0only references from inside the group④ freedstill in usevariablelive ordercount 1 → 1a reference from outside the group③ marked alive② subtract every reference that comes from inside the group
Figure 1 — Finding garbage without knowing the roots. After internal references are subtracted, the cycle's counts fall to zero, so nothing outside points at it. The live order keeps a positive count, because a variable outside the group points at it.

The clever part is step 2. The collector never needs a list of all the program's variables. It works out "referenced from outside" purely by subtraction.

Generations ​

Scanning every container on every collection would be slow. So the collector uses the observation from 3.4 called the generational hypothesis: most objects die young. Containers are kept in three groups called generations.

  • New containers go into generation 0.
  • When generation 0 has had 2,000 more allocations than deallocations, it is collected. Survivors move to generation 1.
  • Every 10 collections of generation 0, generation 1 is collected, and its survivors move to generation 2.
  • Generation 2, the long-lived objects, is collected even less often.
python
import gc
print(gc.get_threshold())
text
(2000, 10, 10)

There is a small story here worth knowing. Python 3.14.0 shipped a new incremental collector, designed to cut long pauses by scanning old objects a slice at a time. Several companies then reported much higher memory use in production, so Python 3.14.5 reverted to the generational collector from 3.13. It is a good example of a performance change that looked right in benchmarks and failed under real workloads.

Turning the collector off ​

Since reference counting frees most objects anyway, some services switch the cycle collector off. In 2017 Instagram disabled it in their Django web servers. Each collection was touching every old object, which on Linux forced the operating system to copy memory pages that worker processes had been sharing. That is the copy-on-write behaviour from Part 2. Disabling it saved about 10% of memory across their fleet. Python 3.7 then added gc.freeze(), which moves every existing object into a permanent group the collector ignores. Servers call it after start-up, before creating worker processes, to get the same benefit safely.

If you disable the collector, cycles really do leak, so only do it with measurements and with code that avoids cycles.

3. __del__ and weak references ​

__del__ looks like a destructor, but in Python it is a risky tool.

  • With reference counting it usually runs promptly, but for objects in a cycle it runs only when the cycle collector gets to them, which may be much later.
  • Exceptions raised inside it are printed and ignored.
  • At interpreter shutdown, module globals it relies on may already be gone.

So do not put important cleanup in __del__. Use a context manager (3.13.9) to close files and connections deterministically.

Weak references ​

Sometimes you want to refer to an object without keeping it alive. A cache of recently loaded orders is the classic case: if nothing else in the program uses an order any more, the cache should not be the only thing holding it in memory. A weak reference points at an object without adding to its count.

python
import weakref

class Order:
    def __init__(self, order_id):
        self.order_id = order_id
    def __del__(self):
        print(f"  freed {self.order_id}")

cache = weakref.WeakValueDictionary()          # (1)
order = Order("ORD-9")
cache["ORD-9"] = order
print(list(cache.keys()))

del order                                      # (2)
print(list(cache.keys()))
text
['ORD-9']
  freed ORD-9
[]
  • (1) A dict whose values are held weakly.
  • (2) When the last strong reference goes, the order is freed, and the cache entry disappears by itself.

weakref.ref(obj) gives a single weak reference, which you call to get the object, or None if it has died. Weak references are also how you break unwanted cycles, for example a child that refers back to its parent weakly.

4. Where the memory comes from, and why it stays high ​

Calling the operating system for memory on every small object would be slow. CPython has its own allocator for small objects, called pymalloc. It requests large chunks, called arenas, of 1 MiB each on modern 64-bit builds. It divides each arena into pools, and each pool into fixed-size blocks for one size class, such as all 32-byte objects. Allocating a small object is then just taking a free block from the right pool, which is very fast. Objects larger than 512 bytes go straight to the system allocator.

This design has one visible side effect. An arena can only be returned to the operating system when every block in it is free. A long-running server that briefly held a million objects, then freed most of them, may keep many partly used arenas forever. Its memory, as seen by tools like top, stays high even though Python is not using it. That is one reason servers such as Gunicorn can restart each worker after a set number of requests.

Finding real leaks with tracemalloc ​

A true memory leak in Python is almost always a reference you forgot about: a module-level list that keeps growing, an unbounded cache, a @lru_cache on a method (3.13.5), or a closure holding a large object. The standard library's tracemalloc records where each allocation came from, so you can compare two snapshots.

python
import tracemalloc

tracemalloc.start()
seen_requests = []                              # (1)

def handle_request(request_id):
    seen_requests.append(f"request-{request_id}-" * 20)

before = tracemalloc.take_snapshot()
for request_id in range(5000):
    handle_request(request_id)
after = tracemalloc.take_snapshot()

for stat in after.compare_to(before, "lineno")[:1]:     # (2)
    print(stat)
text
leak_demo.py:7: size=1489 KiB (+1489 KiB), count=5001 (+5001), average=305 B
  • (1) A module-level list that is never cleared: the classic leak.
  • (2) Comparing the snapshots points straight at line 7, the append, with 5,001 new objects and 1.5 MB of growth.

In production, take a snapshot at start-up and another after an hour of traffic, and compare them in the same way.

5. The Global Interpreter Lock ​

Now the opening story. Reference counts are changed constantly. If two threads both changed the same object's count at the same moment, one update could be lost (3.13.14 shows exactly how). A lost decrement leaks the object. A lost increment frees an object that is still in use, which crashes the interpreter.

Python added threads in 1992. The simplest way to make reference counting safe was one lock around the whole interpreter: a thread must hold the Global Interpreter Lock to run Python bytecode. That one decision gave Python three things.

  1. Fast single-threaded code. No per-object locking at all.
  2. A simple C extension interface. Extension authors did not have to think about thread safety in the interpreter's internals, which is a big reason Python gained so many C libraries.
  3. No parallel Python bytecode. Only one thread runs Python code at any instant.

When the GIL is released ​

The GIL is not held forever by one thread. It is handed over in two situations.

  • Every 5 milliseconds by default, a running thread is asked to release it so another thread gets a turn. You can see the setting with sys.getswitchinterval(), which returns 0.005.
  • Whenever a thread waits or works outside Python. Blocking I/O, such as reading a socket, waiting for a database, time.sleep, or reading a file, releases the GIL during the wait. So do many C libraries while they crunch numbers: NumPy, hashlib on large data, zlib compression and image libraries.

That explains which programs threads help.

  • I/O-bound work, such as calling APIs, querying databases or downloading files, spends most of its time waiting with the GIL released. Threads overlap those waits beautifully. A web scraper with 20 threads really is about 20 times faster.
  • CPU-bound pure Python, such as loops, parsing or calculations in Python code, needs the GIL the whole time. Threads take turns, and you get no speed-up.

Measuring it ​

Here is a pure-Python CPU task, run once on one thread and then split across four threads.

python
import sys
import threading
import time

def count_down(n):
    while n > 0:
        n -= 1

N = 20_000_000
start = time.perf_counter()
count_down(N)
one_thread = time.perf_counter() - start

workers = [threading.Thread(target=count_down, args=(N // 4,)) for _ in range(4)]
start = time.perf_counter()
for worker in workers:
    worker.start()
for worker in workers:
    worker.join()
four_threads = time.perf_counter() - start

print(f"GIL enabled: {sys._is_gil_enabled()}")
print(f"1 thread : {one_thread:.2f}s")
print(f"4 threads: {four_threads:.2f}s")

Run on an 8-core laptop with normal CPython 3.14:

text
GIL enabled: True
1 thread : 0.61s
4 threads: 0.61s

Four threads, four times the cores, and exactly the same time. That is the opening story, measured. For CPU-bound work on standard CPython, use processes, which each have their own interpreter and their own GIL, as 3.13.14 shows.

The GIL does not make your code thread-safe ​

A common misunderstanding is that the GIL means you never need locks. It does not. The GIL can switch threads between any two bytecode instructions, and an innocent line like stock -= 1 is several instructions: read, subtract, write. Two threads can both read 5, both write 4, and one sale is lost. The GIL protects the interpreter's internals, not your data. 3.13.14 shows this race happening and how to fix it.

6. Removing the GIL: free-threaded Python ​

People tried to remove the GIL for twenty years. In 1999, a patch replaced it with fine-grained locks and made single-threaded code about twice as slow, so it was rejected. Guido's rule became: remove the GIL only if single-threaded code does not get slower. Larry Hastings's "Gilectomy" project, started in 2016, hit the same wall.

The breakthrough came from Sam Gross at Meta, who in 2021 published a working "nogil" fork using several new ideas. Each object records which thread owns it, so the owning thread can update its count cheaply without atomic instructions. Other threads use a separate, slower counter. Immortal objects skip counting entirely. And a thread-safe memory allocator called mimalloc replaces pymalloc. The Steering Council accepted it as PEP 703 in 2023, with a careful plan.

  • 3.13 (2024): an optional, experimental free-threaded build, with a t suffix, as in python3.13t.
  • 3.14 (2025): officially supported but still optional (PEP 779). According to the 3.14 release notes, the single-threaded slowdown is now roughly 5–10%.
  • Later: possibly the default, once the ecosystem of C extensions is ready.

Here is the same benchmark on the free-threaded 3.14 build, on the same laptop.

text
$ python3.14t gil_demo.py
GIL enabled: False
1 thread : 0.65s
4 threads: 0.26s

The single thread is a little slower, as promised. But four threads now really run in parallel, 2.5 times faster. Even in this build you can switch the GIL back on with -X gil=1, and some C extensions that are not yet marked as safe will switch it back on automatically when they are imported.

Subinterpreters ​

A second, separate approach runs several interpreters inside one process, each with its own GIL. Python 3.12 gave each subinterpreter its own GIL (PEP 684), and Python 3.14 added the concurrent.interpreters module (PEP 734) to use them from Python code. They share no Python objects. They communicate by passing data, much like processes, but they start faster and use less memory. Think of them as lighter processes, not as shared-memory threads.

What the interviewer will push on ​

"How does Python manage memory?" Reference counting frees most objects immediately. A generational cycle collector cleans up reference cycles among container objects. Small objects come from the pymalloc allocator. The tell of depth is explaining why the cycle collector is needed with a two-object cycle, and how it finds garbage by subtracting internal references.

"What is the GIL, and why does Python have it?" A lock that lets only one thread execute Python bytecode at a time. It exists because reference counting is not thread-safe without it, and one lock kept single-threaded code fast and C extensions simple. The common wrong answer is "Python is single-threaded". It has real OS threads. They just do not run Python bytecode in parallel.

"Then why use threads in Python at all?" Because the GIL is released during blocking I/O and inside many C libraries. Threads work very well for I/O-bound work. For CPU-bound pure Python, use processes, or the free-threaded build.

"Does the GIL make counter += 1 thread-safe?" No. It is several bytecode instructions, and a thread switch can happen between them. Use a Lock. This question separates people who have debugged a race from those who have read a slogan.

"What is the status of removing the GIL?" PEP 703. Experimental free-threaded build in 3.13. Officially supported but optional in 3.14 with a 5–10% single-thread cost. Not yet the default. Mention subinterpreters with their own GIL as a separate approach.

Something to volunteer that nobody asks for: Python 3.14.0 shipped an incremental garbage collector, and 3.14.5 reverted it after production reports of much higher memory use. It shows that you follow what actually happens in releases, and that you know garbage-collector changes are judged by real workloads, not benchmarks.

Recall

  • CPython frees objects by reference counting, immediately when the count hits zero. sys.getrefcount adds one for its own argument. Since 3.12, None, booleans and small ints are immortal.
  • Counting cannot free reference cycles. The cycle collector scans container objects, subtracts internal references, keeps anything still referenced from outside and frees the rest. It uses three generations with thresholds (2000, 10, 10). 3.14.0's incremental collector was reverted in 3.14.5.
  • gc.freeze() after start-up avoids copy-on-write in forked workers. Do not rely on __del__ for cleanup. Use context managers. A weak reference (weakref.ref, WeakValueDictionary) does not keep an object alive.
  • pymalloc gives small objects blocks from pools inside 1 MiB arenas. Memory may stay high after objects are freed. Find a memory leak with tracemalloc snapshot comparisons.
  • The Global Interpreter Lock lets one thread run bytecode at a time, because reference counting is not thread-safe. It is released every 5 ms and during I/O and many C calls. Threads help I/O-bound work, not CPU-bound Python. The GIL does not make your code thread-safe.
  • Free-threaded Python (PEP 703): experimental in 3.13, supported but optional in 3.14, about 5–10% slower single-threaded and truly parallel. Subinterpreters each have their own GIL.

Self-test: Why is customer.partner = order; order.partner = customer a problem for reference counting? How does the cycle collector know an object is referenced from outside without a list of variables? Why does a four-thread CPU loop take the same time as one thread? Why does a threaded web scraper still get faster?

Quiz Bank ​

FoundationalWhen exactly is a Python object freed?

In CPython, an object is freed the moment its reference count drops to zero, for example when the last name bound to it is deleted or rebound, or when the list holding it is cleared. If the object is part of a reference cycle, its count never reaches zero on its own. It is freed later, when the cycle collector runs on the generation it belongs to, or when gc.collect() is called. Immortal objects such as None are never freed. These timings are CPython behaviour, not language guarantees, so code should not depend on them for cleanup.

AppliedA Django service's memory grows every hour and drops only on restart. How would you investigate?

First decide whether it is a real leak or allocator behaviour. Track the number of live objects with len(gc.get_objects()) and Python-level allocated memory with tracemalloc over time. If Python's own numbers stay flat while the process size grows, it is fragmentation in partly used arenas or a C extension, and recycling workers after N requests is a pragmatic fix. If they grow, take a tracemalloc snapshot after warm-up and another an hour later and compare by line. That usually points at an unbounded module-level structure: a cache dict without a size limit, a list of seen IDs, @lru_cache on methods holding instances, or objects kept alive by a long-lived closure or signal handler. Fix the reference, and use weakref or a bounded cache where retention is intended.

InterviewExplain why Python threads do not speed up CPU-bound code, and what you would use instead.

In standard CPython, a thread must hold the Global Interpreter Lock to execute bytecode, so only one thread runs Python code at any moment. CPU-bound pure Python never waits, so threads just take turns every 5 ms, and the total time stays the same or gets slightly worse from switching. Alternatives: multiprocessing or ProcessPoolExecutor, which give each worker its own interpreter and GIL on separate cores, at the cost of pickling data between processes. Moving the hot loop into C-backed code such as NumPy, which releases the GIL, also works. On Python 3.14, the free-threaded build runs threads in parallel for real, if every C extension you use supports it.

StaffYour team wants to switch production to the free-threaded build of Python 3.14. What would you check before agreeing?

Check that every C extension in the dependency tree declares free-threading support. Otherwise importing it re-enables the GIL and you pay the overhead for nothing. Benchmark real workloads, not micro-benchmarks: single-threaded paths get about 5–10% slower, so the gain only exists if the service actually runs CPU-bound Python in parallel threads. A typical async web service mostly waits on I/O and may see no benefit. Audit your own code for data races that the GIL used to hide by accident, such as shared dicts and counters updated without locks, because they become far more likely to cause bugs. Roll out behind a canary with memory and latency dashboards, since the allocator and reference-counting scheme differ. Keep the standard build as the rollback.

Next: you now know why threads behave the way they do. 3.13.14 puts that to work: the race inside += 1, every synchronisation tool in the threading module, thread and process pools, and how to choose between them.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Volume I — Technology
Part 0 — Retention & Notation
Part 1 — Foundations of Computation
Part 2 — Operating Systems
2.1 The Kernel
2.2 Processes
2.3 Threads & Scheduling
2.4 Concurrency Under the Hood
2.5 Memory Management
2.6 File Systems
2.7 Input/Output
2.8 Linux for Engineers
2.9 Windows & macOS in Practice
Part 3 — Languages, Compilers & Runtimes
3.6 JavaScript & V8
3.7 TypeScript
3.8 Node.js Internals
3.13 Python & CPython
3.14 FastAPI & Production Python
Part 4 — Data Structures & Algorithms
4.0 Classic Problems — Before NeetCode
4.4 Arrays & Hashing — Problems
4.5 Two Pointers
4.6 Sliding Window
4.8 Stack — Problems
4.9 Linked List — Problems
4.11 Binary Search — Problems
4.13 Trees
4.14 Tree — Problems
4.15 Trie — Problems
4.17 Heap — Problems
4.18 Backtracking — Problems
4.19 Graphs
4.20 Graph — Problems
4.21 Advanced Graph — Problems
4.23 1-D DP — Problems
4.24 2-D DP — Problems
4.26 Greedy — Problems
4.27 Interval — Problems
4.28 Math & Geometry — Problems
4.30 Bit Manipulation — Problems
Part 5 — Networking & the Web
5.3 IP
5.4 Transport
5.6 HTTP
Part 6 — Frontend Engineering
6.1 Browser Internals
6.2 HTML & CSS Mastery
6.3 JavaScript in the Browser
6.4 React, from First Component to Full Shop
6.5 Frameworks & Rendering
6.8 UI Engineering & Patterns
Part 7 — Databases & Storage
7.2 SQL Mastery
7.3 Storage Engine Internals
7.4 Transactions
7.5 NoSQL Taxonomy
7.8 Analytics & Big Data
Part 8 — Security, Identity & Cryptography
8.2 Cryptography
8.3 Protocols of Trust
8.4 Identity: AuthN & AuthZ
8.5 Application Security
8.6 Infrastructure Security
Part 9 — Software Design (LLD)
9.2 OOP Foundations
9.3 Design Principles
9.4 Design Patterns
9.5 Concurrency
9.6 API Design
9.7 LLD Interview Machines
9.9 Node/Express Production Playbook
Part 10 — Distributed Systems & HLD
10.7 Consistency & Consensus
10.8 Messaging & Microservices
10.14 Caching
Building blocks from the design seat
Part 11 — System Design Case Studies
Part 12 — AI & ML Engineering
12.1 What AI Engineering Is
12.2 The Math You Need
12.3 Classical Machine Learning
12.4 Data for ML
12.5 Deep Learning
12.6 Embeddings & Vector Search
12.7 Transformers & LLMs
12.8 Building with LLMs
12.9 Retrieval-Augmented Generation
12.10 Agents
12.11 Adapting Models
12.12 Production AI
12.13 Multimodal AI
12.14 AI Security & Governance
12.15 Case Studies & Capstone
Part 13 — Cloud, DevOps & Infrastructure
13.11 Git, from First Commit to Team Workflow
Part 14 — Containers: Docker & Kubernetes
14.1 Foundations
14.2 Docker
14.3 Kubernetes
14.4 Kubernetes on Azure
Part 15 — The Craft & Career
Nothing picked yet