Skip to content

Volume ITechnology

3.13.14 — Threads, Processes & concurrent.futures ​

A flash sale puts ten units of a phone on sale. Fifty customers click "Buy" in the same second. The service handles each click in its own thread. When the dust settles, the database says sixteen phones were sold, and the stock count reads minus six. Six customers have paid for phones that do not exist. Nothing crashed, no error was logged, and the code looked obviously correct. This page shows exactly how that happens, then covers every tool Python gives you to run work concurrently and safely: threads and their locks, queues, thread pools, process pools, and the newest option, interpreter pools. It finishes with a simple way to choose between them.

3.13.13 explained the Global Interpreter Lock. Keep its two lessons in mind. Threads help when work waits, and do not help pure-Python work that computes. And the GIL does not protect your data. Part 2's concurrency chapter covers what happens underneath: atomic CPU instructions in 2.4.1 and how locks are built in 2.4.2. This page is about doing it in Python.

1. Starting a thread ​

A thread is a separate path of execution inside the same process, sharing the same memory. The threading module creates one around any function.

python
import threading
import time

def send_receipt(order_id):
    time.sleep(0.2)                                             # (1) pretend to call an email API
    print(f"  receipt sent for {order_id}")

worker = threading.Thread(target=send_receipt, args=("ORD-7",))   # (2)
worker.start()                                                  # (3)
print("checkout response returned")
worker.join()                                                   # (4)
text
checkout response returned
  receipt sent for ORD-7
  • (1) sleep stands in for a slow network call. It releases the GIL while it waits.
  • (2) Create the thread, naming the function and its arguments.
  • (3) start() begins running it alongside the main thread. The main thread carries on at once, which is why its print appears first.
  • (4) join() waits for the thread to finish.

A thread created with daemon=True is killed abruptly when the main program exits, without running any cleanup. That is fine for a background heartbeat. It is dangerous for anything that writes data, because the write can be cut off halfway.

2. The race: how ten phones became sixteen ​

Here is the flash sale, reduced to its core.

python
import threading
import time

stock = 10
sold = 0

def buy():
    global stock, sold
    if stock > 0:                    # (1) check
        time.sleep(0.001)            # (2) e.g. reserve payment
        stock -= 1                   # (3) act
        sold += 1

customers = [threading.Thread(target=buy) for _ in range(50)]
for customer in customers:
    customer.start()
for customer in customers:
    customer.join()
print("stock:", stock, "sold:", sold)
text
stock: -6 sold: 16

Your exact numbers will vary from run to run, which is part of what makes races so hard to debug. Here is one way it goes wrong.

  1. Thread A checks stock > 0 at line (1). There is 1 left, so the check passes.
  2. Thread A pauses at line (2) to reserve the payment. While it waits, the GIL passes to other threads.
  3. Thread B checks stock > 0. Thread A has not decremented yet, so B also sees 1, and also passes.
  4. Both threads reach line (3). Both decrement. Stock is now −1, and two customers paid for one phone.

This pattern is called check-then-act: a decision based on a value that another thread can change before you act on it. The section of code that must run as one indivisible step, here lines (1) to (3), is called a critical section.

Even += 1 is not one step ​

You might think the problem is only the sleep. It is not. Look at the bytecode for a single increment.

python
import dis

counter = 0
def record_sale():
    global counter
    counter += 1

dis.dis(record_sale)
text
  4           RESUME                   0

  6           LOAD_GLOBAL              0 (counter)
              LOAD_SMALL_INT           1
              BINARY_OP               13 (+=)
              STORE_GLOBAL             0 (counter)
              LOAD_CONST               1 (None)
              RETURN_VALUE

One line of Python is four instructions: load the value, load 1, add, store the result. If a thread is paused after the load but before the store, another thread can load the same old value, and one of the two increments is lost.

With the standard GIL build, this particular race is rare in practice, because CPython only switches threads at certain points, such as loops and function calls. Here are four threads each adding one a million times.

text
$ python counter_race.py          # standard build
counter: 4000000

$ python3.14t counter_race.py     # free-threaded build
counter: 1227506

On the standard build the answer happened to be right. On the free-threaded build from 3.13.13, where threads really run at the same time, almost 70% of the increments were lost. Code that was "correct by luck" for twenty years breaks the moment the luck is removed. The lesson: any shared value changed by more than one thread needs a lock, whatever build you run on.

3. Locks ​

A lock, also called a mutex (short for mutual exclusion), is held by at most one thread at a time. A thread that tries to take a lock someone else holds waits until it is released. Wrapping the critical section in a lock makes it run as one step.

python
import threading
import time

stock = 10
sold = 0
stock_lock = threading.Lock()                # (1)

def buy():
    global stock, sold
    with stock_lock:                         # (2)
        if stock > 0:
            time.sleep(0.001)
            stock -= 1
            sold += 1

customers = [threading.Thread(target=buy) for _ in range(50)]
for customer in customers:
    customer.start()
for customer in customers:
    customer.join()
print("stock:", stock, "sold:", sold)
text
stock: 0 sold: 10
  • (1) One lock guards the stock and the count.
  • (2) with takes the lock on entry and releases it on exit, even if an exception is raised. Never call acquire() and release() by hand when with will do. A forgotten release after an exception freezes every other thread forever.

Exactly ten sold, every time. But notice the cost. The sleep now happens inside the lock, so all fifty customers queue up one by one. Keep critical sections as short as possible, and never do slow I/O while holding a lock if you can avoid it. In a real shop, the correct fix lives in the database: an atomic UPDATE products SET stock = stock - 1 WHERE id = ? AND stock > 0, which lets the database do the check and the act as one step. Part 7 covers this properly.

The other synchronisation tools ​

The threading module has several more tools, each for one specific situation.

ToolUse when
Lockone thread at a time in a section
RLockthe same thread may re-enter
Semaphore(n)at most n threads at once
Eventone thread signals "ready" to others
Conditionwait until some state changes
Barrier(n)n threads wait for each other

An RLock, or re-entrant lock, can be taken again by the thread that already holds it. A plain Lock taken twice by the same thread waits for itself forever, which is a deadlock. A Semaphore is the right tool for "a payment gateway allows at most 5 calls at once". An Event is the simplest way for one thread to tell others to start or stop.

python
import threading
import time

gateway_slots = threading.Semaphore(2)                    # (1)
shutdown = threading.Event()                              # (2)

def charge(order_id):
    with gateway_slots:
        print(f"  charging {order_id}")
        time.sleep(0.1)

def heartbeat():
    while not shutdown.wait(timeout=0.05):                # (3)
        pass
    print("  heartbeat stopped")

beat = threading.Thread(target=heartbeat)
beat.start()
workers = [threading.Thread(target=charge, args=(f"ORD-{n}",)) for n in range(4)]
for worker in workers:
    worker.start()
for worker in workers:
    worker.join()
shutdown.set()                                            # (4)
beat.join()
text
  charging ORD-0
  charging ORD-1
  charging ORD-2
  charging ORD-3
  heartbeat stopped
  • (1) At most two threads may be inside the with block at once. The other two wait for a free slot. You would see the first two lines appear together, then the second two 0.1 seconds later.
  • (2) An Event starts "not set".
  • (3) wait(timeout) returns True as soon as the event is set, or False after the timeout. This loop checks for shutdown twenty times a second without wasting CPU.
  • (4) Setting the event wakes every waiting thread.

Deadlocks ​

A deadlock happens when two threads each hold one lock and wait for the other's. Thread A holds the orders lock and wants inventory. Thread B holds inventory and wants orders. Neither will ever let go. The standard prevention is a fixed lock order: every piece of code that needs both locks takes orders first, then inventory. 2.4.3 covers the four conditions for deadlock in depth.

4. Queues: share by passing messages ​

Locks protect shared data. A better design often avoids sharing in the first place: threads pass work to each other through a queue. queue.Queue is a thread-safe first-in, first-out list with all the locking built in. This is the producer-consumer pattern.

python
import queue
import threading

orders = queue.Queue(maxsize=100)                    # (1)

def packer(name):
    while True:
        order_id = orders.get()                      # (2)
        if order_id is None:                         # (3)
            orders.task_done()
            break
        print(f"  {name} packed {order_id}")
        orders.task_done()                           # (4)

packers = [threading.Thread(target=packer, args=(f"packer-{n}",)) for n in range(2)]
for p in packers:
    p.start()
for n in range(4):
    orders.put(f"ORD-{n}")                           # (5)
for _ in packers:
    orders.put(None)
orders.join()                                        # (6)
print("all orders packed")
text
  packer-0 packed ORD-0
  packer-1 packed ORD-1
  packer-0 packed ORD-2
  packer-1 packed ORD-3
all orders packed
  • (1) maxsize bounds the queue. If producers are faster than consumers, put waits instead of letting memory grow without limit. That is backpressure: a slow consumer automatically slows the producer down.
  • (2) get waits until an item is available.
  • (3) A special value, often called a sentinel or poison pill, tells each worker to stop. We send one per worker.
  • (4) task_done reports that one item is finished.
  • (5) The producer adds work.
  • (6) join on the queue waits until every item has been marked done.

The order of the printed lines can vary between runs. With a queue, the only shared object is the queue itself, which is already safe. That removes a whole class of bugs.

5. concurrent.futures: pools instead of hand-made threads ​

Creating threads by hand, collecting results and handling their exceptions is repetitive. The concurrent.futures module, added in Python 3.2, gives you a thread pool: a fixed set of worker threads that run whatever tasks you submit, and hand back futures. A future is an object representing a result that is not ready yet. You can ask it for the result, which waits if needed.

Here, a checkout page needs exchange rates for four currencies, and each call takes 0.2 seconds.

python
import time
from concurrent.futures import ThreadPoolExecutor

def fetch_rate(currency):
    time.sleep(0.2)                                            # (1)
    return currency, {"USD": 83.2, "EUR": 90.1, "GBP": 105.4, "JPY": 0.56}[currency]

currencies = ["USD", "EUR", "GBP", "JPY"]

start = time.perf_counter()
rates = [fetch_rate(c) for c in currencies]
print(f"sequential: {time.perf_counter() - start:.2f}s")

start = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as pool:                # (2)
    rates = dict(pool.map(fetch_rate, currencies))             # (3)
print(f"thread pool: {time.perf_counter() - start:.2f}s", rates)
text
sequential: 0.80s
thread pool: 0.20s {'USD': 83.2, 'EUR': 90.1, 'GBP': 105.4, 'JPY': 0.56}
  • (1) The slow part is waiting, which releases the GIL.
  • (2) with shuts the pool down cleanly at the end, waiting for running tasks.
  • (3) map runs the function on each item in the pool and returns the results in input order.

Four waits overlapped, so the total time is that of one wait. When you want results as they finish, use submit and as_completed.

python
from concurrent.futures import ThreadPoolExecutor, as_completed

with ThreadPoolExecutor(max_workers=4) as pool:
    futures = {pool.submit(fetch_rate, c): c for c in currencies}   # (1)
    for future in as_completed(futures):                            # (2)
        currency, rate = future.result()                            # (3)
        print(f"  {currency}: {rate}")
  • (1) submit returns a future immediately for each call.
  • (2) as_completed yields futures in the order they finish, so a fast result is not stuck behind a slow one.
  • (3) result() returns the value. If the function raised an exception in the worker thread, result() re-raises it here, in your thread. Exceptions are not lost, as long as you call result().

6. Processes: real parallelism for CPU work ​

For CPU-bound pure Python, threads do not help, because of the GIL. A process is a separate running program with its own memory and its own interpreter, and so its own GIL. ProcessPoolExecutor has exactly the same interface as the thread pool, but runs tasks in separate processes on separate cores.

python
import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def checksum(n):                                         # pure-Python CPU work
    total = 0
    for i in range(n):
        total = (total * 31 + i) % 1_000_003
    return total

if __name__ == "__main__":                               # (1)
    jobs = [15_000_000] * 4
    for label, pool_class in [("threads", ThreadPoolExecutor), ("processes", ProcessPoolExecutor)]:
        start = time.perf_counter()
        with pool_class(max_workers=4) as pool:
            list(pool.map(checksum, jobs))
        print(f"{label:9}: {time.perf_counter() - start:.2f}s")

On an 8-core laptop, where the four jobs run one after another in 3.97 seconds:

text
threads  : 3.87s
processes: 1.96s
  • (1) This guard is required with processes on Windows and macOS. Each worker process starts a fresh interpreter and imports your module. Without the guard, each worker would create its own pool, and so on, forever.

Threads gave nothing. Processes halved the time. Not quite four times faster, because starting processes and moving data between them costs time, and because this laptop's eight "cores" are four physical cores with two hardware threads each.

The price of processes: pickling ​

Processes share no memory. Every argument sent to a worker, and every result sent back, is converted to bytes with the pickle module, Python's built-in object serialiser, then copied through a pipe and rebuilt on the other side. Three consequences follow.

  1. Big arguments are slow. Sending a 500 MB list to each worker means pickling and copying 500 MB each time. Send small inputs, such as file names or ID ranges, and let workers load their own data.
  2. Not everything can be pickled. Lambdas, nested functions, open files, locks and database connections cannot.
python
from concurrent.futures import ProcessPoolExecutor

if __name__ == "__main__":
    with ProcessPoolExecutor(2) as pool:
        print(list(pool.map(lambda n: n * 2, [1, 2])))
text
PicklingError: Can't pickle <function <lambda> at 0x00000244CF641170>: it's not found as __main__.<lambda>

Pickle stores a function by name, so the worker can import it. A lambda has no importable name. Use a normal top-level def. 3. Changes in a worker stay in the worker. A worker appending to a global list changes its own copy only. Return results instead.

How workers start: spawn, fork and forkserver ​

There are three ways to start a worker process.

  • spawn starts a brand-new interpreter and imports your module. It is safe but slower to start. It is the default on Windows and macOS.
  • fork, Linux only, copies the parent process instantly, including all its memory. It is fast, but it is dangerous if the parent has other threads running, because only the forking thread is copied, and locks held by the other threads stay locked forever in the child.
  • forkserver starts one clean server process early, and forks workers from that clean server instead of from your busy main process.

Python 3.14 changed the default on Linux from fork to forkserver, precisely because of those fork-with-threads deadlocks. Code that relied on workers silently inheriting global variables from the parent must now pass that data explicitly.

Interpreter pools: the newest option ​

Python 3.14 added InterpreterPoolExecutor, which runs tasks in subinterpreters inside the same process, each with its own GIL (3.13.13). It has the same interface. In the same benchmark it took 1.68 seconds, a little faster than processes, because starting an interpreter is cheaper than starting a process. Data still has to be copied between interpreters, and many C extensions do not support subinterpreters yet. Treat it as promising rather than a default.

7. Choosing the right tool ​

The decision depends on one question: is the work waiting, or computing?

WorkToolWhy
Many network/database callsasynciothousands of waits, one thread
Some blocking I/O, sync librariesThreadPoolExecutorGIL released while waiting
CPU-heavy pure PythonProcessPoolExecutorone GIL per process
CPU-heavy, NumPy-style C codethreads often enoughC code releases the GIL
CPU-heavy, free-threaded buildthreadsno GIL

The first row is the subject of the next page. When a program waits on hundreds or thousands of connections, as a web server does, even threads become expensive. Each one costs memory for its stack and time for the operating system to switch between them. asyncio handles all those waits inside one thread, with no locks needed for most code. FastAPI is built on it.

What the interviewer will push on ​

"Is counter += 1 thread-safe in Python?" No. It compiles to load, add and store, and a thread switch between them loses an update. Show dis. The strong answer adds that the standard build rarely shows it because of where CPython switches threads, while the free-threaded build loses most updates. So the rule is: lock shared mutable state.

"Threads or processes for this task?" Ask whether the work waits or computes. I/O-bound: threads or asyncio. CPU-bound pure Python: processes. Mention pickling costs and the if __name__ == "__main__" guard.

"What is a deadlock, and how do you prevent one?" Two threads each holding a lock the other needs. Prevent it with a global lock order, timeouts on acquire, fewer locks, or message passing with queues. Mention that RLock exists for a thread that re-enters its own lock.

"How do you get an exception out of a worker thread?" With concurrent.futures, future.result() re-raises it in the caller. With a bare threading.Thread, an exception only prints a traceback and the thread dies, so you must catch it inside the thread and report it, for example through a queue.

"What changed for multiprocessing in Python 3.14?" The default start method on Linux became forkserver instead of fork, to avoid deadlocks from forking a process that has threads. And InterpreterPoolExecutor was added.

Something to volunteer that nobody asks for: the right fix for overselling stock is usually not a Python lock at all, because a lock only protects one process, and production runs many. The check and the update must happen in one atomic database statement, such as UPDATE ... SET stock = stock - 1 WHERE stock > 0. That shows you think about the whole system, not just one process.

Recall

  • threading.Thread(target=..., args=...), then start() and join(). Daemon threads die abruptly at exit.
  • Check-then-act and x += 1 are races: several bytecodes with possible switches in between. Protect each critical section with a lock using with lock:. The free-threaded build exposes races the GIL used to hide.
  • RLock for re-entry, Semaphore(n) to limit concurrency, Event for signals, Condition and Barrier for coordination. Prevent a deadlock with a fixed lock order.
  • queue.Queue(maxsize=...) gives the thread-safe producer-consumer pattern with backpressure. Use a sentinel to stop workers.
  • A thread pool (ThreadPoolExecutor) returns futures. map keeps input order, as_completed gives finish order, and result() re-raises worker exceptions.
  • ProcessPoolExecutor gives real CPU parallelism, one GIL per process. Arguments and results are copied with pickle, so no lambdas, and keep data small. Use the __main__ guard. Linux's default start method is forkserver since 3.14. InterpreterPoolExecutor is new in 3.14.
  • Waiting → asyncio or threads. Computing → processes, C code or the free-threaded build.

Self-test: Walk through how two threads can sell the last phone twice. Why might counter += 1 give the right total on the standard build and the wrong one on the free-threaded build? What problem does maxsize on a queue solve? Why can't a lambda be sent to a process pool?

Quiz Bank ​

FoundationalWhat is the difference between a thread and a process?

A thread runs inside a process and shares all of that process's memory with its other threads, so sharing data is easy but must be protected with locks. In CPython, threads also share one GIL. A process has its own memory and its own interpreter, so processes cannot accidentally corrupt each other's data and can run Python on separate cores in parallel. The costs are slower start-up and the need to copy data between processes, which Python does by pickling.

AppliedA script downloads 500 product images one by one and takes 10 minutes. How would you speed it up, and what limits would you add?

The work is I/O-bound, so overlapping the waits helps. Use a ThreadPoolExecutor with a bounded number of workers, such as 20, and as_completed to save each image as it arrives. Or use asyncio with an HTTP client that supports it. Add limits: a maximum number of concurrent requests so you do not overload the image server or get rate-limited, a timeout per request, a small number of retries with backoff for transient errors, and per-item error handling so one failed image does not stop the batch. Expect roughly a 20-times speed-up, until bandwidth or the server becomes the bottleneck.

InterviewImplement a thread-safe counter class and explain why each part is needed.
python
import threading

class SafeCounter:
    def __init__(self):
        self._value = 0
        self._lock = threading.Lock()

    def increment(self, by=1):
        with self._lock:
            self._value += by

    @property
    def value(self):
        with self._lock:
            return self._value

self._value += by is a read, an add and a write, so without the lock two threads can read the same old value and one update is lost. with self._lock makes that sequence a single critical section and releases the lock even if an exception occurs. Reading also takes the lock, so a reader never sees a half-finished update in more complex versions, and so the code stays correct on free-threaded builds. For a simple statistic, itertools.count() or a per-thread counter merged later avoids lock contention.

StaffA team runs CPU-heavy report generation inside their FastAPI web workers and response times for all other endpoints spike. What do you recommend?

The reports hold the GIL, or occupy the event loop if called from async code, so ordinary requests in the same worker wait behind them. First, move the work out of the request path. The endpoint should enqueue a job and return 202 with a job ID. Separate worker processes, managed by a task queue such as Celery or RQ, or a ProcessPoolExecutor owned by a dedicated service, do the heavy work and store the result for later download. If it must stay in-process for now, run it with loop.run_in_executor on a ProcessPoolExecutor, never in the default thread pool. Then make the report itself cheaper: push aggregation into the database, or use vectorised libraries that release the GIL. Size the web tier and the worker tier independently, since their bottlenecks are different.

Next: threads overlap waits, but each costs memory and switching time, and a web server waits on thousands of connections. 3.13.15 shows how asyncio handles all of them in a single thread, and how async and await really work underneath. It is the foundation of FastAPI.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Volume I — Technology
Part 0 — Retention & Notation
Part 1 — Foundations of Computation
Part 2 — Operating Systems
2.1 The Kernel
2.2 Processes
2.3 Threads & Scheduling
2.4 Concurrency Under the Hood
2.5 Memory Management
2.6 File Systems
2.7 Input/Output
2.8 Linux for Engineers
2.9 Windows & macOS in Practice
Part 3 — Languages, Compilers & Runtimes
3.6 JavaScript & V8
3.7 TypeScript
3.8 Node.js Internals
3.13 Python & CPython
3.14 FastAPI & Production Python
Part 4 — Data Structures & Algorithms
4.0 Classic Problems — Before NeetCode
4.4 Arrays & Hashing — Problems
4.5 Two Pointers
4.6 Sliding Window
4.8 Stack — Problems
4.9 Linked List — Problems
4.11 Binary Search — Problems
4.13 Trees
4.14 Tree — Problems
4.15 Trie — Problems
4.17 Heap — Problems
4.18 Backtracking — Problems
4.19 Graphs
4.20 Graph — Problems
4.21 Advanced Graph — Problems
4.23 1-D DP — Problems
4.24 2-D DP — Problems
4.26 Greedy — Problems
4.27 Interval — Problems
4.28 Math & Geometry — Problems
4.30 Bit Manipulation — Problems
Part 5 — Networking & the Web
5.3 IP
5.4 Transport
5.6 HTTP
Part 6 — Frontend Engineering
6.1 Browser Internals
6.2 HTML & CSS Mastery
6.3 JavaScript in the Browser
6.4 React, from First Component to Full Shop
6.5 Frameworks & Rendering
6.8 UI Engineering & Patterns
Part 7 — Databases & Storage
7.2 SQL Mastery
7.3 Storage Engine Internals
7.4 Transactions
7.5 NoSQL Taxonomy
7.8 Analytics & Big Data
Part 8 — Security, Identity & Cryptography
8.2 Cryptography
8.3 Protocols of Trust
8.4 Identity: AuthN & AuthZ
8.5 Application Security
8.6 Infrastructure Security
Part 9 — Software Design (LLD)
9.2 OOP Foundations
9.3 Design Principles
9.4 Design Patterns
9.5 Concurrency
9.6 API Design
9.7 LLD Interview Machines
9.9 Node/Express Production Playbook
Part 10 — Distributed Systems & HLD
10.7 Consistency & Consensus
10.8 Messaging & Microservices
10.14 Caching
Building blocks from the design seat
Part 11 — System Design Case Studies
Part 12 — AI & ML Engineering
12.1 What AI Engineering Is
12.2 The Math You Need
12.3 Classical Machine Learning
12.4 Data for ML
12.5 Deep Learning
12.6 Embeddings & Vector Search
12.7 Transformers & LLMs
12.8 Building with LLMs
12.9 Retrieval-Augmented Generation
12.10 Agents
12.11 Adapting Models
12.12 Production AI
12.13 Multimodal AI
12.14 AI Security & Governance
12.15 Case Studies & Capstone
Part 13 — Cloud, DevOps & Infrastructure
13.11 Git, from First Commit to Team Workflow
Part 14 — Containers: Docker & Kubernetes
14.1 Foundations
14.2 Docker
14.3 Kubernetes
14.4 Kubernetes on Azure
Part 15 — The Craft & Career
Nothing picked yet