Skip to content

Volume ITechnology

3.13.10 — Exceptions & Error Handling ​

At 2 a.m. the order service starts returning "Internal Server Error" for one customer in fifty. The log says only KeyError: 'sku'. Which request? Which payload? Which of the four places that read sku? Nobody knows, because a helper three levels down caught the original error and raised a new one without passing the old one along, and another layer caught everything and logged one line. Good error handling is not about avoiding exceptions. It is about making sure that when something fails, the failure is caught at the right place, carries the right context and does not hide the real cause. This page covers Python's exception machinery from the bottom up, and the habits that make a 2 a.m. log useful.

1. The shape of try ​

An exception is an object that represents an error. Raising one stops the normal flow of the program and travels up through the callers until some code catches it. If nothing catches it, the program prints a traceback and stops.

A try statement has four possible parts. The order in which they run is worth seeing once.

python
def show_flow(raise_error):
    try:
        print("  try")
        if raise_error:
            raise ValueError("bad quantity")       # (1)
    except ValueError as error:
        print("  except:", error)                  # (2)
    else:
        print("  else")                            # (3)
    finally:
        print("  finally")                         # (4)

show_flow(False)
print("--")
show_flow(True)
text
  try
  else
  finally
--
  try
  except: bad quantity
  finally
  • (1) raise creates the exception and starts it travelling.
  • (2) except ValueError catches a ValueError or any subclass of it. as error names the exception object.
  • (3) else runs only if the try block raised nothing.
  • (4) finally runs always: after success, after a caught exception, after an uncaught one, and even after a return inside the try.

Why does else exist? Because it keeps the try block small. Put only the line that might fail in try, and the code that should run after it succeeds in else. Then an exception raised by the follow-up code is not accidentally caught by an except meant for the first line.

python
import json

raw_body = '{"sku": "mug", "qty": 2}'
try:
    order = json.loads(raw_body)          # (1) only this line can raise JSONDecodeError
except json.JSONDecodeError:
    order = None
else:
    print("parsed", order["sku"])         # (2) a KeyError here is NOT swallowed by the except above
text
parsed mug

2. The exception hierarchy ​

Exceptions are classes, arranged in a tree. except SomeClass catches that class and all of its subclasses. So you need to know the tree to know what an except really catches. Figure 1 shows the part you will meet every day.

BaseExceptionSystemExitKeyboardInterruptGeneratorExitExceptionValueErrorTypeErrorLookupErrorArithmeticErrorOSErrorRuntimeErrorKeyErrorIndexErrorZeroDivisionErrorFileNotFoundErrorConnectionErrorTimeoutErrornot errors: signals to stop
Figure 1 — The everyday exception tree. Amber classes inherit straight from BaseException and are not errors. Everything a program should normally handle sits under Exception.

The most important split is at the top.

  • Exception is the base of every error: bad values, missing keys, failed network calls. Your own exceptions inherit from it.
  • A few classes inherit directly from BaseException, because they are not errors. They are instructions to stop. KeyboardInterrupt is raised when you press Ctrl+C. SystemExit is raised by sys.exit(). GeneratorExit is how a generator is closed (3.13.9).

That split is why except Exception: is the widest catch you should normally write. It catches every error but still lets Ctrl+C and sys.exit() stop the program. A bare except: with no class, or except BaseException:, also swallows those, so the program cannot be stopped cleanly. Linters flag bare except: for exactly this reason.

The middle layer gives you useful groups. except LookupError catches both KeyError and IndexError. except OSError catches every operating-system failure: missing files, refused connections, timeouts. ConnectionError and TimeoutError are themselves subclasses of OSError.

You can catch several unrelated classes in one clause with a tuple, except (ValueError, TypeError) as error:. Python 3.14 also lets you drop the brackets when there is no as, as in except ValueError, TypeError:.

3. Raising your own exceptions ​

A service should define its own exception classes for its own failures. That lets callers catch "anything that went wrong with an order" without also catching unrelated bugs.

python
class OrderError(Exception):                                 # (1)
    """Base class for every order-processing failure."""

class OutOfStock(OrderError):                                # (2)
    def __init__(self, sku, requested, available):
        super().__init__(f"{sku}: requested {requested}, only {available} left")   # (3)
        self.sku = sku                                       # (4)
        self.requested = requested
        self.available = available

try:
    raise OutOfStock("mug", requested=5, available=2)
except OrderError as error:                                  # (5)
    print(type(error).__name__, "|", error, "|", error.available)
text
OutOfStock | mug: requested 5, only 2 left | 2
  • (1) One base class for the whole area. It inherits from Exception, never from BaseException.
  • (2) Specific failures inherit from the base.
  • (3) The message passed to super().__init__ is what str(error) shows.
  • (4) Extra attributes carry structured data. A FastAPI error handler can turn error.sku and error.available into a clean JSON response without parsing the message text.
  • (5) Catching the base class catches every specific failure.

The rule of thumb: define a small hierarchy per area of the system, such as one base for orders, one for payments and one for inventory, and give each exception the data a handler would need.

4. Chaining: never lose the original cause ​

Now the 2 a.m. story. A helper turns a low-level error into a higher-level one, which is good practice: callers of parse_order should not need to know about JSON internals. The question is what happens to the original error.

python
import json

class InvalidPayload(Exception):
    pass

def parse_order(raw):
    try:
        return json.loads(raw)
    except json.JSONDecodeError as error:
        raise InvalidPayload("order body is not valid JSON") from error    # (1)

try:
    parse_order("{bad")
except InvalidPayload as error:
    print(repr(error))
    print("cause:", repr(error.__cause__))                                  # (2)
text
InvalidPayload('order body is not valid JSON')
cause: JSONDecodeError('Expecting property name enclosed in double quotes: line 1 column 2 (char 1)')
  • (1) raise NewError(...) from original creates an exception chain. It sets the new exception's __cause__ to the original.
  • (2) The original is still attached, with its full detail.

When this chained exception is printed as a traceback, Python shows both, with a line between them.

text
Traceback (most recent call last):
  File "orders.py", line 8, in parse_order
    return json.loads(raw)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "orders.py", line 13, in <module>
    parse_order("{bad")
  File "orders.py", line 10, in parse_order
    raise InvalidPayload("order body is not valid JSON") from error
InvalidPayload: order body is not valid JSON

Read it from the bottom: the high-level failure, and above it the exact low-level cause and line. That is the log the on-call engineer needed.

Implicit chaining, and from None ​

Even without from, Python remembers. If you raise a new exception while handling another one, the original is stored as __context__, and the traceback says "During handling of the above exception, another exception occurred". That wording suggests a second bug inside the error handler, which is confusing when the new exception was raised on purpose. So:

  • Use raise New(...) from error when you are deliberately translating an error. The traceback then says "direct cause", which is accurate.
  • Use raise New(...) from None when the original adds nothing useful and would only confuse the reader, for example a KeyError inside a lookup helper that already produces a perfectly clear message. This sets __suppress_context__, and the original is hidden from the traceback.
python
def find_price(prices, sku):
    try:
        return prices[sku]
    except KeyError:
        raise LookupError(f"no price for {sku!r}") from None      # (1)

try:
    find_price({"mug": 450}, "lamp")
except LookupError as error:
    print(error, "| original hidden:", error.__suppress_context__)
text
no price for 'lamp' | original hidden: True
  • (1) The KeyError adds nothing that the new message does not already say.

Adding context with add_note ​

Sometimes you do not want a new exception at all. You only want to add where the failure happened. Python 3.11 added add_note, which attaches extra lines to an existing exception.

python
import traceback

try:
    raise ValueError("price must be positive")
except ValueError as error:
    error.add_note("while importing row 1042 of prices.csv")   # (1)
    traceback.print_exc()
text
Traceback (most recent call last):
  File "import_prices.py", line 4, in <module>
    raise ValueError("price must be positive")
ValueError: price must be positive
while importing row 1042 of prices.csv
  • (1) The note is printed under the error message. The exception's type is unchanged, so every existing except ValueError still catches it.

This is the cleanest fix for the bulk-import case: the row number goes into the traceback without wrapping the error in a new class.

5. Re-raising ​

Inside an except block, a bare raise re-raises the exception being handled, with its original traceback untouched.

python
import logging

def charge(order_id):
    raise ConnectionError("gateway timeout")

try:
    try:
        charge("ORD-7")
    except ConnectionError:
        logging.warning("charge failed for ORD-7, will be retried by the queue")   # (1)
        raise                                                                     # (2)
except ConnectionError as error:
    print("caller still sees:", error)
text
WARNING:root:charge failed for ORD-7, will be retried by the queue
caller still sees: gateway timeout
  • (1) This layer adds a log line with context it knows about.
  • (2) Then it passes the same exception up. Writing raise error would work too, but bare raise is the idiom.

"Log it and re-raise" is fine at a boundary that adds context. Doing it at every layer produces the same error logged five times for one failure, which makes logs harder to read, not easier.

6. Exception groups: several errors at once ​

Some code legitimately fails in several ways at the same time. Validating an order form might find three problems. Running ten network requests in parallel might see four of them fail. Raising only the first error hides the other problems. Python 3.11 added ExceptionGroup, an exception that holds a list of exceptions, and a new except* syntax to handle them by type.

python
def validate(order):
    problems = []
    if order["qty"] <= 0:
        problems.append(ValueError("qty must be positive"))
    if "@" not in order["email"]:
        problems.append(ValueError("email is invalid"))
    if order["sku"] not in {"mug", "pen"}:
        problems.append(LookupError(f"unknown sku {order['sku']!r}"))
    if problems:
        raise ExceptionGroup("order is invalid", problems)        # (1)

try:
    validate({"qty": 0, "email": "asha.example.com", "sku": "lamp"})
except* ValueError as group:                                      # (2)
    print("value errors:", [str(e) for e in group.exceptions])
except* LookupError as group:                                     # (3)
    print("lookup errors:", [str(e) for e in group.exceptions])
text
value errors: ['qty must be positive', 'email is invalid']
lookup errors: ["unknown sku 'lamp'"]
  • (1) The group carries a message and the list of individual errors.
  • (2) except* takes the matching errors out of the group, wraps them in a smaller group, and runs this block.
  • (3) Unlike a normal try, several except* blocks can run for one raise. Each handles its share. Anything left unmatched is re-raised as a smaller group.

You will meet exception groups most often from asyncio.TaskGroup (3.13.15), which runs several tasks at once and raises an ExceptionGroup if more than one fails.

7. finally and return: two traps ​

finally always runs, even when the try block returns.

python
def save_order():
    try:
        return "saved"
    finally:
        print("  closing the connection")       # (1)

print(save_order())
text
  closing the connection
saved
  • (1) The return value is computed first and held. Then finally runs. Then the function actually returns.

The trap is a return inside finally. It replaces the pending result, and worse, it silently discards any exception that was on its way out.

python
def import_batch():
    try:
        raise ValueError("row 1042 is corrupt")
    finally:
        return "done"                  # (1)

print(import_batch())
text
import_batch.py:5: SyntaxWarning: 'return' in a 'finally' block
done
  • (1) The ValueError vanishes and the caller is told everything went fine.

Python 3.14 now warns about this at compile time, as described in PEP 765, and the same applies to break and continue in a finally block. The rule: finally is for cleanup only, never for control flow.

8. EAFP versus LBYL ​

There are two styles for dealing with something that might be missing. The Python community names them with two abbreviations.

  • LBYL, look before you leap: check first, then act. if sku in stock: qty = stock[sku].
  • EAFP, easier to ask forgiveness than permission: act, and handle the failure. try: qty = stock[sku] except KeyError: ....

Python generally prefers EAFP, for two real reasons.

  1. It avoids race conditions. Checking os.path.exists(path) and then opening the file leaves a gap where another process can delete it. Opening it and catching FileNotFoundError has no gap. The same applies to a key that another thread removes between your check and your read.
  2. It is fast when failure is rare. Since Python 3.11, a try block that raises nothing costs almost nothing, which is called zero-cost exceptions. The cost appears only when something is actually raised.
python
import timeit

no_try = timeit.timeit("x = 1", number=1_000_000)
try_no_raise = timeit.timeit("try:\n    x = 1\nexcept ValueError:\n    pass", number=1_000_000)
try_and_raise = timeit.timeit("try:\n    raise ValueError\nexcept ValueError:\n    pass", number=1_000_000)
print(f"plain: {no_try:.3f}s  try, no raise: {try_no_raise:.3f}s  raise and catch: {try_and_raise:.3f}s")
text
plain: 0.010s  try, no raise: 0.011s  raise and catch: 0.130s

So the honest rule is: use try freely when failure is rare. When failure is common, as with a cache miss on most lookups, a check or a method like dict.get(sku, 0) is cleaner and faster. The question is how often it fails, not which style is "more Pythonic".

9. Habits for real services ​

  1. Catch narrowly. Name the exceptions you can actually handle. except Exception belongs only at the outermost boundary: the request handler, the job runner or the queue consumer. There it should log the full traceback and turn the failure into a clean response.
  2. Never except: pass. A swallowed exception is the most expensive bug there is, because it leaves no trace. If ignoring an error is really correct, use contextlib.suppress(SpecificError) so the intent is visible.
  3. Translate at the boundary between layers, with from. The database layer raises database errors. The service layer turns them into OrderError subclasses. The HTTP layer turns those into status codes. 3.14.5 shows the FastAPI side.
  4. Log the traceback, not just the message. logger.exception("charge failed", extra={"order_id": order_id}) inside an except block records the full chain. logger.error(str(error)) records one line and throws away the cause.
  5. Do not use exceptions for normal control flow in hot code. A loop that raises and catches on most iterations is slow and hard to read.

What the interviewer will push on ​

"What is the difference between except Exception and a bare except?" A bare except catches BaseException, which includes KeyboardInterrupt, SystemExit and GeneratorExit, so it can stop a program from shutting down. except Exception catches only errors. The tell is naming those three classes and explaining why they sit outside Exception.

"What do else and finally do in a try statement?" else runs only when the try block raised nothing, which keeps the try block narrow. finally always runs. Bonus: a return inside finally swallows exceptions, and 3.14 warns about it.

"What does raise ... from do?" It sets __cause__, so the traceback shows the original error as the direct cause. Contrast it with implicit chaining through __context__, and with from None. The common weak answer does not know that implicit chaining exists at all.

"EAFP or LBYL?" The good answer is not "EAFP because Pythonic". It names the race condition that LBYL has with files and shared state, the near-zero cost of a try that does not raise since 3.11, and the real cost of raising when failure is common.

"How do you handle multiple failures from concurrent tasks?" ExceptionGroup and except*, with asyncio.TaskGroup as the usual source. Explain that several except* blocks can run for one raise.

Something to volunteer that nobody asks for: add_note() lets you attach context such as a row number or an order ID to an exception without changing its type, so every existing except clause still matches. It is the cleanest fix for "the log says KeyError and nobody knows which record".

Recall

  • try → except (if it raised and matched) or else (if nothing was raised) → finally (always). Keep the try block to the line that can fail.
  • except X catches X and its subclasses. Exception is the base of errors. SystemExit, KeyboardInterrupt and GeneratorExit sit directly under BaseException, so a bare except: swallows them. Useful groups: LookupError, OSError, ArithmeticError.
  • Define a small hierarchy of your own Exception subclasses per area, carrying structured attributes.
  • raise New(...) from error sets __cause__ (an exception chain). Raising inside except without from sets __context__. from None hides the original. add_note() adds context without changing the type. A bare raise re-raises with the original traceback.
  • ExceptionGroup holds several exceptions. except* splits it by type, and several except* blocks can run.
  • return in finally swallows exceptions (3.14 warns).
  • EAFP avoids check-then-act races. Zero-cost exceptions make a non-raising try almost free, but raising costs about ten times a plain statement.

Self-test: In what order do try, except, else and finally run when nothing is raised? Why does KeyboardInterrupt not inherit from Exception? What is the difference between __cause__ and __context__? When can two except* blocks both run? Why is os.path.exists() followed by open() a race?

Quiz Bank ​

FoundationalWhy should your own exception classes inherit from Exception and not BaseException?

Because code everywhere uses except Exception as the "catch every error" boundary, for example in web frameworks, job runners and logging middleware. An exception inheriting from BaseException would slip past those handlers and crash the process. BaseException is reserved for signals that are deliberately meant to escape normal error handling, such as KeyboardInterrupt, SystemExit and GeneratorExit.

AppliedA bulk importer catches every exception per row and logs str(error). Support cannot tell which rows failed or why. How would you fix it?

Log with context and the full traceback. Inside the per-row except, call logger.exception("row failed", extra={"row": index, "order_id": row.get("order_id")}), which records the traceback and its chain. Or attach the row number with error.add_note(f"row {index}") before logging. Catch narrowly: expected data problems, such as ValueError or a custom InvalidRow, are collected into a rejected-rows report with their reasons. Unexpected exceptions are logged with the traceback and counted. If the unexpected count passes a threshold, stop the job, because that usually means a bug, not bad data.

InterviewWhat does this print, and why? def f(): try: return 1 finally: return 2

It prints 2. When the try block executes return 1, the value 1 is computed and held, then the finally block runs before the function actually returns. The return 2 in finally replaces the pending return value. The same thing would silently discard an exception raised in the try block. Python 3.14 emits a SyntaxWarning for return, break or continue in finally (PEP 765), because this behaviour hides errors. finally should only clean up.

StaffDesign the error-handling strategy for a FastAPI service with a database layer, a domain service layer and HTTP routes.

Give each layer its own vocabulary and translate at each boundary with raise ... from error. The repository layer catches driver errors, such as a unique-constraint violation, and raises domain-meaningful errors like DuplicateOrder. The service layer raises a small hierarchy, such as OrderError with OutOfStock and InvalidState, carrying structured attributes. The HTTP layer registers exception handlers that map each domain error to a status code and a stable JSON error body with a machine-readable code, without exposing internal messages. Exactly one place, the outermost handler or middleware, catches unexpected Exception, logs it once with the traceback and a request ID, and returns a generic 500. Concurrent work uses TaskGroup and handles ExceptionGroup with except*. Retries live in one place, for transient errors only, on idempotent operations.

Next: every example so far lived in one file. 3.13.11 explains what really happens on import: where Python looks, why a module runs only once, how circular imports break, and how virtual environments keep each project's packages separate.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Volume I — Technology
Part 0 — Retention & Notation
Part 1 — Foundations of Computation
Part 2 — Operating Systems
2.1 The Kernel
2.2 Processes
2.3 Threads & Scheduling
2.4 Concurrency Under the Hood
2.5 Memory Management
2.6 File Systems
2.7 Input/Output
2.8 Linux for Engineers
2.9 Windows & macOS in Practice
Part 3 — Languages, Compilers & Runtimes
3.6 JavaScript & V8
3.7 TypeScript
3.8 Node.js Internals
3.13 Python & CPython
3.14 FastAPI & Production Python
Part 4 — Data Structures & Algorithms
4.0 Classic Problems — Before NeetCode
4.4 Arrays & Hashing — Problems
4.5 Two Pointers
4.6 Sliding Window
4.8 Stack — Problems
4.9 Linked List — Problems
4.11 Binary Search — Problems
4.13 Trees
4.14 Tree — Problems
4.15 Trie — Problems
4.17 Heap — Problems
4.18 Backtracking — Problems
4.19 Graphs
4.20 Graph — Problems
4.21 Advanced Graph — Problems
4.23 1-D DP — Problems
4.24 2-D DP — Problems
4.26 Greedy — Problems
4.27 Interval — Problems
4.28 Math & Geometry — Problems
4.30 Bit Manipulation — Problems
Part 5 — Networking & the Web
5.3 IP
5.4 Transport
5.6 HTTP
Part 6 — Frontend Engineering
6.1 Browser Internals
6.2 HTML & CSS Mastery
6.3 JavaScript in the Browser
6.4 React, from First Component to Full Shop
6.5 Frameworks & Rendering
6.8 UI Engineering & Patterns
Part 7 — Databases & Storage
7.2 SQL Mastery
7.3 Storage Engine Internals
7.4 Transactions
7.5 NoSQL Taxonomy
7.8 Analytics & Big Data
Part 8 — Security, Identity & Cryptography
8.2 Cryptography
8.3 Protocols of Trust
8.4 Identity: AuthN & AuthZ
8.5 Application Security
8.6 Infrastructure Security
Part 9 — Software Design (LLD)
9.2 OOP Foundations
9.3 Design Principles
9.4 Design Patterns
9.5 Concurrency
9.6 API Design
9.7 LLD Interview Machines
9.9 Node/Express Production Playbook
Part 10 — Distributed Systems & HLD
10.7 Consistency & Consensus
10.8 Messaging & Microservices
10.14 Caching
Building blocks from the design seat
Part 11 — System Design Case Studies
Part 12 — AI & ML Engineering
12.1 What AI Engineering Is
12.2 The Math You Need
12.3 Classical Machine Learning
12.4 Data for ML
12.5 Deep Learning
12.6 Embeddings & Vector Search
12.7 Transformers & LLMs
12.8 Building with LLMs
12.9 Retrieval-Augmented Generation
12.10 Agents
12.11 Adapting Models
12.12 Production AI
12.13 Multimodal AI
12.14 AI Security & Governance
12.15 Case Studies & Capstone
Part 13 — Cloud, DevOps & Infrastructure
13.11 Git, from First Commit to Team Workflow
Part 14 — Containers: Docker & Kubernetes
14.1 Foundations
14.2 Docker
14.3 Kubernetes
14.4 Kubernetes on Azure
Part 15 — The Craft & Career
Nothing picked yet