Appearance
3.13.10 — Exceptions & Error Handling
At 2 a.m. the order service starts returning "Internal Server Error" for one customer in fifty. The log says only KeyError: 'sku'. Which request? Which payload? Which of the four places that read sku? Nobody knows, because a helper three levels down caught the original error and raised a new one without passing the old one along, and another layer caught everything and logged one line. Good error handling is not about avoiding exceptions. It is about making sure that when something fails, the failure is caught at the right place, carries the right context and does not hide the real cause. This page covers Python's exception machinery from the bottom up, and the habits that make a 2 a.m. log useful.
1. The shape of try
An exception is an object that represents an error. Raising one stops the normal flow of the program and travels up through the callers until some code catches it. If nothing catches it, the program prints a traceback and stops.
A try statement has four possible parts. The order in which they run is worth seeing once.
python
def show_flow(raise_error):
try:
print(" try")
if raise_error:
raise ValueError("bad quantity") # (1)
except ValueError as error:
print(" except:", error) # (2)
else:
print(" else") # (3)
finally:
print(" finally") # (4)
show_flow(False)
print("--")
show_flow(True)text
try
else
finally
--
try
except: bad quantity
finally(1)raisecreates the exception and starts it travelling.(2)except ValueErrorcatches aValueErroror any subclass of it.as errornames the exception object.(3)elseruns only if thetryblock raised nothing.(4)finallyruns always: after success, after a caught exception, after an uncaught one, and even after areturninside thetry.
Why does else exist? Because it keeps the try block small. Put only the line that might fail in try, and the code that should run after it succeeds in else. Then an exception raised by the follow-up code is not accidentally caught by an except meant for the first line.
python
import json
raw_body = '{"sku": "mug", "qty": 2}'
try:
order = json.loads(raw_body) # (1) only this line can raise JSONDecodeError
except json.JSONDecodeError:
order = None
else:
print("parsed", order["sku"]) # (2) a KeyError here is NOT swallowed by the except abovetext
parsed mug2. The exception hierarchy
Exceptions are classes, arranged in a tree. except SomeClass catches that class and all of its subclasses. So you need to know the tree to know what an except really catches. Figure 1 shows the part you will meet every day.
BaseException and are not errors. Everything a program should normally handle sits under Exception.The most important split is at the top.
Exceptionis the base of every error: bad values, missing keys, failed network calls. Your own exceptions inherit from it.- A few classes inherit directly from
BaseException, because they are not errors. They are instructions to stop.KeyboardInterruptis raised when you press Ctrl+C.SystemExitis raised bysys.exit().GeneratorExitis how a generator is closed (3.13.9).
That split is why except Exception: is the widest catch you should normally write. It catches every error but still lets Ctrl+C and sys.exit() stop the program. A bare except: with no class, or except BaseException:, also swallows those, so the program cannot be stopped cleanly. Linters flag bare except: for exactly this reason.
The middle layer gives you useful groups. except LookupError catches both KeyError and IndexError. except OSError catches every operating-system failure: missing files, refused connections, timeouts. ConnectionError and TimeoutError are themselves subclasses of OSError.
You can catch several unrelated classes in one clause with a tuple, except (ValueError, TypeError) as error:. Python 3.14 also lets you drop the brackets when there is no as, as in except ValueError, TypeError:.
3. Raising your own exceptions
A service should define its own exception classes for its own failures. That lets callers catch "anything that went wrong with an order" without also catching unrelated bugs.
python
class OrderError(Exception): # (1)
"""Base class for every order-processing failure."""
class OutOfStock(OrderError): # (2)
def __init__(self, sku, requested, available):
super().__init__(f"{sku}: requested {requested}, only {available} left") # (3)
self.sku = sku # (4)
self.requested = requested
self.available = available
try:
raise OutOfStock("mug", requested=5, available=2)
except OrderError as error: # (5)
print(type(error).__name__, "|", error, "|", error.available)text
OutOfStock | mug: requested 5, only 2 left | 2(1)One base class for the whole area. It inherits fromException, never fromBaseException.(2)Specific failures inherit from the base.(3)The message passed tosuper().__init__is whatstr(error)shows.(4)Extra attributes carry structured data. A FastAPI error handler can turnerror.skuanderror.availableinto a clean JSON response without parsing the message text.(5)Catching the base class catches every specific failure.
The rule of thumb: define a small hierarchy per area of the system, such as one base for orders, one for payments and one for inventory, and give each exception the data a handler would need.
4. Chaining: never lose the original cause
Now the 2 a.m. story. A helper turns a low-level error into a higher-level one, which is good practice: callers of parse_order should not need to know about JSON internals. The question is what happens to the original error.
python
import json
class InvalidPayload(Exception):
pass
def parse_order(raw):
try:
return json.loads(raw)
except json.JSONDecodeError as error:
raise InvalidPayload("order body is not valid JSON") from error # (1)
try:
parse_order("{bad")
except InvalidPayload as error:
print(repr(error))
print("cause:", repr(error.__cause__)) # (2)text
InvalidPayload('order body is not valid JSON')
cause: JSONDecodeError('Expecting property name enclosed in double quotes: line 1 column 2 (char 1)')(1)raise NewError(...) from originalcreates an exception chain. It sets the new exception's__cause__to the original.(2)The original is still attached, with its full detail.
When this chained exception is printed as a traceback, Python shows both, with a line between them.
text
Traceback (most recent call last):
File "orders.py", line 8, in parse_order
return json.loads(raw)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "orders.py", line 13, in <module>
parse_order("{bad")
File "orders.py", line 10, in parse_order
raise InvalidPayload("order body is not valid JSON") from error
InvalidPayload: order body is not valid JSONRead it from the bottom: the high-level failure, and above it the exact low-level cause and line. That is the log the on-call engineer needed.
Implicit chaining, and from None
Even without from, Python remembers. If you raise a new exception while handling another one, the original is stored as __context__, and the traceback says "During handling of the above exception, another exception occurred". That wording suggests a second bug inside the error handler, which is confusing when the new exception was raised on purpose. So:
- Use
raise New(...) from errorwhen you are deliberately translating an error. The traceback then says "direct cause", which is accurate. - Use
raise New(...) from Nonewhen the original adds nothing useful and would only confuse the reader, for example aKeyErrorinside a lookup helper that already produces a perfectly clear message. This sets__suppress_context__, and the original is hidden from the traceback.
python
def find_price(prices, sku):
try:
return prices[sku]
except KeyError:
raise LookupError(f"no price for {sku!r}") from None # (1)
try:
find_price({"mug": 450}, "lamp")
except LookupError as error:
print(error, "| original hidden:", error.__suppress_context__)text
no price for 'lamp' | original hidden: True(1)TheKeyErroradds nothing that the new message does not already say.
Adding context with add_note
Sometimes you do not want a new exception at all. You only want to add where the failure happened. Python 3.11 added add_note, which attaches extra lines to an existing exception.
python
import traceback
try:
raise ValueError("price must be positive")
except ValueError as error:
error.add_note("while importing row 1042 of prices.csv") # (1)
traceback.print_exc()text
Traceback (most recent call last):
File "import_prices.py", line 4, in <module>
raise ValueError("price must be positive")
ValueError: price must be positive
while importing row 1042 of prices.csv(1)The note is printed under the error message. The exception's type is unchanged, so every existingexcept ValueErrorstill catches it.
This is the cleanest fix for the bulk-import case: the row number goes into the traceback without wrapping the error in a new class.
5. Re-raising
Inside an except block, a bare raise re-raises the exception being handled, with its original traceback untouched.
python
import logging
def charge(order_id):
raise ConnectionError("gateway timeout")
try:
try:
charge("ORD-7")
except ConnectionError:
logging.warning("charge failed for ORD-7, will be retried by the queue") # (1)
raise # (2)
except ConnectionError as error:
print("caller still sees:", error)text
WARNING:root:charge failed for ORD-7, will be retried by the queue
caller still sees: gateway timeout(1)This layer adds a log line with context it knows about.(2)Then it passes the same exception up. Writingraise errorwould work too, but bareraiseis the idiom.
"Log it and re-raise" is fine at a boundary that adds context. Doing it at every layer produces the same error logged five times for one failure, which makes logs harder to read, not easier.
6. Exception groups: several errors at once
Some code legitimately fails in several ways at the same time. Validating an order form might find three problems. Running ten network requests in parallel might see four of them fail. Raising only the first error hides the other problems. Python 3.11 added ExceptionGroup, an exception that holds a list of exceptions, and a new except* syntax to handle them by type.
python
def validate(order):
problems = []
if order["qty"] <= 0:
problems.append(ValueError("qty must be positive"))
if "@" not in order["email"]:
problems.append(ValueError("email is invalid"))
if order["sku"] not in {"mug", "pen"}:
problems.append(LookupError(f"unknown sku {order['sku']!r}"))
if problems:
raise ExceptionGroup("order is invalid", problems) # (1)
try:
validate({"qty": 0, "email": "asha.example.com", "sku": "lamp"})
except* ValueError as group: # (2)
print("value errors:", [str(e) for e in group.exceptions])
except* LookupError as group: # (3)
print("lookup errors:", [str(e) for e in group.exceptions])text
value errors: ['qty must be positive', 'email is invalid']
lookup errors: ["unknown sku 'lamp'"](1)The group carries a message and the list of individual errors.(2)except*takes the matching errors out of the group, wraps them in a smaller group, and runs this block.(3)Unlike a normaltry, severalexcept*blocks can run for one raise. Each handles its share. Anything left unmatched is re-raised as a smaller group.
You will meet exception groups most often from asyncio.TaskGroup (3.13.15), which runs several tasks at once and raises an ExceptionGroup if more than one fails.
7. finally and return: two traps
finally always runs, even when the try block returns.
python
def save_order():
try:
return "saved"
finally:
print(" closing the connection") # (1)
print(save_order())text
closing the connection
saved(1)The return value is computed first and held. Thenfinallyruns. Then the function actually returns.
The trap is a return inside finally. It replaces the pending result, and worse, it silently discards any exception that was on its way out.
python
def import_batch():
try:
raise ValueError("row 1042 is corrupt")
finally:
return "done" # (1)
print(import_batch())text
import_batch.py:5: SyntaxWarning: 'return' in a 'finally' block
done(1)TheValueErrorvanishes and the caller is told everything went fine.
Python 3.14 now warns about this at compile time, as described in PEP 765, and the same applies to break and continue in a finally block. The rule: finally is for cleanup only, never for control flow.
8. EAFP versus LBYL
There are two styles for dealing with something that might be missing. The Python community names them with two abbreviations.
- LBYL, look before you leap: check first, then act.
if sku in stock: qty = stock[sku]. - EAFP, easier to ask forgiveness than permission: act, and handle the failure.
try: qty = stock[sku] except KeyError: ....
Python generally prefers EAFP, for two real reasons.
- It avoids race conditions. Checking
os.path.exists(path)and then opening the file leaves a gap where another process can delete it. Opening it and catchingFileNotFoundErrorhas no gap. The same applies to a key that another thread removes between your check and your read. - It is fast when failure is rare. Since Python 3.11, a
tryblock that raises nothing costs almost nothing, which is called zero-cost exceptions. The cost appears only when something is actually raised.
python
import timeit
no_try = timeit.timeit("x = 1", number=1_000_000)
try_no_raise = timeit.timeit("try:\n x = 1\nexcept ValueError:\n pass", number=1_000_000)
try_and_raise = timeit.timeit("try:\n raise ValueError\nexcept ValueError:\n pass", number=1_000_000)
print(f"plain: {no_try:.3f}s try, no raise: {try_no_raise:.3f}s raise and catch: {try_and_raise:.3f}s")text
plain: 0.010s try, no raise: 0.011s raise and catch: 0.130sSo the honest rule is: use try freely when failure is rare. When failure is common, as with a cache miss on most lookups, a check or a method like dict.get(sku, 0) is cleaner and faster. The question is how often it fails, not which style is "more Pythonic".
9. Habits for real services
- Catch narrowly. Name the exceptions you can actually handle.
except Exceptionbelongs only at the outermost boundary: the request handler, the job runner or the queue consumer. There it should log the full traceback and turn the failure into a clean response. - Never
except: pass. A swallowed exception is the most expensive bug there is, because it leaves no trace. If ignoring an error is really correct, usecontextlib.suppress(SpecificError)so the intent is visible. - Translate at the boundary between layers, with
from. The database layer raises database errors. The service layer turns them intoOrderErrorsubclasses. The HTTP layer turns those into status codes. 3.14.5 shows the FastAPI side. - Log the traceback, not just the message.
logger.exception("charge failed", extra={"order_id": order_id})inside anexceptblock records the full chain.logger.error(str(error))records one line and throws away the cause. - Do not use exceptions for normal control flow in hot code. A loop that raises and catches on most iterations is slow and hard to read.
What the interviewer will push on
"What is the difference between except Exception and a bare except?" A bare except catches BaseException, which includes KeyboardInterrupt, SystemExit and GeneratorExit, so it can stop a program from shutting down. except Exception catches only errors. The tell is naming those three classes and explaining why they sit outside Exception.
"What do else and finally do in a try statement?" else runs only when the try block raised nothing, which keeps the try block narrow. finally always runs. Bonus: a return inside finally swallows exceptions, and 3.14 warns about it.
"What does raise ... from do?" It sets __cause__, so the traceback shows the original error as the direct cause. Contrast it with implicit chaining through __context__, and with from None. The common weak answer does not know that implicit chaining exists at all.
"EAFP or LBYL?" The good answer is not "EAFP because Pythonic". It names the race condition that LBYL has with files and shared state, the near-zero cost of a try that does not raise since 3.11, and the real cost of raising when failure is common.
"How do you handle multiple failures from concurrent tasks?" ExceptionGroup and except*, with asyncio.TaskGroup as the usual source. Explain that several except* blocks can run for one raise.
Something to volunteer that nobody asks for: add_note() lets you attach context such as a row number or an order ID to an exception without changing its type, so every existing except clause still matches. It is the cleanest fix for "the log says KeyError and nobody knows which record".
Recall
try→except(if it raised and matched) orelse(if nothing was raised) →finally(always). Keep thetryblock to the line that can fail.except XcatchesXand its subclasses.Exceptionis the base of errors.SystemExit,KeyboardInterruptandGeneratorExitsit directly underBaseException, so a bareexcept:swallows them. Useful groups:LookupError,OSError,ArithmeticError.- Define a small hierarchy of your own
Exceptionsubclasses per area, carrying structured attributes. raise New(...) from errorsets__cause__(an exception chain). Raising insideexceptwithoutfromsets__context__.from Nonehides the original.add_note()adds context without changing the type. A bareraisere-raises with the original traceback.- ExceptionGroup holds several exceptions.
except*splits it by type, and severalexcept*blocks can run. returninfinallyswallows exceptions (3.14 warns).- EAFP avoids check-then-act races. Zero-cost exceptions make a non-raising
tryalmost free, but raising costs about ten times a plain statement.
Self-test: In what order do try, except, else and finally run when nothing is raised? Why does KeyboardInterrupt not inherit from Exception? What is the difference between __cause__ and __context__? When can two except* blocks both run? Why is os.path.exists() followed by open() a race?
Quiz Bank
FoundationalWhy should your own exception classes inherit from Exception and not BaseException?
Because code everywhere uses except Exception as the "catch every error" boundary, for example in web frameworks, job runners and logging middleware. An exception inheriting from BaseException would slip past those handlers and crash the process. BaseException is reserved for signals that are deliberately meant to escape normal error handling, such as KeyboardInterrupt, SystemExit and GeneratorExit.
AppliedA bulk importer catches every exception per row and logs str(error). Support cannot tell which rows failed or why. How would you fix it?
Log with context and the full traceback. Inside the per-row except, call logger.exception("row failed", extra={"row": index, "order_id": row.get("order_id")}), which records the traceback and its chain. Or attach the row number with error.add_note(f"row {index}") before logging. Catch narrowly: expected data problems, such as ValueError or a custom InvalidRow, are collected into a rejected-rows report with their reasons. Unexpected exceptions are logged with the traceback and counted. If the unexpected count passes a threshold, stop the job, because that usually means a bug, not bad data.
InterviewWhat does this print, and why? def f(): try: return 1 finally: return 2
It prints 2. When the try block executes return 1, the value 1 is computed and held, then the finally block runs before the function actually returns. The return 2 in finally replaces the pending return value. The same thing would silently discard an exception raised in the try block. Python 3.14 emits a SyntaxWarning for return, break or continue in finally (PEP 765), because this behaviour hides errors. finally should only clean up.
StaffDesign the error-handling strategy for a FastAPI service with a database layer, a domain service layer and HTTP routes.
Give each layer its own vocabulary and translate at each boundary with raise ... from error. The repository layer catches driver errors, such as a unique-constraint violation, and raises domain-meaningful errors like DuplicateOrder. The service layer raises a small hierarchy, such as OrderError with OutOfStock and InvalidState, carrying structured attributes. The HTTP layer registers exception handlers that map each domain error to a status code and a stable JSON error body with a machine-readable code, without exposing internal messages. Exactly one place, the outermost handler or middleware, catches unexpected Exception, logs it once with the traceback and a request ID, and returns a generic 500. Concurrent work uses TaskGroup and handles ExceptionGroup with except*. Retries live in one place, for transient errors only, on idempotent operations.
Next: every example so far lived in one file. 3.13.11 explains what really happens on import: where Python looks, why a module runs only once, how circular imports break, and how virtual environments keep each project's packages separate.