Skip to content

Volume ITechnology

3.14.5 — Errors, Middleware, Lifespan & Background Work ​

The order service is in production. A customer reports that checkout "just said error". Support asks which request it was. Nobody can tell, because no log line carries anything that links it to the customer's request. The mobile team complains that errors come back in three different JSON shapes, depending on which endpoint failed. And every few weeks, a deploy leaves database connections open until the database refuses new ones. None of these problems belong to any single endpoint. They belong to every request, or to the server's start and end. This page covers the four FastAPI tools for that kind of work: exception handlers, middleware, the lifespan, and background tasks.

All the examples here build one service, app5, piece by piece.

1. Errors: one shape for every failure ​

3.13.10 recommended a small hierarchy of domain exceptions, translated into HTTP at the edge. FastAPI's exception handlers are that edge. An exception handler is a function that FastAPI calls when a given exception type escapes an endpoint. It turns the exception into a response.

HTTPException: the built-in one ​

You have already used it: raise HTTPException(status_code=404, detail="order 99 not found"). FastAPI's built-in handler turns it into {"detail": "order 99 not found"} with status 404. It is fine for simple cases, but it couples your business logic to HTTP. A service function that raises HTTPException cannot be reused from a background job or a command-line script, where status codes mean nothing.

Domain exceptions with your own handler ​

Better: business code raises business exceptions, and one handler maps them to HTTP.

python
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

class OrderError(Exception):                                        # (1)
    status_code = 400
    code = "order_error"

class OutOfStock(OrderError):
    status_code = 409                                               # (2)
    code = "out_of_stock"

    def __init__(self, sku: str, available: int):
        super().__init__(f"only {available} of {sku} left")
        self.sku = sku
        self.available = available

app = FastAPI()

@app.exception_handler(OrderError)                                  # (3)
async def order_error_handler(request: Request, exc: OrderError):
    return JSONResponse(
        status_code=exc.status_code,
        content={"error": {"code": exc.code, "message": str(exc),
                           "request_id": request_id_var.get()}},    # (4)
    )
  • (1) The base class for every order failure. It carries a default status and a stable machine-readable code.
  • (2) 409 Conflict means "the request is valid, but conflicts with the current state", which fits "not enough stock".
  • (3) One handler covers OrderError and every subclass, because exception matching follows the class hierarchy.
  • (4) Every error body includes the request ID, which section 3 sets up. Now a customer's "it just said error" comes with an ID support can search for.

The business code knows nothing about HTTP.

python
from pydantic import BaseModel

class OrderIn(BaseModel):
    sku: str
    quantity: int

STOCK = {"mug": 2}

@app.post("/orders", status_code=201)
async def create_order(order: OrderIn):
    available = STOCK.get(order.sku, 0)
    if order.quantity > available:
        raise OutOfStock(order.sku, available)        # a plain domain exception
    return {"order_id": "ORD-9"}
text
$ curl -X POST localhost:8000/orders -d '{"sku": "mug", "quantity": 5}'
{"error":{"code":"out_of_stock","message":"only 2 of mug left","request_id":"4c2a11cf"}}    ← 409

Reshaping the 422 ​

Validation errors from 3.14.3 have FastAPI's own shape: {"detail": [...]}. To give clients one consistent error format, override the handler for RequestValidationError.

python
from fastapi.exceptions import RequestValidationError

@app.exception_handler(RequestValidationError)
async def validation_handler(request: Request, exc: RequestValidationError):
    fields = [
        {"field": ".".join(str(part) for part in error["loc"][1:]),     # (1)
         "message": error["msg"]}
        for error in exc.errors()
    ]
    return JSONResponse(status_code=422,
                        content={"error": {"code": "invalid_request", "fields": fields}})
text
$ curl -X POST localhost:8000/orders -d '{"sku": "mug", "quantity": "lots"}'
{"error":{"code":"invalid_request","fields":[{"field":"quantity","message":"Input should be a valid integer, unable to parse string as an integer"}]}}
  • (1) loc starts with where the value came from, such as "body" or "query". Dropping it gives clean field paths like lines.0.quantity.

Now every failure, whether from validation or from business rules, has the shape {"error": {"code": ..., ...}}. A client can switch on code without parsing messages.

Unexpected exceptions ​

What about a plain bug, such as dividing by zero? Starlette's outermost layer, ServerErrorMiddleware, catches any exception nobody else handled, logs the traceback, and returns a plain 500 Internal Server Error with no details, so internal information never leaks to clients. In development, FastAPI(debug=True) shows the traceback in the browser instead. Never turn that on in production.

2. Middleware: code that wraps every request ​

Middleware is code that runs around every request: before the request reaches the router, and after the response comes back. It is the right place for anything that applies to all endpoints, such as request IDs, timing, logging, compression and security headers.

Technically, a middleware is an ASGI app that wraps another ASGI app (3.14.1). It receives the request first, can change it or answer immediately, passes it inward, and then sees the response on its way out.

The decorator form ​

The quickest way is @app.middleware("http"). Here is the request-context middleware that fixes the "which request was it?" problem.

python
import contextvars
import time
import uuid

request_id_var = contextvars.ContextVar("request_id", default="-")      # (1)
log = []

@app.middleware("http")
async def request_context(request: Request, call_next):
    request_id = request.headers.get("x-request-id") or uuid.uuid4().hex[:8]   # (2)
    token = request_id_var.set(request_id)
    start = time.perf_counter()
    log.append(f"[{request_id}] -> {request.method} {request.url.path}")
    try:
        response = await call_next(request)                              # (3)
    finally:
        request_id_var.reset(token)                                      # (4)
    response.headers["x-request-id"] = request_id                        # (5)
    elapsed = (time.perf_counter() - start) * 1000
    log.append(f"[{request_id}] <- {response.status_code} in {elapsed:.0f} ms")
    return response
text
$ curl -X POST localhost:8000/orders -H "X-Request-ID: req-42" -d '{"sku": "mug", "quantity": 1}'
{"order_id":"ORD-9"}                 ← 201, response header x-request-id: req-42

log: ['[req-42] -> POST /orders', '[req-42] <- 201 in 2 ms']
  • (1) A context variable from 3.13.15. Each request runs in its own context, so concurrent requests never see each other's IDs.
  • (2) Reuse the caller's ID if a proxy or another service sent one, so a request can be followed across services. Otherwise make a new one.
  • (3) call_next passes the request inward, through the router, dependencies and endpoint, and waits for the response.
  • (4) Restore the context variable afterwards.
  • (5) Return the ID to the client, so support can ask a customer for it.

With a logging filter that reads request_id_var, every log line written anywhere during the request carries the ID (3.14.9 sets that up).

One trap: if the endpoint raises an exception that no handler catches, it passes straight up through call_next. Our "response" log line never runs for crashes, which are exactly the requests you most want logged. To log those, catch the exception around call_next, log it with the request ID, and re-raise.

Middleware order: an onion ​

Each middleware wraps everything added before it. Here is the stack FastAPI actually built for our service, outermost first.

python
from fastapi.middleware.cors import CORSMiddleware
from fastapi.middleware.gzip import GZipMiddleware

app.add_middleware(GZipMiddleware, minimum_size=1000)                  # (1)
app.add_middleware(CORSMiddleware, allow_origins=["https://shop.example.com"],
                   allow_methods=["GET", "POST"], allow_headers=["*"],
                   allow_credentials=True)                             # (2)
text
ServerErrorMiddleware        ← always outermost: last-resort 500
CORSMiddleware               ← added last, so outermost of yours
GZipMiddleware
request_context              ← the @app.middleware("http") function
ExceptionMiddleware          ← runs your exception handlers
… dependencies' exit stack, router, your endpoint
  • (1) GZip compresses responses larger than 1,000 bytes when the client supports it.
  • (2) CORS, explained next.

The rule: the last middleware added is the outermost. A request passes through the list top to bottom, and the response comes back bottom to top. Two consequences matter.

  1. Exception handlers run inside your middleware, so by the time request_context sees the response, a domain error has already become a clean 409. That is why the middleware could log every status code except true crashes.
  2. CORS must be outside anything that might reject a request, or browsers will report a confusing CORS error instead of the real one.

CORS in one paragraph ​

Browsers enforce a rule called the same-origin policy: JavaScript on https://shop.example.com may not read responses from https://api.example.com, a different origin, unless the API says it allows that. CORS, Cross-Origin Resource Sharing, is how the API says so, through response headers. For requests that could change data, the browser first sends a preflight: an OPTIONS request asking "may I?". Part 8 explains why the rule exists. Here is what CORSMiddleware does with a preflight.

text
$ curl -X OPTIONS localhost:8000/orders -H "Origin: https://shop.example.com" -H "Access-Control-Request-Method: POST"
← 200  access-control-allow-origin: https://shop.example.com
       access-control-allow-methods: GET, POST
       access-control-allow-credentials: true

$ curl -X OPTIONS localhost:8000/orders -H "Origin: https://evil.example" -H "Access-Control-Request-Method: POST"
← 400  Disallowed CORS origin

List the exact origins you trust. allow_origins=["*"] together with allow_credentials=True would let any website make logged-in requests on a user's behalf, and the CORS standard forbids that combination for exactly that reason. CORS protects browsers only. It is not authentication, and a script running outside a browser ignores it completely.

Pure ASGI middleware ​

@app.middleware("http") is built on Starlette's BaseHTTPMiddleware, which is convenient but adds overhead and has limits with streaming responses and context variables in some situations. For heavy-traffic, cross-cutting work, write middleware as a plain ASGI app, the shape from 3.14.1.

python
class SecurityHeaders:
    def __init__(self, app):
        self.app = app                                          # (1)

    async def __call__(self, scope, receive, send):
        if scope["type"] != "http":
            return await self.app(scope, receive, send)

        async def send_with_headers(message):                   # (2)
            if message["type"] == "http.response.start":
                message["headers"].append((b"x-content-type-options", b"nosniff"))
                message["headers"].append((b"strict-transport-security",
                                           b"max-age=31536000; includeSubDomains"))
            await send(message)

        await self.app(scope, receive, send_with_headers)       # (3)

app.add_middleware(SecurityHeaders)
  • (1) The middleware is given the inner app when the stack is built.
  • (2) It wraps send, so it can add headers to the response's start message as it passes by. The body is untouched, so streaming keeps working.
  • (3) It passes the request inward with the wrapped send.

Both headers tell browsers to be stricter. Part 8 explains them.

3. The lifespan: start-up and shutdown ​

Some resources should be created once per process, not once per request: a database connection pool, a shared HTTP client, a machine-learning model loaded into memory, a cache warmed from the database. They must also be closed cleanly on shutdown, or a deploy leaves connections hanging. That is the job of the lifespan, an async context manager (3.13.15) that wraps the whole life of the app.

python
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
import httpx

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.http = httpx.AsyncClient(timeout=5.0)          # (1)
    app.state.exchange_rates = {"USD": 83.2}                 # (2)
    log.append("startup: open http client and db pool")
    yield                                                    # (3)
    await app.state.http.aclose()                            # (4)
    log.append("shutdown: close pool")

app = FastAPI(lifespan=lifespan)

@app.get("/rates")
async def rates(request: Request):
    return request.app.state.exchange_rates                  # (5)
text
log at start-up:  ['startup: open http client and db pool']
$ curl localhost:8000/rates
{"USD":83.2}
log at shutdown:  ['shutdown: close pool']
  • (1) One HTTP client for the whole process. It keeps connections to other services open and reuses them. Creating a new client per request would redo the TCP and TLS handshakes every time, which is slow.
  • (2) app.state is a simple namespace for shared objects.
  • (3) Everything before yield runs when the server starts, before it accepts any request. Everything after runs when it shuts down.
  • (4) Close what you opened.
  • (5) Endpoints, or better, dependencies, read shared objects from request.app.state.

Under the hood, the server sends the special ASGI lifespan messages from 3.14.1: lifespan.startup before serving, and lifespan.shutdown when it receives a stop signal. Older code uses @app.on_event("startup") and "shutdown" decorators. They are deprecated, because two separate functions cannot share local variables or guarantee cleanup the way one context manager can.

Each worker process runs its own lifespan. With four Uvicorn workers there are four pools and four HTTP clients. Size database pools with that in mind (3.14.6).

4. Background tasks: work after the response ​

After an order is created, the customer should get a confirmation email. Sending it takes a second, and the customer should not wait for it. FastAPI's BackgroundTasks runs a function after the response has been sent.

python
from fastapi import BackgroundTasks

def send_confirmation_email(email: str, order_id: str):
    log.append(f"background: emailed {email} about {order_id}")

@app.post("/orders", status_code=201)
async def create_order(order: OrderIn, background: BackgroundTasks):     # (1)
    available = STOCK.get(order.sku, 0)
    if order.quantity > available:
        raise OutOfStock(order.sku, available)
    background.add_task(send_confirmation_email, "asha@example.com", "ORD-9")   # (2)
    log.append("endpoint returns")
    return {"order_id": "ORD-9"}
text
log: ['[req-42] -> POST /orders', 'endpoint returns', '[req-42] <- 201 in 2 ms',
      'background: emailed asha@example.com about ORD-9']
  • (1) Declare a parameter of type BackgroundTasks, and FastAPI injects one.
  • (2) Register a function and its arguments. It runs after the response.

The log proves the order: the middleware saw the 201 response go out, then the email was sent. A plain def task runs in the thread pool, and an async def task runs on the event loop.

When background tasks are not enough ​

BackgroundTasks runs inside the same process, in memory. That gives it hard limits.

  1. Nothing survives a crash. If the process restarts after the response but before the task runs, the task is lost and nobody knows.
  2. No retries. If the email provider is down, the task fails once and is gone.
  3. It competes with requests. A heavy task uses the same CPU, event loop and thread pool as live traffic.

So use BackgroundTasks only for small work where an occasional loss is acceptable, such as updating a "last seen" timestamp or warming a cache. Some work must happen, such as charging a card, sending a receipt or updating stock. For that, use a real task queue. The endpoint stores a job in a durable queue. Common choices are Redis with Celery, RQ or ARQ, or a message broker. Separate worker processes pick jobs up, retry them on failure, and survive deploys. The strongest version writes the job in the same database transaction as the order itself, a design called the transactional outbox, so the order and its follow-up work can never disagree. Part 10 covers it.

What the interviewer will push on ​

"How do you return consistent error responses across a FastAPI service?" Domain exceptions with status and code attributes, one @app.exception_handler for the base class, an override for RequestValidationError, and a request ID in every error body. The tell is keeping HTTPException out of business logic, so services stay reusable outside HTTP.

"In what order does middleware run?" Like an onion: the last added is outermost. Requests go in from the outside and responses come back out. Exception handlers run inside user middleware, and ServerErrorMiddleware is always outermost. Give one consequence, such as CORS needing to be outside anything that rejects requests.

"What is the lifespan for, and why not create resources per request?" Shared, expensive resources such as connection pools, HTTP clients and ML models, created once per process before serving traffic and closed cleanly on shutdown. Per-request creation repeats handshakes and exhausts connections. Mention that each worker process has its own lifespan.

"BackgroundTasks or Celery?" BackgroundTasks runs in-process after the response, with no durability, no retries and shared resources. A task queue gives durable, retried work in separate workers. Anything that must happen goes in a queue, ideally written in the same transaction as the triggering change.

"What does CORS protect against?" It stops a malicious website's JavaScript from reading responses from your API in a user's browser. It is enforced by browsers, not servers, so it is not authentication. Never combine wildcard origins with credentials.

Something to volunteer that nobody asks for: an @app.middleware("http") function never sees the response for an unhandled exception, because the exception passes straight up through call_next. If your access log is written there, crashes silently disappear from it. Catch around call_next, log with the request ID, and re-raise.

Recall

  • Exception handlers turn exceptions into responses. Raise domain exceptions carrying a status and a code, handle the base class once, override RequestValidationError for a consistent 422, and include a request ID. Unhandled exceptions become a plain 500 from ServerErrorMiddleware.
  • Middleware wraps every request. @app.middleware("http") with call_next is easy. Pure ASGI middleware that wraps send is faster and stream-safe. The last added is outermost, and exception handlers run inside user middleware.
  • Request IDs use a context variable and an x-request-id header. Crashes skip the code after call_next unless you catch them.
  • CORS lets listed browser origins read your API. Preflights are OPTIONS requests. Never use * with credentials. It is not authentication.
  • The lifespan is an asynccontextmanager: before yield is start-up, after is shutdown. Store shared clients and pools on app.state. One lifespan per worker process.
  • BackgroundTasks runs after the response, in the same process, with no durability or retries. Use a task queue for work that must happen.

Self-test: Why should business code raise OutOfStock instead of HTTPException(409)? With CORS added last and GZip added first, which sees the request first? Why does a crash not appear in a log written after call_next? What is lost if the process restarts right after a response that queued a background task?

Quiz Bank ​

FoundationalWhere should a shared httpx.AsyncClient for calling a payment provider be created, and why?

In the lifespan, once per process, stored on app.state and closed after the yield. The client keeps a pool of open connections to the provider and reuses them across requests, so each call skips the TCP and TLS handshakes. Creating a client per request throws that pool away every time, adds latency, and under load can exhaust sockets. A dependency can then read it from request.app.state and inject it into endpoints.

AppliedMobile developers say some errors arrive as {'detail': ...}, others as {'error': ...}, and some as plain text. How do you make them consistent?

Pick one error envelope, for example {"error": {"code", "message", "fields", "request_id"}}. Register handlers for every source: your domain base exception, RequestValidationError for 422s, Starlette's HTTPException so 404s from unknown routes and any remaining HTTPException raises match, and a catch-all Exception handler that logs the traceback and returns a generic 500 in the same envelope without internal details. Replace HTTPException raises in services with domain exceptions. Document the envelope and its codes in the OpenAPI schema, so client teams can rely on it.

InterviewWrite middleware that measures request duration and adds it as a response header, and explain its limitations.
python
import time

@app.middleware("http")
async def timing(request, call_next):
    start = time.perf_counter()
    response = await call_next(request)
    response.headers["server-timing"] = f"app;dur={(time.perf_counter() - start) * 1000:.1f}"
    return response

It wraps the whole inner stack, so it measures routing, dependencies, the endpoint and exception handlers. Limitations: for a streaming response, call_next returns when the headers are ready, not when the body finishes, so long streams look fast. An unhandled exception skips the header code entirely. BaseHTTPMiddleware adds per-request overhead. For accurate, low-overhead timing on every response type, write it as pure ASGI middleware that records the time at http.response.start and when the final body message has more_body set to false.

StaffAfter an order is placed, the service must charge the card, reserve stock and email the customer. A teammate puts all three in BackgroundTasks. What do you propose?

BackgroundTasks is in-memory and runs after the response. A crash or deploy at the wrong moment loses the charge or the stock reservation silently, there are no retries, and the work competes with live traffic. Split by criticality. Charging and stock reservation are part of placing the order: do them, or a reliable equivalent, before responding, or model the order as pending and move it through states driven by a durable workflow. Write an outbox row in the same database transaction as the order, and have a worker process publish it to a queue. Workers perform each step idempotently, using the order ID as the idempotency key with the payment provider, retry with backoff, and move failures to a dead-letter queue with alerts. The confirmation email is a downstream consumer of the "order paid" event. BackgroundTasks stays for trivial, best-effort work only.

Next: the lifespan opened a database pool, and the dependency in 3.14.4 handed out sessions. 3.14.6 puts a real database behind the service with SQLAlchemy 2.0: models, async sessions, queries, the N+1 problem and migrations.

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Volume I — Technology
Part 0 — Retention & Notation
Part 1 — Foundations of Computation
Part 2 — Operating Systems
2.1 The Kernel
2.2 Processes
2.3 Threads & Scheduling
2.4 Concurrency Under the Hood
2.5 Memory Management
2.6 File Systems
2.7 Input/Output
2.8 Linux for Engineers
2.9 Windows & macOS in Practice
Part 3 — Languages, Compilers & Runtimes
3.6 JavaScript & V8
3.7 TypeScript
3.8 Node.js Internals
3.13 Python & CPython
3.14 FastAPI & Production Python
Part 4 — Data Structures & Algorithms
4.0 Classic Problems — Before NeetCode
4.4 Arrays & Hashing — Problems
4.5 Two Pointers
4.6 Sliding Window
4.8 Stack — Problems
4.9 Linked List — Problems
4.11 Binary Search — Problems
4.13 Trees
4.14 Tree — Problems
4.15 Trie — Problems
4.17 Heap — Problems
4.18 Backtracking — Problems
4.19 Graphs
4.20 Graph — Problems
4.21 Advanced Graph — Problems
4.23 1-D DP — Problems
4.24 2-D DP — Problems
4.26 Greedy — Problems
4.27 Interval — Problems
4.28 Math & Geometry — Problems
4.30 Bit Manipulation — Problems
Part 5 — Networking & the Web
5.3 IP
5.4 Transport
5.6 HTTP
Part 6 — Frontend Engineering
6.1 Browser Internals
6.2 HTML & CSS Mastery
6.3 JavaScript in the Browser
6.4 React, from First Component to Full Shop
6.5 Frameworks & Rendering
6.8 UI Engineering & Patterns
Part 7 — Databases & Storage
7.2 SQL Mastery
7.3 Storage Engine Internals
7.4 Transactions
7.5 NoSQL Taxonomy
7.8 Analytics & Big Data
Part 8 — Security, Identity & Cryptography
8.2 Cryptography
8.3 Protocols of Trust
8.4 Identity: AuthN & AuthZ
8.5 Application Security
8.6 Infrastructure Security
Part 9 — Software Design (LLD)
9.2 OOP Foundations
9.3 Design Principles
9.4 Design Patterns
9.5 Concurrency
9.6 API Design
9.7 LLD Interview Machines
9.9 Node/Express Production Playbook
Part 10 — Distributed Systems & HLD
10.7 Consistency & Consensus
10.8 Messaging & Microservices
10.14 Caching
Building blocks from the design seat
Part 11 — System Design Case Studies
Part 12 — AI & ML Engineering
12.1 What AI Engineering Is
12.2 The Math You Need
12.3 Classical Machine Learning
12.4 Data for ML
12.5 Deep Learning
12.6 Embeddings & Vector Search
12.7 Transformers & LLMs
12.8 Building with LLMs
12.9 Retrieval-Augmented Generation
12.10 Agents
12.11 Adapting Models
12.12 Production AI
12.13 Multimodal AI
12.14 AI Security & Governance
12.15 Case Studies & Capstone
Part 13 — Cloud, DevOps & Infrastructure
13.11 Git, from First Commit to Team Workflow
Part 14 — Containers: Docker & Kubernetes
14.1 Foundations
14.2 Docker
14.3 Kubernetes
14.4 Kubernetes on Azure
Part 15 — The Craft & Career
Nothing picked yet