Appearance
Async, Streaming and Performance
22. When should an endpoint be async def, and when plain def?
The short answer
Use async def only when everything slow inside it is awaited with async libraries, such as httpx.AsyncClient, asyncpg, async SQLAlchemy or redis.asyncio. Use plain def when it calls blocking code, such as requests, a synchronous database driver or a slow file operation: FastAPI runs def endpoints, and def dependencies, in a thread pool, so they do not block the event loop. When in doubt, plain def is the safe default. The one thing you must never do is call blocking code inside an async def.
Measured: ten concurrent requests to endpoints that each wait 0.5 seconds, one Uvicorn worker:
python
@app.get("/async")
async def with_await():
await asyncio.sleep(0.5) # non-blocking wait → 10 requests in 0.6 s
@app.get("/sync")
def plain_def():
time.sleep(0.5) # blocking, but in the thread pool → 0.6 s
@app.get("/blocking-async")
async def blocking_in_async():
time.sleep(0.5) # blocking the event loop → 5.0 s #Reading it: the first two finished all ten requests in about the time of one, because the waits overlapped. The third ran them one after another: while one request slept, the single event loop could do nothing else, including serving the other nine, or health checks.
| Endpoint calls | Write it as |
|---|---|
| Only awaited async libraries | async def |
| Blocking libraries | def |
| A mix | async def, with blocking parts wrapped in await run_in_threadpool(func) or Asyncer's asyncify |
| CPU-heavy work | Neither: a process pool or a task queue (question 24) |
23. What happens if you call blocking code inside an async endpoint, and how do you fix it?
The short answer
The blocking call holds the event loop, so every request on that worker waits, as the measurement in question 22 shows: ten 0.5-second requests took 5 seconds instead of 0.6. Throughput collapses and latency spikes under load. The fixes: switch to an async library; change the endpoint to plain def; or wrap just the blocking part with await run_in_threadpool(fn, *args) from fastapi.concurrency, or asyncify from Asyncer. For CPU-heavy work, threads do not help on standard CPython, so use a process pool or a background worker.
| Blocking call | Async replacement |
|---|---|
requests.get | httpx.AsyncClient().get |
psycopg2, sync SQLAlchemy | asyncpg, psycopg async, SQLAlchemy AsyncSession |
redis.Redis | redis.asyncio.Redis |
time.sleep | await asyncio.sleep |
boto3 | aioboto3, or the thread pool |
| Large file reads | anyio file APIs, or the thread pool |
python
from fastapi.concurrency import run_in_threadpool
@app.post("/invoices/{order_id}/pdf")
async def render_invoice(order_id: int, session: Session):
order = await orders.get(session, order_id) # async I/O
pdf = await run_in_threadpool(render_pdf_blocking, order) # (1)
return Response(pdf, media_type="application/pdf")- Only the blocking library call goes to a thread; the rest stays async.
The asyncio side of this is in blocking code in async functions.
24. What are background tasks, and when do you need a real task queue instead?
The short answer
BackgroundTasks runs a function after the response is sent, in the same process: useful for small, quick, non-critical work like sending a confirmation email or writing an audit record, so the client does not wait. But it is not durable: if the process restarts or crashes, the task is lost; there are no retries; and heavy work still uses the same worker's CPU. For anything important, slow or retryable, such as payments, video processing or reports, use a task queue with separate workers, such as Celery, ARQ, Dramatiq or a cloud queue.
python
from fastapi import BackgroundTasks
def send_receipt(email: str) -> None:
... # takes about 200 ms
@app.post("/orders", status_code=201)
async def create_order(order: OrderIn, bg: BackgroundTasks, user: CurrentUser):
saved = await orders.create(user["id"], order)
bg.add_task(send_receipt, user["email"]) # (1)
return saved # (2)- Queued to run after the response.
- The client gets
201immediately; in the measured run the response took under 10 ms, and the email function ran 200 ms later.
BackgroundTasks | Task queue: Celery, ARQ, Dramatiq | |
|---|---|---|
| Runs in | The web worker, after the response | Separate worker processes |
| Survives a restart | No | Yes, the queue keeps it |
| Retries | No | Yes, with backoff |
| Scales separately | No | Yes |
| Use for | Small, best-effort follow-ups | Anything that must happen |
Queues in general: message queues.
25. How do you stream responses: Server-Sent Events, JSON Lines and bytes?
The short answer
FastAPI streams when the endpoint yields values. For Server-Sent Events, set response_class=EventSourceResponse from fastapi.sse and yield objects, which are sent as JSON data: fields, or ServerSentEvent instances for full control of event, id and retry. For JSON Lines, one JSON object per line, declare a return type such as AsyncIterable[Item] and yield items. For raw bytes, such as a large file or generated CSV, set response_class to StreamingResponse or a subclass and yield chunks. Streaming keeps memory flat and lets clients see results immediately, such as tokens from an AI model.
python
from collections.abc import AsyncIterable
from fastapi.sse import EventSourceResponse, ServerSentEvent
@app.get("/orders/{order_id}/events", response_class=EventSourceResponse)
async def order_events(order_id: int) -> AsyncIterable[ServerSentEvent]:
async for status in order_status_feed(order_id): # (1)
yield ServerSentEvent(data={"status": status}, event="status", id=status)What the browser receives:
text
event: status
data: {"status": "paid"}
id: paid
event: status
data: {"status": "shipped"}
id: shippedpython
@app.get("/products/export")
async def export_products(session: Session) -> AsyncIterable[ProductOut]: # (2)
async for product in products.stream_all(session):
yield product- Each yielded event goes out immediately; the connection stays open until the generator ends or the client disconnects.
- JSON Lines: each product is validated against
ProductOut, serialised by Pydantic, and written as one line, so exporting a million products never holds them all in memory.
SSE versus WebSockets is compared in real-time options.
26. How do WebSockets work in FastAPI?
The short answer
Declare @app.websocket("/ws/...") with a WebSocket parameter, await websocket.accept(), then loop on receive_text or receive_json and send_text or send_json. Handle WebSocketDisconnect to clean up. Dependencies work too, for example to authenticate from a query parameter or cookie, since browsers cannot set custom headers on WebSocket connections. Each connection lives in one worker process, so to broadcast across workers or servers, use Redis Pub/Sub or a broker. Use SSE instead when only the server needs to push.
python
from fastapi import WebSocket, WebSocketDisconnect
class Rooms:
def __init__(self):
self.members: dict[str, set[WebSocket]] = {}
async def join(self, room: str, ws: WebSocket):
await ws.accept()
self.members.setdefault(room, set()).add(ws)
def leave(self, room: str, ws: WebSocket):
self.members.get(room, set()).discard(ws)
async def broadcast(self, room: str, message: dict):
for ws in list(self.members.get(room, ())):
await ws.send_json(message)
rooms = Rooms()
@app.websocket("/ws/orders/{order_id}")
async def order_chat(ws: WebSocket, order_id: str, user: WsUser): # (1)
await rooms.join(order_id, ws)
try:
while True:
text = await ws.receive_text() # (2)
await rooms.broadcast(order_id, {"from": user.id, "text": text})
except WebSocketDisconnect:
rooms.leave(order_id, ws) # (3)WsUseris a dependency that authenticates from a cookie or a short-lived token in the query string.- Wait for the next message without blocking other connections.
- Always clean up when the client goes away.
27. How do you add middleware in FastAPI?
The short answer
Middleware wraps every request and response. Add built-in ones with app.add_middleware(...): CORSMiddleware for browser cross-origin rules, GZipMiddleware for compression, TrustedHostMiddleware to reject unexpected Host headers, HTTPSRedirectMiddleware. Write your own with @app.middleware("http"), an async function that receives the request and call_next, for timing, request IDs or headers. Middleware runs for every route, so keep it light; use dependencies for logic that needs typed inputs or applies to only some routes.
python
import time, uuid
from fastapi.middleware.cors import CORSMiddleware
app.add_middleware(
CORSMiddleware,
allow_origins=["https://shop.example"], # (1)
allow_credentials=True,
allow_methods=["GET", "POST", "PATCH", "DELETE"],
allow_headers=["Authorization", "Content-Type"],
)
@app.middleware("http")
async def request_context(request: Request, call_next):
request_id = request.headers.get("x-request-id") or str(uuid.uuid4())
start = time.perf_counter()
response = await call_next(request) # (2)
response.headers["x-request-id"] = request_id
response.headers["server-timing"] = f"app;dur={(time.perf_counter() - start) * 1000:.1f}"
return response- An explicit list of origins, never
*with credentials (CORS). - Everything before
call_nextruns on the way in, everything after on the way out.
Order: middleware added last runs first on the request. Add CORS so it wraps everything, including error responses.
28. How do you tune FastAPI performance?
The short answer
Most of the time goes to I/O, so first make every endpoint correctly async or sync (question 22), fix slow queries and N+1 loading, add caching, and reuse connection pools and HTTP clients from the lifespan. Then: declare return types, so Pydantic serialises in Rust instead of jsonable_encoder in Python; run several Uvicorn workers, about one per CPU core; keep responses small with pagination; move heavy work to queues; and profile under realistic load. Do not switch to ORJSONResponse, which is deprecated and no longer faster.
| Area | Do |
|---|---|
| Endpoint style | async def with async libraries, or plain def |
| Serialisation | Return types or response_model, so Pydantic's Rust core does it |
| Connections | One engine and one HTTP client per worker, created in the lifespan |
| Database | Indexes, selectinload to avoid N+1, pagination (N+1 in SQLAlchemy) |
| Processes | fastapi run --workers 4, or one process per container with several replicas |
| Heavy work | Task queue, or a process pool |
| Measure | OpenTelemetry traces, py-spy or the 3.15 Tachyon profiler on a live worker |
Workers versus containers: on a VM, run --workers equal to the cores. On Kubernetes, run one worker per container and scale the number of pods, so the orchestrator handles restarts and health per process (cluster versus containers).