Appearance
Functions, Scope and Decorators
11. What are *args and **kwargs, and what kinds of parameters can a function have?
The short answer
*args collects any extra positional arguments into a tuple; **kwargs collects any extra keyword arguments into a dict. The names are convention; the * and ** do the work. A full signature can also mark parameters positional-only, everything before a /, and keyword-only, everything after a bare * or after *args. The same symbols unpack when calling: f(*items) spreads a list into positional arguments and f(**options) spreads a dict into keyword arguments.
python
def create_order(customer_id, /, *items, express=False, **meta): # (1)
return customer_id, items, express, meta
print(create_order(42, "shoes", "socks", express=True, source="app"))
# (42, ('shoes', 'socks'), True, {'source': 'app'})
create_order(customer_id=42)
# TypeError: create_order() missing 1 required positional argument: 'customer_id' (2)customer_idis positional-only because it comes before/.itemsgathers extra positional values.expresscomes after*items, so it can only be passed by name.metagathers any other named values.- A surprise worth knowing.
customer_id=42is not rejected as "positional-only"; it is swallowed by**meta, and Python then complains that the positional argument is missing.
| Syntax | Meaning |
|---|---|
def f(a, b) | Positional or keyword |
def f(a, /, b) | a positional-only |
def f(a, *, b) | b keyword-only |
def f(*args) | Extra positional values as a tuple |
def f(**kwargs) | Extra keyword values as a dict |
f(*seq, **mapping) | Unpack when calling |
12. Why is a mutable default argument dangerous?
The short answer
Default values are evaluated once, when the function is defined, not each time it is called. So a default like basket=[] is one list shared by every call that does not pass its own. Appending to it in one call leaves the item there for the next call, a classic bug. The fix is to use None as the default and create a fresh object inside the function.
python
def add_item(item, basket=[]):
basket.append(item)
return basket
print(add_item("apple")) # ['apple']
print(add_item("pear")) # ['apple', 'pear'] the same list again
def add_item(item, basket=None):
if basket is None:
basket = [] # a new list on every call
basket.append(item)
return basketWhy Python works this way. def is a statement that runs once and stores the default value on the function object, in add_item.__defaults__. Nothing re-runs it later. The same rule makes def f(now=datetime.now()) capture the time when the module loaded, not when f is called.
13. How does variable scope work in Python, and what do global and nonlocal do?
The short answer
Python looks up a name in the LEGB order: Local, inside the current function; Enclosing, the functions around it; Global, the module; Built-in, names like len. A key rule: if a function assigns to a name anywhere in its body, that name is local to the whole function, so reading it before the assignment raises UnboundLocalError. global x makes assignments go to the module-level x; nonlocal x makes them go to the nearest enclosing function's x. Loops and if blocks do not create a new scope.
python
x = "global"
def outer():
x = "enclosing"
def inner():
return x # (1)
return inner()
print(outer()) # enclosing
count = 0
def broken():
count += 1 # (2)
broken()
# UnboundLocalError: cannot access local variable 'count' where it is not associated with a value
total = 0
def bump():
global total # (3)
total += 1- Lookup walks outward: not local, found in the enclosing function.
count += 1is an assignment, socountis local everywhere inbroken, and it has no value yet when read.globalpoints the name at the module variable. Use it rarely: shared mutable globals make code hard to follow.
14. What are closures, and what is the late-binding trap?
The short answer
A closure is a function that remembers variables from the scope where it was created, even after that scope has finished. A counter factory can return an inner function that keeps incrementing its own private n, using nonlocal. The trap is late binding: closures remember the variable, not its value at the time, so functions created in a loop all see the final value of the loop variable. Fix it by binding the value at creation, for example with a default argument lambda i=i: i, or with functools.partial.
python
def counter():
n = 0
def inc():
nonlocal n # (1)
n += 1
return n
return inc
next_id = counter()
next_id(); print(next_id()) # 2
handlers = [lambda: i for i in range(3)]
print([h() for h in handlers]) # [2, 2, 2] (2)
handlers = [lambda i=i: i for i in range(3)]
print([h() for h in handlers]) # [0, 1, 2] (3)nonlocallets the inner function update the enclosingn, which lives on aftercounter()returns.- All three lambdas look up
iwhen called, after the loop ended withi = 2. - A default argument is evaluated when the lambda is created, capturing 0, 1 and 2.
JavaScript has the same closures (closures) and the same loop trap (the var loop).
15. What are first-class functions, lambdas, map and filter?
The short answer
Functions are first-class objects: you can store them in variables, pass them as arguments, return them from other functions, and keep them in data structures. A lambda is a small anonymous function limited to a single expression, handy for key= arguments: sorted(orders, key=lambda o: o["total"]). map(f, items) applies f to each item and filter(pred, items) keeps items where pred is true; both return lazy iterators. In everyday Python, comprehensions are usually clearer than map and filter with lambdas.
python
orders = [{"id": 1, "total": 4000}, {"id": 2, "total": 2550}, {"id": 3, "total": 9000}]
biggest = sorted(orders, key=lambda o: -o["total"])[0]["id"] # 3
names = list(map(str.upper, ["a", "b"])) # ['A', 'B']
odd = list(filter(lambda n: n % 2, range(6))) # [1, 3, 5]
handlers = {"paid": send_receipt, "refunded": send_refund_email} # functions in a dict
handlers[event["type"]](event) # dispatch by namePrefer comprehensions for readability: [n for n in range(6) if n % 2] reads more plainly than the filter version.
16. What are decorators, and how do they work?
The short answer
A decorator is a function that takes a function and returns a replacement, usually a wrapper that adds behaviour before or after calling the original, such as timing, logging, caching, retries or permission checks. @timed above def f is exactly f = timed(f). Always apply @functools.wraps(fn) to the wrapper, so the result keeps the original's name, docstring and signature for debugging and tools. Accept *args, **kwargs in the wrapper so it works with any function.
python
import functools, time
def timed(fn):
@functools.wraps(fn) # (1)
def wrapper(*args, **kwargs): # (2)
start = time.perf_counter()
try:
return fn(*args, **kwargs) # (3)
finally:
print(f"{fn.__name__} took {(time.perf_counter() - start) * 1000:.0f} ms") # (4)
return wrapper
@timed
def slow_total(n):
time.sleep(0.12)
return n * 2
print(slow_total(21)) # prints "slow_total took 120 ms", then 42
print(slow_total.__name__) # slow_total, thanks to wrapswrapscopies__name__,__doc__and more fromfntowrapper. Without it, every decorated function would be calledwrapperin logs and stack traces.- Accept anything, so the decorator fits any signature.
- Call the original and return its result.
finallyreports the time even iffnraises.
Decorators you already use: @property, @staticmethod, @classmethod, @dataclass, @functools.lru_cache, and FastAPI's @app.get("/orders").
17. How do you write a decorator that takes arguments?
The short answer
Add one more layer: a factory function that takes the arguments and returns the decorator. @retry(times=3) first calls retry(times=3), which returns a decorator, which is then applied to the function. So there are three nested levels: the factory with the settings, the decorator that receives the function, and the wrapper that runs on each call. Use functools.wraps on the innermost wrapper as usual.
python
import functools
def retry(times=3, on=(ConnectionError,)): # (1) the factory
def decorator(fn): # (2) the decorator
@functools.wraps(fn)
def wrapper(*args, **kwargs): # (3) runs per call
for attempt in range(1, times + 1):
try:
return fn(*args, **kwargs)
except on as err:
if attempt == times:
raise # (4)
print(f"attempt {attempt} failed: {err}")
return wrapper
return decorator
calls = 0
@retry(times=3)
def fetch_rates():
global calls
calls += 1
if calls < 3:
raise ConnectionError("reset")
return "ok"
print(fetch_rates())
# attempt 1 failed: reset
# attempt 2 failed: reset
# ok- The factory captures
timesandonin a closure. - The decorator receives the function being decorated.
- The wrapper does the retrying each time the function is called.
- On the last attempt, re-raise the original exception with its traceback. In real code, add a backoff delay between attempts (retries and backoff).
18. What are the most useful tools in functools?
The short answer
lru_cache and cache memoise a function's results by its arguments, so repeated calls with the same inputs are instant; arguments must be hashable. partial creates a new function with some arguments already filled in. wraps is for writing decorators. reduce folds a sequence into one value. cached_property computes an attribute once per instance. singledispatch chooses an implementation by the type of the first argument.
python
from functools import lru_cache, partial, reduce
@lru_cache(maxsize=None)
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(80)) # 23416728348467685, instantly
print(fib.cache_info()) # CacheInfo(hits=78, misses=81, maxsize=None, currsize=81)
with_vat = partial(lambda price, rate: round(price * (1 + rate)), rate=0.2)
print(with_vat(1000)) # 1200
print(reduce(lambda a, b: a * b, [1, 2, 3, 4])) # 24Reading the cache info: without caching, fib(80) would make about 37 trillion calls. With it, each n from 0 to 80 is computed once, 81 misses, and every repeated call is a hit.
Cache with care: maxsize=None never evicts, which can leak memory for unbounded inputs; set a size. Do not cache methods whose results depend on mutable state, and remember the cache holds references to arguments.
19. How are arguments passed to functions in Python?
The short answer
Python passes references to objects, sometimes called "pass by object reference" or "pass by assignment". The parameter becomes a new name for the same object the caller passed. If the function mutates a mutable object, such as appending to a list, the caller sees the change. If the function rebinds the parameter to a new object, the caller does not, because only the local name changed. It is neither classic pass-by-value nor pass-by-reference.
python
def add_tag(tags):
tags.append("sale") # (1) mutates the caller's list
def rename(name):
name = "new" # (2) rebinds the local name only
tags = ["new"]; add_tag(tags)
title = "old"; rename(title)
print(tags, title) # ['new', 'sale'] old- Same list object, changed in place, so the caller sees
'sale'. - A new string bound to the local name
name. The caller'stitlestill points to'old'.
Practical rule: do not mutate arguments unless that is the function's clear purpose; return a new value instead, which avoids surprises for the caller.