Skip to content

Interview PrepIteration · Core

Iterators, Generators and Context Managers ​

20. What is the difference between an iterable and an iterator? ​

The short answer

An iterable is anything you can loop over: it has an __iter__ method that returns an iterator. Lists, strings, dicts, sets, files and ranges are iterables. An iterator is the object that actually produces values one at a time: it has __next__, which returns the next value or raises StopIteration when finished, and its __iter__ returns itself. A list can be looped over many times; an iterator is used up after one pass. A for loop calls iter() on the iterable, then next() until StopIteration.

python
it = iter([10, 20])
print(next(it), next(it), next(it, "done"))   # 10 20 done

What for really does:

python
items = ["shoes", "socks"]
iterator = iter(items)              # (1)
while True:
    try:
        item = next(iterator)       # (2)
    except StopIteration:           # (3)
        break
    print(item)
  1. Ask the iterable for a fresh iterator.
  2. Pull one value.
  3. The end is signalled by an exception, which the loop handles for you.
IterableIterator
Has__iter____iter__ and __next__
Exampleslist, str, dict, rangeiter(list), a generator, a file object
Loop twiceYesNo, it is exhausted

21. What are generators, and how does yield work? ​

The short answer

A generator function is a function containing yield. Calling it does not run the body; it returns a generator object, which is an iterator. Each next() runs the body until the next yield, hands out that value, and pauses, keeping all local variables. The next next() resumes right after the yield. When the function returns, the generator raises StopIteration. Generators produce values lazily, use almost no memory, can represent infinite sequences, and make iterators trivial to write.

python
def steps():
    print("start");  yield 1
    print("middle"); yield 2
    print("end")

gen = steps()          # nothing printed yet
print(next(gen))       # start, then 1
print(next(gen))       # middle, then 2
next(gen)              # end, then StopIteration

A generator as a class's iterator:

python
class Countdown:
    def __init__(self, start):
        self.start = start
    def __iter__(self):              # (1)
        n = self.start
        while n > 0:
            yield n
            n -= 1

print(list(Countdown(3)))            # [3, 2, 1]
  1. Making __iter__ a generator gives the class a fresh iterator every time it is looped over, with no __next__ to write by hand.

22. Why are generators memory-efficient, and how do you build lazy pipelines? ​

The short answer

A generator holds only its current state, not all its values, so processing a 10 GB log file line by line uses the same small amount of memory as a 10 KB one. Chain generators into a pipeline: each stage pulls one item from the previous stage, transforms or filters it, and yields it on, so only one item is in flight at a time. Nothing runs until the final consumer, such as a for loop or sum, asks for values. The trade-offs: a generator can be consumed only once, and you cannot index it or ask its length.

python
def read_orders(lines):
    for line in lines:                       # (1)
        sku, qty = line.split(",")
        yield sku, int(qty)

def large_only(rows, min_qty):
    for sku, qty in rows:                    # (2)
        if qty >= min_qty:
            yield sku

with open("orders.csv") as f:
    for sku in large_only(read_orders(f), 5):   # (3)
        print(sku)
  1. A file object is itself a lazy iterator of lines, so the whole file is never loaded.
  2. Each stage pulls from the previous one, one item at a time.
  3. The loop drives everything. With the lines A1,2, B7,9 and C3,12, this prints B7 and C3.

The memory difference, measured on CPython 3.14: a list of 1 million squares takes about 8.4 MB; the generator expression (n * n for n in range(1_000_000)) is a 200-byte object.

Batches from any iterator:

python
from itertools import islice

def chunks(iterable, size):
    it = iter(iterable)
    while batch := list(islice(it, size)):   # walrus: assign and test
        yield batch

print(list(chunks(range(7), 3)))    # [[0, 1, 2], [3, 4, 5], [6]]

Since Python 3.12, itertools.batched(iterable, 3) does the same, yielding tuples.

23. What do yield from and send() do? ​

The short answer

yield from other delegates to another iterable or generator: it yields every value from it in turn, which makes recursive generators simple, such as flattening nested lists or walking a tree. gen.send(value) resumes a paused generator and makes the yield expression evaluate to that value, so values can flow into a generator as well as out. A generator must first be advanced to its first yield, with next(gen), before you can send to it. send was the foundation of early coroutines; today async/await covers that use.

python
def flatten(items):
    for x in items:
        if isinstance(x, list):
            yield from flatten(x)      # (1)
        else:
            yield x

print(list(flatten([1, [2, [3, 4]], 5])))   # [1, 2, 3, 4, 5]

def running_average():
    total = count = 0
    average = None
    while True:
        value = yield average          # (2)
        total += value
        count += 1
        average = total / count

avg = running_average()
next(avg)                              # (3)
print(avg.send(10), avg.send(20), avg.send(60))   # 10.0 15.0 30.0
  1. Hand over to the inner generator until it is exhausted, then continue.
  2. yield both sends out the current average and receives the next value.
  3. Prime the generator, running it to the first yield.

24. What is a context manager, and how does the with statement work? ​

The short answer

A context manager sets something up and guarantees cleanup, even if an error happens. with open(path) as f: opens the file and always closes it. Under the hood, with calls the object's __enter__, binds its return value to the as name, runs the block, and then always calls __exit__(exc_type, exc, traceback), with the exception details if the block raised. If __exit__ returns True, the exception is suppressed; otherwise it continues upwards. Typical uses: files, locks, database transactions, temporary settings and timers.

python
import time

class Timer:
    def __enter__(self):
        self.start = time.perf_counter()
        return self                                        # (1)
    def __exit__(self, exc_type, exc, tb):
        ms = (time.perf_counter() - self.start) * 1000
        print(f"took {ms:.0f} ms, error={exc_type.__name__ if exc_type else None}")   # (2)
        return False                                       # (3)

with Timer():
    time.sleep(0.05)          # took 50 ms, error=None

with Timer():
    raise ValueError("bad sku")   # took 0 ms, error=ValueError, then the ValueError propagates
  1. The value bound by as.
  2. __exit__ runs in both cases, and can see whether an exception happened.
  3. Return False so the exception is not swallowed.
UseExample
Fileswith open("report.csv") as f:
Lockswith lock:
Database transactionswith conn.transaction():
Several at oncewith open(a) as src, open(b, "w") as dst:
Async resourcesasync with httpx.AsyncClient() as client:

25. How does contextlib.contextmanager help you write context managers? ​

The short answer

@contextmanager turns a generator function into a context manager, so you do not need a class. Code before the yield is the setup, __enter__; the yielded value is what as receives; code after the yield is the cleanup, __exit__. Wrap the yield in try/except/finally to handle errors raised inside the with block, and re-raise unless you intend to suppress them. contextlib also provides suppress, closing, ExitStack and nullcontext.

python
from contextlib import contextmanager

@contextmanager
def transaction(conn):
    print("BEGIN")                 # (1)
    try:
        yield conn                 # (2)
        print("COMMIT")            # (3)
    except Exception:
        print("ROLLBACK")          # (4)
        raise

with transaction(db):
    pass                           # BEGIN, COMMIT

with transaction(db):
    raise RuntimeError("fail")     # BEGIN, ROLLBACK, then RuntimeError
  1. Setup runs when the with block starts.
  2. The yielded value goes to as, if used; the with body runs at this point.
  3. The block finished normally: commit.
  4. The block raised: the exception is thrown into the generator at the yield, so roll back and re-raise.

ExitStack handles a variable number of resources, for example opening every file in a list and closing them all, even if opening the fifth fails.

26. Which itertools functions are worth knowing? ​

The short answer

itertools gives fast, lazy building blocks for iteration. islice takes a slice of any iterator; batched splits into fixed-size groups; chain joins iterables; accumulate gives running totals; groupby groups consecutive items by a key, so sort first; pairwise yields neighbours; product, permutations and combinations generate arrangements; count, cycle and repeat make infinite iterators. They save memory and often replace nested loops.

python
import itertools as it

list(it.batched("ABCDEFG", 3))           # [('A', 'B', 'C'), ('D', 'E', 'F'), ('G',)]
list(it.chain([1, 2], [3]))              # [1, 2, 3]
list(it.accumulate([1, 2, 3, 4]))        # [1, 3, 6, 10]
list(it.pairwise([1, 2, 4, 7]))          # [(1, 2), (2, 4), (4, 7)]
list(it.product("ab", [1, 2]))           # [('a', 1), ('a', 2), ('b', 1), ('b', 2)]

rows = [("Eng", "Ravi"), ("Eng", "Lena"), ("Sales", "Tom")]
{k: [n for _, n in g] for k, g in it.groupby(rows, key=lambda r: r[0])}
# {'Eng': ['Ravi', 'Lena'], 'Sales': ['Tom']}

The groupby trap: it only groups items that are next to each other. Unsorted input with Eng, Sales, Eng gives two separate Eng groups. Sort by the same key first, or use a defaultdict(list).

Combine chapters into one PDF

Pick the chapters you want. They print in the order you pick them.

Interview Prep
Python
Foundations
Functions
Iteration
Objects
Internals
Errors and packaging
Concurrency
Modern Python
Coding
Other topics
Nothing picked yet