Appearance
3.13.11 — Modules, Packages & Environments
A new engineer clones the order service, runs python app/main.py, and gets ImportError: attempted relative import with no known parent package. They "fix" it by adding the folder to sys.path. Next, two modules that import each other start failing with cannot import name 'Order' from partially initialized module. Then a library upgrade for one project breaks another project on the same laptop. Every one of these problems comes from not knowing what import actually does, or where Python looks for packages. This page walks the import system step by step, then explains packages, circular imports, virtual environments and how a modern Python project is laid out.
1. What import does, step by step
A module is a file of Python code, and at run time it is an object of type module whose attributes are the file's top-level names. When Python runs import shop.pricing, it goes through these steps.
- Check the cache. Python looks in
sys.modules, a dict from module name to module object. If the module is already there, it is reused and nothing else happens. - Find it. If not cached, Python asks a list of finders, stored in
sys.meta_path, whether they know whereshop.pricinglives. Each finder returns a module spec, a small record saying "here is the module, and here is the loader that can create it". - Create the module object. The loader creates an empty module object and fills in attributes such as
__name__,__file__and__spec__. - Add it to the cache first. The new, still-empty module is stored in
sys.modulesbefore its code runs. Remember this step. It is the reason circular imports behave the way they do. - Run the code. The loader compiles the file to bytecode, or loads the cached
.pycfrom 3.13.1, and executes it with the module's namespace as its globals. Every top-leveldef,classand assignment becomes a module attribute. - Bind the name. Finally, the
importstatement binds a name in the importing module:shopforimport shop.pricing, oradd_taxforfrom shop.pricing import add_tax.
Here is a small shop package that lets us watch those steps.
text
project/
demo.py
shop/
__init__.py ← print(" running shop/__init__.py"); from .pricing import add_tax
pricing.py ← print(" running shop/pricing.py"); TAX_RATE = 0.18; def add_tax(...)
payments/
__init__.py
upi.pypython
# demo.py
import sys
import shop # (1)
import shop.pricing # (2)
import shop # (3)
print(shop.add_tax(100))
print("shop" in sys.modules, sys.modules["shop.pricing"] is shop.pricing) # (4)
print(type(shop), shop.__name__)
print(shop.pricing.__spec__.name, type(shop.pricing.__spec__.loader).__name__) # (5)text
running shop/__init__.py
running shop/pricing.py
118.0
True True
<class 'module'> shop
shop.pricing SourceFileLoader(1)Importing a package runs its__init__.py, which itself importspricing. Both prints appear.(2)shop.pricingis already insys.modules, so nothing runs.(3)Nor here. A module's code runs once per process, however many times and from however many files it is imported.(4)Every importer receives the same module object from the cache.(5)The spec records the module's name and the loader that created it.SourceFileLoaderloads ordinary.pyfiles.
That single-run behaviour is why a module works as a natural singleton. A settings object or a database engine created at the top of a module is shared by everyone who imports it. It is also why a slow top-level statement, such as connecting to a database at import time, slows down every program that imports the module, even one that never uses it.
Where Python looks: sys.path and the finders
sys.meta_path holds three standard finders, tried in order.
BuiltinImporterfinds modules compiled into the interpreter itself, such assysandmath.FrozenImporterfinds modules frozen into the interpreter binary for fast startup, such as parts of the import system itself.PathFindersearches the folders listed insys.path, in order, for a matching file or folder.
sys.path is built at startup from:
- the folder of the script you ran, or the current folder for
python -candpython -m; - the folders in the
PYTHONPATHenvironment variable, if set; - the standard library folders;
- the
site-packagesfolder, where installed packages live.
The first match wins. That explains a classic mistake. Name your own file random.py or json.py, and every import json in your project, and in every library you use, finds your file first. Python 3.13 and later even add a hint to the error message: "consider renaming … if it has the same name as a library you intended to import".
importlib.reload
Because modules run once, editing a file does not affect a running program. importlib.reload(module) runs the module's code again in the same module object. It is handy in an interactive session, but objects created before the reload still point at the old classes and functions. In real services, restart the process instead.
2. Packages
A package is a folder of modules that can be imported as one name. A regular package is a folder containing an __init__.py file. When the package is imported, __init__.py runs, and whatever it defines becomes the package's attributes. Common uses are:
- re-exporting the public names, as
shop/__init__.pydoes withadd_tax, so callers writefrom shop import add_taxinstead of reaching into internal modules; - defining
__all__; - leaving the file empty, which is perfectly fine.
A package has a __path__ attribute, the list of folders to search for its submodules. That attribute is what makes it a package and not just a module.
Python 3.3 added namespace packages (PEP 420): a folder without __init__.py can still be imported as a package. Their purpose is to let one package name be split across several folders, even several installed distributions. A company might install acme.billing and acme.shipping separately, both under the acme name. For your own application code, include __init__.py. Leaving it out by accident creates a namespace package, which works until a tool such as a type checker or a test runner treats the folder differently.
Absolute and relative imports
Inside a package, there are two ways to name a sibling module.
python
# shop/payments/upi.py
from shop.pricing import add_tax # (1) absolute
from ..pricing import add_tax # (2) relative
def charge(amount):
return f"UPI charge {add_tax(amount)}"(1)An absolute import names the full path from the top of the project.(2)A relative import starts with dots. One dot means "this package", two dots mean "the parent package". It is resolved using the module's__package__attribute, here"shop.payments".
Now the opening error. Run that file directly and it fails.
text
$ python shop/payments/upi.py
ImportError: attempted relative import with no known parent package
$ python -m shop.payments.upi
running shop/__init__.py
running shop/pricing.pyWhen you run a file by its path, Python treats it as a standalone script named __main__, with no package. So .. means nothing. python -m shop.payments.upi runs the same file as the module shop.payments.upi, imported through the normal system from the project root. Its package is known, so relative imports work. The correct fix is to run code with -m, or through an installed entry point (section 6), and never to edit sys.path by hand.
__all__ and import *
from module import * imports a module's public names. Which names are public? If the module defines a list called __all__, exactly those. Otherwise, every top-level name that does not start with an underscore.
python
from shop.pricing import *
print("add_tax" in dir(), "TAX_RATE" in dir(), "_round_paise" in dir())text
True True Falseimport * is fine in an interactive session and in a package's __init__.py when re-exporting. In ordinary code it hides where names come from and can silently overwrite your own names. Linters warn about it. Even if nobody uses *, __all__ is still worth defining, because it documents the public API, and tools such as documentation generators read it.
3. Circular imports
Two modules that import each other are a common source of confusing errors. Here is the classic case: an order knows its customer, and a customer can find its latest order.
python
# shop/orders.py
from shop.customers import Customer # (1)
class Order:
def __init__(self, customer):
self.customer = customerpython
# shop/customers.py
from shop.orders import Order # (2)
class Customer:
def latest_order(self):
return Order(self)text
$ python -c "import shop.orders"
ImportError: cannot import name 'Order' from partially initialized module 'shop.orders' (most likely due to a circular import)Walk through it using step 4 from section 1.
import shop.orderscreates an emptyshop.ordersmodule, adds it tosys.modulesand starts running its code.- Line
(1)importsshop.customers. That module is created, cached, and starts running. - Line
(2)asks forOrderfromshop.orders. The module is found in the cache, so it is not run again. But it is still stuck at line(1), andclass Orderhas not been executed yet. - So
Orderdoes not exist on the module, and the import fails. The message calls it partially initialized, which is exactly right.
Step 4 in section 1 is what stops this from being an infinite loop. Without the early caching, each module would import the other forever.
There are four fixes, from best to worst.
- Remove the cycle. Often two modules that need each other are really one idea split badly, or both need a third thing. Move the shared piece into its own module that both import.
- Import the module, not the name.
import shop.ordersincustomers.py, and useshop.orders.Orderinside the method. The module object already exists in the cache, and the nameOrderis looked up later, when the method actually runs. By then both modules have finished loading. - Import inside the function that needs it. That delays the import until the first call. It works, but it hides a dependency and repeats a dict lookup on every call.
- Import only for type hints. If one module needs the other only for annotations, put the import under
if TYPE_CHECKING:. That block never runs at all, but type checkers read it. 3.13.12 shows it.
4. Virtual environments: one set of packages per project
Installed packages live in a site-packages folder. If every project on your laptop shares one, then project A needing pydantic version 1 and project B needing version 2 cannot both work. A virtual environment, or venv, gives each project its own site-packages while sharing the Python interpreter.
text
$ python -m venv .venv # or: uv venv
$ ls .venv
pyvenv.cfg Scripts/ (bin/ on Linux and macOS) Lib/site-packages/How does it work? There is no container or copy of Python. There are just three ideas.
.venv/Scripts/python(or.venv/bin/python) is a small link or launcher that points at the real interpreter.- Next to it sits
pyvenv.cfg. When Python starts, it looks for this file one folder up from its own executable. If it finds one, it knows it is running inside a venv. - It then sets
sys.prefixto the venv folder and puts the venv'ssite-packagesonsys.pathinstead of the global one.
Here is a real pyvenv.cfg.
text
home = C:\Users\asha\AppData\Local\Python\pythoncore-3.14-64
implementation = CPython
version_info = 3.14.6
include-system-site-packages = falsehome is where the real interpreter lives. include-system-site-packages = false means the global packages are invisible. From inside Python you can always check where you are.
python
import sys
print(sys.prefix != sys.base_prefix) # (1)text
True(1)Inside a venv,sys.prefixis the venv andsys.base_prefixis the real Python installation.
"Activating" a venv with source .venv/bin/activate only puts .venv/bin first on your shell's PATH, so that typing python finds the venv's interpreter. Activation is a convenience. Running .venv/bin/python app.py directly works just as well, and is what Docker images and servers usually do.
Installing packages is covered in 3.10. The short version: pip install downloads and unpacks packages into the active environment's site-packages. uv, written in Rust, does the same job ten to a hundred times faster, also creates venvs, manages Python versions and keeps a lockfile. It has become the default choice for new projects.
5. A modern project layout
Since PEP 518 (2016) and PEP 621 (2020), a Python project describes itself in one file, pyproject.toml, written in the TOML format from 3.10. Here is a FastAPI service's file.
toml
[project]
name = "order-service"
version = "1.4.0"
requires-python = ">=3.12"
dependencies = [ # (1)
"fastapi>=0.115",
"sqlalchemy[asyncio]>=2.0",
"pydantic-settings>=2.0",
]
[project.optional-dependencies]
dev = ["ruff", "mypy"] # (2)
[project.scripts]
order-service = "order_service.cli:main" # (3)
[build-system]
requires = ["hatchling"] # (4)
build-backend = "hatchling.build"(1)The packages the service needs at run time, with version ranges.(2)Extra groups installed only on request, here developer tools.(3)An entry point. Installing the project creates a command calledorder-servicethat importsorder_service.cliand callsmain().(4)The build backend, the tool that turns the project into an installable package. Hatchling, setuptools, flit and uv's own backend all read this same file.
And the folder layout most teams use, called the src layout:
text
order-service/
pyproject.toml
uv.lock ← exact versions of every dependency
src/
order_service/
__init__.py
main.py ← creates the FastAPI app
cli.py
orders/
__init__.py
models.py
service.pyWhy put the package inside src/? Because then the project folder itself is not on sys.path, so import order_service only works after the package is properly installed into the venv. In development you install it in editable mode with uv pip install -e . or pip install -e ., which makes the venv point at your source folder so edits take effect immediately. This catches packaging mistakes, such as a forgotten file, on your laptop instead of in production. It also removes the temptation to fix imports by editing sys.path.
6. Running code the right way
Putting the pieces together, here are the three correct ways to run code in a package.
python -m order_service.mainruns a module by its import name, with the package context set up, so relative imports work.order-serviceruns the entry point command created at install time.uvicorn order_service.main:appasks a server to import your module by name and fetch theappobject from it. 3.14.1 explains what Uvicorn does with it.
All three import your code by its package name from an installed environment. None of them need sys.path hacks.
What the interviewer will push on
"What happens when you import a module twice?" It runs once. The second import finds it in sys.modules and returns the cached object. Strong candidates add the consequence: modules act as singletons, and slow work at the top of a module slows every importer.
"Why do circular imports fail with 'partially initialized module'?" Because a module is added to sys.modules before its code runs. The second module receives the half-finished first module and asks for a name that has not been defined yet. List the fixes in order: remove the cycle, import the module rather than the name, import lazily, or use TYPE_CHECKING.
"What is the difference between a module and a package?" A package is a module with a __path__, usually a folder with __init__.py. Mention namespace packages and why application code should still use __init__.py.
"How does a virtual environment work?" pyvenv.cfg next to the interpreter link, sys.prefix pointing at the venv, and the venv's own site-packages on sys.path. Activation only edits PATH. The common wrong answer is that a venv copies Python or isolates the operating system.
"Why does python app/main.py fail with a relative import error but python -m app.main works?" Running by path makes the file __main__ with no package. -m imports it by its package name, so __package__ is set and the dots can be resolved.
Something to volunteer that nobody asks for: the src layout plus an editable install means import order_service only works through the installed package, so packaging bugs show up on your machine instead of in the deployed container. And it removes every reason to touch sys.path.
Recall
importcheckssys.modules, asks the finders insys.meta_pathfor a module spec, has the loader create the module, caches it before running it, runs its code once and binds the name.sys.path= script folder,PYTHONPATH, standard library,site-packages. First match wins, so never name a filejson.py.- A package is a module with
__path__: a folder with__init__.py. Namespace packages have no__init__.py. Relative imports use__package__. Run package code withpython -m. __all__lists public names forimport *and documentation.- Circular imports see a partially initialized module. Fix by removing the cycle, importing the module instead of the name, importing lazily, or using
TYPE_CHECKING. - A virtual environment is a
pyvenv.cfgplus its ownsite-packages.sys.prefix != sys.base_prefixinside one. Activation only changesPATH. pyproject.tomlholds the project's metadata, dependencies, entry points and build backend. Use the src layout with an editable install.
Self-test: Which import step makes circular imports fail instead of looping forever? Why does a module behave like a singleton? What is the one file that makes a folder a venv? Why does the src layout catch packaging mistakes?
Quiz Bank
FoundationalYou edit a module while a Python program is running. Why doesn't the running program see the change?
The module's code ran once, when it was first imported, and the resulting module object is cached in sys.modules. Later imports, and all existing references, use that cached object. Python never re-reads the file on its own. importlib.reload re-runs the code in the same module object, but objects created earlier still refer to the old functions and classes. Development servers such as uvicorn --reload instead watch the files and restart the whole process.
AppliedA script named random.py in your project crashes with AttributeError: module 'random' has no attribute 'choice' inside a third-party library. Explain.
The folder of the script you run is placed first on sys.path. When the library executes import random, the path finder finds your random.py before the standard library's random module. Your file is imported as random and cached in sys.modules, so every module in the process, including the library, now gets your file, which has no choice function. Rename the file, and delete any random.cpython-*.pyc in __pycache__ next to it.
InterviewTwo modules in a package import each other's classes at the top. Give two fixes that keep both classes where they are.
First, import the module instead of the name: in customers.py, write import shop.orders at the top and use shop.orders.Order inside methods. The module object exists in sys.modules from the start of the import, and the attribute Order is looked up only when the method runs, after both modules have finished loading. Second, if Order is only needed for type annotations, write from typing import TYPE_CHECKING and put from shop.orders import Order inside if TYPE_CHECKING:. That code never runs, but type checkers read it, and annotations are not evaluated at run time in 3.14. A third option, importing inside the method, also works but hides the dependency.
StaffYour company has twenty Python services and every one handles dependencies differently. What standard would you set, and why?
One pyproject.toml per service, using PEP 621 metadata and a pinned build backend. One tool, such as uv, for creating environments and resolving dependencies, with a committed lockfile so every machine and every Docker build installs exactly the same versions. Top-level dependencies are declared with sensible ranges, while the lockfile pins everything below them. Use the src layout with an editable install for development, and entry points instead of loose scripts. CI checks that the lockfile is up to date and runs a vulnerability audit. Docker images install from the lockfile only, into a venv inside the image. The payoff is that "works on my machine" disappears, upgrades become reviewable lockfile changes, and a security fix can be rolled out to all twenty services the same way.
Next: the TYPE_CHECKING trick only makes sense once you know what type hints are and who reads them. 3.13.12 covers static typing in Python from the first PEP to the features FastAPI depends on.