Appearance
Docker Compose
122. What is Docker Compose, and what problems does it solve?
The short answer
Docker Compose lets you describe a whole multi-container application in one YAML file: its services, networks, volumes and settings. One command, docker compose up, then creates everything in the right order, and docker compose down removes it. It replaces long lists of docker run commands, so a whole team, and CI, can start the same environment the same way every time.
Without Compose, running the shop locally means typing something like this, in the right order, every time:
bash
docker network create shop
docker volume create pgdata
docker run -d --name db --network shop -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=dev postgres:16
docker run -d --name cache --network shop redis:7
docker run -d --name api --network shop -e DATABASE_URL=postgres://postgres:dev@db/shop -p 8080:8080 shop/api:dev
docker run -d --name web --network shop -p 3000:80 shop/web:devWith Compose, the same setup is a file in the repository:
yaml
# compose.yaml
services:
db:
image: postgres:16
environment:
POSTGRES_PASSWORD: dev
volumes:
- pgdata:/var/lib/postgresql/data
cache:
image: redis:7
api:
build: ./checkout-api
environment:
DATABASE_URL: postgres://postgres:dev@db/shop
ports: ["8080:8080"]
depends_on: [db, cache]
web:
build: ./web
ports: ["3000:80"]
volumes:
pgdata: {}bash
docker compose up -d # create the network, volume and four containers
docker compose logs -f api
docker compose down # remove containers and the network, keep the volumeWhat it solves:
- One source of truth. The environment is described in Git, reviewed like code, and the same for everyone.
- Fast onboarding. A new developer clones the repository and runs one command.
- Automatic wiring. Compose creates a network for the project, and every service can reach every other by its service name, such as
db(question 116). - Repeatable test environments. CI can start real dependencies, run checks against them, and tear them down.
- Simple single-server deployments, for small apps that do not need a cluster.
A version note. Compose today is a plugin of the Docker CLI, run as docker compose. The older standalone Python program, docker-compose with a hyphen, is retired. Modern Compose files also no longer need the version: line at the top.
123. How is Docker Compose different from Docker Engine and Kubernetes?
The short answer
Docker Engine runs containers on one machine, one command at a time. Compose is a client tool on top of it: it reads a YAML file and makes the same Engine calls for you, still on one machine. Kubernetes is a different kind of system altogether: it runs containers across many machines and constantly works to keep them in the state you described, replacing failures, moving work off broken machines and rolling out updates safely.
| Docker Engine | Docker Compose | Kubernetes | |
|---|---|---|---|
| What it is | Container runtime and API | A tool that drives the Engine | A cluster orchestrator |
| Machines | One | One | Many |
| How you describe work | Commands | One YAML file | Many YAML objects |
| Keeps fixing state | No | No, only when you run it | Yes, all the time |
| Survives a machine dying | No | No | Yes |
| Rolling updates with health checks | No | Limited | Yes |
| Autoscaling | No | No | Yes |
The key difference is who is watching. Compose acts only when you run a command. After docker compose up returns, Compose is not running anywhere. If a container dies later, only its restart policy can help. Kubernetes runs control loops that watch forever: if a pod disappears, a controller notices within seconds and creates a new one, on another machine if needed (question 332).
They share ideas. A Compose service is roughly a Kubernetes Deployment plus a Service. Compose networks and service names play the role of Kubernetes Services and DNS. Compose volumes are like persistent volume claims. That is why Compose is a gentle first step towards Kubernetes. A tool called Kompose can even translate a Compose file into Kubernetes manifests as a starting point, though the result always needs human review.
124. What are the key sections in a Compose file, and how do services, networks, volumes, ports, and environment variables relate?
The short answer
The main top-level sections are services, networks, volumes, secrets and configs. Each service describes one kind of container: its image or build, ports, environment, volumes and networks. Services refer to networks and volumes by name, and those names are defined in the top-level sections. Every service also becomes a DNS name that the others can use to reach it.
yaml
name: shop # (1)
services:
api: # (2)
build:
context: ./checkout-api # (3)
image: shop/api:dev
ports:
- "127.0.0.1:8080:8080" # (4)
environment: # (5)
DATABASE_URL: postgres://app@db:5432/shop
LOG_LEVEL: ${LOG_LEVEL:-info} # (6)
env_file: [api.env]
networks: [back] # (7)
volumes:
- uploads:/app/uploads # (8)
secrets: [db_password] # (9)
depends_on:
db:
condition: service_healthy # (10)
restart: unless-stopped
deploy:
resources:
limits: { cpus: "1.0", memory: 512M } # (11)
db:
image: postgres:16
networks: [back]
volumes: [pgdata:/var/lib/postgresql/data]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d shop"]
interval: 5s
retries: 10
networks:
back: {} # (12)
volumes:
pgdata: {}
uploads: {}
secrets:
db_password:
file: ./secrets/db_password.txt # (13)- The project name. Every container, network and volume Compose creates is prefixed with it, so two projects never collide.
- A service called
api. This name is also its DNS name on the project's networks. - Build the image from this folder.
image:names the result. - Publish container port 8080 on the host's loopback only (question 114).
- Environment variables for the container. Note that it reaches the database simply as
db, the other service's name. - Interpolation: Compose replaces
${LOG_LEVEL:-info}with the value ofLOG_LEVELfrom your shell or from a.envfile next to the Compose file, or withinfoif it is unset. - The networks this service joins, by name. They must be defined in the top-level
networkssection. - A named volume, defined at the bottom, mounted at
/app/uploads. - A secret, mounted as a file at
/run/secrets/db_password(question 128). - Start
apionly afterdbreports healthy (question 125). - Resource limits for the container (questions 97 and 98).
- Top-level definitions that services refer to by name.
- The secret's content comes from a file that is kept out of Git.
How the pieces relate: services are the containers. Networks decide who can talk to whom. Volumes decide where data lives. Ports decide what is reachable from outside. Environment variables and secrets pass in settings. A useful command shows the final, fully resolved file, after all interpolation and merging:
bash
docker compose config125. What is the difference between depends_on ordering and an actual application readiness check?
The short answer
A plain depends_on only controls the order in which containers are started. Compose starts the database container, and as soon as it is running, starts the app, even though PostgreSQL inside may need several more seconds before it accepts connections. Readiness means the app inside is actually able to serve. You get that by giving the dependency a health check and using depends_on with condition: service_healthy.
What plain depends_on does:
yaml
services:
api:
depends_on: [db]Compose creates and starts db before api. That is all. "Started" means the container's process has been launched. PostgreSQL then spends a few seconds, or much longer on the first run, creating its files and getting ready. If api connects during that time, it gets "connection refused" and may crash.
What a readiness condition does:
yaml
services:
db:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d shop"] # (1)
interval: 5s
timeout: 3s
retries: 10
api:
depends_on:
db:
condition: service_healthy # (2)
restart: true # (3)pg_isreadyis PostgreSQL's own tool for asking "are you accepting connections yet?" It exits with 0 when the answer is yes.- Compose waits until
db's health status ishealthybefore startingapi. - If
dbis restarted by a Compose command,apiis restarted after it too.
The other conditions:
| Condition | Waits until |
|---|---|
service_started | The container is started, the default |
service_healthy | Its health check passes |
service_completed_successfully | It has run and exited with 0 |
Why the app must still retry. All of this only happens at start-up. If the database restarts in the middle of the day, depends_on does nothing. A well-built app retries its connection with a growing delay instead of crashing. Kubernetes has no depends_on at all, so this habit matters even more there.
126. How would you wait for a database to become ready before running application initialization?
The short answer
Give the database a health check that tests real readiness. Run the initialization, such as database migrations, as its own one-off service that waits for the database to be healthy. Then start the app only after that one-off service has completed successfully. And keep a retry with backoff inside the app, for the times the database disappears later.
yaml
services:
db:
image: postgres:16
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
POSTGRES_DB: shop
secrets: [db_password]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d shop"] # (1)
interval: 3s
retries: 20
migrate:
image: shop/api:dev
command: ["node", "dist/migrate.js"] # (2)
depends_on:
db: { condition: service_healthy }
restart: "no"
api:
image: shop/api:dev
depends_on:
migrate: { condition: service_completed_successfully } # (3)
ports: ["127.0.0.1:8080:8080"]
secrets:
db_password:
file: ./secrets/db_password.txt- The database is "healthy" only when it accepts connections.
- A separate, short-lived service from the same image runs the migrations, then exits. It waits for the healthy database first.
- The API starts only if the migrations finished with exit code 0. If they failed, the API never starts, which is exactly right: running new code against an old schema causes confusing errors.
Retry with backoff in the app itself, in TypeScript:
ts
async function connectWithRetry(pool: Pool, attempts = 10): Promise<void> {
for (let i = 1; i <= attempts; i++) {
try {
await pool.query('SELECT 1') // (1)
return
} catch (err) {
const waitMs = Math.min(500 * 2 ** i, 10_000) // (2)
console.warn(`database not ready (try ${i}/${attempts}), retrying in ${waitMs} ms`)
await new Promise((r) => setTimeout(r, waitMs))
}
}
throw new Error('database never became ready') // (3)
}- A cheap query that succeeds only if the database really answers.
- Wait longer after each failure, doubling up to a cap of 10 seconds. This is exponential backoff: it gives the database time to recover, without hammering it with connection attempts.
- Give up after the last attempt and crash with a clear message, so the restart policy or the orchestrator can deal with it.
An older approach is a "wait-for" shell script in the entrypoint that loops until a port opens. It works, but an open port does not always mean the database is ready, and it adds a shell to the image. The health check approach is better.
127. How do you configure separate development and production settings with Compose without duplicating everything?
The short answer
Keep everything shared in one base file, compose.yaml. Put development-only settings in compose.override.yaml, which Compose merges in automatically. Put production settings in a separate file such as compose.prod.yaml, and combine them explicitly with -f compose.yaml -f compose.prod.yaml. Use variables with defaults for values that change, such as image tags, and check the merged result with docker compose config.
yaml
# compose.yaml: shared by all environments
services:
api:
image: shop/api:${TAG:-dev} # (1)
environment:
DATABASE_URL: postgres://app@db:5432/shop
depends_on:
db: { condition: service_healthy }yaml
# compose.override.yaml: development only, merged automatically
services:
api:
build: ./checkout-api # (2)
volumes:
- ./checkout-api/src:/app/src # (3)
ports:
- "127.0.0.1:9229:9229" # (4)
environment:
LOG_LEVEL: debugyaml
# compose.prod.yaml: production only, chosen explicitly
services:
api:
restart: unless-stopped
deploy:
resources:
limits: { cpus: "2", memory: 1G } # (5)
environment:
LOG_LEVEL: info- The image tag comes from a variable, defaulting to
dev. Production setsTAG=1.4.2. - In development, build locally instead of pulling.
- Bind-mount the source code so edits appear instantly.
- Open Node.js's debugger port, on loopback only.
- Production gets limits and a restart policy.
bash
docker compose up -d # development: base + override
TAG=1.4.2 docker compose -f compose.yaml -f compose.prod.yaml up -d # production: base + prod
docker compose -f compose.yaml -f compose.prod.yaml config # see the merged resultHow merging works: single values, such as image or restart, are replaced by the later file. Maps, such as environment, are merged key by key. Lists such as ports and volumes are combined. Running config before you deploy removes any doubt.
Two other tools: extends lets one service inherit another service's settings, even from another file. include pulls a whole other Compose file into the project, which helps split a large setup into parts owned by different teams.
128. How should secrets and persistent data be handled in a Compose-based local development environment?
The short answer
Use Compose's secrets feature, which mounts each secret as a file under /run/secrets/, and keep the real secret files out of Git. Use only development credentials locally, never production ones. Keep persistent data, such as the database, in named volumes, and remember that docker compose down keeps them while docker compose down -v deletes them.
Secrets:
yaml
services:
api:
image: shop/api:dev
environment:
DB_PASSWORD_FILE: /run/secrets/db_password # (1)
secrets: [db_password, payment_test_key]
secrets:
db_password:
file: ./secrets/db_password.txt # (2)
payment_test_key:
environment: PAYMENT_TEST_KEY # (3)- The app is told where the secret file is, and reads the file. This
_FILEnaming convention is used by many official images, such as PostgreSQL'sPOSTGRES_PASSWORD_FILE. The value never appears indocker inspect, unlike a plain environment variable. - The secret comes from a local file. Put
secrets/in.gitignore, and commit an example file with fake values so people know what to create. - Or from a variable in the developer's own shell.
Rules for local secrets:
- Development credentials only. A laptop is not a safe place for production keys.
- Never commit them. Add a secret scanner as a pre-commit hook (question 14).
- Use test modes of third-party services, such as a payment provider's test key, which cannot move real money.
Persistent data:
bash
docker compose down # removes containers and networks; named volumes stay
docker compose down -v # also deletes named volumes: the local database is gone- Keep the database on a named volume, so recreating containers keeps the data.
- Seed data: the official PostgreSQL image runs any
.sqlor.shfiles it finds in/docker-entrypoint-initdb.d/, but only the first time, when the data folder is empty. Mount your seed scripts there to get a ready-made development database. - When you want a clean start,
down -vis the reset button. Make sure everyone knows what it deletes.
129. What are the limitations of Docker Compose for production orchestration at scale?
The short answer
Compose runs everything on one machine, so it cannot spread work across servers, and if that machine dies, everything dies with it. It has no control loop that keeps fixing things, no rolling updates gated on health, no autoscaling, no cluster-wide load balancing, and only simple file-based secrets. It is excellent for development and for small single-server apps, but larger systems need an orchestrator such as Kubernetes.
What Compose cannot do:
- Use more than one machine. One host's CPU and memory are the ceiling.
- Survive the host dying. There is no other machine to move containers to.
- Keep fixing things on its own. Compose acts only when you run it. It does not notice that a container has been unhealthy for an hour.
- Roll out updates safely. By default,
docker compose upwith a new image stops the old container and starts the new one. That means a short outage. Nothing checks that the new version is healthy before the old one is gone. - Scale automatically.
--scale api=3runs three copies, but only on that one host, and nothing adds copies when traffic grows. - Balance load properly. Several copies share one name through DNS round-robin, without health-aware load balancing.
- Handle secrets and access control seriously. Secrets are plain files on the host, and anyone who can run
dockeron it can do anything. - Run many teams safely on shared machines. There are no namespaces, quotas or permissions per team.
Where Compose is still a fine choice in production: a small internal tool, a single-server app, or a side project, typically on one virtual machine with restart: unless-stopped, backups of its volumes, and a reverse proxy in front. When you outgrow it, the next steps are managed container services such as Azure Container Apps, AWS ECS or Google Cloud Run (questions 559 and 562), or Kubernetes.
475. What are Compose profiles and Compose Watch, and how do they help local development?
The short answer
Profiles let you mark some services as optional, so they start only when you ask for them, such as a database admin tool or a mock payment server. Compose Watch watches your source files and, when they change, either copies them into the running container, restarts it, or rebuilds the image, depending on the rule you set. Together they keep the everyday docker compose up small and the edit-and-see loop fast.
Profiles:
yaml
services:
api:
image: shop/api:dev
db:
image: postgres:16
pgadmin:
image: dpage/pgadmin4
profiles: [tools] # (1)
payment-mock:
image: shop/payment-mock:dev
profiles: [mocks]bash
docker compose up -d # (2)
docker compose --profile tools up -d # (3)pgadmin, a web tool for browsing the database, belongs to thetoolsprofile.- Starts only services with no profile:
apianddb. - Also starts the
toolsprofile. You can name several profiles.
Compose Watch:
yaml
services:
api:
build: ./checkout-api
develop:
watch:
- action: sync # (1)
path: ./checkout-api/src
target: /app/src
- action: rebuild # (2)
path: ./checkout-api/package.json
- action: sync+restart # (3)
path: ./checkout-api/config
target: /app/configbash
docker compose watch- sync: when a file in
srcchanges, copy it into the container. A file-watching dev server inside then reloads it. - rebuild: when
package.jsonchanges, rebuild the image and recreate the container, because new dependencies must be installed. - sync+restart: copy the files, then restart the container, for settings the app only reads at start-up.
Why Watch can beat a bind mount: on Docker Desktop, bind mounts cross a slow file-sharing layer between your laptop and the Linux VM, and permissions can get confusing (question 94). Watch copies only the changed files into the container, so it is fast and has no ownership surprises.
476. How is Docker Swarm different from Kubernetes, and why is it rarely chosen today?
The short answer
Docker Swarm is the orchestrator built into Docker Engine. You turn it on with one command, and deploy multi-container apps from Compose-style files across several machines, with rolling updates, secrets and built-in load balancing. Kubernetes does the same jobs and far more, with a much larger and more flexible system. Swarm is simpler to run, but Kubernetes became the industry standard, with managed versions on every cloud and a huge ecosystem of tools, so most teams choose it.
bash
docker swarm init # (1)
docker stack deploy -c compose.yaml shop # (2)
docker service scale shop_api=5 # (3)
docker service update --image shop/api:1.5 shop_api # (4)- Turns this machine into a Swarm manager. Other machines join as workers with a token it prints.
- Deploys a Compose file as a stack of services across the cluster.
- Runs five copies of the API, spread across the machines.
- A rolling update to a new image.
| Docker Swarm | Kubernetes | |
|---|---|---|
| Set-up | One command | Many components, or a managed service |
| Configuration | Compose files | Many kinds of objects |
| Extensibility | Limited | Custom resources and operators |
| Ecosystem | Small | Very large |
| Managed cloud offerings | Rare | Every major cloud |
| Learning curve | Gentle | Steep |
Why it is rarely chosen now. Around 2017 the industry settled on Kubernetes. Cloud providers built managed Kubernetes services, and tools for monitoring, security, deployment and storage were built for Kubernetes first. Docker's own products moved their focus to developer tools. Swarm still works and is still maintained, and for a small team running a handful of services on a few machines, its simplicity can be the right trade. But in interviews and job descriptions, "orchestration" now almost always means Kubernetes.