Appearance
Inside the Docker Engine
30. What are Docker Engine, Docker CLI, and Docker daemon?
The short answer
Docker Engine is the whole client-and-server system that builds and runs containers. The Docker CLI is the client: the docker command you type, which turns your command into an API request. The Docker daemon, dockerd, is the server: a background process that receives those requests and does the real work of managing images, containers, networks and volumes.
Docker is built like a restaurant. You do not walk into the kitchen to cook. You tell a waiter what you want, and the kitchen does the work.
- The Docker CLI is the waiter. When you type
docker run nginx, thedockerprogram does not start any container itself. It reads your command, turns it into an HTTP request, and sends that request to the daemon. - The Docker daemon,
dockerd, is the kitchen. It runs in the background all the time. It pulls images, builds images, creates containers, sets up networks and volumes, and keeps track of everything. To actually start containers, it hands work tocontainerd, which question 32 explains. - Docker Engine is the whole restaurant: the daemon, its API, and the CLI together.
Because the two halves talk over an API, they do not have to be on the same machine. Your CLI on a laptop can control a daemon on a server. Question 464 shows how.
You can see both halves and their versions with one command:
bash
docker versionClient:
Version: 29.0.0
API version: 1.52
Server: Docker Engine - Community
Engine:
Version: 29.0.0
API version: 1.52 (minimum version 1.44)
containerd:
Version: 2.1.4
runc:
Version: 1.3.0The output has a Client section and a Server section. If the daemon is not running, you get the client section followed by an error such as "Cannot connect to the Docker daemon". That error is the clearest proof that the CLI is only a messenger. The exact version numbers will differ on your machine.
31. How does the Docker CLI communicate with the Docker daemon?
The short answer
Over a REST API, which is HTTP requests and JSON replies. On Linux, these requests go through a Unix socket file, /var/run/docker.sock, by default. The CLI can also reach a daemon over SSH or over TCP protected by TLS. Anyone who can write to that socket can control the daemon, which means they can control the whole machine.
A Unix socket is a special file that two programs on the same machine use to talk to each other. It works like a network connection, but it never leaves the machine. The daemon listens on /var/run/docker.sock. The CLI writes HTTP requests into it.
You can prove that it is plain HTTP by skipping the CLI completely:
bash
curl --unix-socket /var/run/docker.sock http://localhost/containers/json # (1)json
[{"Id":"8f3c...","Names":["/checkout-api-1"],"Image":"checkout-api:1.4","State":"running"}](1) sends an HTTP GET request through the socket, asking for the list of running containers. The reply is JSON. This is exactly what docker ps does before it formats the result as a table.
The ways a CLI can reach a daemon:
| Address | Example | Use |
|---|---|---|
| Unix socket | unix:///var/run/docker.sock | Default on Linux |
| Named pipe | npipe:////./pipe/docker_engine | Default on Windows |
| SSH | ssh://deploy@build-01 | Remote, safest option |
| TCP with TLS | tcp://build-01:2376 | Remote, needs certificates |
| Plain TCP | tcp://build-01:2375 | Never on a network |
The CLI picks the address from the DOCKER_HOST environment variable, or from the current Docker context (question 464).
The security point interviewers want to hear. The socket file belongs to root and to a group called docker. Adding a user to the docker group lets them run docker without sudo. But that user can now run something like docker run -v /:/host ..., which mounts the host's whole disk into a container where they are root. So membership in the docker group is equal to root access. Plain TCP on port 2375 is even worse: anyone on the network gets that power with no password at all. Attackers scan the internet for open port 2375 to install cryptocurrency miners. Question 140 covers the related danger of mounting the socket into a container.
32. What roles do containerd and runc play in a typical Linux container startup path?
The short answer
dockerd hands the job to containerd. containerd manages images and the container's life: it pulls and stores layers, prepares the filesystem, and starts a small shim process for each container. The shim calls runc. runc is a small program that asks the Linux kernel to create the namespaces and cgroup, sets up the root filesystem, starts the app's process, and then exits. The shim stays behind as the container's parent.
The figure shows the full chain for docker run, from your keyboard to a running process.
containerd is a daemon that manages containers for a higher-level tool. Its jobs:
- Images. Pull layers from registries, check their hashes, and store them.
- Snapshots. Unpack layers and prepare each container's stacked filesystem.
- Life cycle. Create, start, stop and delete containers, and report their status.
It exposes a gRPC API. gRPC is a fast way for programs to call each other's functions over a connection. Docker uses containerd, and so does Kubernetes, directly, with no Docker in between.
The shim, containerd-shim-runc-v2, is a tiny process started for each container. It becomes the container's parent process. It keeps the container's standard input and output open, and it waits to collect the exit code when the app ends. Because the shim, not the daemon, is the parent, you can restart or upgrade dockerd and containerd without stopping running containers.
runc is the low-level runtime. It is a command-line program, not a daemon. containerd writes a folder with the container's root filesystem and a config.json describing it, in the format of the OCI runtime specification (question 461). Then runc does the kernel work:
- It creates new namespaces with the
clonesystem call: process, network, mount, hostname, IPC and cgroup namespaces. - It places the process in a cgroup with the memory and CPU limits.
- It switches the process's root to the container's filesystem, using
pivot_root. - It drops Linux capabilities and applies a seccomp filter (question 101).
- It replaces itself with your application, using the
execsystem call.
After that, runc's work is done and it exits. Your app is now running, as process 1 inside its own namespaces.
Why they ask: knowing this chain lets you debug at the right level. A registry error is containerd's world. A "permission denied" during start is usually runc meeting a security setting.
33. What is the difference between Docker Engine and the container runtime?
The short answer
A container runtime is the part that actually creates and runs containers. There are two levels: a low-level runtime such as runc, which talks to the kernel, and a high-level runtime such as containerd, which manages images and container life cycles. Docker Engine sits on top of these and adds the developer features: building images, networks, volumes, Compose support and a friendly API.
People use the word "runtime" loosely, so name the levels clearly:
| Level | Examples | Job |
|---|---|---|
| Platform | Docker Engine, Podman | Builds, networks, volumes, CLI |
| High-level runtime | containerd, CRI-O | Images, life cycle, API |
| Low-level runtime | runc, crun, gVisor, Kata | Create the isolated process |
- Low-level runtimes follow the OCI runtime specification. runc is the standard one. crun does the same job and is written in C, so it is smaller and starts faster. gVisor (
runsc) and Kata Containers give stronger isolation, using a fake kernel or a tiny VM (question 20). - High-level runtimes handle everything around running: pulling and storing images, preparing filesystems, tracking state. containerd and CRI-O both implement the CRI, the Container Runtime Interface, which is how Kubernetes talks to a runtime. Question 321 explains the CRI.
- Docker Engine is a full platform built on containerd. It adds
docker build(through BuildKit), user-friendly networks with built-in DNS, named volumes, logging drivers, and the API thedockerCLI and Docker Compose use.
Why this matters. Kubernetes only needs a high-level runtime. Since Kubernetes 1.24, it talks to containerd or CRI-O directly and does not use Docker Engine at all. Your images still run unchanged, because every level follows the same OCI standards. Docker Engine is a great tool for building and running containers by hand. It is not required to run them in production.
34. What is a container registry, and how is it different from a running Docker host?
The short answer
A registry is a server that stores and serves images over HTTP. You push images to it and pull images from it. It never runs containers. A Docker host is a machine running the Docker daemon, which pulls images from registries into its local store and runs containers from them. The registry is the warehouse; the host is the shop floor.
A good comparison is an app store and a phone. The app store holds apps for millions of people but runs none of them. Your phone downloads an app and runs it.
A registry:
- Stores images as manifests and layers, organised into repositories. A repository is one image name, such as
shop/checkout-api, holding many tags. - Offers an HTTP API defined by the OCI distribution specification (question 461), so any client can push and pull.
- Handles login and permissions: who may push, who may pull.
- Often adds extras: vulnerability scanning, copying images to other regions, and rules that delete old images.
- Examples: Docker Hub, GitHub Container Registry (
ghcr.io), Azure Container Registry, Amazon ECR, Google Artifact Registry, and Harbor, which you can host yourself.
A Docker host:
- Runs
dockerd, which keeps a local image store, a cache of the images it has pulled or built. - Creates and runs containers from images in that local store.
- Never serves images to other machines, unless you also run a registry on it.
The normal flow ties them together:
bash
docker build -t myacr.azurecr.io/shop/checkout-api:1.4 . # (1)
docker push myacr.azurecr.io/shop/checkout-api:1.4 # (2)
docker pull myacr.azurecr.io/shop/checkout-api:1.4 # (3)- Build the image on a CI machine. The name includes the registry's address,
myacr.azurecr.io, so Docker knows where it belongs. - Push it. Docker uploads only the layers the registry does not already have.
- On any other host, pull it. Then run containers from it.
A detail worth saying. The first part of an image name is the registry. nginx is short for docker.io/library/nginx. Docker Hub is only the default when no registry is named. In production, always use full names, so nobody accidentally pulls from the wrong place.
35. What is the difference between a Docker image tag and an image digest?
The short answer
A tag is a human-friendly name, such as nginx:1.27, that points at an image, and it can be moved to point at a different image later. A digest, such as nginx@sha256:5ef7..., is the hash of the image's manifest. It identifies one exact image and can never point at anything else. Tags are for people. Digests are for machines that must get exactly the same bytes every time.
A tag is a label on a box. You can peel it off and stick it on another box. When the nginx team releases a patch, they push a new image and move the 1.27 tag to it. Pull nginx:1.27 today and next month, and you may get two different images.
A digest is the box's fingerprint. It is the SHA-256 hash of the image's manifest, and the manifest lists the hashes of everything inside. Change any byte anywhere and the digest changes. So nginx@sha256:5ef7... will return the same bytes forever, or fail if that image was deleted. It cannot quietly become something else.
When you pull by tag, Docker tells you the digest it got:
bash
docker pull nginx:1.271.27: Pulling from library/nginx
Digest: sha256:5ef79149e0ec84a7a9f9284c3f91aa3c20608f8391f5445eabe92ef07dbda03c
Status: Downloaded newer image for nginx:1.27You can pin the exact image by digest, and you can keep the tag for readability at the same time:
bash
docker pull nginx:1.27@sha256:5ef79149e0ec84a7a9f9284c3f91aa3c20608f8391f5445eabe92ef07dbda03c # (1)(1) uses both. When a digest is present, Docker uses the digest and only treats the tag as a comment for humans.
Two subtleties that impress interviewers:
- One tag, many digests, for different processors. For a multi-platform image, the tag points at an image index, a list of one manifest per platform. The index has its own digest, and each platform's manifest has its own digest too. Pinning the index digest still lets each machine pick its own platform. Question 82 covers multi-platform images.
- The image ID is not the digest.
docker imagesshows anIMAGE ID. That is the hash of the image's config, not of its manifest. The registry digest is the hash of the manifest. They are two different objects, so the two hashes never match. On newer Docker versions that store images in containerd, the ID you see is the manifest digest instead. If someone is confused why the numbers differ, this is why.
What to use in production: deploy by digest, or by a tag you never move (such as the commit ID) and record its digest. Question 87 covers the risks of moving tags.
36. What happens when you run docker run nginx for the first time?
The short answer
Docker expands the name to docker.io/library/nginx:latest, sees the image is not stored locally, and pulls it. It gets a login token from Docker Hub and fetches the manifest list. It picks the manifest for your processor. Then it downloads and checks each layer. Then it creates the container, with a writable layer and a network address, and starts it: runc builds the namespaces and cgroup and launches nginx as process 1. The terminal stays attached to nginx's output. No port is published, so you cannot reach it from the host yet.
This is one of the most common interview questions. The more steps you can name in order, the better. The figure shows the whole journey.
1. The name is expanded. nginx has no registry, no namespace and no tag, so Docker fills in the defaults: docker.io/library/nginx:latest. library is the namespace Docker Hub uses for official images.
2. Create, and the image is missing. The CLI asks the daemon to create a container. The daemon looks in its local image store, does not find the image, and replies "no such image". The CLI prints Unable to find image 'nginx:latest' locally and asks the daemon to pull it.
3. The pull. This is the longest part:
- Authentication. Docker Hub requires a token even for public images. The daemon asks Docker Hub's token service for an anonymous token that may only pull
library/nginx. - The manifest list. It asks for the manifest of tag
latest. nginx is a multi-platform image, so it gets back an image index, a list with one entry per processor type. - Pick the platform. It chooses the entry matching this machine, for example
linux/amd64, and fetches that manifest. - The config and the layers. It downloads the config and every layer it does not already have, several at a time. Each layer's SHA-256 is checked against the manifest, so a corrupted or tampered download is rejected.
- Unpack. Layers are decompressed into the storage driver, ready to be stacked.
4. Create the container. The daemon now creates the container. It gets a random ID and a random name, such as nostalgic_turing. It gets a new writable layer. It also gets a network interface on the default bridge network, with a private IP address such as 172.17.0.2. It copies the defaults from the image's config: the command nginx -g 'daemon off;', port 80 as exposed, and the stop signal.
5. Start it. containerd starts a shim, and the shim runs runc. runc creates the namespaces and the cgroup, mounts the layered filesystem as /, sets the hostname to the container's short ID, drops capabilities, applies the seccomp filter, and replaces itself with nginx's entry script. nginx now runs as process 1 inside the container.
6. Attach. Because you did not pass -d, the CLI stays attached and streams nginx's output to your terminal. Pressing Ctrl+C sends the signal on to nginx, which stops, and the container exits.
The gotcha to mention: nginx is now listening on port 80 inside its own network namespace. Your browser on the host cannot reach it, because no port was published. You need docker run -p 8080:80 nginx. Question 58 explains why EXPOSE alone does not do this.
37. What changes if the requested image already exists locally?
The short answer
The whole pull is skipped. Docker creates and starts the container straight from the local copy, usually in well under a second. Docker does not check the registry for a newer image, even for the tag latest. So a local latest can be weeks out of date. To refresh it, run docker pull or use docker run --pull=always.
When the image is already in the local store, steps ② to ⑤ from question 36 disappear. The daemon goes straight to creating the container and starting it.
The surprise is that Docker does not look for updates. The default pull rule for docker run is --pull=missing: pull only if the image is missing. If you pulled nginx:latest three months ago, docker run nginx today uses that three-month-old image. Nobody tells you.
The three pull rules:
| Option | Behaviour |
|---|---|
--pull=missing | Default. Pull only if not stored locally |
--pull=always | Always ask the registry; download if the digest changed |
--pull=never | Never pull; fail if missing |
A middle case: part of the image is already there. Sometimes you pull a new version and some of its layers are already stored, from an older version or a shared base image. Then Docker downloads only the missing layers. You see this in the pull output:
3ae0c06b4d3a: Already exists
f8a5d1e2b7c9: Pull completeAlready exists means that layer's hash matched one already stored, so no bytes were downloaded.
The production lesson. The same "stale cache" problem appears in Kubernetes as imagePullPolicy. If a node already has checkout-api:latest, it may run an old version while the pipeline thinks it deployed a new one. Question 353 covers this. The cure is the same everywhere: deploy unique tags or digests, never a moving tag, so "already exists locally" can only ever mean "exactly the right image".
464. What is a Docker context, and how do you point the CLI at a remote Docker engine?
The short answer
A Docker context is a saved, named set of connection details for a Docker daemon. You create one for each daemon you use, such as your laptop and a remote build server, and switch between them with docker context use. Every docker command then runs against the selected daemon. For remote daemons, SSH is the simplest safe way to connect.
Because the CLI only sends API requests (question 31), it can talk to any daemon it can reach. A context saves the details so you do not type them each time.
bash
docker context create build-server --docker "host=ssh://deploy@build-01.internal" # (1)
docker context ls # (2)
docker context use build-server # (3)
docker ps # (4)
docker --context default ps # (5)- Creates a context named
build-server. It connects over SSH as the userdeploy. Docker runs the API over the SSH connection, so no extra port is opened and SSH keys do the authentication. - Lists all contexts. The current one is marked with an asterisk.
- Makes
build-serverthe default for all following commands. - Now lists containers on
build-01, not on your laptop. - Runs one command against a different context, without switching.
You can also choose per terminal with environment variables: DOCKER_CONTEXT=build-server, or DOCKER_HOST=ssh://deploy@build-01.internal.
Where this is useful:
- Building images on a powerful remote machine instead of a laptop.
- Switching between Docker Desktop and other local engines.
- Checking a remote server's containers without logging in to it.
Two cautions worth saying:
- It is easy to forget which context is active. A
docker system prunemeant for your laptop could clean a shared server. Check withdocker context show, or put the current context in your shell prompt. - Builds upload the build context over the network. A remote
docker buildsends your whole project folder to the remote daemon first. A good.dockerignorematters even more here. Question 60 covers it.