Docker

Area: Containers and deployment
Related: Container Images, PostgreSQL, Redis, graphile-worker, Kubernetes, Linux namespaces and cgroups

What is it?

Docker packages an app with everything it needs to run (a filesystem with the runtime, libraries and your code) into an image, and runs that image as an isolated process called a container. The same image runs the same way on your laptop, in CI and in production.

A container is not a small virtual machine. There’s no second kernel. It’s an ordinary Linux process that the host kernel has been told to lie to: it sees its own filesystem, its own process list, its own network interfaces, and it has limits on CPU and memory.

A container is a normal process with a private view of the system. An image is a stack of read-only filesystem layers plus a bit of config.

Why it exists

Before containers, “deploy” meant a server with a specific OS version, a specific Node or Python, system libraries installed by hand or by Chef/Puppet, and config files that drifted. Two apps on one box fought over library versions. “Works on my machine” was a real category of bug.

VMs fixed isolation but cost a full OS per app: gigabytes of disk, seconds to minutes to boot, and RAM reserved for each guest kernel.

Linux already had the parts for lighter isolation (chroot since the 1980s, namespaces and cgroups in the 2000s, LXC around 2008). Docker, released in 2013, made them usable: one file describing the image, a layered format that’s cheap to ship, a registry to push and pull from, and one command to run it. The image format and runtime were later standardised by the Open Container Initiative (OCI), so images built by Docker run on containerd, Podman, Kubernetes and the cloud container services.

Containers vs VMs

	Container	VM
Kernel	Shared with the host	Its own
Starts in	Milliseconds (it’s a process)	Seconds or more
Size	Megabytes (only userland files)	Gigabytes
Isolation	Kernel features. A kernel bug can break out.	Hardware virtualisation. Much stronger.
Runs a different OS	No. Linux containers need a Linux kernel.	Yes

That last row is why Docker Desktop on macOS and Windows runs a hidden Linux VM. Your containers run inside it. Bind mounts from your Mac’s filesystem cross that VM boundary, which is why file-heavy work (like node_modules on a bind mount) is slower there than on Linux.

How it works underneath

The process chain

docker CLI ──REST over /var/run/docker.sock──▶ dockerd
   dockerd ──gRPC──▶ containerd          (image pulls, container lifecycle)
   containerd ──▶ containerd-shim ──▶ runc   (OCI runtime: sets up namespaces/cgroups, execs your process)
   runc exits, the shim stays as the parent of your container's PID 1

The shim is why you can restart dockerd without killing running containers. runc is a small program that reads an OCI runtime spec (a JSON file describing namespaces, mounts, limits) and starts the process.

Namespaces: what the process can see

Each namespace gives the process its own copy of one kernel resource:

Namespace	Isolates	Effect
`pid`	Process ids	Your app is PID 1 and can’t see host processes
`net`	Network stack	Own interfaces, IPs, routing table, ports
`mnt`	Mount table	Own root filesystem
`uts`	Hostname	Own hostname
`ipc`	Shared memory, semaphores	No cross-talk with host IPC
`user`	User and group ids	Root inside can map to an unprivileged user outside (not on by default)
`cgroup`	cgroup view	Sees only its own cgroup tree

You can see the same mechanism without Docker: sudo unshare --pid --fork --mount-proc bash and then ps aux shows only two processes.

cgroups: what the process can use

Control groups limit and account resources. docker run --memory 512m --cpus 1.5 writes values into the container’s cgroup (on cgroup v2, files like memory.max and cpu.max).

Memory: going over the limit triggers the kernel OOM killer inside that cgroup. The container exits with code 137 (128 + SIGKILL’s 9). docker inspect shows "OOMKilled": true.
CPU: a limit is a quota per period. Over it, the process is throttled, not killed. That shows up as latency spikes, not errors.
The process still sees the host’s total RAM and CPU count in /proc/meminfo and nproc unless the runtime adjusts it. Runtimes that size themselves from host values (older JVMs, older Node heap defaults, thread pools) over-allocate and get OOM-killed. Modern runtimes read cgroup limits, but check yours.

Images and layers

The full format (manifests, digests, registries) is in Container Images. The short version: an image is:

a config JSON (env vars, entrypoint, working dir, exposed ports, the history),
an ordered list of layers, each a tarball of filesystem changes,
a manifest tying them together.

Everything is content-addressed by SHA-256. A layer’s id is the hash of its contents, so two images that share a base layer store and download it once.

At run time the storage driver (overlay2 on most Linux hosts) stacks the layers with OverlayFS:

merged/   ← what the container sees as /
  upperdir   (writable, per container, thrown away with the container)
  lowerdir   (image layers, read-only, shared by all containers of that image)

Reading a file finds the topmost layer that has it. Writing to a file from a lower layer triggers copy-up: the whole file is copied into upperdir first. Deleting a file writes a “whiteout” marker. Two results:

Deleting a file in a later layer doesn’t shrink the image. The bytes are still in the earlier layer.
Writing heavily to big files in the container layer is slow. Databases must use volumes.

Tags and digests

node:22-alpine is a tag, a movable pointer. The publisher can re-point it tomorrow. node@sha256:3f1c... is a digest and never changes. Pin digests (or let a bot like Renovate update them) when you need reproducible builds. latest is just a tag that happens to be the default name. It doesn’t mean newest.

A tag can also point to a multi-platform index: one manifest per architecture (linux/amd64, linux/arm64). Docker pulls the one matching your machine.

Writing a Dockerfile

# syntax=docker/dockerfile:1
FROM node:22-slim AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm ci
 
FROM deps AS build
COPY . .
RUN npm run build && npm prune --omit=dev
 
FROM node:22-slim AS runtime
ENV NODE_ENV=production
WORKDIR /app
COPY --from=build --chown=node:node /app/node_modules ./node_modules
COPY --from=build --chown=node:node /app/dist ./dist
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

What each part is doing:

Layer cache order. BuildKit (the default builder since Docker Engine 23) caches each step. A step is reused if its instruction and inputs haven’t changed. For COPY the input is a checksum of the copied files. Copying only the lockfile before npm ci means editing source code doesn’t reinstall dependencies. Copy . first and every code change reinstalls everything.

Multi-stage builds. Compilers, dev dependencies and source stay in earlier stages. The final image only gets what COPY --from brings in. That often takes an image from around a gigabyte to a couple of hundred megabytes, and removes tools an attacker could use.

Cache mounts. RUN --mount=type=cache keeps npm’s download cache between builds without baking it into a layer.

Secrets. Never ARG NPM_TOKEN or ENV a secret. Build args are visible in docker history, and a secret written in one layer stays in that layer even if a later one deletes it. Use RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci and pass --secret id=npmrc,src=$HOME/.npmrc.

.dockerignore. The build context (everything in the directory) is sent to the builder. Without ignoring node_modules, .git and .env, builds are slow, cache busts on every commit, and your .env ends up in the image.

node_modules
.git
.env*
dist
*.log

Base image choice.

Base	Size	Trade-off
`node:22` (Debian)	Large	Everything installed, easy debugging
`node:22-slim`	Medium	Debian with fewer packages. Usual default.
`node:22-alpine`	Small	musl libc instead of glibc. Some native modules and DNS behaviour differ.
Distroless	Small	No shell, no package manager. Hard to exec into, small attack surface.

`RUN` vs `CMD` vs `ENTRYPOINT`

RUN executes at build time and produces a layer.
CMD is the default command at run time. docker run image other-cmd replaces it.
ENTRYPOINT is the fixed program. CMD then becomes its default arguments.

Use the exec form (["node", "server.js"]), not the shell form (node server.js). The shell form runs /bin/sh -c "node server.js", so sh is PID 1 and your app is its child, which leads to the next problem.

PID 1 and signals

docker stop sends SIGTERM to PID 1, waits 10 seconds (--time / stop_grace_period), then sends SIGKILL.

The kernel treats PID 1 specially: signals with no handler installed are ignored instead of taking their default action. So:

If sh -c is PID 1, it may not forward SIGTERM to your app. Your app never shuts down cleanly and gets killed after 10 s. Every deploy drops in-flight requests.
If your app is PID 1 and doesn’t handle SIGTERM, same result.
PID 1 also has to reap zombie child processes. Apps that spawn children (headless browsers, shell-outs) leak zombies if PID 1 doesn’t wait().

Fixes: exec form CMD, a real handler, and docker run --init (or init: true in Compose), which runs a tiny init (tini) as PID 1 to forward signals and reap zombies.

process.on("SIGTERM", async () => {
  server.close();          // stop accepting new connections
  await drainJobs();       // finish in-flight work
  await db.end();
  process.exit(0);
});

npm start as CMD has the same issue: npm is PID 1 and handles signals its own way. Run node directly.

Networking

The default network is a Linux bridge (docker0). Each container gets a virtual ethernet pair with one end in its namespace and a private IP.
User-defined bridge networks (what Compose creates) add an embedded DNS server at 127.0.0.11, so containers reach each other by service name: postgres://db:5432. The legacy default bridge has no name resolution.
-p 8080:3000 publishes container port 3000 on host port 8080 using iptables NAT rules (plus a userland proxy in some setups).
Inside a container, localhost is the container itself. Your app connecting to localhost:5432 won’t find Postgres in another container or on the host. Use the service name, or host.docker.internal (Docker Desktop, or with --add-host=host.docker.internal:host-gateway on Linux).
An app listening on 127.0.0.1 inside the container can’t be reached through a published port. Listen on 0.0.0.0.
-p 5432:5432 binds on all host interfaces by default. Docker’s iptables rules are inserted before rules from tools like ufw, so a firewall you think blocks 5432 may not. Bind explicitly: -p 127.0.0.1:5432:5432.

Storage: volumes, bind mounts, tmpfs

Type	Example	Use for
Named volume	`-v pgdata:/var/lib/postgresql/data`	Database files, anything that must outlive the container. Managed by Docker.
Bind mount	`-v "$PWD":/app`	Live-editing code in dev. Mirrors a host path.
tmpfs	`--tmpfs /tmp`	Scratch data in RAM

The container’s own writable layer dies with docker rm. A database without a volume loses its data the first time you recreate the container.

Bind mounts bring UID trouble: files created by a container running as root show up root-owned on your Linux host, and a container running as node (UID 1000) may not be able to write to a host directory owned by someone else.

Docker Compose for local dev

services:
  api:
    build: .
    ports: ["127.0.0.1:3000:3000"]
    environment:
      DATABASE_URL: postgres://app:app@db:5432/app
      REDIS_URL: redis://cache:6379
    depends_on:
      db: { condition: service_healthy }
      cache: { condition: service_started }
    init: true
 
  db:
    image: postgres:17
    environment: { POSTGRES_USER: app, POSTGRES_PASSWORD: app, POSTGRES_DB: app }
    volumes: [pgdata:/var/lib/postgresql/data]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 2s
      retries: 15
 
  cache:
    image: redis:7
    command: ["redis-server", "--appendonly", "yes"]
 
volumes:
  pgdata:

depends_on alone only waits for the container to start, not for Postgres to accept connections. Use a healthcheck with condition: service_healthy, and still retry connections in the app.
docker compose down keeps volumes. down -v deletes them.
docker compose watch (or a bind mount plus a file watcher) syncs code changes without rebuilding.

Security

Root in the container is root on the kernel. Without user namespaces, UID 0 inside is UID 0 outside, limited by dropped capabilities, seccomp and AppArmor/SELinux profiles. A kernel bug or a misconfiguration (--privileged, mounting /) is a host compromise. Run as a non-root USER.
The docker group is root-equivalent. Anyone who can talk to the daemon can run docker run -v /:/host ... and own the host.
Mounting /var/run/docker.sock into a container gives that container root on the host. CI runners and “Docker-in-Docker” setups do this often. Treat it as such.
Rootless mode and Podman run the daemon (or no daemon) as an unprivileged user, which removes that class of problem at some cost in features and networking speed.
Tighten at run time: --read-only, --cap-drop ALL (add back only what’s needed), --security-opt no-new-privileges, resource limits.
Scan images for known CVEs (Docker Scout, Trivy, Grype) and rebuild often, because base images get patched under the same tag.

Performance

Startup is the process start plus image pull. Pulls dominate on fresh nodes, so smaller images and shared base layers matter more than anything inside the container.
CPU and memory overhead of the container itself is close to zero. It’s the same kernel scheduling the same process.
Network through the bridge and NAT adds a little latency. --network host removes it on Linux at the cost of isolation.
Filesystem writes to the overlay layer are slower than a volume, and copy-up of large files is expensive.
Bind mounts on macOS go through the VM’s file sharing (VirtioFS in current Docker Desktop). Large node_modules trees on a bind mount are the classic slowdown. Keep node_modules in a named volume or inside the image.
Build speed comes from cache hits: Dockerfile order, .dockerignore, cache mounts, and remote cache (--cache-to/--cache-from) in CI so each run doesn’t start cold.

Docker and the alternatives

Tool	What it is	When
Docker Engine / Desktop	Daemon + CLI + builder + Compose	Local dev, simple servers
Podman	Daemonless, rootless-first, Docker-compatible CLI	Where a root daemon isn’t allowed
containerd + nerdctl	The runtime Docker itself uses, without dockerd	Lean hosts, Kubernetes nodes
Kubernetes	Schedules containers across many machines	Many services, autoscaling, self-healing
Managed services (Cloud Run, ECS/Fargate, Fly, Render)	Run your image without managing hosts	Most small and mid-size apps
Buildpacks / Nixpacks	Build an image without writing a Dockerfile	When the default build is good enough

Kubernetes stopped using Docker as its runtime (dockershim was removed in 1.24), but it runs Docker-built images unchanged, because they’re OCI images.

A real-world example

A SaaS web app with an API, a background worker and a Postgres database:

One Dockerfile, two commands. The same image runs the API (node dist/server.js) and the worker (node dist/worker.js), so they can never drift in dependencies.
Local dev is docker compose up: Postgres and Redis with named volumes and health checks, the API with a bind mount and a watcher. New engineers don’t install either database on their laptop.
CI builds with BuildKit using a registry cache, runs tests inside the built image (so tests see exactly what production gets), scans it, and pushes it tagged with the git SHA.
Builds target linux/amd64 explicitly with docker buildx build --platform linux/amd64, because developers are on Apple Silicon and the servers aren’t. The first deploy without this failed with exec format error.
Production runs the image on a managed container service with memory limits. The app handles SIGTERM, stops taking new requests, finishes in-flight jobs, and exits within the grace period, so deploys don’t drop requests.
Migrations run as a one-off container from the same image before the new version takes traffic.

Gotchas

localhost inside a container means the container.
Listening on 127.0.0.1 inside the container makes published ports useless.
Shell-form CMD or npm start as PID 1 swallows SIGTERM. Deploys then wait 10 s and SIGKILL.
Exit code 137 means killed by SIGKILL, usually the OOM killer.
Images built on Apple Silicon are arm64 by default. On an amd64 server: exec format error.
depends_on doesn’t wait for readiness.
Deleting files in a later layer doesn’t make the image smaller.
Secrets in ARG, ENV or a copied .env are readable from the image forever.
No .dockerignore → huge build context, .env in the image, no cache hits.
-p 5432:5432 exposes the port publicly and can bypass ufw.
Database container without a volume loses data on recreate.
Alpine’s musl libc breaks some native modules and changes DNS behaviour.
:latest and other tags move. Production should deploy a SHA tag or digest.
docker system df and docker system prune exist because images, build cache and dangling volumes quietly fill disks.
Timezone, locale and CA certificates aren’t in minimal images unless you add them, which breaks date handling and outbound HTTPS in odd ways.

Practice

Run docker run --rm -it alpine sh, then ps aux, hostname, ip addr. Compare with the host. Then reproduce the PID namespace with unshare.
Write a Dockerfile for a small Node API in the naive order (COPY . . then npm ci). Time a rebuild after a one-line code change. Reorder it and time again.
Convert it to a multi-stage build and compare docker images sizes and docker history output.
Make the app ignore SIGTERM, time docker stop, then fix it with exec form plus a handler. Confirm docker stop returns at once.
Run a container with --memory 64m and allocate memory until it dies. Check the exit code and OOMKilled.
Build a Compose file with the API, Postgres and Redis. Remove the healthcheck and make the API crash on startup because Postgres isn’t ready, then fix it.
Find a layer containing a “deleted” secret: add a file in one RUN, delete it in the next, then docker save the image and find the file in the tarball.
Build for both linux/amd64 and linux/arm64 with buildx and inspect the resulting index with docker buildx imagetools inspect.

Sources worth reading

Docker docs: Dockerfile reference, Build cache, Multi-stage builds, Networking overview, Storage drivers
OCI Image Format and Runtime specifications (github.com/opencontainers)
man 7 namespaces, man 7 cgroups, and the kernel’s OverlayFS documentation
Julia Evans’ zines and blog posts on containers and namespaces
Why you need an init process inside your Docker container (the tini README covers PID 1 and zombie reaping)
Docker security docs: Rootless mode, Seccomp security profiles, Docker daemon attack surface

Things

Explorer

Docker

What is it?

Why it exists

Containers vs VMs

How it works underneath

The process chain

Namespaces: what the process can see

cgroups: what the process can use

Images and layers

Tags and digests

Writing a Dockerfile

`RUN` vs `CMD` vs `ENTRYPOINT`

PID 1 and signals

Networking

Storage: volumes, bind mounts, tmpfs

Docker Compose for local dev

Security

Performance

Docker and the alternatives

A real-world example

Gotchas

Practice

Sources worth reading

Graph View

Table of Contents

Backlinks

Things

Explorer

Docker

What is it?

Why it exists

Containers vs VMs

How it works underneath

The process chain

Namespaces: what the process can see

cgroups: what the process can use

Images and layers

Tags and digests

Writing a Dockerfile

RUN vs CMD vs ENTRYPOINT

PID 1 and signals

Networking

Storage: volumes, bind mounts, tmpfs

Docker Compose for local dev

Security

Performance

Docker and the alternatives

A real-world example

Gotchas

Practice

Sources worth reading

Graph View

Table of Contents

Backlinks

`RUN` vs `CMD` vs `ENTRYPOINT`